Measuring Experiments in LLMs: A/B Tests and Automated Testing

Name: Measuring Experiments in LLMs: A/B Tests and Automated Testing
Start: 2026-04-10T08:00:00+00:00
End: 2026-04-10T08:25:00+00:00
Location: SMK

Talk 25 min April 10, 2026 at 08:00

Özge Çinko

Kader Miyanyedi

Even small changes in LLMs can impact output quality, safety, and user experience. In this talk, we’ll show how to log experiments with Langfuse, automate tests with Pytest, and enrich them using Hypothesis-generated random data scenarios. Participants will learn how to use code, tests, and data-driven A/B tests to improve LLM development.