Data Analyst · Mid-level · System design

A company runs dozens of A/B tests at the same time, and analysts calculate the results by hand in notebooks. Design the data side of an experimentation platform that calculates results automatically.

~10 min by voice

How to approach it

The interviewer wants to see whether you can turn manual analysis into a reliable pipeline: the right source of assignments, unified metric definitions, scalable computation and correct statistics with experiment health checks.

Have you seen this question in a real interview?

Answers0

Avg. score—

Strong—

Practiced0

Topics

ab testing experimentation platform

Similar questions

What are the most common mistakes people make when running A/B tests? Junior How would you plan an A/B test of a new checkout button in an online store? Junior You're planning an A/B test of a new onboarding flow, and the baseline conversion is ten percent. How would you figure out how many users you need and how long the test should run? Mid-level A product manager looks at A/B test results every day and stops the test as soon as the p-value drops below 0.05. What's wrong with that, and how would you fix it? Senior Design an internal A/B testing platform for a product with ten million users and a hundred concurrent experiments. What components would it consist of? Senior

Answer by voice

The interviewer will ask this question and listen. Think out loud and ask for a hint if you need one. When you're done, pause or tap “That's my answer”. In about a minute, your score and the points you covered show up here.

Best with headphones · ~10 min