Data Analyst · Mid-level · Theory

You're planning an A/B test of a new onboarding flow, and the baseline conversion is ten percent. How would you figure out how many users you need and how long the test should run?

~4 min by voice

How to approach it

They're checking whether you can plan an experiment before launch: calculate the sample size from the statistical parameters and the business goal, convert it into a duration given real traffic and seasonality, and fix the plan in advance rather than peeking at the results.

Have you seen this question in a real interview?

Answers0

Avg. score—

Strong—

Practiced0

Topics

ab testing statistical power

Similar questions

A company runs dozens of A/B tests at the same time, and analysts calculate the results by hand in notebooks. Design the data side of an experimentation platform that calculates results automatically. Mid-level What are the most common mistakes people make when running A/B tests? Junior How would you plan an A/B test of a new checkout button in an online store? Junior A product manager looks at A/B test results every day and stops the test as soon as the p-value drops below 0.05. What's wrong with that, and how would you fix it? Senior Design an internal A/B testing platform for a product with ten million users and a hundred concurrent experiments. What components would it consist of? Senior

Answer by voice

The interviewer will ask this question and listen. Think out loud and ask for a hint if you need one. When you're done, pause or tap “That's my answer”. In about a minute, your score and the points you covered show up here.

Best with headphones · ~4 min