Your team has redesigned the pricing page of a project-management tool, and about 6k eligible visitors reach that page each day. Before anyone flips the switch, the interviewer asks: how many users do you need, and how long will the test run? Answering it is called sample sizing.

You pick the smallest effect worth detecting, decide how often you are willing to be fooled, and the math tells you how many visitors each arm needs. Then you turn that count into whole weeks on the calendar. When the answer comes back as months of traffic, the fix is to raise the smallest effect worth detecting, switch to a more sensitive metric, or skip the test, not to run a smaller sample that cannot see the effect.

This question follows almost every A/B design prompt in product data science and analytics loops, usually right after you name the metric, and it takes 5 to 10 minutes. For a mid-level role, the bar is naming the four inputs, computing a number per arm for a conversion metric, and rounding the runtime to full weeks. Senior candidates are also expected to justify the effect size from the business case, calculate the extra time needed for an unequal traffic split, and say what to give up when the answer comes back too long. For engineers, knowing the inputs and how the sample scales is usually enough.

The tutorial assumes you can compute the standard error of a difference between two rates, as covered in Hypothesis Testing and Confidence Intervals, and it plans one fixed-length test of one primary metric with two arms. Shrinking a metric's variance and stopping a test early are separate sections of this course.