1. Quasi-experiments when you can't randomize

"The change already launched to everyone. How would you estimate its impact?" Interviewers ask this right after you propose an A/B test and they take it away.

The answer is a quasi-experiment: a way to estimate a change's effect without random assignment, by building a stand-in for what would have happened without the change. This tutorial teaches three designs that do that, difference-in-differences (DiD), synthetic control, and regression discontinuity (RD), and the one assumption each design depends on. If you can still randomize, even by whole city, a randomized test beats all three. These designs are for when that option is gone.

It rarely fills a round, but it shows up in product data science cases when a launch was rolled out by city, by a legal deadline, or by an eligibility rule. A mid-level answer explains why a before/after number misleads and computes a DiD. A senior answer also checks the assumption on data already in hand, uses synthetic control with placebo tests for a single market, runs RD with a manipulation check, and picks the design from how the change was rolled out.

The running example is a food delivery app in 50 United States cities with about 3M weekly active customers, measured by weekly orders per active customer. It made three changes, each rolled out differently. On March 1 it lowered the free-delivery minimum from $35 to $25 in 20 launch cities, while 30 waiting cities got it on May 1 because operations could not staff more. It launched a priority-delivery tier only in Denver. And it gives Gold status, free delivery on every order, to customers with 10 or more orders in the previous 90 days.

You should already know what a confidence interval is. Hypothesis Testing covers it. Propensity-score matching and instrumental variables are real tools too, but they go beyond interview depth and are not covered here.