1. Bayesian A/B testing

Your team runs a pricing page redesign, and the product lead says: "Stop sending me p-values. Tell me the probability the new page is better, and what we risk if we ship it." Bayesian A/B testing is the analysis that answers in those terms.

You treat each version's true signup rate as unknown and start with a belief about its plausible values. You update that belief with the signups you observed. Then you decide whether to ship based on the chance that the new page beats the old one and the expected loss in signup rate if you pick wrong. It changes the question you answer, not the evidence: on a large test it agrees with the p-value, and it does not make stopping at the first good-looking read safe.

Interviewers raise it as a follow-up to a standard analysis question, often phrased as "your team wants probability that B beats A instead of p-values, what changes?" It is an optional topic in most experimentation rounds. A mid-level answer should be able to update a posterior and compute the probability to beat control. A senior answer also chooses a prior and defends it, ships on expected loss rather than on probability alone, and says plainly what switching methods does not fix.

The tutorial assumes you know the frequentist read covered in Hypothesis Testing and Confidence Intervals: standard error, p-values, and confidence intervals. It stays with one finished comparison of a conversion metric, from prior to ship decision.