1. Chaos engineering and game days

The interviewer points at your photo-sharing upload service, with its upload API on 15 instances across 3 zones, and asks how you know it survives losing a zone. "The redundancy is there" is a belief. Chaos engineering turns it into evidence by breaking part of the running system on purpose, in a controlled way, to see whether it keeps working. You find the weakness on a quiet afternoon, not during a real outage. A game day runs the same kind of failure with the on-call team involved, so it tests people and runbooks as well as machines.

The principle has four moves. Write down what normal looks like in numbers, such as upload success staying at or above 99.9%. That is the steady state. Predict that it holds when one named part fails. Inject the failure into a small slice of traffic or capacity, which bounds the blast radius, with a stop condition ready. Then compare what happened with the prediction.

Use it when a team has added redundancy it never exercised, before a launch that depends on surviving a failure, or when on-call has never rehearsed one. Skip it when there is no SLO (service level objective) or dashboard you trust, because you cannot tell what normal is. Skip it too for a weakness you already know will fail, such as a single-zone deployment. Proving it breaks costs real failed uploads, so fix it first.

This is a moderate-weight follow-up, usually 2 to 3 minutes of a reliability round right after you claim redundancy, so keep the answer short. Mid-level candidates write a hypothesis, name a small blast radius, and say what stops the experiment. Senior candidates also work out how many failed uploads the blast radius could cost against the error budget, argue staging vs production, and add a game day for the people side. The chapters follow the order you would make those decisions in.