1. Embeddings and retrieval

Almost every recommendation, search, or ads prompt reaches the same question: how do you find the few hundred best candidates out of tens of millions of items, fast enough to sit on a request path? Embedding-based retrieval is the standard answer. It turns users and items into vectors in one space, so that nearby vectors mean "this user would probably like this item," and then finds the nearest item vectors with an index built for that lookup. The index is only worth building at that scale. With a few hundred thousand items, comparing the user vector against every item vector takes milliseconds on one machine, so a plain scan is simpler and exact. The ranking stage after it belongs to Model serving.

We'll use one example throughout: the home page of a music streaming app. It has 50M tracks and 15M daily listeners. Each track is a 128-number float32 vector, which is 512 bytes per track and 25.6GB for the catalog. Retrieval must return 500 candidate tracks to the ranker in 20ms at p99.

This section is optional depth for many rounds. Plenty of first-layer machine learning (ML) interviews stop at "retrieve candidates with embeddings," and a mid-level answer explains what an embedding is, why a two-tower model lets item vectors be computed ahead of time, and that an approximate index replaces a full scan.

Search- and ranking-heavy rounds go one level further, and a senior answer compares an inverted file index (IVF) with a Hierarchical Navigable Small World (HNSW) graph using numbers, measures recall against exact search, and plans how the index is rebuilt when the model changes.