NewThe Learning module is now liveSee what's new
Intermediate
Training Data and Labels
20 min readChapter 2Core
1Training data and labelsChapter 1

A supervised model learns by copying answers. You give it past examples, each one a set of inputs paired with the right answer, called the label, and it learns to produce that answer for new inputs. The collection of those examples is the training data. Picture a payments processor that scores every card transaction for fraud in under 100ms.

To train that model, someone has to say, for millions of past transactions, "this one was fraud" or "this one was not." Where that answer comes from, how wrong it can be, and how you turn it into a clean set of examples is what this tutorial is about.

The usual default is to train on an outcome the business already logs: here, the chargeback, a cardholder's dispute of a charge. Analysts can label thousands of rows, not a billion, so hand-labeling is used for evaluation and hard cases rather than as the default training source. Labels copied from existing fraud rules are used only to bootstrap a first model before real outcomes exist.

In a machine learning (ML) system design round, "Where do your labels come from?" usually arrives right after you frame the problem, and the conversation often stays there for 5-10 minutes. Mid-level candidates are expected to name a label source, say why its weakness matters, and handle a rare positive class. Senior candidates are expected to raise label delay, the model biasing its own future labels, and leakage before the interviewer asks.

We will use the fraud model throughout: about 0.1% of transactions are fraud, and the confirmed label is a chargeback that mostly arrives within 60 days, with a tail to 120. You will come away able to audit the labels for any new prompt, whether it is fraud, spam, or click prediction: where they come from, what is wrong with them, and how to sample them into a training set an interviewer would trust. Choosing the prediction target itself, designing features, and splitting data for evaluation each have their own sections in this course.

Your progressLogin to save progress
Spot an issue or have a suggestion?
Feature Engineering  |

Discussion

Log in to join the discussion.