Data Leakage and Preparation Checklist

What is data leakage?

Data leakage happens when information that would not be available at prediction time influences model training or preprocessing. It produces unrealistically high validation scores and models that fail in production.

Common sources of leakage

Source	Example
Preprocessing before splitting	Scaling the whole dataset, so test-set statistics leak into training
Target leakage	A feature computed from the target, such as "refund issued" when predicting churn
Duplicates across splits	The same row in both train and test
Time leakage	Using future rows to predict the past in time-series data
Group leakage	The same patient or customer in both train and test

Rules to avoid leakage

1. Split first, then fit transformers on the training data only.
2. Never build features from the target column.
3. Remove duplicates before splitting.
4. For time-ordered data, split by time.
5. Use pipelines so cross-validation re-fits preprocessing inside every fold.

Data preparation checklist

• I understand every column and the target
• Duplicates removed, data types and formats fixed
• Missing values handled deliberately (not just dropped)
• Outliers examined and treated based on their cause
• Data split before fitting any transformer
• Categorical variables encoded, numeric features scaled where needed
• Class balance checked
• Everything wrapped in a reproducible pipeline
• Every decision documented