Data leakage happens when information from outside the training data — often, indirectly, from the test set itself — improperly influences a model, producing evaluation scores that look excellent but don't reflect real-world performance.
# WRONG: scaling before splitting leaks test-set statistics into the training process
from sklearn.preprocessing import StandardScaler
X_scaled = StandardScaler().fit_transform(X) # fit on ALL data, including future test set
X_train, X_test, y_train, y_test = train_test_split(X_scaled, y) # too late — leak already happened
# CORRECT: split first, then fit preprocessing only on the training portion
X_train, X_test, y_train, y_test = train_test_split(X, y)
scaler = StandardScaler().fit(X_train) # fit ONLY on training data
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test) # test data only ever transformed, never fit
# A subtler leak: a feature that wouldn't be available at real prediction time
# e.g. predicting "will this loan default" using a feature like "total interest paid" —
# that number doesn't exist until AFTER the loan outcome is already known
The scaling example is the most common, mechanical form of leakage: fitting any preprocessing step (scaler, encoder, imputer) on the full dataset before splitting means the training process has effectively "seen" statistics derived from the test set, inflating evaluation scores in a way that won't hold up on genuinely new, unseen data. The fix, shown above and using a Pipeline (covered earlier) makes this nearly automatic, is simple but easy to forget: always split first, then fit every preprocessing step only on the training data.
A subtler and often more damaging form of leakage involves features that wouldn't actually be available at prediction time in a real deployment — a feature computed using information from after the event being predicted, or derived in a way that implicitly encodes the target itself. This kind of leakage doesn't show up as an obvious coding mistake; it requires actually understanding the data's provenance and asking, for every feature, "would this value genuinely be known at the moment a real prediction needs to be made?" A model riddled with this kind of leakage often shows suspiciously, almost too-good-to-be-true performance during evaluation — a strong signal worth investigating rather than celebrating, since real-world deployment performance for such a model is reliably far worse than what evaluation suggested.