A model trained on historical data will faithfully learn whatever patterns — including unfair or discriminatory ones — exist in that data, since machine learning has no inherent understanding of fairness unless it's specifically checked for and addressed.
# Checking model performance across different subgroups, not just overall
for group in df["demographic_group"].unique():
mask = X_test["demographic_group"] == group
group_accuracy = model.score(X_test[mask], y_test[mask])
print(f"{group}: accuracy = {group_accuracy:.3f}, n = {mask.sum()}")
# a model that looks accurate overall might perform much worse for specific subgroups
# Checking for proxy variables — features that correlate strongly with a protected attribute
print(df.groupby("zip_code")["protected_attribute"].value_counts(normalize=True))
Historical bias is baked into data whenever past decisions reflected unfair patterns — training a hiring model on a company's past hiring decisions will reproduce whatever bias, conscious or not, influenced those past decisions, even if the protected attribute itself (gender, race, etc.) is deliberately excluded from the model's features.
This is exactly why simply removing a protected attribute from the feature set doesn't reliably solve the problem: other features can act as proxies, correlating strongly enough with the excluded attribute that the model effectively reconstructs and uses it indirectly — zip code frequently correlating with race or income level is a commonly cited real-world example. Checking model performance and error rates broken down by subgroup, not just in aggregate, is essential for catching this — a model with 95% overall accuracy could easily be 99% accurate for one group and 70% for another, a disparity invisible in the single overall number, and unacceptable in most real applications.
Responsible ML practice means actively auditing for these issues before deployment rather than only after a problem is publicly discovered — checking subgroup performance, questioning training data's historical origins and possible biases, and being appropriately skeptical of results that seem to confirm existing stereotypes or advantage the already-advantaged. This is as much a domain-knowledge and ethical-judgment skill as a technical one, and it's one of the most consequential things separating careful, trustworthy ML work from careless work that quietly automates existing injustice, treating fairness as a genuine, ongoing responsibility rather than a box to check once at the end of a project.