A dataset is imbalanced when one class (the majority) has far more samples than another (the minority). This is common in real applications:
- Fraud detection — 0.1% of transactions are fraudulent.
- Medical diagnosis — most patients do not have the rare disease.
- Spam / intrusion detection — malicious cases are rare.
- Equipment failure and churn prediction — failures and churners are the minority.
A standard model trained on such data tends to favour the majority class, and the minority class — which is usually the one we care about — gets ignored.
Why accuracy is misleading
Suppose 990 of 1000 patients are healthy. A model that always predicts "healthy" scores 99% accuracy but detects zero sick patients.
import numpy as np
from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score, recall_score
y_true = np.array([0] * 990 + [1] * 10) # 1% positives
X_dummy = np.zeros((1000, 1))
dummy = DummyClassifier(strategy="most_frequent").fit(X_dummy, y_true)
y_pred = dummy.predict(X_dummy)
print("Accuracy:", accuracy_score(y_true, y_pred)) # 0.99
print("Recall :", recall_score(y_true, y_pred)) # 0.0 <- finds no positives
Detecting imbalance
import pandas as pd
print(y.value_counts())
print(y.value_counts(normalize=True).round(3))
y.value_counts().plot(kind="bar")
Levels of imbalance
| Ratio (majority : minority) | Level |
|---|---|
| Up to 4 : 1 | Mild |
| 4 : 1 to 100 : 1 | Moderate |
| Above 100 : 1 | Severe |
Why it hurts learning
- The loss function is dominated by majority samples.
- Decision boundaries shift toward the minority class.
- Too few minority examples to learn its patterns.
- Overlap between classes makes the problem harder still.
Overview of solutions
| Level | Techniques |
|---|---|
| Metrics | Precision, recall, F1, PR-AUC instead of accuracy |
| Data | Random under/over-sampling, SMOTE, collect more data |
| Algorithm | Class weights, cost-sensitive learning, balanced ensembles |
| Decision | Threshold tuning |
| Reframing | Anomaly detection |
Create a demo dataset
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from collections import Counter
X, y = make_classification(
n_samples=10000, n_features=10, n_informative=5,
weights=[0.95, 0.05], flip_y=0, random_state=42,
)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, stratify=y, random_state=42 # stratify!
)
print(Counter(y_train), Counter(y_test))