Imputation replaces missing values with estimated ones so that the dataset keeps its size. The filled values may not always represent the true data, so choose carefully.
1. Mean, median and mode imputation
Replace missing values with the mean, median or mode of the variable. It is simple, but it does not account for relationships between variables.
mean_imputation = df["Marks"].fillna(df["Marks"].mean())
median_imputation = df["Marks"].fillna(df["Marks"].median())
mode_imputation = df["Marks"].fillna(df["Marks"].mode().iloc[0])
print("Imputation using Mean:\n", mean_imputation)
print("Imputation using Median:\n", median_imputation)
print("Imputation using Mode:\n", mode_imputation)
df['Marks'].fillna(df['Marks'].mean())fills with the mean..median()fills with the middle value, robust to outliers..mode().iloc[0]fills with the most frequent value;.iloc[0]picks the first mode.
| Statistic | Best for |
|---|---|
| Mean | Numeric, roughly symmetric data without strong outliers |
| Median | Numeric, skewed data or data with outliers |
| Mode | Categorical or discrete data |
Constant (e.g. "Unknown") | When "missing" itself is a meaningful category |
Pros: easy, fast, suitable for numeric variables. Cons: may distort the true distribution and shrink variance.
2. Filling categorical columns
df["City"] = df["City"].fillna(df["City"].mode().iloc[0]) # most frequent city
df["Address"] = df["Address"].fillna("Unknown") # constant label
3. Forward fill and backward fill
These methods use the closest available value in the column. They are useful for ordered or time-series data.
forward_fill = df["Marks"].ffill()
backward_fill = df["Marks"].bfill()
print("Forward Fill:\n", forward_fill)
print("Backward Fill:\n", backward_fill)
- Forward fill copies the last valid observation forward.
- Backward fill pulls the next valid observation backward.
Older tutorials write
fillna(method="ffill"). That form is deprecated in recent pandas versions, so use.ffill()and.bfill().
Pros: preserves order and local patterns. Cons: inaccurate when gaps are large or neighbouring values differ a lot.
4. Imputation the safe way (no leakage)
from sklearn.impute import SimpleImputer
from sklearn.model_selection import train_test_split
X_train, X_test = train_test_split(df[["Marks", "Rank"]], test_size=0.25, random_state=42)
imputer = SimpleImputer(strategy="median")
X_train_imp = imputer.fit_transform(X_train) # median learned from TRAIN only
X_test_imp = imputer.transform(X_test) # applied to test data