Removing rows or columns with missing entries is the simplest way to handle missing data. It leaves a clean dataset, but it can reduce the sample size and introduce bias if the data is not missing at random.
Drop rows with any missing value
dropna() removes every row that has at least one missing entry.
df_cleaned = df.dropna()
print("DataFrame after removing rows with missing values:")
print(df_cleaned)
print("Rows before:", len(df), "| after:", len(df_cleaned))
With our sample data, only the rows without any missing value remain.
More control over dropna()
df.dropna(axis=1) # drop columns that contain any NaN
df.dropna(how="all") # drop rows where every value is NaN
df.dropna(subset=["Marks"]) # drop rows only if Marks is missing
df.dropna(thresh=7) # keep rows with at least 7 non-null values
| Parameter | Effect |
|---|---|
axis=0 (default) | Drop rows |
axis=1 | Drop columns |
how="any" | Drop if any value is missing |
how="all" | Drop only if all values are missing |
subset=[...] | Look only at the listed columns |
thresh=n | Keep rows that have at least n non-null values |
Pros and cons
| Pros | Cons |
|---|---|
| Simple and quick | Loss of data |
| Results in a fully complete dataset | Possible bias when data is not MCAR |
| No assumptions about the missing value | Can leave too few samples for training |
When is removal a good idea?
- Only a very small percentage of rows are affected (for example under 5%) and the data is MCAR.
- A column is mostly empty (for example over 60% missing) and adds little value.
- The missing value is in the target column — such rows cannot be used for supervised training.