Feature Scaling in Practice

The golden rule: fit on train only

Scaling before splitting lets test-set information (its min, max, mean and std) leak into training. Always:

1. Split the data.
2. Fit the scaler on the training data.
3. Transform train, validation, test and any future data with the same fitted scaler.

Complete Titanic example

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler, StandardScaler

df = pd.read_csv("Titanic-Dataset.csv")
X = df[["Pclass", "Age", "SibSp", "Parch", "Fare"]].copy()
X["Age"] = X["Age"].fillna(X["Age"].median())
y = df["Survived"]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

num_cols = ["Age", "SibSp", "Parch", "Fare"]

mm = MinMaxScaler()
X_train_mm, X_test_mm = X_train.copy(), X_test.copy()
X_train_mm[num_cols] = mm.fit_transform(X_train[num_cols])
X_test_mm[num_cols] = mm.transform(X_test[num_cols])

ss = StandardScaler()
X_train_ss, X_test_ss = X_train.copy(), X_test.copy()
X_train_ss[num_cols] = ss.fit_transform(X_train[num_cols])
X_test_ss[num_cols] = ss.transform(X_test[num_cols])

print(X_train_ss[num_cols].describe().loc[["mean", "std"]].round(2))

Use a pipeline

A pipeline guarantees the correct order automatically, including inside cross-validation.

from sklearn.pipeline import make_pipeline
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import cross_val_score

model = make_pipeline(StandardScaler(), KNeighborsClassifier())
print(cross_val_score(model, X_train, y_train, cv=5).mean())

Scaling only some columns

Use ColumnTransformer so binary flags and one-hot columns are left untouched.

from sklearn.compose import ColumnTransformer

ct = ColumnTransformer(
    [("scale", StandardScaler(), ["Age", "Fare"])],
    remainder="passthrough",          # leave the other columns unchanged
)
X_train_ct = ct.fit_transform(X_train)
X_test_ct = ct.transform(X_test)

Impact on a distance-based model

knn = KNeighborsClassifier()

print("Without scaling:", knn.fit(X_train, y_train).score(X_test, y_test))
print("With scaling   :", knn.fit(X_train_ss, y_train).score(X_test_ss, y_test))

Typically the scaled version scores clearly higher, because the large-valued Fare no longer dominates the distance.

Saving the scaler for production

import joblib

joblib.dump(ss, "scaler.pkl")           # save the fitted scaler
scaler = joblib.load("scaler.pkl")      # reuse it when serving predictions

Practical tips

• Do not scale the target, binary flags or one-hot encoded columns (usually unnecessary).
• Handle outliers first, or choose RobustScaler.
• Tree-based models do not need scaling; skipping it keeps features interpretable.
• Never fit a scaler on validation or test data.
• Check the result with .describe(): mean about 0 and std about 1 for standardization, min 0 and max 1 for Min-Max.

Quick decision guide

Situation	Scaler
Default for most classical ML	StandardScaler
Neural network or need a bounded range	MinMaxScaler
Many outliers	RobustScaler
Sparse data	MaxAbsScaler
Text or cosine similarity	Normalizer
Heavily skewed feature	PowerTransformer or a log transform