Standardization (Z-score Scaling)

Standardization transforms each feature to have mean 0 and standard deviation 1. It centers the data around the mean and scales it by the standard deviation, making it suitable for algorithms that assume centered or roughly Gaussian data.

Formula

Z = (X - μ) / σ

where:

• X = the data value
• μ = the mean of the feature
• σ = the standard deviation of the feature

A z-score of +2 means the value is two standard deviations above the mean.

Worked example

Values [10, 20, 30]: mean = 20, std ≈ 8.16.

X	(X - 20) / 8.16	Z
10	-10 / 8.16	-1.22
20	0 / 8.16	0.00
30	+10 / 8.16	+1.22

In scikit-learn

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_train_std = X_train.copy()
X_test_std = X_test.copy()

X_train_std[num_cols] = scaler.fit_transform(X_train[num_cols])   # learn mean/std from train
X_test_std[num_cols] = scaler.transform(X_test[num_cols])         # reuse them for test

print("Mean:", scaler.mean_.round(2))
print("Std :", scaler.scale_.round(2))

scaler.mean_ and scaler.scale_ store the learned mean and standard deviation.

Checking the result

print(X_train_std[num_cols].mean().round(3))   # ~0
print(X_train_std[num_cols].std().round(3))    # ~1

Manual implementation

col = X_train["Age"]
z = (col - col.mean()) / col.std(ddof=0)       # ddof=0 matches scikit-learn

Pros and cons

Pros	Cons
Works well for SVM, logistic regression, PCA, neural networks	Output is not bounded to a fixed range
Less sensitive to outliers than Min-Max	Still affected by extreme outliers (mean and std shift)
Preserves outlier information	Best when data is roughly bell-shaped

Normalization vs. Standardization

	Min-Max normalization	Standardization
Output range	Fixed, typically [0, 1]	Unbounded, mean 0, std 1
Outlier sensitivity	High	Moderate
Assumes a distribution?	No	Works best with roughly Gaussian data
Typical use	Neural nets, images	Most classical ML algorithms

Rule of thumb: use standardization by default. Use Min-Max when you need a bounded range.