Feature Transformation

Feature transformation changes the shape, scale or distribution of an existing feature so that patterns become easier to learn.

1. Log and power transforms for skewed data

Many real-world features (income, price, counts) are right-skewed. A log transform compresses the long tail.

import numpy as np

df["Fare_log"] = np.log1p(df["Fare"])
df["Fare_sqrt"] = np.sqrt(df["Fare"])

print("Skew before:", round(df["Fare"].skew(), 2))
print("Skew after :", round(df["Fare_log"].skew(), 2))

Transform	Use when
log1p(x)	Right-skewed, non-negative data
sqrt(x)	Mild right skew, counts
x ** 2	Left skew
Box-Cox / Yeo-Johnson	Automatic choice of the best power

2. Binning (discretization)

Convert a continuous variable into categories.

df["AgeGroup"] = pd.cut(
    df["Age"],
    bins=[0, 12, 18, 35, 60, 120],
    labels=["Child", "Teen", "YoungAdult", "Adult", "Senior"],
)

df["FareQuartile"] = pd.qcut(df["Fare"], q=4, labels=["Q1", "Q2", "Q3", "Q4"])   # equal-frequency bins

Binning can capture non-linear effects (children and seniors behave differently) and reduces the influence of outliers, but it discards detail.

3. Polynomial and interaction features

Let linear models capture curves and combined effects.

from sklearn.preprocessing import PolynomialFeatures

poly = PolynomialFeatures(degree=2, include_bias=False)
X_poly = poly.fit_transform(df[["Age", "Fare"]].fillna(0))
print(poly.get_feature_names_out(["Age", "Fare"]))   # Age, Fare, Age^2, Age Fare, Fare^2

4. Interaction features by hand

df["Age_x_Pclass"] = df["Age"] * df["Pclass"]
df["Sex_Pclass"] = df["Sex"] + "_" + df["Pclass"].astype(str)     # categorical interaction

5. Reciprocal, ratio and difference features

df["FarePerAge"] = df["Fare"] / (df["Age"] + 1)       # +1 avoids division by zero
df["AgeDiffFromMean"] = df["Age"] - df["Age"].mean()

6. Transformations in a pipeline

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PowerTransformer, StandardScaler

pipe = Pipeline([
    ("power", PowerTransformer(method="yeo-johnson")),
    ("scale", StandardScaler()),
])

Tips

• Check the distribution (histogram) before and after the transform.
• Fit transformers such as PowerTransformer on the training set only.
• Tree-based models rarely need log or polynomial transforms; linear models and neural nets benefit the most.
• Too many polynomial features cause overfitting; combine them with regularization or feature selection.