Best Practices and Ideas by Data Type

Feature engineering ideas by data type

Data type	Ideas
Numeric	Ratios, differences, logs, squares, bins, rolling means
Categorical	Grouping rare levels, frequency counts, target encoding, interactions
Date-time	Year, month, day, weekday, hour, holiday flags, time since an event
Text	Length, word counts, TF-IDF, embeddings
Geographic	Distance to a landmark, region, cluster id
Time series	Lags, rolling averages, differences, seasonality flags

Basic text features

from sklearn.feature_extraction.text import TfidfVectorizer

reviews = pd.DataFrame({"text": ["Great product, fast delivery", "Bad quality!!! Never again"]})

reviews["n_chars"] = reviews["text"].str.len()
reviews["n_words"] = reviews["text"].str.split().str.len()
reviews["n_exclaim"] = reviews["text"].str.count("!")

tfidf = TfidfVectorizer(stop_words="english")
X_text = tfidf.fit_transform(reviews["text"])

Time-series features

sales = pd.DataFrame({"sales": [100, 120, 90, 130, 150, 140]},
                     index=pd.date_range("2026-01-01", periods=6, freq="D"))

sales["lag_1"] = sales["sales"].shift(1)                       # yesterday's value
sales["roll_mean_3"] = sales["sales"].rolling(3).mean()        # 3-day moving average
sales["diff_1"] = sales["sales"].diff()                        # change from yesterday

Bringing it all together

A single ColumnTransformer can apply different feature engineering steps to different columns.

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, PowerTransformer, StandardScaler

preprocess = ColumnTransformer([
    ("num_skewed", Pipeline([("imp", SimpleImputer(strategy="median")),
                             ("pt", PowerTransformer())]), ["Fare"]),
    ("num", Pipeline([("imp", SimpleImputer(strategy="median")),
                      ("sc", StandardScaler())]), ["Age", "FamilySize"]),
    ("cat", OneHotEncoder(handle_unknown="ignore"), ["Sex", "Embarked", "Title"]),
])

Best practices

1. Start from domain knowledge — ask what a human expert would look at.
2. Build features inside a pipeline / training data so no leakage occurs.
3. Test every new feature: does cross-validation performance improve?
4. Prefer simple, explainable features first.
5. Watch for multicollinearity and an explosion in the number of columns.
6. Never use information from the future or from the target to build a feature.
7. Document how every feature is created so it can be reproduced in production.
8. Keep an eye on train/serve consistency: the same code must run when the model is deployed.

Common pitfalls

Pitfall	Effect
Target leakage (features built from the label)	Great validation score, useless in production
Group statistics computed on the full dataset	Test information leaks in
Encoding unseen categories incorrectly	Errors or wrong predictions on new data
Too many polynomial or interaction terms	Overfitting
Forgetting to apply the same transformations to test data	Inconsistent predictions

Automated feature engineering

Libraries such as Featuretools (deep feature synthesis), tsfresh (time-series features) and feature-engine can generate many candidate features automatically. You still need to filter them with feature selection and validate them with cross-validation.

Key takeaways

• Feature engineering injects domain knowledge into the dataset and is often the biggest lever for model quality.
• Common tools: combining, extracting, transforming, binning and encoding.
• Always validate new features with cross-validation and avoid leakage.