Most machine learning algorithms work with numbers, so categorical features (text labels) must be encoded.
Types of categorical data
| Type | Description | Example |
|---|---|---|
| Nominal | No natural order | Color, city, gender |
| Ordinal | Has a meaningful order | Low < Medium < High, education level |
Encoding methods at a glance
| Method | How it works | Best for |
|---|---|---|
| Label / Ordinal | Integers 0, 1, 2, ... | Ordinal categories, tree models |
| One-Hot | One binary column per category | Nominal, low cardinality |
| Binary | Category id written in binary digits | Medium cardinality |
| Frequency / Count | Replace with the occurrence count | High cardinality |
| Target (mean) | Replace with the mean of the target per category | High cardinality (with care) |
1. Ordinal encoding
from sklearn.preprocessing import OrdinalEncoder
import pandas as pd
sizes = pd.DataFrame({"Size": ["Small", "Large", "Medium", "Small"]})
oe = OrdinalEncoder(categories=[["Small", "Medium", "Large"]])
sizes["Size_enc"] = oe.fit_transform(sizes[["Size"]]).astype(int)
print(sizes)
2. One-hot encoding
from sklearn.preprocessing import OneHotEncoder
ohe = OneHotEncoder(handle_unknown="ignore", sparse_output=False)
encoded = ohe.fit_transform(df[["Sex", "Embarked"]].fillna("S"))
print(ohe.get_feature_names_out())
encoded_df = pd.DataFrame(encoded, columns=ohe.get_feature_names_out())
handle_unknown="ignore" prevents errors when new data contains an unseen category. drop="first" avoids the dummy variable trap in linear models.
3. pandas shortcut
df_dummies = pd.get_dummies(df, columns=["Sex", "Embarked"], drop_first=True)
4. Frequency encoding
freq = df["Embarked"].value_counts(normalize=True)
df["Embarked_freq"] = df["Embarked"].map(freq)
5. Target encoding
Replaces each category with the average target value of that category. Powerful, but it can leak the target if computed carelessly.
from sklearn.preprocessing import TargetEncoder # scikit-learn 1.3+
te = TargetEncoder(smooth="auto", random_state=42)
train_enc = te.fit_transform(X_train[["Embarked"]].fillna("S"), y_train) # cross-fitted on train
test_enc = te.transform(X_test[["Embarked"]].fillna("S"))
Warnings
Avoid plain label encoding on unordered categories in linear or distance-based models: it invents a false order (for example "Dhaka" < "Khulna" < "Sylhet").
Compute target encoding only on training folds (or use scikit-learn's
TargetEncoder, which uses internal cross-fitting).
High-cardinality columns
Columns such as ZipCode or ProductID can have thousands of values. Options: frequency encoding, target encoding, grouping rare categories into "Other", or using embeddings.