Most ML algorithms expect numeric input on comparable scales, and categorical (text) columns need to be converted to numbers before a model can use them at all.
from sklearn.preprocessing import StandardScaler, OneHotEncoder
# Scaling: many algorithms are sensitive to features being on wildly different scales
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train) # learns mean/std from TRAINING data only
X_test_scaled = scaler.transform(X_test) # applies that SAME transformation to test data
# Encoding: convert a text category into numeric columns
encoder = OneHotEncoder(sparse_output=False)
city_encoded = encoder.fit_transform(df[["city"]]) # "Dhaka" -> [1,0,0], "Sylhet" -> [0,1,0], ...
import pandas as pd
df_dummies = pd.get_dummies(df, columns=["city"]) # pandas shortcut for the same idea
StandardScaler transforms each feature to have mean 0 and standard deviation 1 — essential for algorithms like k-nearest neighbors, SVMs, and neural networks that compute distances or gradients, since a feature ranging in the millions (income) would otherwise completely dominate a feature ranging 0-1 (a ratio), regardless of each feature's actual importance. Tree-based models (decision trees, random forests) are a notable exception — they don't require scaling, since they split on threshold comparisons rather than distances.
A critical, easy-to-get-wrong rule: always call .fit_transform() on the training data and .transform() (not .fit_transform() again) on the test data — fitting the scaler separately on test data leaks information about the test set's distribution into preprocessing, a subtle form of the "data leakage" problem covered later, which silently inflates apparent model performance.
One-hot encoding converts a categorical column with k distinct values into k separate binary (0/1) columns, one per category — necessary because most algorithms can't meaningfully interpret an arbitrary numeric label like city = 2 as anything other than "greater than city = 1," a relationship that doesn't actually exist between unordered categories. pd.get_dummies() is Pandas' more concise shortcut for the same transformation, commonly used interchangeably with scikit-learn's OneHotEncoder in practice.