A random forest trains many decision trees on randomized variations of the data and combines their predictions — an "ensemble" approach that fixes a single decision tree's biggest weakness: its tendency to overfit.
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(
n_estimators=100, # number of trees in the forest
max_depth=10, # still limits each individual tree
random_state=42,
)
model.fit(X_train, y_train)
predictions = model.predict(X_test)
probabilities = model.predict_proba(X_test)
importances = pd.Series(model.feature_importances_, index=X_train.columns)
print(importances.sort_values(ascending=False).head())
Each tree in the forest is trained on a different random subset of the training data (a technique called bagging, short for bootstrap aggregating) and considers only a random subset of features at each split — this deliberate randomness means individual trees make different mistakes from each other, so when their predictions are averaged (or, for classification, combined by majority vote), those individual errors tend to cancel out, producing a smoother, more reliable overall prediction than any single tree.
This is the core insight behind ensembling: a group of models that each make somewhat different mistakes performs better combined than any one of them alone, provided their errors aren't all identical. Random forests are one of the most reliable, low-effort "default" choices for tabular classification and regression problems — they handle non-linear relationships and feature interactions well, need relatively little preprocessing (no scaling required, since they're tree-based), and are noticeably more resistant to overfitting than a single unconstrained decision tree, at the cost of being somewhat less directly interpretable than one tree alone.