Hyperparameters (like a decision tree's max_depth, or a random forest's n_estimators) are settings chosen before training, not learned from data — finding good values for them systematically, rather than by guesswork, is what tuning accomplishes.
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV
from sklearn.ensemble import RandomForestClassifier
param_grid = {
"n_estimators": [50, 100, 200],
"max_depth": [5, 10, 20, None],
"min_samples_split": [2, 5, 10],
}
grid_search = GridSearchCV(
RandomForestClassifier(random_state=42),
param_grid,
cv=5, # cross-validation within each combination
scoring="accuracy",
n_jobs=-1, # use all available CPU cores in parallel
)
grid_search.fit(X_train, y_train)
print(grid_search.best_params_) # the best combination found
print(grid_search.best_score_) # its cross-validated score
best_model = grid_search.best_estimator_ # ready-to-use model with the best settings
random_search = RandomizedSearchCV(
RandomForestClassifier(random_state=42),
param_grid, n_iter=20, cv=5, random_state=42, # tries 20 random combinations, not all of them
)
GridSearchCV exhaustively tries every combination of the specified hyperparameter values, evaluating each with cross-validation (covered earlier) — thorough, but the number of combinations grows multiplicatively with each additional parameter or value, quickly becoming computationally expensive for anything beyond a small grid.
RandomizedSearchCV instead samples a fixed number of random combinations (n_iter) from the specified ranges rather than testing every single one — counterintuitively, this often finds nearly-as-good hyperparameters in a fraction of the time, especially when many hyperparameters matter little and only a few actually drive performance, since random sampling explores the space more broadly per unit of computation than an exhaustive grid does.
n_jobs=-1 parallelizes the search across all available CPU cores, since each parameter combination's cross-validation is independent of the others — a straightforward way to meaningfully speed up what's otherwise a computationally expensive process. In practice, RandomizedSearchCV is often used first for a broad, fast exploration, followed by a smaller, more focused GridSearchCV around the promising region it identified.