Despite its name, logistic regression is a classification algorithm — it predicts the probability of a data point belonging to a given class, and decision trees offer a very different, rule-based approach to the same task.
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier, plot_tree
import matplotlib.pyplot as plt
log_model = LogisticRegression()
log_model.fit(X_train, y_train)
probabilities = log_model.predict_proba(X_test) # probability of each class, not just the label
predictions = log_model.predict(X_test) # the predicted class itself
tree_model = DecisionTreeClassifier(max_depth=4, random_state=42)
tree_model.fit(X_train, y_train)
plt.figure(figsize=(15, 8))
plot_tree(tree_model, feature_names=X_train.columns, class_names=True, filled=True)
Logistic regression applies a sigmoid function to a linear combination of features, squashing the output into a 0-1 probability — .predict_proba() returns that raw probability for each class, while .predict() applies a threshold (0.5 by default) to produce a final class label. Like linear regression, it's fast, interpretable via its coefficients, and a solid baseline — but shares the same core limitation of only capturing linear decision boundaries.
A decision tree instead learns a sequence of if/else questions about the features ("is income > $50,000? if yes, is age > 30? ..."), splitting the data at each step to separate classes as cleanly as possible — genuinely capable of capturing non-linear relationships and feature interactions that logistic regression cannot. max_depth controls how many questions deep the tree can go; an unrestricted tree can keep splitting until it perfectly memorizes the training data, which sounds good but is a textbook case of overfitting — capturing noise specific to the training set rather than a generalizable pattern, and performing poorly on new data as a result. Decision trees are also the direct building block for the ensemble methods (random forests, gradient boosting) covered later in this notebook.