Accuracy alone can be dangerously misleading, especially on imbalanced data — a model that always predicts "not fraud" is 99% "accurate" on a dataset where fraud is 1% of transactions, while being completely useless.
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
from sklearn.metrics import confusion_matrix, classification_report
import seaborn as sns
predictions = model.predict(X_test)
print("Accuracy:", accuracy_score(y_test, predictions))
print("Precision:", precision_score(y_test, predictions)) # of predicted positives, how many were right?
print("Recall:", recall_score(y_test, predictions)) # of actual positives, how many were caught?
print("F1:", f1_score(y_test, predictions)) # harmonic mean of precision and recall
print(classification_report(y_test, predictions)) # all of the above, per class, in one table
cm = confusion_matrix(y_test, predictions)
sns.heatmap(cm, annot=True, fmt="d", xticklabels=["Pred: No", "Pred: Yes"], yticklabels=["Actual: No", "Actual: Yes"])
The confusion matrix breaks predictions into four categories: true positives (correctly predicted positive), true negatives (correctly predicted negative), false positives (predicted positive but actually negative), and false negatives (predicted negative but actually positive) — every other classification metric is computed directly from these four counts.
Precision answers "of everything the model flagged as positive, what fraction actually was?" — important when false positives are costly (flagging a legitimate transaction as fraud, annoying a real customer). Recall answers "of everything that was actually positive, what fraction did the model catch?" — important when false negatives are costly (missing an actual fraud case, or a real disease diagnosis). These two metrics trade off against each other — a model can trivially achieve 100% recall by flagging everything as positive, at the cost of terrible precision, so the two must be weighed together based on which type of mistake is actually worse for the specific problem.
F1 score is the harmonic mean of precision and recall, a single number balancing both when neither should be prioritized outright — useful for comparing models overall, but always check precision and recall individually before trusting F1 alone, since two very different precision/recall trade-offs can produce the same F1 score.