Choosing a classification model
When you pick a prediction model, the algorithm's name matters less than what you're predicting and how you'll judge the result. Let the shape of your data, the cost of each kind of mistake, and your validation plan steer the choice.
Will this customer cancel? Will this loan go delinquent? Will this test come back positive? There's no shortage of models for questions like these, from logistic regression to random forests to XGBoost, but the most complex one isn't automatically the best.
Start by pinning down exactly what you're predicting, and plan to judge every model on data it never saw during training. Which cases you can't afford to miss will also decide which evaluation metric makes sense.
This guide won't walk through how each algorithm works or how to read its output; each method has its own page for that. Instead, it lays out how to match a model to your question and your data, and how to validate the comparison so you can trust it.
Four questions to answer before you pick a model
Define the prediction problem before you start comparing algorithms.
A map for choosing a classifier
Where to start, based on what you're predicting and what your data are like.
Rather than committing to one algorithm up front, set a simple model as your baseline and compare several candidates on the same data split and the same metrics. Which candidates belong on that list depends on how much data you have, what kinds of variables you're working with, and how much missingness and class imbalance you're dealing with.
Start with the basic classifiers
A simple baseline is the yardstick for judging anything more complex.
When you want probabilities and interpretable effects
Logistic regression is the workhorse for binary classification. You can read each predictor's relationship to the log-odds directly, and a well-specified model gives you predicted probabilities, too. There are extensions for outcomes with more than two categories.
When you want rules you can see
A decision tree splits cases with a series of if-then conditions on the predictors, so it's easy to explain to anyone. The catch: let the tree grow too deep and it starts memorizing the training data.
When you want to use distributions or distances
Naive Bayes is a fairly simple model built on probability assumptions. kNN classifies a new case by looking at its nearest neighbors. LDA and QDA assume a distribution for each class and estimate the boundary between them. Check each method's assumptions, and whether you have enough data for it, as you compare.
When the relationships are complex
Tree-based ensembles are the usual contenders for tabular data.
When you want to pool many trees
Random forests and bagging train many models and combine their predictions. The result is usually more stable than a single decision tree, but you lose the ability to show the whole decision process as one simple set of rules.
When you want to fix errors one step at a time
GBM, XGBoost, LightGBM, and AdaBoost belong to the boosting family. XGBoost and LightGBM in particular are strong contenders on tabular data, but getting that performance takes more than tuning: you also need to check for overfitting and validate fairly.
When you want to combine different models' strengths
Stacking feeds the predictions of several base models into another model. If you reuse the base models' predictions on their own training data, information leaks, so build the stack on cross-validated (out-of-fold) predictions.
| Approach | Typical methods | What to watch |
|---|---|---|
| Single model | Logistic regression, decision tree | Easy to interpret; good baseline |
| Bagging family | Random forest, bagging | Lower variance, more stable |
| Boosting family | GBM, XGBoost, LightGBM, AdaBoost | Captures complex patterns; needs tuning, risk of overfitting |
| Model combination | Stacking | Cross-validation design; preventing leakage |
When your data or goal is unusual
Class imbalance, time order, and how you'll use the probabilities all change the choice.
When positive cases are rare
When the cases you care about are rare, as with fraud or disease, look past accuracy to precision, recall, F1, and PR-AUC. Which one to prioritize depends on what a false alarm costs compared with a missed case. You can also choose the threshold that turns a predicted probability into a yes/no call on its own terms.
When records are linked by time or by person
If several records from the same patient or customer end up in both the training and test sets, performance will look better than it really is. When you're forecasting the future, keep the time order intact. When people or sites mustn't overlap between sets, split the data by group.
When you'll use the probabilities themselves
If the scores will be used to rank risk or set priorities, right-or-wrong isn't enough; the calibration of the probabilities matters too. Check that among people the model rates as high risk, the event actually happens at about the predicted rate.
When 12% of customers churn, how many of them the model actually catches can matter more than a high accuracy score. Pushing recall up means fewer churners slip through, but it can also mean more retention offers sent to people who were never leaving, so weigh it against precision.
What to check before you evaluate
Trust results from a setup that mirrors real use, not training scores.
| Check | Why it matters |
|---|---|
| Data leakage | No information from the evaluation data should reach training, whether through preprocessing, feature selection, or tuning. |
| Data splitting | Pick a random, time-based, or group split, whichever matches how the model will actually be used. |
| Cross-validation | Don't rely on a single split; look at how much performance varies across folds. |
| Class imbalance | Beyond accuracy and ROC-AUC, check PR-AUC, precision, and recall. |
| Overfitting | Compare training performance with validation and holdout performance; a big gap is a warning sign. |
| Interpretation and deployment | Consider explainability, run time, the cost of errors, and how the data may drift after deployment. |
Use the validation data to compare and tune models, and save the final holdout set for one last evaluation. If you peek at the holdout score and keep adjusting the model to improve it, you've effectively trained on the holdout too.
All 15 classification methods at a glance
Each method's own page covers how it works and how to read its results.
Here are the most common classification algorithms. They differ in how they handle binary versus multiclass problems, what preprocessing they need, and whether they output probabilities.
A good prediction model comes with a good validation plan.
The first decisions aren't about algorithms. They're about what you're predicting, what each kind of error costs, and how you'll validate. Pin those down before you reach for anything complex, then compare candidates on the same data split and metrics, under conditions that match real use.
Build a baseline with logistic regression or a decision tree, then bring in random forests and boosting if you need them. With heavily imbalanced data, check how many of the events you care about the model actually catches. If the data have a time or group structure, split the evaluation data to match. Guard against leakage during preprocessing and tuning, and make the final call based on performance on data the model never trained on.