Choosing a regression model
Look at what kind of outcome you have and how your data are structured, then pick the regression model that fits your goal.
Explaining how much a customer spends and predicting whether they'll buy at all sound like similar jobs. Can the same regression handle both? And what if your outcome is the number of doctor visits, or a satisfaction rating? Do you need yet another method?
Regression looks at how a set of variables relates to an outcome. Which model fits, though, depends on what kind of values the outcome takes and on how the data were collected. This guide doesn't rehash the math or how to read the output. It focuses on how to choose, and on what sets each method apart.
Five questions to answer before you choose
Start with the outcome, then look at how the data are structured and what you want from the analysis
What kind of outcome do you have?
Is it continuous, categorical, or a count?
Are the observations independent?
If you measured the same people repeatedly, or the data are grouped by school or hospital, the model has to account for it.
Does the distribution have any quirks?
Look for an unusually large number of zeros, a long tail of large values, or a lot of variability.
Explanation or prediction?
Interpretable coefficients, variable selection, and stable predictions can pull in different directions. Decide which matters most.
Is the measurement range limited?
Check whether some true values went unobserved because they hit an upper or lower limit.
Lots of zeros don't automatically call for a zero-inflated model. Ask why the zeros show up so often, and check how well an ordinary count model already accounts for them.
A decision map for regression
Start from the type of outcome, then move to an extension if your data need one
Start with the type of outcome
Everything begins with the value you're trying to explain
| Outcome | Examples | Consider first |
|---|---|---|
| Continuous | Test scores, monthly spending | Linear regression and its extensions |
| Two categories | Bought or not, readmitted or not | Binary logistic regression |
| Several unordered categories | How people commute | Multinomial logistic regression |
| Ordered categories | Satisfied, neutral, dissatisfied | Ordinal logistic regression |
| Count | Number of visits, number of accidents | Poisson or negative binomial regression |
| Positive, skewed continuous | Medical bills (always above $0) | Gamma regression or another GLM |
| Censored values | Values stuck at the top of the scale | Tobit regression or another censored model |
Two columns of numbers can mean very different things. A 1–5 satisfaction rating may really be an ordered category, while 0–5 visits per month is count data. Don't choose linear regression just because the outcome was entered as a number.
The main model families
Your goal and the type of outcome point you to a family of models
Explaining a continuous outcome
Linear regression is the natural starting point. But if the relationship curves, if you care about a particular quantile rather than the mean, or if you have repeated measurements, you'll want a different model.
When you have many predictors, or they overlap
Ridge, lasso, and elastic net all put a penalty on the coefficients. Where they differ is in how much they shrink the coefficients and whether they set some of them exactly to zero. They're especially useful for prediction, paired with cross-validation.
When the outcome is a category
It's not just how many categories there are; it also matters whether they have a natural order. Binary, multinomial, and ordinal logistic regression are not interchangeable.
Counts of events and skewed data
When you count events, the exposure time or the number of chances to observe an event often differs from one unit to the next, and the model needs to account for that. For a positive continuous outcome with a long right tail, a gamma GLM is often a better choice than an ordinary linear model with normal errors.
Excess zeros and measurement limits
A zero-inflated model treats the process that produces zeros separately from the process that produces counts. Tobit regression is for values that exist but are only partly observed because they hit an upper or lower measurement limit. The two solve different problems from the start.
Data structure and goals change the choice
Even with the same outcome, how it was observed and what you're asking both matter.
Many predictors, or strongly correlated ones
If high correlations among predictors make the coefficients unstable, or you have a lot of candidate predictors, consider ridge, lasso, or elastic net. Each penalizes the coefficients differently, so weigh predictive performance against whether you also want variable selection.
The same subjects measured more than once
When observations come in clusters, like each student's grades over several terms or patients nested within hospitals, the independence assumption is in trouble. A mixed-effects regression, which builds in differences between groups or individuals, is the usual answer.
Prominent outliers or skew
If a few observations have an outsized pull on the linear regression fit, consider robust regression. If you care about the median or an upper quantile rather than the mean, consider quantile regression. And for a positive, right-skewed outcome, a gamma GLM is another candidate.
Lots of zeros, or a limited measurement range
If the extra zeros in your count data come from a separate process, look at a zero-inflated model, but remember that lots of zeros alone don't make one necessary. If values were censored at a limit, a Tobit model may fit. Keep that distinct from truncation, where cases beyond the limit never make it into the sample at all.
Checks that apply to every model
Choosing a model doesn't mean you're ready to fit it
- UnitsConfirm the scale, units, and reference categories of the outcome and predictors.
- IndependenceDon't ignore repeated observations on the same person, similarity within groups, or time order.
- Functional formCheck linearity on the scale of the model's link function and for the predictors as you've entered them, not just as a straight line in the raw data.
- Distribution and varianceLook for overdispersion in counts, residual patterns and heteroscedasticity in continuous data, and extreme values.
- Missing data and model sizeReview how missing data are handled, the sample size, the number of predictors, and the number of events in each category.
- Model evaluationFor explanation, look at the coefficients, their uncertainty, and diagnostics. For prediction, look at performance on held-out validation data.
Even when a relationship shows up in the data, you still have to think separately about confounders, selection bias, which variable came first, and the study design.
Where should you start reading?
Find the family that matches your situation, then go to the detailed guide
| If your data look like this | Start with |
|---|---|
| Explaining a score or a dollar amount | Simple or multiple linear regression |
| Many predictors, or highly correlated ones | Ridge, lasso, elastic net |
| Passed or failed, chosen or not | Binary logistic regression |
| Three or more possible answers | Multinomial or ordinal logistic regression |
| Number of times an event happened | Poisson or negative binomial regression |
| Zeros seem to come from a separate process | Zero-inflated regression |
| Values stop at the edge of what can be measured | Tobit regression |
The detailed guide for each method, linked in the lists above, covers its assumptions, a worked example, and how to read the results.
Choosing a regression model starts with the outcome.
Then narrow it down by looking at how the data are structured, how they're distributed, and what you want the analysis to do.
Not every outcome can be handled with linear regression. A yes/no purchase calls for logistic regression, counts of events for Poisson or negative binomial regression, and censored values for Tobit regression, because each kind of outcome needs different assumptions. Even within a family, how the sample was drawn and how the variables relate lead to further choices.
Above all, first be clear about what your outcome represents, then check whether the observations are dependent, how they're distributed, whether measurement was limited, and what the study is for. More important than a method's name is understanding what it assumes about your data.