Reading the current lesson
Learn
Interactive·

Logistic & Probit Regression

How much does purchase probability change as ad exposure increases?

A purchase outcome is either 0 or 1, but the quantity of interest is the probability of purchase. A straight linear model can predict values below 0 or above 1 and assumes an unsuitable constant error variance.

Logistic and probit regression transform probability to an unbounded latent scale, fit a linear predictor there, and map the result back between 0 and 1.

Key question

How does a predictor shift purchase probability, and how should that shift be summarized?

1

Concept

Why is linear regression insufficient?

Binary regression produces an S-shaped probability curve rather than an unrestricted line

Logistic regression

log(p/(1−p)) = β₀ + β₁x

The model is linear in log-odds, not directly in probability. Converting back produces predictions constrained to the 0–1 range.

The S-shaped curve

0 < p̂ < 1
10

The curve flattens at both ends, so the predicted probability stays in range.

Straight in log-odds

logit(p)
logit(p)

On the logit scale the same relationship becomes a straight line.

The classification threshold

0.50
0.50

The model returns a probability; the cutoff is what turns it into a decision.

2

Why It Matters

How do logit and probit differ?

Logit and probit use different S-shaped links but often give similar fitted probabilities

ModelLinkPractical distinction
Logitlog-oddscoefficients exponentiate to odds ratios and are widely used
Probitinverse normal CDFoften motivated by an underlying latent normal threshold

Their coefficient scales differ, so raw coefficients should not be compared directly across the two models.

3

How It Works

How is an odds ratio interpreted?

Exponentiating a logistic coefficient gives an odds ratio

Advertising exposure

If β=0.40, the odds ratio is exp(0.40)=1.49. A one-unit increase in exposure is associated with approximately 49% higher odds, holding the other predictors constant.

Odds are not probability

An odds ratio of 1.49 does not mean probability increases by 49 percentage points. The probability change depends on the starting probability.

4

Example

Why is the probability change not constant?

The same coefficient produces different probability changes at different baselines

An exposure increase may barely change probability near 0 or 1 but produce a much larger change near the middle of the S-curve. Report predicted probabilities or marginal effects when practical interpretation matters.

  • •
    Coefficients describe change on the link scale.
  • •
    Odds ratios describe multiplicative change in odds.
  • •
    Marginal effects describe probability change at specified values or averaged across observations.
  • •
    Classification thresholds are a separate decision from fitting the probability model.
5

Interactive

Change the link and coefficient

Change the link function and coefficient

Switch between logit and probit and change the coefficient. Compare curve steepness, predicted probability, and how probability change depends on the current x value.

η

0.60

Probability

64.6%

Odds ratio

2.23

xProbability

What to watch

Probability effects are largest near the middle of the curve and smaller near the boundaries, even when the link-scale coefficient is constant.

Key takeaways

  • •
    Logistic and probit regression model binary outcomes through a bounded probability curve.
  • •
    Logit coefficients can be exponentiated into odds ratios; probit coefficients use a latent normal scale.
  • •
    Odds ratios are not percentage-point changes in probability.
  • •
    Use predicted probabilities or marginal effects for practical interpretation.
Next: Count-data regression