Interactive·

Regression Diagnostics: R² & Residuals

A high R² does not guarantee a good model when residual patterns remain.

An R² of 0.90 in a housing-price regression can look impressive. Yet if the residuals trace a curve or a few observations pull the fitted line, one number is not enough to trust the model.

Regression diagnostics ask whether linearity, constant variance, residual distribution, and influential observations are reasonably consistent with the model.

Key question

How can we examine both what the model explained and the structure it left behind?

Concept

What does R² tell us?

R² measures how much the model reduced error relative to using the sample mean alone

Coefficient of determination

R² = 1 − SSE/SST

Value	Interpretation
R² = 0	similar error to a mean-only model
R² = 0.60	60% of the outcome variation is explained within the sample
R² = 1	every observed value is fitted perfectly in the sample

Caution

R² does not guarantee causality, accurate coefficients, or strong prediction on new data.

What R² measures

SSE / SST

How much of the spread around the mean the model removed.

Residuals with no shape

healthy

Scattered evenly above and below zero is what you want to see.

Residuals with a curve

warning

A curve means the straight-line form missed part of the relationship.

Why It Matters

What do residuals reveal?

A residual is the observed value minus the predicted value

One home

If the actual price is $520,000 and the predicted price is $500,000, the residual is +$20,000. A positive residual means the model underpredicted the actual value.

•
A curved pattern suggests a nonlinear relationship that the straight-line model missed.
•
A funnel shape suggests that residual spread changes with the fitted value.
•
Runs of the same residual sign over time suggest possible autocorrelation.
•
Very large residuals or high leverage observations should be checked for influence.

How It Works

Which diagnostic plots should we inspect?

Each diagnostic view targets a different problem

Diagnostic	What it checks
Residuals vs. fitted	nonlinearity and heteroskedasticity
Q–Q plot	severe departures from residual normality
Cook’s distance	observations that strongly change the fitted coefficients
Standardized residuals	potential outliers
Residual order plot or ACF	autocorrelation

Do not diagnose a model from one plot alone. Combine graphs, statistics, and subject-matter context.

Example

Can a high R² still be misleading?

A model can have a high R² and still be wrong

A straight line fitted to a curved relationship may retain obvious curvature in the residuals despite a high R². Two unrelated time series that trend upward together can also produce a misleadingly high R².

Adjusted R²

R² almost always rises when predictors are added. Adjusted R² penalizes model size, but it does not replace residual diagnostics.

Interactive

Change curvature, outliers, and noise

Change curvature, an outlier, and noise

First increase noise and observe R². Then add curvature and an outlier and compare how the residual plot changes.

R²

0.939

RMSE

3.54

Nonlinearity0.0Outlier size0Random variation3

What to watch

Models with similar R² values can have completely different residual patterns. Inspect explained variation and leftover structure separately.

Key takeaways

•
R² is the proportion of variation explained in the sample, not a universal model score.
•
Residuals are observed minus predicted and reveal patterns the model left behind.
•
Well-behaved residuals are scattered around zero without a clear systematic structure.
•
Interpret R², adjusted R², residual plots, and influence diagnostics together.

Next: Heteroskedasticity and autocorrelation