Polynomial Regression

Does a used car’s price drop by the same amount every time it gets a year older? The price may fall quickly right after the car leaves the dealership, while the drop may be smaller for older cars.

Simple linear regression describes the relationship between two variables as a straight line. But if the change in real data is closer to a curve, a straight line alone can’t describe the relationship well enough.

Polynomial regression describes such curved relationships by adding squared or cubed terms of the explanatory variable to the regression equation. It isn’t just about drawing a nice curve; it is an analysis that checks whether the shape of the change is supported by the data.

In this lesson we use car age and sale price as an example to compare a straight line with a quadratic curve, and look at how to interpret the regression coefficients, explanatory power, residuals and overfitting.

Even if a used car’s price goes down as it gets older, it doesn’t go down by the same amount every year. The price may fall quickly right after the car leaves the dealership, and the drop may get smaller as time goes on.

Polynomial regression is a method that captures such nonlinear relationships by including squared, cubed and similar terms of the explanatory variable in the regression equation. Because the model is still linear in its coefficients, it can be seen as an extension of ordinary linear regression.

You can consider adding polynomial terms when a scatter plot shows a gentle curve or a bend, or when a regular pattern remains in the residuals after fitting a straight line.

But you shouldn’t keep adding higher-order terms just because the data look curved. Consider the purpose of the analysis, the range of the sample, interpretability, and predictive performance on validation data together.

The quadratic regression equation is ŷ = β₀ + β₁x + β₂x². β₁ is not a fixed change for a one-unit increase in x; it is part of a slope that depends on β₂ and on the current value of x.

The slope of the quadratic model is β₁ + 2β₂x. So the same one-year difference can mean a different expected price change for a 1-year-old car than for an 8-year-old car.

Suppose we recorded the car age (0–10 years) and sale price ($ thousands) of 60 hypothetical used cars of the same model and trim. The example data were generated to illustrate polynomial regression and are not real market prices.

Linear regression assumes that the expected sale price always changes by the same amount when a car gets one year older. A quadratic model, on the other hand, allows the rate of decline to level off as the car ages.

In this example, the blue curve is steep at first and flatter later on, which shows that even though the price keeps falling, the drop can get smaller as the car gets older. If the curve starts to rise at the end of the graph, you should think carefully about whether that really reflects rising prices in the market.

A higher degree can fit the training data better, but that doesn’t mean it is more accurate on new data. It is best to start comparing from the simplest model your research purpose needs.

In the hypothetical data actually computed, the p-value of the age² term was 5.057e-14. This result evaluates the contribution of the squared term under that model and its assumptions; it does not prove a causal effect.

If the same data are used, you can use a nested-model F-test for adding the quadratic term, or a test of its regression coefficient. If prediction is the goal, you should also check the cross-validation error, and you shouldn’t pick the quadratic model straight away just because R² went up a little.

In particular, polynomial regression and multinomial logistic regression, where the dependent variable has several categories, are different analyses that only sound alike.

In ordinary least squares estimation, check the independence of the observations, the form of the conditional mean function, constant residual variance, influential observations, and so on. For the usual small-sample inference, such as coefficient tests and confidence intervals, normality of the errors can also matter.

Because x and x² can be highly correlated, subtracting the mean from age before forming the squared term helps with interpreting the coefficients and with numerical stability. Centering, however, does not improve the model’s predictions themselves.

A polynomial may fit smoothly within the range of ages observed, but outside that range its predictions can rise or fall sharply. In particular, predicting the price of a 20-year-old car from data that go up to 10 years is risky.

Check outliers and influential points, residual plots, cross-validation error, and the balance between the amount of data and the degree together. If a complex model is needed, it can also help to compare it with other approaches such as splines.

Sample write-up (hypothetical data): To assess the nonlinear relationship between car age and used car sale price, a quadratic polynomial regression including age and age squared was performed. The overall model was statistically significant, F(2, 57) = 497.25, p < .001, R² = 0.946. The age-squared term indicated a curved relationship with price (b = 0.440, p < .001).

In a paper or report, add as needed the standard errors and confidence intervals of the regression coefficients, the models compared, the results of residual diagnostics, and the prediction range. When summarizing the results, describe them within the scope that the curved model describes the observed relationship between age and price.

It can be used to explore relationships where the rate of change may not be constant, such as price and age, age and performance, advertising spending and sales, or exercise time and fatigue.

But polynomial regression is not always the best answer. It is important to make the research question clear before the analysis, and to judge whether a curve is needed through visualization and residual diagnostics.

In this example, we compared a linear model and a quadratic model on the same used car data. Including the squared term makes it possible to express a relationship in which the rate of decline in the expected price changes with the car’s age.

But a curve fitting the data better does not mean the relationship is causal or that it holds outside the observed range. When interpreting, check the model comparison, the residuals and the prediction range together.