Instrumental Variables (IV) Analysis

Do residents who live near a subway station use their cars less? Or is it that people who drive a lot chose to live in areas with poor public transit in the first place? What should we do if unobserved factors such as income or job location affect both variables at once?

When there is an endogeneity problem between an explanatory variable and an outcome, instrumental variables (IV) analysis is the approach that tries to identify the causal effect by using a third variable that meets certain conditions.

Ordinary regression estimates the statistical relationship holding other conditions constant, but because of omitted variables or reverse causality it can’t be read directly as a causal effect. Instrumental variables analysis uses a variable that affects the explanatory variable but has no effect on the outcome other than through the explanatory variable. The hardest part, though, is judging whether these conditions actually hold.

The instrumental variables approach is used for education and wages, health care use, policy effects, transportation research and more. In this lesson we’ll use an example that analyzes public transit access and car use, with the layout of historical rail lines as a candidate instrument, to look at the principle, key assumptions, analysis steps, interpretation of results and cautions.

Instrumental variables analysis is an identification strategy that, when an explanatory variable X may be related to the error term, uses an instrument Z to exploit only a particular part of the variation in X. What we care about is the causal effect of X on Y; Z helps us estimate that effect.

In the example we set X = public transit access, Y = frequency of car use, Z = distance to historical rail lines. Historical rail lines are only a candidate in this example; we shouldn’t assume in advance that they are a valid instrument.

People who use their cars less may prefer neighborhoods with convenient transit. Also, local density, income, parking costs, job locations, urban planning and so on can affect both access and use at the same time. In this case the ordinary regression coefficient can mix in influences other than the effect of transit.

If it’s hard to observe every possible confounder perfectly, you can examine whether there is a suitable instrument that creates exogenous differences in access. But if the instrument is inappropriate, you can end up with results that are even more wrong than ordinary regression.

In the simple case of a single instrument and a single explanatory variable, the Wald estimator can be understood as the difference in mean Y by Z ÷ the difference in mean X by Z. It rescales the change in Y by how much Z changes X.

With continuous variables and several control variables, the effect is usually estimated with two-stage least squares (2SLS) or similar methods. The instrumental variable refers to the conditions and logic for identifying the causal effect, and 2SLS is one of the estimation methods that compute the effect under those conditions.

Suppose a hypothetical study observed 240 residential areas in several cities. Current transit access is measured with a 0–10 index, and car use is recorded as weekly car trips per household. The data below are a hypothetical setup for learning.

‘Distance to historical rail lines’ may be related to current transit access, but it could also affect car use directly through the path of urban development or land use. So being a historical variable doesn’t by itself guarantee exogeneity.

① Draw the causal paths first

Lay out the relationships among transit access, car use, income, population density and the historical rail layout. Look for any other path from Z to Y.

② Check first-stage relevance

With access as the dependent variable, estimate its relationship with the instrument and the control variables. Check for a weak-instrument problem at the same time.

③ Compare IV estimates with ordinary regression results

Estimate the IV effect with an appropriate method such as 2SLS, and show the estimates and confidence intervals side by side with the ordinary regression results. A difference between them doesn’t automatically make either one true.

④ Discuss possible violations of the assumptions

Carry out sensitivity analyses with different control variables, sample definitions and choices of instrument.

The F = 18.6 above is an illustrative number. People often check whether F exceeds 10, but that isn’t a universal safety threshold. If there is heteroscedasticity or a clustered structure, you should use appropriate weak-instrument diagnostics and critical values.

The following are separate illustrative hypothetical estimates to explain how to read a results table. They were not produced by fitting actual data and do not represent statistical test results.

The 2SLS estimate of −0.42 is interpreted as car use falling by an average of 0.42 trips per week when access rises by 1 point. But to read this figure as a causal effect, the instrumental variable assumptions must be valid, and whether the effect generalizes to the whole population has to be judged separately.

The IV confidence interval can be wider than the OLS one. The price of addressing confounding can be lower precision in the estimate. Don’t choose a model just by looking for a small p-value or a large coefficient.

Are historical rail lines really exogenous?

Historical railways may have influenced commercial centers, road development, population concentration and long-term urban expansion. If these paths still affect car use today, the exclusion restriction can be violated.

Is the instrument strong enough?

A weak instrument can make estimates unstable and undermine the validity of the usual confidence intervals. Consider appropriate inference methods that are robust to weak instruments.

Did you account for spatial correlation and group structure?

Areas within the same city may not be independent of one another. Depending on the data structure, consider clustered standard errors or spatial correlation.

Are the target of estimation and the units clear?

Check the units of the access index, how use per household is defined and which group changes because of the instrument, and limit the scope over which the findings are generalized.

IV and 2SLS aren’t completely separate, competing analyses. IV is a bundle of design and assumptions for identifying a causal effect, and 2SLS is the standard estimation technique for linear IV models.

The report should include how the data were collected and the sample, definitions of the outcome and the endogenous variable, the institutional and historical grounds for choosing the instrument, the first-stage coefficient and weak-instrument diagnostics, the IV estimate and confidence interval, how standard errors were computed, and sensitivity analyses.

Don’t report that “a causal effect was demonstrated” just because the relevance of the IV was confirmed. Describe the weaknesses of the exclusion restriction, possible alternative paths and the scope of interpretation as well.

The instruments in these examples are candidates a study might examine. This is not a list guaranteeing that the exclusion restriction holds in each study.

Even if a negative correlation is observed between transit access and car use, it can’t be read directly as the causal effect of public transit. Instrumental variables analysis is a way of finding an exogenous path through which access varies, to reduce problems caused by omitted variables or reverse causality. In the process you can look at candidates that may be related to the explanatory variable, such as the historical rail layout, but being related isn’t enough to make the analysis work.

What matters most is how convincingly you can support the exclusion restriction, that there is no other path by which the instrument affects car use, and the independence assumption. You also need to check for weak-instrument problems, the spatial structure of the data, the population the estimate targets and the uncertainty. If you can’t find a suitable instrument, it’s more sound to state the limits of identifying the causal effect clearly than to present IV estimates.