Random effects model

Why does academic achievement differ from school to school? At some schools students’ average scores are consistently high, while at others they may be relatively low. These differences may be related not only to a school’s education programs but also to many other factors, such as the local environment, the student body and school culture.

Suppose a researcher surveys 60 schools over five years, comparing the budget spent on education programs with students’ average scores. How scores changed when the budget rose at the same school matters, but the differences in budget and scores between schools can also be useful information for the study.

A standard panel data method you can use here is the random effects model.

In data that observe the same units over several time points, the random effects model treats the differences that exist between schools, firms or individuals as random individual effects. This lets you analyze the relationship between the explanatory variables and the outcome using not only change over time within groups but also differences between groups.

However, the random effects model requires an important assumption about the relationship between the unobserved individual effects and the explanatory variables. In this article we’ll cover the basic principle of the random effects model, a worked example, how to interpret the regression coefficients and variance components, how it differs from the fixed effects model, model choice and the key assumptions.

For example, suppose we observe the average scores of 60 schools for five years. Why average scores differ across schools may not be explained by the education program budget alone, because school culture, local characteristics and long-standing educational environments can also play a role.

The random effects model includes these school-level differences in the model as an individual effect. The term “random effects” here means that, instead of estimating each school’s distinct characteristics as a separate fixed intercept, the model treats them as individual effects that follow a particular probability distribution.

The basic random effects model assumes these individual effects are uncorrelated with the explanatory variables. If this assumption is appropriate, the regression coefficients can be estimated using both the differences between schools and the changes over time within the same school.

But if unobserved school characteristics are related to the education budget, the basic random effects estimates can be biased. So when applying a random effects model, it is important to understand the model’s basic assumptions.

Panel data need an identifier variable that distinguishes the units and a time variable that marks when each observation was made. For example, suppose we collected the education program budget and average scores of three schools.

There are two kinds of differences in these data. First, the education budget and scores change over time within the same school. Second, budgets and scores also differ between schools.

For example, school B has both a higher education budget and a higher average score than school A. At the same time, within school A, scores rose in the years when the budget increased. Under conditions such as the individual effects being uncorrelated with the explanatory variables, the random effects model uses both kinds of information.

The symbols mean the following.

In this model, u_i represents the persistent difference for each school. For example, school A may score higher on average even after the observed explanatory variables are taken into account, and school C may score relatively lower. These school-level differences are captured by the random effect.

The basic random effects model usually imposes the following condition.

This means the school-level random effect is not systematically related to that school’s explanatory variables over the whole observation period. If unobserved school characteristics also influence decisions about the education budget, this assumption may not hold.

The fixed effects model controls for each school’s time-invariant characteristics with school fixed effects. The random effects model, by contrast, represents school-level differences as random individual effects.

For example, suppose a school’s academic climate influences decisions about its education budget. That academic climate may also affect average scores. If academic climate wasn’t measured directly, the unobserved school characteristics may be correlated with the education budget.

In that case the assumption of the basic random effects model can break down. The fixed effects model can control for the influence of an academic climate that doesn’t change within a school. So the choice between the two methods should not simply be based on which model gives the smaller p-value.

The researcher collected each school’s education program budget and students’ annual average scores. If every school was observed every year, there are 300 school-year observations in total.

Variables in the analysis

The research question is as follows.

Assume the analysis included school-level random effects and year dummy variables. The year dummies control for year-to-year changes common to all schools. School location was set up as a variable that does not change during the observation period.

In a random effects model, if the assumption that the individual effects are uncorrelated with the explanatory variables is appropriate, the coefficients of time-invariant variables like this can also be estimated.

Random effects regression results

Assumes a model with year dummy variables; these are not estimates from real data.

The coefficient for the education program budget was 0.85. This means that, under the model assumptions about the other explanatory variables and the school-level random effects, a school with an education program budget $20,000 higher tended to have an average score about 0.85 points higher.

The coefficient for teachers per 100 students was 1.40. Other things being equal, one more teacher was associated with an average score about 1.4 points higher. The coefficient for the average attendance rate was 0.32, so an attendance rate 1 percentage point higher went with an average score about 0.32 points higher.

The coefficient for rural schools was -2.10. This means that, with the other explanatory variables held equal, rural schools had average scores about 2.1 points lower than urban schools.

These results describe a statistical association, though. They should not be interpreted as a causal effect, as if actually increasing the education budget by $20,000 would necessarily raise scores by 0.85 points.

Also, under the basic assumptions, the random effects coefficients are estimates that use both within-school and between-school variation. So it is not appropriate to interpret them, as you would in a fixed effects model, purely in terms of change within the same school.

These are called variance components. The basic one-way random effects model distinguishes the following two variances.

• Variance of the between-school random effect: σ_u²
• Variance of the time-specific error within the same school: σ_ε²

For example, suppose we got the following results.

The between-school variance reflects the variation in persistent differences in average scores from school to school. The within-school error variance is the time-to-time variation that remains after the explanatory variables and the school-level random effects are taken into account. Comparing the two variances shows how the variation not explained by the explanatory variables is split between and within schools.

However, the values of the variance components depend on the measurement units of the dependent variable, so they should not simply be compared with the variances of other outcome variables.

The ICC is a measure of how similar observations in the same group are. In a simple one-way random intercept model it can be calculated as follows.

In the example above, the between-school variance was 16 and the within-school error variance was 24. So the ICC is as follows.

In other words, about 40% of the total variance not explained by the model is due to persistent differences between schools. It can also be understood to mean that the conditional errors of two observations from the same school have a certain correlation. This interpretation applies only to the simplest random intercept model, however.

In models with random slopes or more complex error structures, the within-group correlation can vary with the time of observation or the values of the explanatory variables. Also, a high ICC alone does not tell you that the random effects model’s assumption of no correlation between individual effects and explanatory variables holds.

For example, suppose each school’s location is fixed as either urban or rural throughout the study period.

In a school fixed effects model, the school effects and the location variable overlap completely, so the coefficient for location cannot be estimated separately. The random effects model, by contrast, also uses differences between schools, so it can estimate the location coefficient. This feature can be useful in studies where the relationship between time-invariant characteristics and the outcome matters.

But there is an important condition. If location is related to unobserved school-level educational environments, and those environments also affect average scores, the basic random effects assumption can break down. So the advantage of being able to estimate time-invariant variables does not by itself guarantee that their coefficients are estimated without bias.

What matters most is whether unobserved unit-level characteristics are related to the explanatory variables. For example, suppose the education program budget is related to a school’s organizational culture or its principal’s leadership. If these characteristics also affect average scores but are not included in the data, the random effects model’s no-correlation assumption may be violated.

In that case a fixed effects model may be more appropriate. On the other hand, if the assumption that the individual effects are uncorrelated with the explanatory variables is reasonable, the random effects model can use both within-school and between-school variation and may have an advantage in estimation efficiency.

What to check when choosing a model

You can also use the Hausman test to compare the two methods.

In real research, though, this assumption can be too strong. For example, schools with higher education budgets may also differ in unobserved school culture or leadership. A common approach to this problem is the correlated random effects (CRE) model.

The Mundlak approach takes the correlation between the individual effects and the explanatory variables into account by adding each unit’s mean of the time-varying explanatory variables to the model. For example, you can calculate each school’s average education budget and include it in the model alongside the yearly education budget.

This lets you look separately at budget changes within a school and differences in average budget between schools. Under certain conditions, the within-unit effect of a time-varying explanatory variable can match the fixed effects estimate. But using a CRE model does not automatically solve time-varying confounding or reverse causality.

Also, interpreting the effects of time-invariant variables causally requires additional assumptions.

Correlation between individual effects and explanatory variables

This is the most important assumption of the basic random effects model. If unobserved school characteristics are related to the education budget, the regression coefficients can be biased.

Time-varying confounders

Including random effects does not control for every factor that changes over time. For example, a large shift in a school’s student body in a particular year could affect both the education budget and average scores at the same time.

Reverse causality

The education program budget may be related to average scores, but it is also possible that more budget was allocated to schools with lower scores. Reverse causality like this cannot be solved by an ordinary random effects model alone.

Correlated errors and heteroskedasticity

Repeated observations from the same school may be related across time points. If there is additional error correlation or heteroskedasticity beyond the basic random effects structure, the way standard errors are calculated and the model specification should be reviewed.

Sample composition and number of groups

If there are few schools or the number of observations per school varies greatly, the estimates of the variance components and regression coefficients can be unstable. So it’s a good idea to check the number of units, the number of time points and any missing observations as well.

Using the relationship between school education budgets and average scores from above, you could write it up like this.

In a real study, you could also examine whether the assumption of no correlation between the random effects and the explanatory variables is appropriate, and present comparisons with a fixed effects model or a correlated random effects model. It’s also a good idea to state how the variance components were estimated and how the standard errors were calculated.

It can use not only change over time within units but also differences between units, and under suitable conditions it can also estimate the coefficients of variables that don’t change over time. But the basic random effects model relies on the important assumption that the unobserved individual effects are uncorrelated with the explanatory variables.

So you should not choose a random effects model just because it can use more information than a fixed effects model. The right model should be chosen with the purpose of the analysis, the data structure, the nature of the individual effects and their relationship with the explanatory variables in mind.