Exploratory Factor Analysis (EFA)

Suppose we run a survey to find out how employees rate their work environment. The researcher wrote 12 items to measure job autonomy, relationships with coworkers, work burden and so on.

But will the items split into three areas as the researcher expected? Items written to measure job autonomy may actually be more closely tied to relationships with coworkers, and items thought to belong to different areas may group into a single factor.

In this way, the method that analyzes the correlations among several observed variables to explore the common latent factors behind them is exploratory factor analysis (Exploratory Factor Analysis, EFA).

EFA is used when the structure of survey items is unclear, or when developing a new measurement instrument and you want to find out which factors exist. The goal is to go beyond interpreting individual items one at a time and find the factors that act in common on several items.

In this article we look at the basic principle of exploratory factor analysis, a real-world example, the KMO and Bartlett tests, choosing the number of factors, factor loadings, communalities, rotation methods, cross-loadings, reliability and how to report the results.

For example, suppose we asked employees 12 questions about their work environment. Looking at the responses, items about how work gets done may be highly correlated with one another, and items about coworker relationships may also show similar response patterns.

EFA uses these relationships to explore which factors the items group into. Here, a factor represents a common characteristic that is hard to observe directly.

The basic factor model can be written as follows.

Here, X_i is an observed item, F_j is a common factor, λ_ij is a factor loading, and ε_i represents the item's unique variation and error.

An important feature of EFA is that the relationships between items and factors are not completely fixed before the analysis. The structure the researcher expects and the structure that appears in the actual data may differ, and checking this is itself the purpose of the analysis.

For example, you might expect a newly developed work environment perception scale to contain three factors: autonomy, collaboration and work burden. But whether this division also appears in the actual responses needs to be checked.

On the other hand, if a validated measurement model already exists and it is clearly specified which factor each item belongs to, confirmatory factor analysis (CFA) may be more appropriate.

In real scale development, you can explore a candidate structure with EFA and then examine that structure with CFA on independent data.

However, running CFA on the same data from which the EFA results were obtained can cause the information used to discover the structure to be reused in testing it, so it is best to use a separate sample when possible.

The survey consisted of 12 items, all answered on a Likert scale from 1 "strongly disagree" to 5 "strongly agree."

Survey items

The researcher expects that the items may consist of three factors: job autonomy, collaborative climate and work burden. But rather than forcing the analysis to extract exactly three factors, you can first examine the correlation structure that appears in the actual data.

Typical preliminary diagnostics are the KMO measure of sampling adequacy and Bartlett's test of sphericity.

KMO measure of sampling adequacy

KMO (Kaiser–Meyer–Olkin) is an index that evaluates suitability for factor analysis based on how small the partial correlations are compared with the correlations among the variables. KMO ranges from 0 to 1, and larger values are generally interpreted as indicating a correlation structure that common factors could explain.

Bartlett's test of sphericity

Bartlett's Test of Sphericity tests the null hypothesis that the population correlation matrix is an identity matrix. A significant result is evidence that correlations exist among the variables.

Preliminary diagnostic results

Since KMO is 0.86, it can be interpreted that a correlation structure suitable for factor analysis is likely to exist. Bartlett's test was also significant, so the null hypothesis that the correlation matrix is an identity matrix can be rejected.

However, satisfactory results on these two tests do not prove that a particular number of factors or factor structure is valid. You should also examine the correlations of individual items, the response distributions and the characteristics of the sample.

Extracting too few factors can merge different concepts into one factor. Conversely, extracting too many can include weakly meaningful factors and make the structure hard to interpret.

Typical methods for choosing the number of factors include eigenvalues (Eigenvalue), the scree plot (Scree Plot) and parallel analysis (Parallel Analysis).

Eigenvalue criterion

Traditionally, the Kaiser criterion, which retains factors with eigenvalues greater than 1, has been used. However, this criterion can over- or underestimate the number of factors, so using it alone is hard to recommend.

Scree plot

A scree plot is a graph of the eigenvalues in descending order. You can choose the number of factors by looking at the point where the eigenvalues drop sharply and then level off.

Parallel analysis

Parallel analysis compares the eigenvalues of the actual data with eigenvalues obtained from random data. Typically, the number of factors is chosen by retaining factors whose eigenvalues in the actual data exceed the reference values from the random data.

In this example, assume that after jointly considering parallel analysis, the scree plot and the interpretability of the factor structure, extracting three factors was judged appropriate. In a real analysis, you should consider not only statistical criteria but also whether each factor is theoretically meaningful.

Several extraction methods can be used in EFA; typical ones are principal axis factoring (Principal Axis Factoring) and maximum likelihood (Maximum Likelihood).

Principal axis factoring estimates factors by focusing on the common variance among items. Because it depends relatively little on the normality assumption, it is used with a wide range of survey data. Maximum likelihood estimates factors using a probability model, and under appropriate distributional assumptions it allows statistical tests and model comparisons.

In this example, assume principal axis factoring was used. However, when analyzing Likert items, you can also consider using polychoric correlations (Polychoric Correlation) instead of Pearson correlations, taking into account the number of response categories and the response distributions.

Results can differ depending on the extraction method and correlation matrix used, so it is good to state these settings when reporting the results.

Factor rotation (Factor Rotation) adjusts the orientation of the factors to make the relationships between items and factors easier to interpret. The two main types are orthogonal rotation (Orthogonal Rotation) and oblique rotation (Oblique Rotation).

Orthogonal rotation constrains the correlations between factors to 0. Varimax is the typical method. Oblique rotation, by contrast, allows the factors to correlate. Oblimin and Promax are examples.

Job autonomy, collaborative climate and work burden may well be related to one another. So in this example, assume an Oblimin rotation, which allows the factors to correlate, was used.

With oblique rotation, it is generally best to interpret factor loadings mainly from the pattern matrix (Pattern Matrix) and to check the factor correlation matrix as well.

Generally, the larger the absolute value of an item's loading on a particular factor, the more closely the item is interpreted as being related to that factor. Below is the pattern matrix after extracting three factors and applying oblique rotation.

Factor pattern matrix after rotation

Q1–Q4 showed high loadings on factor 1. Since these items concern autonomy in doing work and making decisions, factor 1 can be interpreted as "job autonomy."

Q5–Q8 showed high loadings on factor 2, and since they are items about information sharing and cooperation, the factor can be named "collaborative climate." Q9–Q12 showed high loadings on factor 3, which can be interpreted as a "work burden" factor.

As this shows, factor names are not decided automatically by the statistical software; the researcher must interpret them based on the content of the items with high loadings.

For example, if an item's communality is 0.65, about 65% of the item's variance is explained by the extracted common factors. A low communality may mean the item is not adequately explained by the factor structure it shares with the other items.

Communalities by item

We can assume that all items showed relatively high communalities. In practice, items with communalities below 0.30 are sometimes reviewed further, but this too is not an absolute criterion for removal.

In particular, with oblique rotation the factors are correlated, so communality must not be calculated simply as the sum of squared pattern coefficients after rotation. The correlations between factors must be taken into account. So in a real analysis, it is good to check the communalities reported by the statistical software together with the factor correlations.

For example, suppose Q4 had a loading of 0.55 on job autonomy and 0.49 on collaborative climate. In this case, it may be unclear whether Q4 measures job autonomy or collaborative climate.

When deciding whether to remove an item, check not only its highest loading but also the gap between it and its loadings on the other factors. For example, you could compare the following two items.

A cross-loading does not necessarily mean the item must be deleted, because the item may actually reflect both concepts.

However, if the scale requires the factors being measured to be clearly distinguished, you can consider revising or removing the item. In that case, rather than deciding on removal by statistical criteria alone, consider the item's conceptual importance and the coverage of the measurement as well.

For example, suppose the following correlations were found among the three factors.

There was a positive correlation between job autonomy and collaborative climate. This means that in environments rated high in job autonomy, the collaborative climate also tends to be rated positively.

Work burden, on the other hand, was negatively correlated with the other two factors. Respondents who rate job autonomy and collaborative climate more positively may tend to rate their work burden lower.

However, the mere existence of correlations between factors does not allow you to conclude that one factor causes another. Also, if a factor correlation is excessively high, you need to examine whether it is appropriate to treat the two factors as separate.

The eigenvalue-greater-than-1 rule is simple but does not always guarantee an appropriate result. It is good to consider parallel analysis, the scree plot and the interpretability of the factor structure together.

Do not confuse principal component analysis (PCA) with EFA.

PCA is a dimension reduction method that constructs new components using the total variance of the observed variables. EFA, by contrast, estimates latent factors that explain the common variance among items. So the two methods differ in purpose and interpretation.

Do not remove items based on factor loadings alone.

If an item has a low loading or a cross-loading, examine the statistical criteria together with the item's content and the concept you intend to measure. Repeatedly removing items to produce the structure you want carries a risk of overfitting.

Consider the measurement level of the response data.

When analyzing ordinal items such as Likert scales, you can consider using a polychoric correlation matrix, depending on the number of categories and the distributions. Also, if there are reverse-worded items, check the direction of responses before the factor analysis.

Do not judge sample size by a simple formula alone.

Rules such as "at least so many times the number of items" are sometimes used, but the required sample size depends on the factor loadings, communalities, number of factors, item structure and so on. So consider the stability of the factor structure as well as the sample size.

Do not treat EFA results as a validated structure.

EFA is a method for exploring possible factor structures in the data. For an important measurement instrument, it is good to examine the structure further on independent data, for example with CFA.

Summary of results

In a real study, it is good to present the final factor loadings, communalities and factor correlation matrix in tables. You can also report Cronbach's alpha or McDonald's omega to evaluate the internal consistency of each factor.

It is especially useful when developing a new survey scale or when it is unclear which concepts the items group into.

In EFA, you first check whether the correlation structure among items is suitable for factor analysis, then choose the number of factors using eigenvalues, the scree plot, parallel analysis and so on. You then extract and rotate the factors and interpret each item's factor loadings, communalities, cross-loadings and so on.

But the appearance of a particular factor structure in EFA does not mean that structure has been finally validated. You need to consider item content and theoretical grounds together, and where necessary it is good to run a confirmatory factor analysis on separate data.

Also, if you plan to combine each factor's items into scale scores based on the factor analysis results, you should examine the reliability of each factor as well.