Canonical Correlation Analysis (CCA)

How closely is a lake’s water quality related to its fish community? Suppose a research team measured water quality in 120 lakes and, at the same sites, measured the number of fish species, the number of fish, and fish biomass.

The researchers want to know not just the correlation between one water quality measure and one fish measure, but how the water quality measures as a whole relate to the fish ecology measures as a whole.

A method you can use here is canonical correlation analysis (CCA). CCA forms a linear combination within each of the two sets of variables and chooses the weights so that the correlation between the two combinations is as large as possible. After finding the first relationship, it goes on to look for the next relationship that does not overlap with the ones already found.

In this lesson we cover how to set up the two variable sets, how canonical variates and canonical correlations are calculated, the difference between loadings and weights, significance tests, interpreting the results, and cautions for the analysis.

The example numbers are made up for illustration.

For example, you can calculate the correlation between a lake’s nitrate concentration and its number of fish species, or between turbidity and fish biomass. But when there are many measures, individual correlation coefficients alone make it hard to grasp the overall relationship.

CCA splits the variables into two sets. With the water quality measures as the first set and the fish ecology measures as the second, it forms weighted sums of the variables in each set to find the two composite scores that are most strongly related to each other. These composite scores are called canonical variates.

CCA differs from ordinary regression, which designates one variable as the dependent variable and predicts it. It explores the symmetric structure of association between two sets of variables, and the results alone do not tell you which set is the cause of the other.

Before the analysis, all measures are checked for units, outliers and missing values. In particular, differences in lake size or survey effort can affect the fish measures, so where needed, standardize the variables appropriately or account for related factors separately.

In this example, each set has 3 variables, so up to 3 pairs of canonical variates can be extracted.

If we call the canonical variate of the first set U and that of the second set V, they are:

Here a and b are the weights given to each variable. CCA chooses the weights so that the correlation between U and V, Corr(U, V), is as large as possible. A normalization constraint is applied to the weights, such as fixing the variance of the canonical variates, so you cannot raise the correlation by making the weights arbitrarily large.

The first pair of canonical variates represents the largest linear correlation between the two sets. The second pair finds the next largest correlation using new axes that are uncorrelated with the earlier canonical variates of the same set. Further possible pairs are extracted in the same order.

The sign of each canonical variate can be flipped arbitrarily. So rather than looking at a single sign in the results table, interpret in which directions the variables move together within a canonical variate.

For example, a first canonical correlation of 0.78 means a strong positive correlation was estimated between the optimal linear combination of the water quality measures and the optimal linear combination of the fish measures.

The squared canonical correlation can be read as the proportion of variance shared by that pair of canonical variates. For the first function, 0.6084 means the two canonical variates share about 60.8% of their variance.

This should not be interpreted as ‘the water quality measures explain 60.8% of the variance in all the fish ecology measures.’ To assess how much of the original variable sets as a whole is explained, you need to look separately at the loadings, the redundancy index, and so on.

In this example each set has 3, so there are at most 3 functions. But being able to extract a function is not the same as it being worth interpreting.

If the first function has a large correlation, the second a moderate one, and the third a very small one, you can decide which functions to interpret by looking at the statistical tests, the loadings and theoretical interpretability together. In particular, even if a canonical correlation comes out large in the sample, it may not replicate in new data, because it is partly the product of chance optimization.

Rather than interpreting every function out of habit, it is good to weigh statistical significance, effect size, the content of the variable sets, and stability under cross-validation together.

Depending on the sample size and distributional assumptions, a chi-square or F approximation is used. The test results below are for understanding the method.

In this example, you could focus your interpretation on the first and second canonical functions and treat the third with caution. But a p-value does not tell you how important a function is in practice, so also check the size of the canonical correlations and the loadings.

Real test statistics must be calculated so that they match the canonical correlations estimated from the same data; the table above does not reproduce software output.

Canonical loadings (also called structure coefficients), on the other hand, are the correlations between the original variables and the canonical variate of their own set.

Weights can become unstable in size and sign when the variables within the same set are highly correlated. So when explaining which variables represent the meaning of a canonical function, people often focus on the canonical loadings.

Cross-loadings are the correlations between the original variables and the canonical variate of the opposite set. They show how strongly each variable is related to the composite score of the other set. Because these three measures are calculated differently and serve different interpretive purposes, they should be clearly distinguished in the results table.

In the water quality set, dissolved oxygen has a high positive loading, while nitrate and turbidity have negative loadings. So the first water quality canonical variate can be interpreted as an axis representing water that is rich in oxygen and low in nitrate and turbidity.

In the fish set, species count and biomass have high positive loadings. So the first fish canonical variate can be interpreted as an axis representing a fish community with high diversity and large size.

Because the first canonical correlation is 0.78, the two combinations of characteristics show a strong positive relationship. But this result alone does not support the causal conclusion that good water quality in a lake necessarily increased fish diversity. There may be other factors that affect both, such as lake size, location, or how well the lake is managed.

For example, in the second function, turbidity on the water quality side and fish density on the fish side might have relatively large loadings. That would let you explore a different kind of relationship from the general level of water quality and community captured by the first function.

However, the second function’s correlation is 0.43, smaller than the first. And even if the function is statistically significant and stable enough, it is better not to force a name onto it if the variables’ loadings have no consistent meaning.

When interpreting a function, look not at the signs but at the combination of variables with relatively large absolute values within the same function, and check whether that combination fits with field knowledge.

That is because how much is actually explained depends on how well each canonical variate represents the original variables of its own set.

The measure that looks at this is the redundancy index. It is calculated by combining the proportion of variance extracted, which shows how much a set’s canonical variate explains its own set’s variables on average, with the squared canonical correlation.

For example, if the average proportion of variance extracted by the first fish canonical variate is 0.61 and the squared first canonical correlation is 0.6084, then 0.61 × 0.6084 ≈ 0.371.

In other words, in this illustrative calculation, the water quality canonical variate reproduces about 37.1% of the variance of the fish variable set on average. In real studies, you should calculate the redundancy index for each canonical function and state that the value can differ depending on the direction.

The 0.61 above is a separate summary value used to show how the calculation works.

Even when variables are measured in different units, the canonical correlations themselves are invariant to suitable linear rescaling, but standardization can be useful for comparing the size of canonical weights and for numerical stability.

In particular, if a set contains many measures that are essentially duplicates, the covariance matrix can become unstable or the weights hard to interpret. If there are too many variables relative to the sample size, there is also a risk of overestimating the canonical correlations.

If there are factors that may affect both sets of variables, such as lake area or survey timing, you can consider treating them as control variables or using partial canonical correlation analysis.

On the other hand, if the same lakes were measured across several seasons, you need to revisit the independence of the observations. The traditional significance tests in CCA can be affected by assumptions such as multivariate normality, so if needed, consider permutation tests or resampling methods.

PCA does not maximize the relationship between two sets. Instead, it finds axes that summarize the variation within one set well. CCA has a different goal in that it maximizes the correlation between two sets. Regression differs in that it clearly designates an outcome variable to predict, or estimates conditional relationships.

Where needed, you can also report the redundancy index and resampling or cross-validation results.

An actual manuscript should add the test statistics produced by the software and the exact sample conditions.

Instead of checking the correlation of each pair of variables one by one, it combines related variables so you can see which combinations move together.

But a high canonical correlation alone does not mean the original variable sets as a whole are well explained, or that a causal relationship has been found. You need to look at the canonical loadings, the redundancy index, the sequential test results and out-of-sample stability together.