Kruskal–Wallis Test
A nonparametric method that compares data from three or more independent groups using ranks to test whether the groups differ.
If the counseling program changes, does user satisfaction change too? What if you compared the satisfaction of people who used basic counseling, tailored counseling, and counseling that includes follow-up care? If responses are on a 1–5 scale, what is the right way to check whether the three groups differ?
The Kruskal–Wallis test is a method for comparing the responses of three or more independent groups using ranks. It is often introduced as the nonparametric alternative to one-way analysis of variance (one-way ANOVA), and it uses the ranks of all observations instead of the raw values.
Where one-way ANOVA focuses on comparing group means, the Kruskal–Wallis test checks whether the groups have the same rank distribution. When the condition that the distributions have a similar shape is met, the result can also be described in terms of a difference in location or median.
It is useful in research that uses ordinal scores or skewed data, such as surveys, health research, and educational assessment. In this article we cover the basic idea, a worked example, ranks and the H statistic, post hoc tests, interpreting the results, how it differs from one-way ANOVA, and points to watch out for.
What is the Kruskal–Wallis test?
A test that compares several independent groups using ranks
The Kruskal–Wallis test evaluates whether the ranks of the observations follow a similar distribution across three or more groups. You can think of it as extending the Mann–Whitney U test for two independent groups to three or more groups, but the key point is that it tests all the groups at once.
What research questions is it used for?
When the outcome is ordinal or the normality assumption is hard to defend
You can consider it when you are comparing three or more independent groups and the outcome can at least be put in order. Examples include comparing satisfaction at three hospitals, preferences for three teaching methods, or wait times in several regions.
| Item | This example |
|---|---|
| Groups | Basic counseling / tailored counseling / counseling + follow-up care |
| Outcome | Service satisfaction, 1–5 points |
| Design | Each respondent experienced only one counseling approach |
| Hypothesis | The three groups have the same satisfaction distribution |
How does the test work?
Convert all observations to ranks, then compare each group’s rank sum
First, pool the observations from all groups and rank them from smallest to largest. When the same score appears more than once, each tied value gets the average of the tied ranks. Then calculate how far each group’s rank sum is from the level you would expect across all groups.
A large H does not automatically tell you which two groups differ. That takes a separate post hoc comparison.
Worked example: comparing satisfaction by counseling approach
A 1–5 satisfaction survey of 45 hypothetical users
Fifteen different users took part in each of the three counseling approaches. After their sessions, they rated their satisfaction from 1 (very dissatisfied) to 5 (very satisfied). Because the same scores appear many times, ties are handled.
| Counseling approach | n | Median | Mean rank |
|---|---|---|---|
| Type A (basic counseling) | 15 | 3.0 | 11.33 |
| Type B (tailored counseling) | 15 | 4.0 | 23.30 |
| Type C (counseling + follow-up) | 15 | 5.0 | 34.37 |
How do you read the test results?
Check the H statistic and the p-value first
| Statistic | Result | Meaning |
|---|---|---|
| Sample size | 45 | Total respondents across the three groups |
| H statistic | 25.60 | Reflects rank differences between groups |
| Degrees of freedom | 2 | Number of groups − 1 |
| p-value | 2.761e-06 | A measure of how likely a difference this large is under the null hypothesis |
In this example, H(2) = 25.60, p < .001. At the .05 significance level, we reject the null hypothesis that all groups have the same satisfaction distribution. However, this result alone does not tell us which groups differ from each other.
Which groups differ?
After the overall test, check pairwise differences with post hoc comparisons
If the overall test is significant, apply a post hoc comparison such as Dunn’s test to see which pairs of groups have large rank differences. When you compare several pairs, the chance of a significant result by luck goes up, so adjust the p-values with the Holm or Bonferroni method.
| Comparison | Unadjusted p | Holm-adjusted p |
|---|---|---|
| Basic vs tailored | 0.0086 | 0.0172 |
| Basic vs follow-up | 0.0000 | 0.0000 |
| Tailored vs follow-up | 0.0151 | 0.0172 |
How do you describe the size of the difference?
Report the effect size along with significance
A p-value alone makes it hard to tell how large a difference is in practical terms. With the Kruskal–Wallis test you can also report a rank-based effect size, ε². Formulas and interpretation guidelines vary across the literature, though, so it is best to state which definition you used.
| Item | Result |
|---|---|
| ε² (based on H) | 0.562 |
| Formula | (H − k + 1) / (N − k) |
| Interpretation | Relative size of the rank-based group difference |
How does it differ from one-way ANOVA?
A test of means versus a rank-based test
| Aspect | One-way ANOVA | Kruskal–Wallis |
|---|---|---|
| What is compared | Group means | Rank distribution of each group |
| Outcome | Mainly continuous | Ordinal or continuous |
| Key assumptions | Independence, normal residuals, equal variances | Independence, values can be ordered |
| Follow-up analysis | Tukey, etc. | Dunn, etc. |
The Kruskal–Wallis test is not a safer or better test than ANOVA in every situation. Choose based on the research question, the measurement level of the data, and the shape of each group’s distribution.
What should you check before the analysis?
Check independence, distribution shape, and ties together
Are the observations independent?
If the same person appears in several groups, or there is a cluster structure within groups, the independence assumption can break down. If the same people were measured at several time points, the Friedman test may be a better fit.
What does each group’s distribution look like?
When the distributions have different shapes, you should not simply interpret the result as a difference in medians. Look at distribution plots for each group as well.
Have you accounted for ties and sample size?
Data with many identical values, such as a 5-point satisfaction rating, need a tie correction. With very small samples the chi-square approximation can be inaccurate, so consider alternatives such as a permutation test.
How do you report the results in writing?
State the numbers and the scope of comparison clearly
A Kruskal–Wallis test was conducted to examine differences in satisfaction scores by counseling approach. The satisfaction distributions differed significantly across groups, H(2) = 25.60, p < .001. Group medians were 3 for basic counseling, 4 for tailored counseling, and 5 for counseling with follow-up care. Specific between-group differences were examined with Dunn’s post hoc tests using the Holm adjustment.
When reporting, give the sample sizes, medians, each group’s distribution, and the adjustment method for post hoc comparisons so that the basis for your interpretation is clear.
How does it differ from other nonparametric tests?
Choose by the number of groups and whether they are independent
| Method | Groups/measurement design | Typical question |
|---|---|---|
| Mann–Whitney U | Two independent groups | Do A and B have different distributions? |
| Wilcoxon signed-rank | Two paired measurements on the same subjects | Is there a before–after change? |
| Kruskal–Wallis | Three or more independent groups | Do A, B, and C differ? |
| Friedman | Three or more time points on the same subjects | Does it change across the three time points? |
What should you watch out for?
A significant result is not the same as a causal effect or a difference in medians
The Kruskal–Wallis test shows a difference between the groups and the outcome, but it does not confirm whether that difference is caused by the counseling approach. In an observational study without random assignment, you need to account for confounders such as user characteristics and prior experience.
Where can the Kruskal–Wallis test be used?
Comparing ordinal or non-normal data from several independent groups
In customer satisfaction surveys you can compare responses by service type, and in education research you can compare course evaluation scores of groups taught with different methods. In health research it can be used to examine differences in pain ratings across treatment groups or in wait times across hospitals.
The starting point for choosing an analysis is to first check the number of groups and whether measurements are repeated, and to be clear about whether the research goal is to compare means or whole distributions.
The Kruskal–Wallis test looks at differences between several independent groups using ranks.
Read the group distributions and post hoc comparisons along with the test statistic.
The Kruskal–Wallis test evaluates differences by converting the observations from three or more groups into ranks. It can be a useful choice when continuous data have trouble meeting the normality assumption or when the outcome is on an ordinal scale. If the test is significant, check the score distribution in each group and use appropriately adjusted post hoc comparisons to see where the differences lie.
Also, because the shapes of the group distributions can differ, it is important not to conclude from the H statistic and p-value alone that the medians differ. Readers understand results more accurately when you choose an analysis that fits the research question and present effect sizes and visualizations as well.
The key to the Kruskal–Wallis test is comparing data from three or more independent groups using ranks to check whether the groups differ. When the result is significant, interpret which groups differ and how by considering the shape of the distributions and the multiple-comparison adjustment together.