Three delivery methods have different averages. Is the separation real or ordinary sampling variation?
A retailer compares three delivery operating methods. Similar orders are assigned across the methods, and the company records the time from dispatch to arrival.
| Delivery method | Orders | Mean delivery time | Standard deviation |
|---|---|---|---|
| Standard | 30 | 42 hr | 9 hr |
| Next-day | 30 | 35 hr | 8 hr |
| Local hub | 30 | 28 hr | 7 hr |
Local-hub delivery has the shortest sample mean. A different set of orders, however, would produce somewhat different averages.
Analysis of variance compares the separation among the group means with the spread of observations within the groups.
Key question
Are the differences among the three delivery methods large relative to the order-to-order variation within each method?
What does ANOVA compare?
Separated means may still be difficult to distinguish when delivery times vary widely within each method. The same mean differences become much clearer when the observations cluster tightly around their group means.
Between-group variation
How far the three delivery-method means are from the grand mean
Within-group variation
How much delivery times differ among orders using the same method
ANOVA summarizes that comparison with an F-statistic.
The F-statistic is a ratio of two sources of variation
F = between-group variation ÷ within-group variation
Between groups
96.4
Within groups
10.0
F = 9.64
How to read F
A larger F-statistic means that separation among the group means is large relative to the ordinary variation within the groups. The p-value describes how unusual an F-statistic this large would be if all population means were equal.
Why not run several t-tests?
Three groups produce three pairwise tests: standard versus next-day, standard versus local hub, and next-day versus local hub. Every additional test creates another opportunity for a chance result to cross the significance threshold.
| Number of tests | Probability of at least one false positive |
|---|---|
| 1 | 5% |
| 3 | About 14% |
| 6 | About 26% |
| 10 | About 40% |
ANOVA begins with one overall null hypothesis: all population means are equal.
What a significant ANOVA means
At least one population mean differs. It does not mean that every group differs from every other group.
Which groups differ?
The omnibus F-test tells us whether the equal-means model is plausible, but it does not identify the groups responsible for the result. A post hoc procedure such as Tukey HSD compares the relevant pairs while controlling the overall error rate.
| Comparison | Mean difference | Adjusted p | Interpretation |
|---|---|---|---|
| Standard − next-day | 7 hr | 0.08 | Not statistically clear |
| Standard − local hub | 14 hr | < .001 | Significant difference |
| Next-day − local hub | 7 hr | .012 | Significant difference |
With a very large sample, even small differences may be statistically significant. Eta squared, η², describes the proportion of total variation associated with group membership.
Different questions
p-value: If all population means were equal, how unusual would this result be?
η²: How much of the total variation is associated with differences among the groups?
What should you check first?
When variances differ sharply or sample sizes are unbalanced, Welch’s ANOVA is generally safer. Games–Howell can then be used for follow-up comparisons.
| Situation | Recommended approach |
|---|---|
| Variances are reasonably similar | One-way ANOVA + Tukey HSD |
| Variances differ or sample sizes are unbalanced | Welch’s ANOVA + Games–Howell |
| Small samples with severe skew or outliers | Consider a robust method or Kruskal–Wallis |
How to read p = .11
❌ The three population means are equal.
✓ The current data do not provide sufficient evidence that at least one population mean differs.
Change the group separation and spread
Adjust the distance among the means, the within-group spread, and the sample size. Watch the F-statistic, p-value, and η² respond.
Move the group means and the within-group spread
The F-statistic rises when the group means separate and falls when observations within each group spread out.
Between groups
367.6
Within groups
26.4
F
13.92
p
<0.0001
η² = 0.242 · proportion of total variation associated with the groups
Key lesson
ANOVA is determined jointly by between-group separation, within-group spread, and sample size.
ANOVA does not merely ask whether the averages differ. It asks whether their separation is large relative to the variation within the groups.
Analysis result
One-way ANOVA · Tukey post hoc test
F-statistic
9.64
Degrees of freedom
2, 87
p-value
< .001
η²
.18
Mean delivery time differed by operating method, F(2, 87) = 9.64, p < .001, η² = .18. Tukey comparisons showed that local-hub delivery was faster than both standard and next-day delivery.