Before trusting a test result, check whether the data structure supports the calculation.
Suppose two distribution centers differ in mean delivery time with p = .03. Before treating that number as a firm conclusion, we should ask how well the data meet the assumptions behind the test.
Common assumptions include independence, normality, and equal variances. Formal tests help, but a single pass-or-fail p-value is rarely enough.
Key question
How much could an assumption violation change the conclusion, and what alternative is appropriate?
What does the normality assumption mean?
Many parametric procedures require means or residuals to behave approximately normally. With larger samples, the central limit theorem can make mean comparisons stable even when raw data are somewhat skewed.
Inspect residuals, not only raw scores
For many t-tests, ANOVA models, and regressions, the normality condition concerns model errors more directly than every raw variable.
Large samples detect trivial departures
A formal normality test can become significant for deviations that have little practical effect on the analysis.
Do not decide from one test alone
Use histograms, Q–Q plots, outlier checks, sample size, and the analysis goal together.
Why does equal variance matter?
The classical independent t-test and traditional ANOVA assume similar group variances. When variances differ strongly and sample sizes are unbalanced, standard errors and p-values can be distorted.
Similar means, very different spreads
A pooled-variance method can become unreliable when group variability differs sharply.
Center A
Mean 12 min · SD 3 min
Center B
Mean 13 min · SD 11 min
If equal variance is doubtful, Welch’s t-test is usually safer for two independent groups. Welch ANOVA provides a corresponding option for several groups.
What should you do when assumptions fail?
Mild non-normality
With enough data and no extreme outliers, t-tests and ANOVA can be robust.
Unequal variances
For independent means, prefer a Welch procedure.
Extreme observations
Investigate the cause and consider robust methods or transformation.
Severe skew + small sample
Consider a rank-based test or bootstrap method.
First rule out data-entry errors and design problems. Wrong units, mixed populations, or repeated records can matter more than the choice between two test formulas.
Independence violations can be the most serious
Treating repeated measurements or clustered observations as independent can make standard errors much too small. Paired, repeated-measures, or mixed models may be required.
How should diagnostic tests be interpreted?
Assumption diagnostics
Independent means comparison
| Diagnostic | Statistic | p-value | Interpretation |
|---|---|---|---|
| Shapiro–Wilk | W = 0.97 | .18 | No clear non-normality |
| Levene | F = 8.42 | .005 | Equal variances not supported |
| Outliers | 2 | — | Inspect original records |
Shapiro–Wilk p = .18 does not prove normality. It means this sample does not show a clear departure. Levene p = .005 provides evidence of unequal variances and supports using a Welch procedure.
| Situation | Recommended approach |
|---|---|
| Significant normality test, large sample, no outliers | Inspect plots; parametric test may still be robust |
| Significant normality test, small sample, severe skew | Consider transformation, robust, or rank method |
| Significant Levene test | Welch t-test or Welch ANOVA |
| Repeated data entered as independent | Paired, repeated-measures, or mixed model |
Change skew, outliers, and variance differences
Adjust the three conditions and watch the means, standard deviations, and diagnostic indicators respond.
Mean A
11.58
Mean B
12.17
SD ratio
1.01
Decision
Equal variance plausible
A
SD 2.15 · skew -0.01
B
SD 2.12 · skew 0.35
Key lesson
The purpose of diagnostics is not to pass or fail data. It is to judge how safe the analysis is and choose a better method when needed.
Good diagnostics ask “how much could this violation change the conclusion?” rather than only “did the data pass?”