ANOVA: Comparing Means Across Multiple Groups

Why Not Just Run Multiple t-Tests?

If you have three groups (A, B, C), you might think of running three t-tests: A vs B, A vs C, B vs C. But this inflates your Type I error rate dramatically.

• ●3 tests at α = 0.05: overall false positive rate ≈ 14%
• ●5 tests at α = 0.05: overall false positive rate ≈ 23%
• ●10 tests at α = 0.05: overall false positive rate ≈ 40%
• ●ANOVA tests all groups simultaneously with one test, keeping α at 0.05

How ANOVA Works

ANOVA partitions the total variability in your data into two components: variability between groups (explained by the group factor) and variability within groups (random error).

A large F-statistic means the variation between group means is much larger than the variation within groups — suggesting the groups genuinely differ. The p-value tells you whether this F is larger than expected by chance.

Reading the ANOVA Table

The ANOVA output table has these key columns:

• ●Source: Between groups (factor) and Within groups (error)
• ●SS (Sum of Squares): Total variability attributed to each source
• ●df: Degrees of freedom — for k groups: between df = k-1, within df = N-k
• ●MS (Mean Square): SS divided by df — the variance estimate for each source
• ●F: Ratio of between-group to within-group variance
• ●p-value: Probability of this F-ratio under H₀ (all group means equal)
• ●η² (eta-squared): Effect size = SS_between / SS_total. 0.01 = small, 0.06 = medium, 0.14 = large

Post-Hoc Tests: Which Groups Differ?

A significant ANOVA F-test tells you that at least one group mean is different, but NOT which specific pairs differ. Post-hoc tests identify the specific differences while controlling for multiple comparisons.

• ●Tukey's HSD: The most common. Controls familywise error rate. Use when all pairwise comparisons are of interest.
• ●Bonferroni: More conservative than Tukey. Good when you have pre-specified a limited number of comparisons.
• ●Dunnett's: Compare each treatment group to a single control group only.
• ●Games-Howell: Use when variances are unequal across groups.

Assumptions of ANOVA

1. 1.Normality within groups: Test with Shapiro-Wilk within each group
2. 2.Homogeneity of variance: Test with Levene's test. If violated, use Welch's ANOVA.
3. 3.Independence: Each observation must be from a different, unrelated subject
4. 4.If assumptions are violated: Use Kruskal-Wallis H test (nonparametric alternative)