Effect Size: Why Statistical Significance Is Not Enough
Understand Cohen's d, odds ratios, and other effect size measures — and why a statistically significant result can still be clinically meaningless.
You'll learn
Why a statistically significant result can still be clinically meaningless, and how to measure true effect magnitude.
Use this when
You interpret any test result — always report an effect size alongside the p-value.
The Problem with Relying on p-Values Alone
A p-value tells you whether your result is compatible with the null hypothesis, given your sample size. With a large enough sample, even a trivial, clinically irrelevant difference will produce p < 0.05. This is the fundamental limitation of significance testing alone.
💡 Significant does not mean important
A trial of 50,000 patients might detect a blood pressure reduction of 0.3 mmHg as statistically significant (p < 0.001). But a 0.3 mmHg reduction has no clinical meaning. Effect size measures capture magnitude independent of sample size.
Common Effect Size Measures
| Measure | Used for | Small | Medium | Large |
|---|---|---|---|---|
| Cohen's d | Difference between two group means | 0.2 | 0.5 | 0.8 |
| r (correlation) | Strength of linear relationship | 0.1 | 0.3 | 0.5 |
| η² (eta-squared) | Proportion of variance explained in ANOVA | 0.01 | 0.06 | 0.14 |
| Odds Ratio (OR) | Ratio of odds in two groups | depends on context | — | — |
| Risk Ratio (RR) | Ratio of risks in two groups | depends on context | — | — |
| Hazard Ratio (HR) | Ratio of instantaneous event rates | depends on context | — | — |
| NNT | Number needed to treat to prevent one event | lower is better | — | — |
Cohen's benchmarks (small/medium/large) were intended as rough guides for psychology research. In clinical research, what constitutes a clinically meaningful difference depends on the specific outcome — a 2-point difference in a 100-point pain scale may be trivial, but a 2-percentage-point reduction in mortality may be enormous.
The Minimal Clinically Important Difference (MCID)
For patient-reported outcomes and many clinical measures, researchers have established the Minimal Clinically Important Difference (MCID) — the smallest change that patients perceive as meaningful. Always compare your effect size to the MCID for your outcome, not to Cohen's generic thresholds.
- ●MCID for WOMAC pain score: ~8–10 points out of 100
- ●MCID for SF-36 quality of life: ~5 points out of 100
- ●MCID for FEV1 in COPD: ~100–140 mL
- ●If your CI excludes the MCID, the treatment may be statistically significant but clinically useless
Reporting Effect Sizes Correctly
All major reporting guidelines (CONSORT, STROBE, APA) require effect sizes with 95% confidence intervals for all primary outcomes. Never report only the p-value.
- ●For two-group comparisons: report the mean difference (or median difference) and 95% CI
- ●For categorical outcomes: report OR or RR (not just the chi-square statistic) with 95% CI
- ●For regression: report the regression coefficient or standardized coefficient with 95% CI
- ●For ANOVA: report η² or ω² in addition to the F statistic
Trusted sources behind this lesson
Read next
Introduction to Hypothesis Testing
Understand null and alternative hypotheses, Type I and Type II errors, the logic of p-values, and the difference between statistical and practical significance.