How to Choose the Right Statistical Test

The Four Questions to Ask First

Selecting the right statistical test comes down to answering four questions about your data and research design:

1. 1.What type is my outcome (dependent) variable? → Continuous, categorical, time-to-event?
2. 2.How many groups am I comparing? → One, two, or three+?
3. 3.Are the groups independent or paired/matched? → Different subjects or the same subjects measured twice?
4. 4.Are parametric assumptions met? → Roughly normal distribution, adequate sample size?

Comparing Continuous Outcomes

• ●One group vs. a known value → One-sample t-test (parametric) or Wilcoxon signed-rank (nonparametric)
• ●Two independent groups → Independent samples t-test (parametric) or Mann-Whitney U (nonparametric)
• ●Two paired/matched groups → Paired t-test (parametric) or Wilcoxon signed-rank (nonparametric)
• ●Three or more independent groups → One-way ANOVA (parametric) or Kruskal-Wallis (nonparametric)
• ●Three or more paired/repeated measurements → Repeated-measures ANOVA (parametric) or Friedman test (nonparametric)

Comparing Categorical Outcomes

• ●Two categorical variables (any cell size ≥ 5) → Chi-square test of independence
• ●Two categorical variables (small expected counts, < 5) → Fisher's exact test
• ●Two groups, binary outcome → Chi-square, risk ratio, odds ratio, or difference in proportions
• ●Ordered categories → Chi-square for trend (Cochran-Armitage) or ordinal logistic regression

Exploring Relationships Between Variables

• ●Two continuous variables, normally distributed → Pearson correlation (r)
• ●Two continuous variables, non-normal or ordinal → Spearman correlation (ρ)
• ●Predict a continuous outcome from one or more predictors → Linear regression
• ●Predict a binary outcome (yes/no) → Logistic regression
• ●Time-to-event outcome → Cox proportional hazards regression (survival analysis)
• ●Outcome with multiple measurements per subject → Mixed-effects models

Checking Parametric Assumptions

Parametric tests (t-test, ANOVA, regression) require:

1. 1.Normality: The outcome (or residuals) should be approximately normally distributed. Check visually with histograms and Q-Q plots; use Shapiro-Wilk for small samples.
2. 2.Homogeneity of variance: Group variances should be roughly equal. Test with Levene's test. For t-tests, Welch's version handles unequal variances automatically.
3. 3.Independence: Observations must be independent unless you are using paired/repeated-measures tests.
4. 4.Outliers: Extreme outliers can distort means and inflate variance. Investigate and report them.

Quick Decision Guide

• ●Comparing means of 2 independent groups → Independent t-test
• ●Comparing means of 2 paired groups → Paired t-test
• ●Comparing means of 3+ groups → ANOVA
• ●Testing relationship between 2 categorical variables → Chi-square
• ●Testing association between 2 continuous variables → Correlation
• ●Predicting a continuous outcome → Linear regression
• ●Predicting a yes/no outcome → Logistic regression
• ●Time to event (death, relapse) → Kaplan-Meier + log-rank + Cox regression