Sample size calculator for ANOVA

Explanation

One-way ANOVA tests the null hypothesis that the means of k ≥ 2 independent groups are equal against the alternative that at least two of them differ. When k = 2 it is equivalent to the two-sample t-test, but for k ≥ 3 it controls the overall type I error rate without requiring multiple pairwise comparisons.

The parameter that summarizes the effect size is Cohen's f, defined as \(f = \sigma_{\text{means}} / \sigma_{\text{residual}}\), where \(\sigma_{\text{means}}\) is the standard deviation of the group means and \(\sigma_{\text{residual}}\) is the variability within groups. f is related to \(\eta^2\) (eta squared) by \(f = \sqrt{\eta^2/(1-\eta^2)}\).

The power calculation is based on the noncentral F distribution: under H₁, the F statistic follows a noncentral F with noncentrality parameter \(\lambda = n \cdot k \cdot f^2\). This calculator searches for the minimum integer n for which the power exceeds the target using an iterative search.

Formula and calculation method

With n observations per group (balanced design):

\( \lambda = n \cdot k \cdot f^2 \)

\( 1-\beta = P\!\left(F'(k-1,\;k(n-1),\;\lambda) > F_{\alpha,\,k-1,\,k(n-1)}\right) \)

n is increased starting from 2 until the calculated power exceeds the target.

• f: Cohen's effect size = \(\sigma_{\text{means}} / \sigma_{\text{residual}}\). Reference: 0.10 (small), 0.25 (medium), 0.40 (large).
• λ: noncentrality parameter; it grows linearly with n and k, and quadratically with f.
• k−1 and k(n−1): numerator and denominator degrees of freedom of the F statistic.

How to calculate f from prior data

If you know the expected means of each group \(\mu_1, \ldots, \mu_k\) and the common residual standard deviation \(\sigma\):

\( \sigma_{\text{means}} = \sqrt{\frac{1}{k}\sum_{i=1}^{k}(\mu_i - \bar{\mu})^2}, \quad f = \frac{\sigma_{\text{means}}}{\sigma} \)

Alternatively, if you have \(\eta^2\) from a prior study: \(f = \sqrt{\eta^2/(1-\eta^2)}\).

Quick setup

• Unknown f: use f = 0.25 (medium) as a conservative starting point.
• f from means and σ: calculate \(\sigma_{\text{means}}\) from the expected means of each group and divide it by the residual σ.
• Number of groups (k): include all the groups in the design, even a control group. Increasing k reduces power per group if n does not also increase.
• Alpha: 0.05 is the standard; 0.01 for confirmatory studies or when multiple post-hoc comparisons are planned.
• Power: 0.80 minimum; 0.90 in studies where post-hoc comparisons are the main goal.

Simple example

A trial comparing 3 diets (k = 3) with a medium effect (f = 0.25), α = 0.05 and power 0.80: n ≈ 52 per group (total ≈ 156). With 4 groups and the same f: n ≈ 45 per group (total ≈ 180) — the total sample grows as groups are added even though the per-group n goes down.

Worked example

An education researcher wants to compare the academic performance achieved with three different teaching methods (\(k = 3\) groups): lecture-based instruction, project-based learning and the flipped classroom. Based on prior studies, the standard deviation within each group is estimated at \(\sigma = 10\) points out of 100. The group means are expected to be approximately 75, 80 and 85 points, with an overall mean of 80.

To calculate Cohen's effect size \(f\), the between-group standard deviation is obtained first:

\( \sigma_{\text{between}} = \sqrt{\frac{(75-80)^2 + (80-80)^2 + (85-80)^2}{3}} = \sqrt{\frac{25 + 0 + 25}{3}} = \sqrt{\frac{50}{3}} \approx 4.08 \)

The effect size is \(f = \sigma_{\text{between}} / \sigma = 4.08 / 10 \approx 0.408\), considered large by Cohen's criteria (\(f \geq 0.40\)). With \(\alpha = 0.05\) and 80% power, the approximation gives a sample size of approximately 21 participants per group, that is, 63 participants in total.

If the differences among methods were more modest — for example means of 76, 80 and 84 — the between-group standard deviation would be \(\sqrt{(16+0+16)/3} \approx 3.27\), with \(f \approx 0.327\) (medium-to-large effect). In that case approximately 33 participants per group (99 in total) would be needed.

Note that the number of groups \(k\) also affects the calculation: for the same \(f\), adding a fourth group increases the numerator degrees of freedom in the ANOVA and changes the required sample size. It is therefore advisable to use the tool to explore different scenarios before planning data collection.

Model assumptions

• The k groups are independent of each other.
• The variable follows a normal distribution within each group (or n is large enough).
• The group variances are equal (homoscedasticity). If not, consider Welch's ANOVA.
• Balanced design (same n per group). For unequal groups, use the harmonic n as an approximation.

Common uses

• Comparison of 3 or more treatments, doses or experimental conditions.
• One-factor factorial designs with several levels.
• Studies where the analysis will be followed by post-hoc comparisons (Tukey, Bonferroni, etc.).
• Experimental psychology, agronomy, pharmacology, education.

How to interpret the result

The value \(n\) is the minimum sample size per group in a balanced one-way ANOVA design with \(k\) groups. The total number of participants to recruit is \(k \times n\). Always round up. If you anticipate losses or dropouts, calculate the per-group recruitment number as \(\lceil n / (1 - \text{loss rate}) \rceil\). A key feature of the ANOVA design is that imbalance between groups (different \(n\) per group) does not invalidate the analysis, but it does reduce power and complicate interpretation; if you anticipate unequal groups, use the formula for unbalanced designs or increase the \(n\) of the smallest group.

Cohen's effect size \(f\) (or the between-means variance \(\sigma_\mu^2\)) is the hardest parameter to specify. A value \(f = 0.10\) is considered small, \(f = 0.25\) medium and \(f = 0.40\) large. Small errors in \(f\) have a quadratic impact on \(n\): if the real effect is 20% smaller than expected, the required \(n\) increases by approximately 56% \((1/0.8^2 \approx 1.56)\). Perform a sensitivity analysis by evaluating \(n\) for different values of \(f\) or of the within-group variance \(\sigma^2\). Keep in mind that the calculated \(n\) protects the overall significance level of the ANOVA F test, but not the post-hoc comparisons: if you plan to make all pairwise comparisons between groups with a Bonferroni correction (\(\alpha/\binom{k}{2}\)), the \(n\) needed for each individual comparison is considerably larger.

When \(n\) is very small per group (< 15), verify that normality of the residuals and homoscedasticity are reasonable, or use the nonparametric equivalent (Kruskal-Wallis test) with its own adjusted \(n\). If the total \(n\), \(kn\), turns out to be unfeasible, consider whether the number of groups \(k\) can be reduced (by merging similar groups) or whether the design could be more efficient as a repeated-measures ANOVA. Once the data have been collected, use the ANOVA calculator for the analysis of variance and the relevant post-hoc tests.

References and further reading

• Wikipedia (en): One-way analysis of variance — theoretical foundation of ANOVA.
• Wikipedia (en): Effect size — Cohen's f — definition and conversion with η².
• Wikipedia (en): Noncentral F-distribution — the distribution used for the power calculation.
• Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum. — chapter 8 on one-way ANOVA.

Frequently asked questions

• What is Cohen's f and how is it obtained? f = σ_means/σ_residual. From η²: f = √(η²/(1−η²)). Reference values: 0.10 small, 0.25 medium, 0.40 large.
• Does it work for ANOVA with unequal groups? This calculator assumes a balanced design. For unequal groups use the harmonic n \(\tilde{n} = k/\sum(1/n_i)\) as an approximation.
• How does increasing k affect the n per group? Adding groups with the same f reduces the n per group because there are more error degrees of freedom, but the total n increases.
• Does the calculated size guarantee power for post-hoc comparisons? No; multiple post-hoc comparisons may require a larger sample or a corrected α.