Sample size calculator for two means

Explanation

This calculator determines the minimum number of observations per group needed to detect a difference between the means of two independent groups. The underlying test is the two-sample Student's t-test: \(H_0\!: \mu_1 = \mu_2\) versus \(H_1\!: |\mu_1 - \mu_2| \geq \Delta\).

Unlike estimating a single mean, here the goal is not precision but power: we want to ensure that, if the true difference is at least \(\Delta\), the test will detect it with the specified probability \(1-\beta\).

The effect size is Cohen's d: \(d = \Delta / \sigma\). Reference values: d = 0.2 (small effect), d = 0.5 (medium), d = 0.8 (large). For d = 0.5 with α = 0.05 and power 0.80 the result is n ≈ 64 per group.

Sample size formula

\( n_1 = \frac{(Z_{\alpha/2}+Z_\beta)^2(\sigma_1^2+\sigma_2^2/k)}{\Delta^2} \)

\( n_2 = k\,n_1 \)

• n1, n2: per-group sample sizes (n2 = k·n1).
• \(\Delta\): minimum clinically or practically relevant difference of means.
• \(\sigma_1, \sigma_2\): expected standard deviations in each group.
• k: allocation ratio n2/n1 (k = 1 for a balanced design).
• \(Z_{\alpha/2}\) and \(Z_\beta\): normal quantiles (two-sided α and power).

Quick setup

• σ1 and σ2: use historical data, a pilot study, or literature references. If you assume equal variances, use the same σ in both fields.
• Δ: define the minimum practically relevant difference — not the expected difference, but the smallest one that would justify an action or decision.
• Alpha (α): 0.05 two-sided is the standard; 0.01 for confirmatory studies.
• Power: 0.80 is the accepted minimum; 0.90 in pivotal studies or when false negatives are costly.
• 1:1 allocation: is always the most efficient when the cost per subject is similar in both groups.

Simple example

You compare the processing time of two lines with σ1 = σ2 = 10, minimum detectable difference Δ = 5, α = 0.05 and power 0.80. Result: ≈ 63 subjects per group (total ≈ 126). Cohen's d is Δ/σ = 5/10 = 0.5 (medium effect).

Worked example

A research group wants to compare the effectiveness of two diets (diet A and diet B) on weight loss after 12 weeks. From a pilot study, the standard deviation of weight loss is \(\sigma \approx 8\) kg in both groups. The researchers consider a difference of at least \(\Delta = 3\) kg between diets to be clinically relevant. They set a power of 80% (\(z_\beta = 0.842\)) and a two-sided significance level \(\alpha = 0.05\) (\(z_{\alpha/2} = 1.960\)), with 1:1 allocation.

For equal-sized groups (\(k = 1\)) and \(\sigma_1 = \sigma_2 = \sigma\), the formula simplifies to:

\( n = \frac{2\sigma^2\,(z_{\alpha/2}+z_\beta)^2}{\Delta^2} = \frac{2 \times 64 \times (1.960 + 0.842)^2}{9} = \frac{128 \times (2.802)^2}{9} = \frac{128 \times 7.851}{9} = \frac{1{,}004.9}{9} \approx 111.7 \rightarrow n = 112 \)

They need 112 participants per group, for a total of 224 participants in the trial. Cohen's d for the effect is \(d = 3/8 = 0.375\), a small-to-medium effect requiring moderate samples.

If the researchers anticipate a 12% dropout rate, they must recruit \(112 / 0.88 \approx 128\) participants per group (256 in total) to preserve the planned power.

Sensitivity analysis (90% power): if the team decides to raise the power to 90% (\(z_\beta = 1.282\)) to reduce the risk of false negatives, the calculation gives: \( n = 2 \times 64 \times (1.960 + 1.282)^2 / 9 = 128 \times (3.242)^2 / 9 = 128 \times 10.510 / 9 \approx 149.5 \rightarrow n = 150 \) per group (300 in total). Going from 80% to 90% power increases the sample size by 34%, a cost the researchers must weigh against the reduction in type II error risk.

Model assumptions

• The two groups are independent of each other.
• The variable is approximately normally distributed in each group (or n is large enough for the CLT to apply).
• The standard deviations σ1 and σ2 are known or reliably estimated.
• Two-sided test. For non-inferiority or equivalence studies, use the dedicated calculator.

Common uses

• Randomized clinical trials with a continuous outcome variable.
• Performance comparison between two systems, processes or pieces of equipment.
• A/B experiments with a quantitative variable (load time, score).
• Educational or psychological intervention studies.

How to interpret the result

The value \(n\) returned by the calculator is the minimum size per group in a balanced design. The total number of participants to recruit is \(2n\) (one per group). Always round up and add a margin for losses: if you expect a dropout rate of \(r\,\%\), the number of subjects to recruit per group is \(\lceil n / (1 - r) \rceil\). The difference \(\Delta\) should be the minimum clinically or practically relevant difference, not simply the expected difference: specifying too small a difference inflates the sample size until it becomes infeasible.

The sensitivity of \(n\) to the parameters is critical in this design. Errors in \(\sigma\) propagate squared: if the true standard deviation is 20% larger than assumed, the required \(n\) increases by 44% \((1.2^2 = 1.44)\). It is therefore essential to run a sensitivity analysis: calculate \(n\) for \(\sigma - 25\%\), \(\sigma\) and \(\sigma + 25\%\), and plan for the least favorable scenario. Likewise, if \(\Delta\) is halved, \(n\) quadruples. Increasing power from 80% to 90% increases \(n\) by roughly 30%.

If the resulting \(n\) is too large to be feasible, the alternatives are: (1) increase \(\Delta\) if there is clinical consensus to do so, (2) accept lower power (e.g., 80% instead of 90%), (3) design the study with repeated measures that exploit within-subject correlation, or (4) use an adaptive design. When \(n\) is very small (< 20 per group), verify that the normality of the data or the size of the deviations does not invalidate the assumptions of the two-sample t-test. Once the data are collected, analyze the results with the hypothesis test calculator for difference of means.

References and further reading

• Wikipedia: Student's t-test — theoretical basis of the underlying test.
• Wikipedia: Cohen's d — standardized effect size for the difference of means.
• Wikipedia: Sample size determination — derivation and variants.
• Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum. — standard reference for effect sizes.

Frequently asked questions

• What is Cohen's d? The standardized effect size: Δ/σ. It allows comparing studies independently of units. d = 0.2 (small), 0.5 (medium), 0.8 (large).
• What if σ1 ≠ σ2? The formula allows for it; enter each standard deviation separately. If the difference is large, consider a Welch test.
• What is statistical power? The probability of rejecting H0 when the true difference is at least Δ.
• When is a paired design preferable? When you can measure the same subject under both conditions. A paired design usually requires far fewer subjects because it removes between-subject variability.