Sample size calculator for the difference of proportions (CI)

Explanation

This calculator determines the minimum number of observations per group needed to estimate the difference \(p_1 - p_2\) between two independent proportions with a maximum absolute margin of error \(E\) and a given confidence level. It is the starting point for planning comparative studies between two groups before collecting the data.

The two key parameters are the expected proportions in each group. If you don't have prior estimates, the conservative scenario is \(p_1 = p_2 = 0.5\), which maximizes the sum \(p_1(1-p_1) + p_2(1-p_2)\) and therefore the sample size. In practice, using proportions informed by pilot data or the literature considerably reduces the required sample.

The formula assumes balanced allocation (the same number of subjects in each group) and random sampling from two very large populations. The margin of error \(E\) applies to the difference \(p_1-p_2\), not to each proportion separately.

Sample size formula

\( n = \frac{Z^2 \cdot \left[p_1(1-p_1) + p_2(1-p_2)\right]}{E^2} \)

• n: minimum number of observations per group (total N = 2n).
• Z: normal quantile — 1.645 (90%), 1.960 (95%), 2.576 (99%).
• \(p_1, p_2\): expected proportions in each group.
• E: maximum absolute margin of error for the difference \(p_1 - p_2\).

Interpreting the margin of error

A margin \(E = 0.10\) means the estimated difference \(\hat{p}_1 - \hat{p}_2\) can differ from the true population difference \(p_1 - p_2\) by at most ±10 percentage points at the chosen confidence level. If the expected difference between groups is small, the margin of error should be small too, which requires larger samples. Always compare the margin of error with the real expected difference: an E larger than that difference makes the study useless for detecting it.

Quick setup

• p₁ and p₂: take them from pilot data or the literature. If you have no information, use 0.5 for both.
• Margin of error E: define it in absolute percentage points. For expected differences of 10–20 pp, a margin of ±5–8 pp is usually reasonable.
• Confidence level: 95% is the usual standard.
• Expected dropout: divide n per group by (1 − dropout rate). With 15% dropout: n_recruit = n / 0.85.
• Unequal groups: if you need a different number in each group, the formula with allocation ratio k = n₂/n₁ is more appropriate.

Worked example

A researcher wants to estimate the difference in treatment adherence rate between two interventions: intervention A has an expected adherence of \(p_1 = 0.60\) and B of \(p_2 = 0.40\), according to data from similar studies. A margin of error of \(E = 0.10\) (±10 pp) is set with a 95% confidence level (\(Z = 1.960\)).

We apply the formula:

\( n = \frac{(1.960)^2 \cdot [0.60 \cdot 0.40 + 0.40 \cdot 0.60]}{(0.10)^2} = \frac{3.8416 \times 0.48}{0.01} = \frac{1.844}{0.01} = 184.4 \rightarrow n = 185 \text{ per group} \)

185 participants per group are needed, i.e. 370 in total. With an expected dropout rate of 12%, the number to recruit per group is 185 / 0.88 ≈ 210, and the total to recruit is 420.

Sensitivity analysis: if the margin is reduced to ±5 pp (E = 0.05), the sample increases to n = 3.8416 × 0.48 / 0.0025 = 737 per group (1,474 total). Defining the margin correctly is critical for the feasibility of the study.

Conservative scenario: with no prior information (\(p_1 = p_2 = 0.5\)), with E = 0.10: n = 3.8416 × 0.50 / 0.01 = 193 per group.

Model assumptions

• Simple random and independent sampling from two very large populations.
• The two samples are independent of each other (no unit-to-unit matching).
• The normal approximation is adequate when \(n_i \hat{p}_i \geq 5\) and \(n_i(1-\hat{p}_i) \geq 5\) in both groups.
• Balanced 1:1 allocation. If groups of different sizes are needed, adjust the formula with the allocation ratio.

Common uses

• Comparing satisfaction, adherence, or conversion rates between two groups or segments.
• Estimating the difference in prevalence between two populations (men vs. women, regions, periods).
• Designing comparative before/after surveys in cross-sectional epidemiological studies.
• Planning descriptive A/B experiments where the goal is to characterize the difference, not to test a hypothesis.

How to interpret the result

The value \(n\) is the minimum number of valid observations per group needed so that the confidence interval for the difference \(p_1 - p_2\) has a maximum half-width of \(E\) at the chosen confidence level. In a balanced design, the total number of participants to recruit is \(2n\). Always round up and add a margin for dropouts by dividing by \((1 - \text{dropout rate})\) per group; with an expected dropout of 10%, plan for \(\lceil n / 0.90 \rceil\) individuals per group.

The proportions \(p_1\) and \(p_2\) are the most critical and uncertain parameters. The variance of the difference depends on \(p_1(1-p_1) + p_2(1-p_2)\), which is maximal when both are close to 0.5. Run a sensitivity analysis varying \(p_1\) and \(p_2\) by ±0.10 around the expected values: when both are extreme (\(<0.10\) or \(>0.90\)), the variance decreases and \(n\) can be smaller than intuitively expected. If you have no prior information, use \(p_1 = p_2 = 0.5\) for the most conservative scenario. Also evaluate different values of \(E\): halving the margin of error quadruples \(n\).

This approach (CI) differs from hypothesis testing: here the goal is to estimate the magnitude of the difference with a given precision, not to decide whether it exists. If you need to detect whether \(p_1 \neq p_2\) with a given power, use the calculator for two-proportion hypothesis testing. Once the data has been collected, build the actual CI with the confidence interval calculator for the difference of proportions. If the expected frequencies in any cell are small (< 5), consider Fisher's exact test.

References and further reading

• Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical Methods for Rates and Proportions (3rd ed.). Wiley.
• Newcombe, R. G. (1998). Interval estimation for the difference between independent proportions, Statistics in Medicine, 17, 873–890.
• Wikipedia: Sample size determination — derivation for the difference of proportions.

Frequently asked questions

• When is it conservative to use p₁=p₂=0.5? Whenever you have no prior information. It maximizes the variance of each group and therefore the required sample size.
• What if the groups are very different in size? If you can't have the same n in each group, efficiency decreases. With ratio k=n₂/n₁, the n of the larger group grows proportionally.
• Does this calculation work for hypothesis testing? Not directly. To detect a difference with a given power and significance level, use the two-proportion calculator for hypothesis testing.