Sample size calculator for paired proportions

Explanation

For paired binary data (yes/no measured twice on the same unit), the appropriate test is McNemar's test. Its core logic is that concordant pairs (0→0 and 1→1) provide no information about change: only discordant pairs determine whether the intervention has an effect.

Discordant pairs come in two types: \(p_{01}\) (the response changed from negative to positive, 0→1) and \(p_{10}\) (changed from positive to negative, 1→0). The effect of interest is the difference \(p_{10} - p_{01}\): if the intervention is effective, \(p_{10}\) will be larger than \(p_{01}\) (more subjects move to a negative response under treatment, or vice versa depending on the design).

The efficiency of the paired design compared with two independent groups depends on concordance: the more concordant pairs there are (subjects who don't change), the less the pairing helps, because those pairs are effectively excluded from the analysis.

Sample size formula

\( n = \left\lceil\frac{\left(Z_{\alpha/2}\sqrt{p_d}+Z_\beta\sqrt{p_d-\Delta^2}\right)^2}{\Delta^2}\right\rceil \)

where \(p_d = p_{01}+p_{10}\) and \(\Delta = |p_{10}-p_{01}|\). Connett et al.'s (1987) formula, more precise than Fleiss's approximation.

• n: total number of pairs (concordant + discordant).
• p₀₁: probability that a pair changes from 0 to 1 (negative response → positive).
• p₁₀: probability that a pair changes from 1 to 0 (positive response → negative).
• Note: the denominator \((p_{10}-p_{01})^2\) requires \(p_{10} \neq p_{01}\); if they are equal, there is no effect to detect.

Quick setup

• p₀₁ and p₁₀: should be estimated from a pilot study or historical data. If you only have the prevalence before (p_before) and the expected effect, you need to make assumptions about the pair structure.
• If p₀₁ ≈ p₁₀: the difference is small and the required sample will be very large — check whether the effect is clinically relevant.
• Relation to prevalences: p_before = p₁₀ + p₁₁ (concordant positives); p_after = p₀₁ + p₁₁. If you know both prevalences and one of the discordant p's, you can compute the other.
• Alpha and power: 0.05 and 0.80 are the standard baseline.

Simple example

Evaluation of a vaccination campaign: p₀₁ = 0.18 (people who get vaccinated after the campaign who hadn't before), p₁₀ = 0.08 (people who stop being vaccinated — very unlikely in practice). With α = 0.05 and power 0.80, the result is ≈ 202 pairs.

Worked example

A clinical diagnostics team wants to compare a new detection method for an infectious disease against the current standard method. Both tests are applied to each patient, forming pairs. A pilot study with 100 patients finds that 15% of pairs are discordant of the type "new+/standard−" (\(p_{12} = 0.15\)) and 8% are discordant of the type "new−/standard+" (\(p_{21} = 0.08\)). The goal is to detect this difference with two-sided \(\alpha = 0.05\) and 80% power.

In McNemar's test only discordant pairs carry information. The total proportion of discordant pairs is \(p_d = p_{12} + p_{21} = 0.23\) and the difference between discordant proportions is \(\Delta = |p_{12} - p_{21}| = 0.07\). With \(z_{\alpha/2} = 1.960\) and \(z_{\beta} = 0.842\):

\( n = \frac{\left(z_{\alpha/2}\sqrt{p_d} + z_{\beta}\sqrt{p_d - \Delta^2}\right)^2}{\Delta^2} \)

Substituting the values: \(\sqrt{p_d} = \sqrt{0.23} \approx 0.4796\) and \(\sqrt{p_d - \Delta^2} = \sqrt{0.23 - 0.0049} = \sqrt{0.2251} \approx 0.4745\). Therefore:

\( n = \frac{(1.960 \times 0.4796 + 0.842 \times 0.4745)^2}{0.07^2} = \frac{(0.9400 + 0.3995)^2}{0.0049} = \frac{(1.3395)^2}{0.0049} = \frac{1.7942}{0.0049} \approx 366.2 \rightarrow 367 \)

367 pairs of patients (each evaluated with both methods) are needed to reach 80% power. This large sample size reflects that the difference to detect (\(\Delta = 0.07\)) is small relative to the proportion of discordant pairs.

If the expected difference were larger — for example \(p_{12} = 0.20\) and \(p_{21} = 0.08\), with \(\Delta = 0.12\) — the sample size would drop considerably, which illustrates the sensitivity of McNemar's test to the effect size among discordant pairs.

Model assumptions

• Pairs are independent of each other.
• Within each pair, the two observations may be correlated (that's the whole point of McNemar's test).
• The values of p₀₁ and p₁₀ are estimated correctly; errors in them directly affect the calculated n.

Common uses

• Evaluation of adherence to treatment or behavior before/after an intervention.
• Comparison of a binary diagnosis by two observers or at two time points.
• Studies of change in dichotomous variables (yes/no, positive/negative) in the same individual.
• Crossover designs with a binary outcome.

How to interpret the result

The value \(n\) is the minimum total number of pairs (including concordant ones, i.e. pairs where both conditions produce the same result). Always round up. Add a margin for loss of complete pairs: if you expect 10% of subjects to not complete both measurements or conditions, recruit \(\lceil n / 0.90 \rceil\) pairs. It is essential to distinguish between the number of pairs and the number of observations: each pair contributes two binary observations, but McNemar's test only uses the discordant pairs.

The most important feature of this design is that statistical power depends fundamentally on the expected proportion of discordant pairs \((p_{01} + p_{10})\), not on the total number of pairs. If subjects respond very consistently under both conditions (high concordance), discordant pairs will be scarce and the study will be underpowered even if the total \(n\) is large. Therefore, when the proportion of discordant pairs is expected to be low (< 20%), many more subjects will need to be recruited to accumulate enough discordant pairs. Run a sensitivity analysis varying \(p_{01}\) and \(p_{10}\) by ±0.05 to see the impact on \(n\).

If the required \(n\) turns out to be unfeasible, the alternatives are: (1) relax the power (e.g., from 90% to 80%), (2) relax the significance requirement (increase \(\alpha\)), or (3) reconsider the study design to increase the expected discordance rate by changing conditions. Once the data has been collected, analyze the results with the McNemar test for paired proportions calculator; if the discordant pair counts are very low (< 25 total), use the exact version of the test.

References and further reading

• Wikipedia: McNemar's test — statistical foundation of the underlying test.
• Wikipedia: Matched pairs — matched pair designs and their applications.
• Fleiss, J. L., Levin, B. and Paik, M. C. (2003). Statistical Methods for Rates and Proportions (3rd ed.). Wiley. — chapter on McNemar's test and sample size calculation.

Frequently asked questions

• Why do only discordant pairs matter? Because concordant pairs (same response in both conditions) provide no information about whether the intervention changed the outcome.
• How do I estimate p₀₁ and p₁₀ without a pilot study? If you know the prevalence before (p_a) and after (p_d) and the correlation between measurements, you can approximate: p₀₁ ≈ p_d·(1−ρ) and p₁₀ ≈ p_a·(1−ρ).
• When is a paired design better than two groups? When the correlation between measurements is high (ρ > 0.3) and you can apply both conditions to the same individual without carryover.
• Is the formula exact? It's the normal approximation. For small n or extreme proportions, use the exact version of McNemar's test.