Sample size calculator for a one-proportion hypothesis test

Explanation

This calculator estimates the sample size for testing a proportion with null hypothesis \(H_0: p=p_0\), significance level \(\alpha\), power \(1-\beta\) and minimum detectable difference \(\Delta=|p_1-p_0|\).

It allows a two-tailed (\(H_a: p\neq p_0\)) or one-tailed (\(H_a: p\geq p_0\), \(H_a: p\leq p_0\)) alternative.

Sample size formula

\( n=\left(\frac{Z_{\alpha^*}\sqrt{p_0(1-p_0)} + Z_{\beta}\sqrt{p_1(1-p_1)}}{\Delta}\right)^2 \)

• \(\alpha^*\): \(\alpha/2\) for two-tailed and \(\alpha\) for one-tailed.
• \(p_0\): proportion under the null hypothesis.
• \(p_1\): proportion under the alternative (based on \(\Delta\) and direction).
• \(1-\beta\): target power.

Quick setup

• p0: the null value you want to test.
• Δ: the minimum clinically or practically relevant difference.
• α: 0.05 is a typical value.
• Power: 0.80 or 0.90 depending on the study's requirements.
• Direction of Ha: choose two-tailed or one-tailed based on the prior hypothesis.

Simple example

If \(p_0=0.20\), \(\Delta=0.05\), \(\alpha=0.05\) and power 0.80, the required sample size is on the order of hundreds of observations.

Worked example

A services company wants to assess whether its customer satisfaction rate has changed from the historical reference value of 30%. The quality department sets up the two-tailed test \(H_0: p = 0.30\) versus \(H_a: p \neq 0.30\), and considers that detecting a change of at least 10 percentage points (up to \(p_1 = 0.40\)) is relevant for decision-making. Power is set at 80% (\(z_\beta = 0.842\)) with \(\alpha = 0.05\) two-tailed (\(z_{\alpha/2} = 1.960\)).

The formula for the one-proportion test uses the variances under \(H_0\) and under \(H_1\) separately. We calculate the terms step by step:

\( n = \left(\frac{z_{\alpha/2}\sqrt{p_0(1-p_0)} + z_\beta\sqrt{p_1(1-p_1)}}{\Delta}\right)^2 \)

With \(p_0 = 0.30\), \(p_1 = 0.40\) and \(\Delta = |p_1 - p_0| = 0.10\): the first term under the square root is \(p_0(1-p_0) = 0.30 \times 0.70 = 0.21\), so \(\sqrt{0.21} = 0.4583\); the second is \(p_1(1-p_1) = 0.40 \times 0.60 = 0.24\), so \(\sqrt{0.24} = 0.4899\). Substituting:

\( n = \left(\frac{1.960 \times 0.4583 + 0.842 \times 0.4899}{0.10}\right)^2 = \left(\frac{0.8983 + 0.4125}{0.10}\right)^2 = \left(\frac{1.3108}{0.10}\right)^2 = (13.108)^2 = 171.8 \rightarrow n = 172 \)

You need 172 respondents for the test to have an 80% probability of detecting a real 10 pp change in satisfaction, controlling type I error at 5%. If the satisfaction campaign has really raised the proportion to 40%, the test will detect it in 8 out of every 10 applications of the procedure.

Practical interpretation: if after collecting 172 responses \(\hat{p} = 0.40\) or higher is obtained, the test will conclude that the proportion has changed significantly from the historical 30%. With fewer than 172 responses, the study would be underpowered and would risk failing to detect the improvement even if it is real.

Scenario with stricter statistical requirements (\(\alpha = 0.01\)): if the company decides to be more conservative about false positives and uses \(\alpha = 0.01\) (\(z_{\alpha/2} = 2.576\)), while keeping the same 80% power: \( n = ((2.576 \times 0.4583 + 0.842 \times 0.4899) / 0.10)^2 = (1.1806 + 0.4125)^2 / 0.01 = (15.931)^2 = 253.8 \rightarrow n = 254 \). Moving from \(\alpha = 0.05\) to \(\alpha = 0.01\) requires 48% more responses to maintain the same power.

Model assumptions

• Random sampling and independence.
• Normal approximation valid for the proportion within the planned n range.

How to interpret the result

The value \(n\) is the minimum number of valid observations needed for the test \(H_0\!: p = p_0\) versus the specified alternative to have the power \((1-\beta)\) at level \(\alpha\). Always round up. If you expect some respondents won't complete the survey or will be excluded by inclusion criteria, divide \(n\) by \((1 - \text{non-response rate})\) to get the number of individuals to contact; with an expected 20% non-response rate, plan for \(\lceil n / 0.80 \rceil\) contacts.

The parameters that most influence \(n\) are the difference \(\Delta = |p_1 - p_0|\) and the proportion under the alternative \(p_1\). The variance function \(p(1-p)\) is maximal at \(p = 0.5\), so when both \(p_0\) and \(p_1\) move away from 0.5, \(n\) decreases. Halving \(\Delta\) increases \(n\) roughly fourfold; increasing power from 80% to 90% increases it by about 30%. The choice between a two-tailed and one-tailed test also matters: a one-tailed test requires a smaller \(n\) for the same power, but is only justified if the effect can occur in only one direction. Run a sensitivity analysis varying \(p_0\) and \(p_1\) by ±0.05 to assess the robustness of the design.

If the resulting \(n\) is unaffordable, consider: (1) increasing \(\Delta\) if the context allows it, (2) reducing power to 80% if 90% had been specified, or (3) switching to a one-tailed test if the scientific rationale justifies it. When \(n\) is small (< 30) and \(p_0\) or \(p_1\) are extreme (< 0.10 or > 0.90), consider the continuity correction or the exact binomial test instead of the normal approximation. Once the data are collected, run the test with the hypothesis test calculator for one proportion; if the goal is to estimate \(p\) with precision, use the sample size calculator for one proportion (estimation).