Calculator
Select the mode based on what you want to compute and enter the parameters.
Reference proportion under the null hypothesis, which the observed result is compared with.
True proportion you expect or want to be able to detect. The larger |p₁ − p₀| is, the easier the effect is to detect.
Number of observations planned or already used in the study. The larger n is, the greater the power for the same effect.
Risk of rejecting H₀ when it is true. The usual value is 0.05.
Use two-tailed if you have no expected direction a priori. The one-tailed test has more power but only detects changes in that direction.
Reference proportion under the null hypothesis, which the observed result is compared with.
True proportion you want to be able to detect. The larger |p₁ − p₀| is, the smaller the required n.
Risk of rejecting H₀ when it is true. The usual value is 0.05.
Desired probability of detecting the effect if it is real. In confirmatory studies at least 0.80 is recommended; if the cost of a false negative is high, use 0.90.
Use two-tailed if you have no expected direction a priori. The one-tailed test needs a smaller sample but only detects changes in that direction.
How to read the chart
The horizontal axis is the standardized Z statistic \(Z = (\hat{p} - p_0)/SE_0\). The two curves show its distribution under two scenarios:
- Blue curve (H₀): distribution of Z if the real proportion were p₀, which follows approximately an N(0,1). The zones shaded in red are the rejection regions; their total area is α.
- Dashed orange curve (H₁): distribution of Z if the real proportion were p₁. It is shifted according to the size of the effect. The green zone — the area under H₁ within the rejection region — is the power (1−β). The unshaded area under the orange curve within the acceptance region is β, the type II error.
The larger |p₁ − p₀| or the sample size n, the more the two curves separate and the greater the power (a wider green zone).
Statistical foundation
For the test \(H_0: p = p_0\) with the normal approximation, the statistic is:
\( Z = \dfrac{\hat{p} - p_0}{\sqrt{p_0(1-p_0)/n}} \sim N(0,\,1) \text{ under } H_0 \)
Under the alternative hypothesis \(H_1: p = p_1\), the same statistic is distributed approximately as:
\( Z \sim N\!\left(\delta,\; \sigma_1^2\right) \quad \text{where} \quad \delta = \frac{(p_1 - p_0)\sqrt{n}}{\sqrt{p_0(1-p_0)}},\quad \sigma_1 = \sqrt{\frac{p_1(1-p_1)}{p_0(1-p_0)}} \)
The parameter \(\delta\) is the non-centrality: how many standard deviations (under H₀) the statistic shifts when p = p₁. The larger \(|\delta|\), the easier it is to distinguish H₁ from H₀.
Power formulas
- Two-tailed (\(H_a: p \neq p_0\)):
\( 1 - \beta \;=\; 1 - \Phi\!\left(\frac{z_{\alpha/2} - \delta}{\sigma_1}\right) + \Phi\!\left(\frac{-z_{\alpha/2} - \delta}{\sigma_1}\right) \)
- Upper one-tailed (\(H_a: p > p_0\)):
\( 1 - \beta \;=\; 1 - \Phi\!\left(\frac{z_\alpha - \delta}{\sigma_1}\right) \)
- Lower one-tailed (\(H_a: p < p_0\)):
\( 1 - \beta \;=\; \Phi\!\left(\frac{-z_\alpha - \delta}{\sigma_1}\right) \)
Sample size formula
To achieve power \(1-\beta\) with level \(\alpha\) and effect \(\Delta = |p_1 - p_0|\):
\( n \;=\; \left\lceil\left(\frac{z_{\alpha^*}\sqrt{p_0(1-p_0)} + z_{1-\beta}\sqrt{p_1(1-p_1)}}{\Delta}\right)^2\right\rceil \)
where \(z_{\alpha^*} = z_{\alpha/2}\) for two-tailed tests and \(z_{\alpha^*} = z_\alpha\) for one-tailed tests.
Worked example
A company wants to check whether its satisfaction rate p₀ = 0.30 has changed after an improvement campaign, expecting it to rise to p₁ = 0.40. A two-tailed test H₀: p = 0.30 is run with α = 0.05 and a sample of n = 172 respondents.
Step 1 — Standard error under H₀:
\( SE_0 = \sqrt{p_0(1-p_0)/n} = \sqrt{0.30 \times 0.70 / 172} \approx 0.03494 \)
Step 2 — Non-centrality δ:
\( \delta = \frac{p_1 - p_0}{SE_0} = \frac{0.10}{0.03494} \approx 2.862 \)
Step 3 — Relative variance under H₁:
\( \sigma_1 = \sqrt{\frac{0.40 \times 0.60}{0.30 \times 0.70}} = \sqrt{\frac{0.24}{0.21}} \approx 1.069 \)
Step 4 — Two-tailed power (\(z_{\alpha/2} = 1.960\)):
\( 1-\beta = 1 - \Phi\!\left(\frac{1.960 - 2.862}{1.069}\right) + \Phi\!\left(\frac{-1.960 - 2.862}{1.069}\right) = 1 - \Phi(-0.844) + \Phi(-4.511) \approx 0.801 \)
With n = 172 and p₁ = 0.40, the power is ≈ 80.1%. If the real proportion has risen to 0.40, the test will correctly detect it in about 8 out of every 10 studies. The type II error is β ≈ 0.199: there is a 20% probability of not detecting the improvement even though it is real.
How to interpret the results
Power (1−β)
A power of 80% means that if the real proportion is p₁, the test will reject H₀ 80% of the time. In confirmatory studies, a power of ≥ 0.80 is recommended; in decisions where the cost of a false negative is high, ≥ 0.90.
Type II error (β)
This is the complement: β = 1 − power. With β = 0.20, one in five studies with the same design would fail to detect the effect even if it exists. Low power makes negative results uninformative: not rejecting H₀ does not mean the effect is null.
Sample size
The computed n is the minimum theoretical value under the normal approximation. In practice, apply a margin for losses: divide n by (1 − expected non-response rate). With 10% losses, plan for ⌈n / 0.90⌉ contacts.
Practical recommendations
- If power is low (< 0.70), you need a larger sample or a bigger effect for the study to be reliable.
- The one-tailed test has more power than the two-tailed test for the same n, but it only detects changes in one direction and must be justified a priori.
- When p₀ or p₁ are extreme (< 0.10 or > 0.90) with a small n, consider the exact binomial test instead of the normal approximation.
- Run a sensitivity analysis varying p₁ ±0.05 to assess the robustness of the design.
Frequently asked questions
- What is statistical power? The probability of rejecting H₀ when the alternative hypothesis is true, that is, the probability of detecting a real effect.
- What is type II error (β)? The probability of not detecting an effect that really exists. It is complementary to power: β = 1 − power.
- What do the critical regions in the chart show? The red zones are the rejection regions under H₀ (total area = α). The green zones are the area of H₁ within those same regions, representing the power (1−β).
- When should I use a two-tailed test versus a one-tailed test? Use two-tailed when you have no expected direction beforehand. The one-tailed test has more power but only detects changes in one direction.