Confidence intervals

Confidence interval for a proportion

Estimate the population proportion from the successes observed in a sample. Includes the Wald, Wilson and Clopper-Pearson (exact) methods.

Parameters

Enter the number of successes and the sample size to get both intervals.

Result pending…

Confidence interval for the proportion (p)

Explanation

When we want to estimate the true proportion of a characteristic in a population (defect rate, percentage of satisfied users, prevalence of a disease…), the confidence interval for a proportion gives us the plausible range for that value.

The sample proportion \(\hat{p} = x/n\) (successes among observations) is the estimator. There are several methods for building the interval; the most common are Wald (normal approximation), Wilson (more accurate for extreme proportions or small samples) and Clopper-Pearson (exact, based directly on the binomial; explained in its own section).

Let \(C\) denote the confidence level and \(\alpha=1-C\) the total area outside the interval. For 95% confidence, \(C=0.95\), \(\alpha=0.05\) and \(\alpha/2=0.025\) in each tail.

Wald method (normal approximation)

Wald applies the normal approximation directly to the sample proportion \(\hat{p}\). It is the simplest interval and the easiest to compute by hand:

\( \hat{p} \pm z_{\alpha/2} \cdot \sqrt{\dfrac{\hat{p}(1-\hat{p})}{n}} \)

It works reasonably well when the sample is large and the proportion isn't close to 0 or 1. As a practical rule, it is advisable to require \(n\hat{p} \geq 5\) and \(n(1-\hat{p}) \geq 5\), although with moderate sample sizes this rule doesn't always guarantee good actual coverage.

Its main limitation is that it centers the interval exactly on \(\hat{p}\) and uses a standard error estimated only from \(\hat{p}\). Because of this it can produce bounds below 0 or above 1, and it can underestimate the uncertainty when there are few successes, few failures, or extreme proportions.

Wilson method (score, recommended)

Wilson starts from the score test for a proportion and works out which values of \(p\) would be compatible with the data. It doesn't simply add and subtract the same margin around \(\hat{p}\); the center of the interval shifts slightly toward less extreme values and the bounds stay within \([0,1]\).

\( \dfrac{\hat{p} + \dfrac{z^2}{2n} \pm z\sqrt{\dfrac{\hat{p}(1-\hat{p})}{n} + \dfrac{z^2}{4n^2}}}{1 + \dfrac{z^2}{n}} \)

Wilson is generally recommended because it better maintains the actual coverage of the confidence level, especially with small samples or proportions close to 0 or 1. It also correctly handles extreme cases such as \(x=0\) or \(x=n\), where Wald can give degenerate or uninformative intervals.

Worked example

In a survey of 200 customers, 134 report being satisfied. \(\hat{p} = 134/200 = 0.67\). With 95% confidence (\(C=0.95\), \(\alpha=0.05\), \(z_{0.025}=1.960\)):

With Wald, the standard error is \(\sqrt{0.67\cdot0.33/200}\approx0.03325\), and the interval is:

\( 0.67 \pm 1.960 \cdot 0.03325 \approx [0.605,\; 0.735] \)

With Wilson, applying the score formula to the same \(x=134\), \(n=200\) and \(z=1.960\), we get:

\( CI_{Wilson} \approx [0.602,\;0.731] \)

In this example both methods are similar because the sample is large and the proportion isn't extreme. If we had observed very few successes, very few failures, or a small sample, the difference would be larger and Wilson would be the more reliable choice.

The Clopper-Pearson method (exact)

Wald and Wilson rely on the normal approximation of \(\hat{p}\). Clopper and Pearson (1934) proposed approximating nothing: the number of successes \(X\) follows a binomial \(B(n, p)\) exactly, and the interval is built from that distribution. That is why it is called exact.

What it consists of

The interval contains all the values of \(p\) for which the observed result, \(x\) successes, is not too extreme. Each end is defined by a binomial tail equal to \(\alpha/2\):

Development and additional details

\[ \begin{aligned} p_L &: \quad P(X \ge x \mid p_L) = \tfrac{\alpha}{2}, \\ p_U &: \quad P(X \le x \mid p_U) = \tfrac{\alpha}{2}. \end{aligned} \]

In other words, \(p_L\) is the smallest value of \(p\) for which getting \(x\) successes or more still has probability \(\alpha/2\), and \(p_U\) is the largest for which getting \(x\) or fewer does. If \(x = 0\), \(p_L = 0\) is taken, and if \(x = n\), \(p_U = 1\).

Both equations always have a single solution: as \(p\) increases, the probability \(P(X \ge x \mid p)\) rises continuously from 0 to 1, and \(P(X \le x \mid p)\) falls from 1 to 0. It is the same idea as the Wilson interval, which gathers the values of \(p\) a test would not reject, but with the exact binomial test instead of the approximate one.

What it is based on: the link between the binomial and the beta

The equations above cannot be solved by hand, but they have a closed-form solution thanks to an identity between the binomial and the beta distribution:

\[ \begin{gathered} P(X \ge x \mid p) = P(B \le p), \\ B \sim \mathrm{Beta}(x,\; n - x + 1). \end{gathered} \]

It is proved with order statistics. Generate \(n\) independent uniform numbers between 0 and 1, \(U_1, \ldots, U_n\). The number of them that fall below \(p\) is a binomial \(B(n, p)\), because each does so with probability \(p\). Sorting them from smallest to largest, \(U_{(1)} \le \cdots \le U_{(n)}\), there are at least \(x\) below \(p\) exactly when the \(x\)-th smallest is below \(p\):

\[ X \ge x \iff U_{(x)} \le p. \]

And the \(x\)-th smallest of \(n\) uniforms follows a \(\mathrm{Beta}(x,\, n - x + 1)\) distribution. Its density is proportional to \(u^{x-1}(1-u)^{n-x}\), because for it to equal \(u\) there must be \(x - 1\) values below \(u\) and \(n - x\) above. Putting the two together, \(P(X \ge x) = P(U_{(x)} \le p)\), which is the identity.

With it, the first equation says that \(p_L\) leaves an area \(\alpha/2\) to its left under a \(\mathrm{Beta}(x,\, n - x + 1)\): it is its \(\alpha/2\) quantile. For the second, use \(P(X \le x) = 1 - P(X \ge x + 1)\) and the same identity with \(x + 1\). The result is:

\[ \begin{aligned} p_L &= \text{quantile } \tfrac{\alpha}{2} \text{ of } \mathrm{Beta}(x,\; n - x + 1), \\ p_U &= \text{quantile } 1 - \tfrac{\alpha}{2} \text{ of } \mathrm{Beta}(x + 1,\; n - x). \end{aligned} \]

This is what this calculator computes. Some of those quantiles can be looked up in the beta distribution table.

When \(x = 0\) it can be worked out by hand: \(P(X \le 0 \mid p) = (1-p)^n\), and setting it equal to \(\alpha/2\),

\[ p_U = 1 - \left(\tfrac{\alpha}{2}\right)^{1/n}. \]

With \(n = 20\) and 95%, \(p_U = 1 - 0.025^{1/20} = 0.168\): seeing no successes in 20 trials is compatible with a true proportion of up to 16.8%.

Why it guarantees coverage

Fix the true value \(p\). The interval misses on the upper side (\(p_U < p\)) only if an \(x\) is observed so small that \(P(X \le x \mid p) < \alpha/2\). Since that probability increases with \(x\), the values of \(x\) that cause the miss form a run \(0, 1, \ldots, x^*\), and the probability of landing in it is \(P(X \le x^* \mid p)\), which is less than \(\alpha/2\). By the same reasoning, the probability of missing on the lower side is also less than \(\alpha/2\). Therefore,

\[ \begin{aligned} P(\text{the interval does not contain } p) &\le \frac{\alpha}{2} + \frac{\alpha}{2} \\ &= \alpha, \end{aligned} \]

that is, the coverage is at least \(1 - \alpha\) for any value of \(p\) and any \(n\). Neither Wald nor Wilson has that guarantee. The price is that the actual coverage is almost never exactly \(1 - \alpha\), but higher: since \(X\) only takes whole values, the binomial tails cannot equal exactly \(\alpha/2\), and the interval always ends up on the safe side. That is why it is called conservative, and it is usually somewhat wider than Wilson's.

Examples

  • 2 successes in 20, at 95%: \(p_L\) is the 0.025 quantile of a \(\mathrm{Beta}(2, 19)\), 0.012; \(p_U\) is the 0.975 quantile of a \(\mathrm{Beta}(3, 18)\), 0.317. Check: with \(p = 0.012\), \(P(X \ge 2) = 0.025\); with \(p = 0.317\), \(P(X \le 2) = 0.025\). Wilson gives \([0.028,\ 0.301]\).
  • 134 successes in 200, at 95% (the worked example): Clopper-Pearson gives \([0.600,\ 0.735]\), Wilson \([0.602,\ 0.731]\) and Wald \([0.605,\ 0.735]\). With large samples and intermediate proportions, the three almost coincide.

Actual coverage of the three methods

Exact probability that a 95% interval contains the true value:

CaseWaldWilsonClopper-Pearson
p = 0.05, n = 2063.9%92.5%98.4%
p = 0.05, n = 10087.7%96.6%98.3%
p = 0.10, n = 5087.9%97.0%97.0%
p = 0.30, n = 2094.7%97.5%97.5%
p = 0.50, n = 3095.7%95.7%95.7%

Averaging over all values of \(p\) with \(n = 20\), Wald covers 85%, Wilson 95% and Clopper-Pearson 98%. Wilson is right on average, but for some values of \(p\) it drops to around 89%; Clopper-Pearson never drops below 95%, at the cost of almost always overshooting. You can see it with simulated samples in the confidence interval simulator.

Assumptions for the CI of a proportion

  • The data must come from Bernoulli trials: each observation is classified as success or failure.
  • The observations must be independent; if sampling without replacement from a finite population, the sample should be small relative to the population or a specific correction should be applied.
  • The probability of success must be the same for all observations within the analyzed group.
  • For Wald, enough successes and failures are needed: \(n\hat p\ge 5\) and \(n(1-\hat p)\ge 5\). Wilson is more reliable when this condition is doubtful.

Differences between Wald, Wilson and Clopper-Pearson

Aspect Wald Wilson Clopper-Pearson
Idea Direct normal approximation of \(\hat{p}\). Score interval: values of \(p\) compatible with the data. Values of \(p\) compatible with the data according to the exact binomial test.
Center Centered on \(\hat{p}\). Center adjusted toward less extreme values. Not built as center ± margin; usually asymmetric.
Bounds Can fall outside \([0,1]\). Stays within \([0,1]\). Stays within \([0,1]\).
When to use it Large samples, non-extreme proportions and quick computation. Recommended for general use; especially if \(n\) is small, \(\hat{p}\) is extreme, or there are few successes/failures. When coverage must be guaranteed not to fall below the nominal level (clinical trials, quality control, regulatory reports).

In short: Wald is useful as a quick, didactic approximation; Wilson is preferable as the main result because it better respects the binomial nature of the data and avoids impossible bounds. Clopper-Pearson is the cautious option: it never covers less than promised, although its intervals are somewhat wider.

When to use each method?

  • Use Wilson by default if you want a robust interval for most situations.
  • Use Wald only as a quick approximation when \(n\) is large and there are enough successes and failures.
  • Avoid Wald if \(x=0\), \(x=n\), \(n\) is small, or \(\hat{p}\) is close to 0 or 1.
  • Use Clopper-Pearson if you need the guarantee that actual coverage does not fall below the nominal level, even if the interval is somewhat wider.
  • Compare the methods if you are in an intermediate zone: if they differ a lot, interpret Wald with caution and prioritize Wilson or Clopper-Pearson.

How to interpret the result

The interval \([L, U]\) is the range of values of the population proportion \(p\) compatible with the observed data at the chosen confidence level. In frequentist terms: if you repeated the sampling many times and built the CI with the same method, a proportion \(C\) of those intervals would contain the true value of \(p\). The width depends on the sample size \(n\) and on the estimated proportion itself \(\hat{p}\): variability is greatest when \(\hat{p} = 0.5\) and decreases as it moves away from that value. In the chart, the green region under the standard normal curve is the confidence zone and the red tails (area \(\alpha/2\) each) delimit the critical values \(\pm z_{\alpha/2}\).

To halve the width of the interval while keeping the same confidence level, you need to quadruple the sample size, since the margin of error is proportional to \(1/\sqrt{n}\). Increasing confidence from 95% to 99% widens the interval because \(z_{0.005} = 2.576 > z_{0.025} = 1.960\); that is, demanding more certainty comes at the cost of a less precise estimate.

Development and additional details

  • Wald vs. Wilson: when the condition \(n\hat{p} \geq 5\) and \(n(1-\hat{p}) \geq 5\) is met at both ends, the two methods usually give very similar results. If the proportion is extreme or the sample is small, Wilson is more reliable because the center of the interval shifts slightly toward less extreme values and the bounds always stay within \([0, 1]\).
  • Connection with hypothesis testing: if a reference value \(p_0\) falls outside the interval, the data would reject \(H_0\!: p = p_0\) at level \(\alpha = 1 - C\) in the equivalent two-sided test. If \(p_0\) falls inside, there isn't enough evidence to reject it.
  • Practical relevance: a very narrow CI indicates high precision, but remember that statistical precision and practical relevance are different concepts. An interval from 67% to 68% is very precise but could be irrelevant in contexts where only exceeding 70% matters.

Frequently asked questions

  • Can I enter p̂ directly instead of x? Enter x = p̂ × n rounded to the nearest integer; the result will be equivalent.
  • What happens if x = 0 or x = n? Wald gives an interval of zero width, which is not useful. Wilson and Clopper-Pearson handle these cases correctly; for example, with 0 successes in 20, at 95%, Clopper-Pearson gives [0, 0.168].
  • When is the normality condition met? When n·p̂ ≥ 5 and n·(1−p̂) ≥ 5. The calculator warns you if it isn't met.

Reference: Binomial proportion confidence interval — Wikipedia. Clopper, C. J. and Pearson, E. S. (1934). The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika, 26(4), 404–413. Brown, L. D., Cai, T. T. and DasGupta, A. (2001). Interval estimation for a binomial proportion. Statistical Science, 16(2), 101–133.