When does the CI indicate a significant difference?

When the value 0 falls outside the interval [L, U], the difference of proportions is significant at the chosen confidence level.

Which difference is calculated?

The calculator estimates p₁ − p₂, that is, the proportion of group 1 minus the proportion of group 2.

When should Newcombe-Wilson be used?

It is recommended when samples are small, proportions are close to 0 or 1, or Wald's normality condition is not met.

CI for the difference of proportions

Calculator

Enter the successes and sample sizes of each group to get the confidence interval for p₁ − p₂.

Successes group 1 (x₁)

Sample size group 1 (n₁)

Successes group 2 (x₂)

Sample size group 2 (n₂)

Confidence level

Method

Custom confidence level (0–1)

Result pending…

Explanation

When comparing two independent groups (for example, conversion in A/B testing, prevalence in two populations, or satisfaction percentage by segment), the confidence interval for the difference of proportions quantifies the size of the difference and its uncertainty.

The calculator estimates \(p_1 - p_2\) from \(\hat{p}_1 = x_1/n_1\) and \(\hat{p}_2 = x_2/n_2\). If the interval contains 0, the observed difference may be compatible with no difference at the chosen confidence level.

Let \(C\) denote the confidence level and \(\alpha=1-C\) the total area outside the interval. For 95% confidence, \(C=0.95\), \(\alpha=0.05\) and \(\alpha/2=0.025\) in each tail.

Wald method (normal approximation)

\( (\hat{p}_1 - \hat{p}_2) \pm z_{\alpha/2} \cdot \sqrt{\dfrac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \dfrac{\hat{p}_2(1-\hat{p}_2)}{n_2}} \)

This is the classic method and is adequate when both groups have enough successes and failures: \(n_i\hat{p}_i \geq 5\) and \(n_i(1-\hat{p}_i) \geq 5\).

Newcombe-Wilson method (recommended)

The Newcombe-Wilson method does not apply the normal approximation directly to the difference \(\hat{p}_1-\hat{p}_2\). It first computes, separately, a Wilson score interval for each proportion: \([L_1, U_1]\) for \(p_1\) and \([L_2, U_2]\) for \(p_2\). The idea comes from Newcombe's (1998) work, which compared several methods for the difference of proportions and highlighted the good performance of Wilson-based methods.

It then combines the distances from each observed proportion to its Wilson limits, keeping the sign of the estimated difference \(d = \hat{p}_1 - \hat{p}_2\). The combination is not simply \([L_1-U_2, U_1-L_2]\): the uncertainties of both groups are added quadratically to build each end of the interval:

\( L = d - \sqrt{(\hat{p}_1-L_1)^2 + (U_2-\hat{p}_2)^2} \)

\( U = d + \sqrt{(U_1-\hat{p}_1)^2 + (\hat{p}_2-L_2)^2} \)

This combination tends to keep the actual coverage of the interval closer to nominal than Wald, especially with small samples or proportions near 0 or 1. That's why it's a good default choice for comparing two independent percentages. This page uses the version without continuity correction, which is the standard form for a fast, reproducible calculator.

Worked example

Group 1 has 84 successes out of 120 observations and group 2 has 57 successes out of 110 observations. \(\hat{p}_1 = 0.70\), \(\hat{p}_2 \approx 0.518\) and the estimated difference is \(0.182\). With 95% confidence (\(C=0.95\), \(\alpha=0.05\), Wald method):

\( 0.182 \pm 1.960 \cdot \sqrt{\frac{0.70\cdot0.30}{120} + \frac{0.518\cdot0.482}{110}} \approx [0.058,\; 0.306] \)

Assumptions for comparing two proportions

The two groups must be independent of each other; there must be no unit-to-unit pairing.
Within each group, every observation must be classified as a success or failure and have the same probability of success within that group.
Observations within each group must be independent.
Wald requires enough successes and failures in both groups: \(n_i\hat p_i\ge 5\) and \(n_i(1-\hat p_i)\ge 5\). If this doesn't hold, Newcombe-Wilson is usually preferable.

Wald or Newcombe-Wilson?

Wald is easy to interpret and generally works well with large samples and non-extreme proportions. Newcombe-Wilson tends to be preferable by default if either group has few successes or few failures, because it uses individual Wilson intervals and combines their uncertainties quadratically, not through a direct subtraction of limits. This reduces the coverage problems of the classic normal interval.

How to interpret the result

The interval \([L, U]\) is the plausible range of the difference of population proportions \(p_1 - p_2\) given the chosen confidence level. If the experiment were repeated many times with samples of the same size, a proportion \(C\) of the intervals built with the same method would contain the true value of \(p_1 - p_2\). The width of the interval reflects precision: larger samples or proportions farther from 0.5 produce narrower intervals; lower variability and larger \(n\) lead to more precise estimates.

The key decision hinges on whether 0 belongs to the interval. If \(0 \in [L, U]\), the data do not rule out \(p_1 = p_2\) at the chosen confidence level; equivalently, the two-sided test \(H_0\!: p_1 = p_2\) would not be rejected at level \(\alpha = 1 - C\). If \(0 \notin [L, U]\), there is evidence of a difference: if \(L > 0\) group 1's proportion is significantly higher; if \(U < 0\) group 2's proportion is significantly higher. The calculator automatically states which of the two cases applies, for both methods when shown together.

Effect magnitude: beyond significance, the center of the interval (\(\hat{p}_1 - \hat{p}_2\)) indicates the estimated difference in percentage points. A significant CI centered close to 0 may have little practical relevance.
Wald vs. Newcombe-Wilson: when Wald's normality condition holds in both groups, both methods usually agree on the conclusion. If either group has few successes or failures, Newcombe-Wilson tends to be more reliable because it uses individual Wilson intervals and combines their uncertainties quadratically.
Reading the chart: the green region under the standard normal curve is the confidence zone and the red tails (area \(\alpha/2\) each) mark the critical values \(\pm z_{\alpha/2}\). The chart visualizes the z values compatible with the data; the CI on the original scale is obtained by mapping those values back to the difference of proportions.

Frequently asked questions

When does the CI indicate a significant difference? When the value 0 falls outside the interval [L, U].
Which difference is calculated? The tool computes p₁ − p₂; if you swap the groups, the sign of the difference changes.
What does it mean if the CI includes 0? That at the chosen confidence level you cannot rule out that both population proportions are equal.

References used

GraphPad: CI for the difference between two proportions — explains the classic Wald-type interval and cites the comparison of Newcombe's methods.
Statology: Confidence Interval for the Difference in Proportions — a popular explanation of the normal/Wald formula.
NCSS/PASS: Confidence Intervals for the Difference Between Two Proportions — documentation of the Wilson score method modified by Newcombe and its bibliographic references.
Newcombe, R. G. (1998). Interval estimation for the difference between independent proportions: comparison of eleven methods, Statistics in Medicine, 17, 873–890.