Online hypothesis test calculator for two proportions

Explanation

This test is used when comparing two independent groups with binary outcomes. Under \(H_0\), it is assumed that \(p_1 = p_2\), and a pooled proportion is used for the standard error, because under the null hypothesis both groups share the same probability of success.

This approach makes it possible to evaluate consistently whether the observed difference reflects sampling fluctuation or a structural change between conditions.

Hypotheses and test statistic

\(H_0: p_1 - p_2 = 0\)

\( z = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})(1/n_1 + 1/n_2)}} \)

\(\hat{p}=\frac{x_1+x_2}{n_1+n_2}\)

• \(\hat{p}_1 = x_1/n_1\): observed sample proportion in group 1.
• \(\hat{p}_2 = x_2/n_2\): observed sample proportion in group 2.
• \(x_1, x_2\): number of successes in each group.
• \(n_1, n_2\): sizes of groups 1 and 2.
• \(\hat{p}\): pooled proportion under \(H_0\) (joint estimate of the proportion common to both groups).

Quick test

Under \(H_0\), both samples share the same unknown population proportion. The natural joint estimate of that common proportion is \(\hat{p}=(x_1+x_2)/(n_1+n_2)\).

That is why the standard error of the test is built with \(\hat{p}\), consistent with the null hypothesis of equality \(p_1=p_2\).

Worked example: comparing conversions

Version A gets 420 conversions out of 10,000 visits and version B gets 480 out of 10,000. Enter the successes and sizes of both groups to get z, the p-value and the decision. Besides significance, check the absolute difference: 4.8% − 4.2% = 0.6 percentage points.

How to interpret the result

Rejecting \(H_0\) (p-value < α) indicates that the observed difference between \(\hat{p}_1\) and \(\hat{p}_2\) is statistically incompatible with the hypothesis of population equality. In practical terms, there is evidence that the two conditions, groups, or versions produce different success rates. However, the size of the difference matters as much as its significance: in A/B testing, a difference of 0.5 percentage points can be statistically significant with massive traffic but have a negligible business impact. Report the estimated difference \(\hat{p}_1 - \hat{p}_2\) and its confidence interval to give context to the result.

Not rejecting \(H_0\) (p-value ≥ α) does not prove that the proportions are equal: it only indicates that the data are compatible with that equality at the chosen level. With small samples, the test may not have enough power to detect real differences. Before concluding there is no effect, check that the sample size was adequate to detect the minimum difference of practical interest.

The z statistic measures how many standard errors the sample difference is from zero, using the pooled proportion \(\hat{p}\) to estimate the variance under \(H_0\). In the chart, the green zone corresponds to the non-rejection region, the red zones are the critical regions, and the amber line marks the observed statistic. To strengthen the interpretation, consider computing the relative risk or the number needed to treat (NNT), which offer a more direct view of the practical impact.

Frequently asked questions

• Is it valid for dependent groups? No; for paired data you need a specific method for paired proportions.
• What exactly does it measure? Whether the observed difference between \(p_1\) and \(p_2\) can be explained by chance under \(H_0\).
• What sample size is recommended? The larger, the better the precision; with small samples the power can be limited.
• What should I report besides the p-value? The difference of proportions, its confidence interval, and the practical context of the decision.
• Is there also a case of known population standard deviation here? Not like in tests for means. For two proportions, the variability under \(H_0\) is estimated with the pooled proportion \(\hat{p}\), so there is no choice between a z-test or a t-test based on \(\sigma\).