Online McNemar test calculator for paired proportions

Explanation

McNemar's test is the standard procedure for testing hypotheses about paired proportions when the response variable is binary (success/failure). It is typically applied in before/after designs, paired case-control studies, and crossover trials where the response of the same individuals to two conditions is compared.

The key point of the test is that only the discordant pairs provide information about the difference between proportions. Concordant pairs (both success or both failure) do not indicate whether one condition is better than the other. This leads to an elegant formulation: under \(H_0\), the discordant pairs would be expected to split evenly between b and c.

2×2 table for paired data

Each pair falls into one of four cells:

• a: success in both conditions (concordant +/+)
• b: success in condition 1, failure in condition 2 (discordant)
• c: failure in condition 1, success in condition 2 (discordant)
• d: failure in both conditions (concordant −/−)

Hypotheses and test statistic

\(H_0\colon p_{\text{discordant},12} = p_{\text{discordant},21}\) (i.e., \(\pi_b = \pi_c\))

Without correction: \( \chi^2 = \dfrac{(b - c)^2}{b + c} \)

With continuity correction: \( \chi^2 = \dfrac{(|b - c| - 1)^2}{b + c} \)

\(df = 1\), right-tail p-value on the \(\chi^2\) distribution

Equivalent on the z scale (sign test): \( z = \dfrac{b - c}{\sqrt{b + c}} \)

Quick interpretation guide

• If the p-value is less than \(\alpha\), \(H_0\) is rejected: there is evidence that the marginal proportions differ.
• If the p-value is greater than or equal to \(\alpha\), there is not enough evidence to reject \(H_0\).
• Use the continuity correction as the default option; omit it only if \(b + c\) is very large (> 100).
• If \(b + c < 25\), consider the exact binomial test on the discordant pairs.

Why do only the discordant pairs matter?

Imagine that 100 people take two drugs in random order and report whether they improve or not. If someone improves with both or with neither, that tells us nothing about which drug is better; it only confirms that this person responds or does not respond. On the other hand, a subject who improves only with drug A (discordant pair b) or only with drug B (discordant pair c) provides direct evidence about the difference. That is why McNemar's test ignores the concordant cells a and d and concentrates all the inference on b and c.

Worked example

In a crossover trial, 100 patients evaluate two drugs. The results table is: a = 45 (improve with both), b = 18 (only drug 1), c = 8 (only drug 2), d = 29 (neither improves). \(H_0\colon \pi_b = \pi_c\) is tested with \(\alpha = 0.05\).

With continuity correction:

\( \chi^2 = \dfrac{(|18 - 8| - 1)^2}{18 + 8} = \dfrac{9^2}{26} = \dfrac{81}{26} \approx 3.115 \)

The p-value with \(df = 1\) is \(p \approx 0.078\). Since \(p = 0.078 > \alpha = 0.05\), \(H_0\) is not rejected.

Conclusion: with this data there is not enough evidence to claim that the two drugs produce different improvement rates at the 5% level, although the result is borderline.

How to interpret the result

Rejecting \(H_0\) (p-value < α) means there is statistical evidence that the marginal proportions of the two conditions are different. In terms of the 2×2 table, the discordant pairs (\(b\) and \(c\)) show an asymmetry that cannot be explained by chance. McNemar's test is especially sensitive to this type of change within the same subjects, which makes it more powerful than comparing two independent proportions when the design is paired.

Not rejecting \(H_0\) (p-value ≥ α) does not imply that the proportions are equal, but rather that the asymmetry between \(b\) and \(c\) is compatible with the random variation expected. With a small number of discordant pairs (\(b+c < 25\)), the chi-square approximation loses reliability: in that case the exact binomial test on the discordant pairs is recommended instead of the approximate statistic.

The McNemar chi-square statistic is computed only from \(b\) and \(c\) (the pairs where the two conditions disagree); the concordant pairs (\(a\) and \(d\)) provide no information about the difference. A large value of the statistic indicates strong asymmetry between \(b\) and \(c\). To assess practical relevance, complement it with the ratio \(b/c\) (marginal odds ratio) or with the difference of marginal proportions \((b-c)/n\) and its confidence interval.

Frequently asked questions

• When is McNemar's test used? When a binary variable is measured twice on the same subjects and you want to compare the marginal proportions.
• What are discordant pairs? The pairs where the two conditions give different results (b and c). Concordant pairs (a and d) provide no information about the difference.
• Why is the continuity correction applied? It adjusts the discrete statistic toward the continuous chi-square distribution, reducing the type I error when b+c is small.
• What is the exact alternative? The exact binomial test on the discordant pairs, recommended when b+c < 25.
• Can McNemar's test be used with more than two categories? Yes, with Bowker's test of symmetry for k×k tables.

Reference: McNemar's test — Wikipedia