Hypothesis tests

Online McNemar test calculator for paired proportions

Test whether the marginal proportions differ in paired binary data, based on the discordant pairs.

Parameters

Enter the discordant pairs of the 2×2 table to get the chi-square statistic, p-value and decision for McNemar's test.

Result pending…

Rejection regions and observed statistic

What is McNemar's test?

It is the test for comparing two proportions when the data are paired: each individual gives a yes/no answer under two conditions or at two times. For example, the same patients on two drugs in a crossover trial, the same person before and after a campaign, two diagnostic tests applied to the same samples, or two raters classifying the same cases. The question is whether the proportion of "yes" changes from one condition to the other.

The test for two independent proportions will not do, because the two answers from the same individual are related: someone who improves on one drug tends to improve on the other. Ignoring that pairing gives wrong p-values.

The 2×2 table for paired data

The data are summarised by counting how many individuals fall into each combination of answers:

Condition 2: yesCondition 2: no
Condition 1: yesa (yes in both)b (only in 1)
Condition 1: noc (only in 2)d (in neither)

With \(N = a + b + c + d\) individuals, the proportion of successes is \(p_1 = (a+b)/N\) under condition 1 and \(p_2 = (a+c)/N\) under condition 2. Their difference is

\( p_1 - p_2 = \dfrac{(a + b) - (a + c)}{N} = \dfrac{b - c}{N} \)

Cells a and d drop out: individuals who answer the same way under both conditions say nothing about which is better. All the information is in the discordant pairs, b and c. That is why the calculator only asks for them.

Hypotheses and test statistic

\(H_0: p_1 = p_2\) (equivalent to both types of change being equally frequent among the discordant pairs)

\( \chi^2 = \dfrac{(b - c)^2}{b + c} \qquad \text{or, with continuity correction,} \qquad \chi^2 = \dfrac{(|b - c| - 1)^2}{b + c} \)

If \(H_0\) is true, each discordant pair is equally likely to be of type b or type c, like tossing a coin. The statistic measures how far the observed split is from 50–50 and, under \(H_0\), approximately follows a chi-square distribution with 1 degree of freedom. It is the square of the statistic \(z = (b - c)/\sqrt{b + c}\).

  • Continuity correction (Edwards): subtracts 1 from \(|b - c|\) to compensate for the counts being whole numbers. It is more conservative and is the default option.
  • Exact version: with few discordant pairs (fewer than about 25), the most reliable approach is the exact binomial p-value: b follows a binomial distribution with parameters \(b + c\) and 0.5. The calculator shows it in the details and warns you when you should use it.

Decision rule

  • If the p-value is less than \(\alpha\), \(H_0\) is rejected: the proportion of successes differs between the two conditions.
  • If the p-value is greater than or equal to \(\alpha\), there is not enough evidence to reject \(H_0\).
  • The direction of the difference is given by the sign of \(b - c\): if b > c, condition 1 has more successes.

Assumptions of the test

  • Paired data: each individual (or matched pair) gives one answer under each condition.
  • A dichotomous response under both conditions.
  • Individuals independent of each other.
  • Enough discordant pairs for the chi-square approximation. With fewer than 25, use the exact p-value.

McNemar's test is the yes/no counterpart of the paired means test. With more than two conditions and a dichotomous response, the generalisation is Cochran's Q test.

Worked example 1: crossover trial with two drugs

In a crossover trial, 100 patients receive two drugs in different periods. 45 improve with both (a), 18 only with drug 1 (b), 8 only with drug 2 (c) and 29 with neither (d). The improvement rate is 63% with drug 1 and 53% with drug 2. We test \(H_0\colon p_1 = p_2\) with \(\alpha = 0.05\) and continuity correction.

\( \chi^2 = \dfrac{(|18 - 8| - 1)^2}{18 + 8} = \dfrac{81}{26} \approx 3.115 \)

With 1 degree of freedom, the critical value is 3.841 and the p-value is \(p \approx 0.078\). \(H_0\) is not rejected. Since there are only 26 discordant pairs, it is worth also looking at the exact p-value, 0.076: the conclusion does not change.

Conclusion: there is not enough evidence that the drugs differ, although the result is close to the threshold and leans towards drug 1 (18 changes against 8). With only 26 discordant pairs, the test detects with 80% probability only differences in which one type of change is at least 3.2 times as frequent as the other; here the observed ratio is 18/8 = 2.25.

In the calculator: b = 18, c = 8 and "With continuity correction". These are the values the page opens with.

Worked example 2: before and after a campaign

200 people are asked whether they know a brand, before and after an advertising campaign. 12 knew it before but not after (b) and 35 did not know it before but did after (c); everyone else answered the same both times. We test with \(\alpha = 0.05\).

\( \chi^2 = \dfrac{(|12 - 35| - 1)^2}{12 + 35} = \dfrac{484}{47} \approx 10.30 \)

The p-value is \(p \approx 0.0013\): \(H_0\) is rejected. Brand awareness changed, in favour of after the campaign, because c > b. The net increase is \((35 - 12)/200 = 11.5\) percentage points. Since this is a before-and-after design without a control group, the change may also be due to other causes.

In the calculator: b = 12, c = 35.

How to interpret the result

Rejecting \(H_0\) (p-value < α) indicates that the proportion of successes differs between the two conditions; the direction is set by which type of discordant pair is more frequent. To judge the size of the change, compute the difference \((b - c)/N\), which needs the total number of individuals, or the paired odds ratio \(b/c\), which the calculator shows with its confidence interval.

Not rejecting \(H_0\) does not prove that the conditions are equal: with few discordant pairs the test has little power.

In the calculator, the result shows the decision with the p-value, the χ² statistic and a sentence stating the conclusion and the direction of the difference. On the chart, the red area is the rejection region on the chi-square distribution and the line marks the observed statistic. Under "Calculation details" you will find the exact p-value, the z statistic, the share of each type of discordant pair and the paired odds ratio.

Power: what matters is the number of discordant pairs

The power of McNemar's test does not depend on the total number of individuals, but on how many pairs are discordant and how unbalanced their split is. That is why, below the conclusion, the calculator shows the minimum detectable effect with your discordant pairs: how many times more frequent one type of change must be than the other for the test to detect it with 80% probability. With 26 discordant pairs you need a ratio of more than 3 to 1; with 100, about 1.8 to 1 is enough.

Planning a study requires the expected proportions of each type of change over the total, which cannot be deduced from b and c alone. You can work it out with the sample size calculator for paired proportions.

Common mistakes

  • Analysing paired data with the test for two independent proportions, comparing the two marginal proportions as if they were different groups.
  • Entering a or d in the discordant fields: the test only uses b and c.
  • Using the chi-square approximation with very few discordant pairs instead of the exact p-value.
  • Forgetting the direction: a significant result indicates a difference, and b against c shows which condition it favours.
  • Attributing the change to the intervention in a before-and-after design without a control group.

Frequently asked questions

  • When is McNemar's test used? To compare two proportions measured on the same individuals, or on matched pairs: before and after, two treatments in a crossover trial, or two diagnostic tests on the same samples.
  • Why does it only use the discordant pairs? Because the difference between the two proportions is (b − c)/N: individuals who answer the same way under both conditions say nothing about which is better.
  • With or without continuity correction? The correction is more conservative and is the usual choice. With few discordant pairs, the most reliable option is the exact binomial p-value, which the calculator shows in the details.
  • When should I use the exact version? When there are fewer than about 25 discordant pairs. It relies on the fact that, under H₀, b follows a binomial distribution with b + c trials and probability 0.5.
  • What hypothesis does it test? That the proportion of successes is the same under both conditions, which is equivalent to both types of discordant pair being equally frequent.
  • How does it differ from the two-proportion test? The two-proportion test assumes independent groups; McNemar takes into account that both answers come from the same individual.
  • What is the paired odds ratio? The ratio b/c: how many times more frequent the change in favour of condition 1 is than the opposite change.
  • What if I have more than two conditions? For three or more conditions with a yes/no response, use Cochran's Q test.

Reference: McNemar's test — Wikipedia