Hypothesis tests

Fisher's exact test calculator (2×2)

Exact independence test for 2×2 tables with small frequencies.

Parameters

Enter the counts of the 2×2 table to get the exact p-value, the odds ratio and the test decision.

Column A Column B
Row 1
Row 2
Result pending…

Hypergeometric distribution under H₀

The chart shows the probability mass function of the hypergeometric distribution: each bar represents the exact probability that cell a takes the value k, given the fixed margins of the table. Bars in red are the ones that contribute to the p-value (the most extreme ones according to the test type); the bar in orange marks the observed value. The p-value is the sum of the red probabilities.

What is Fisher's exact test?

It is the test for deciding whether there is an association between two dichotomous variables arranged in a 2×2 table —treatment or placebo against recovered or not, exposed or not against ill or not— when the sample is small. Unlike the chi-square test and the two-proportion z-test, it does not rely on any normal approximation: it computes the exact probability of every possible table. That makes it the right choice for small trials, rare events, subgroup analyses and whenever an expected frequency is below 5.

Ronald Fisher devised it with the famous experiment of the lady who claimed she could tell whether the milk or the tea had been poured first into her cup: with only 8 cups, the question was how many correct answers were needed to rule out that she was guessing.

The table and the hypothesis

Column AColumn BTotal
Row 1aba + b
Row 2cdc + d
Totala + cb + dn

\(H_0\): rows and columns are independent (\(OR = 1\))

\(H_1\): there is an association (\(OR \neq 1\), \(OR > 1\) or \(OR < 1\))

Usually the rows are the groups (treatment and control) and the columns the outcome (yes and no). Then \(H_0\) says that the proportion of column A is the same in both rows: \(a/(a+b)\) against \(c/(c+d)\).

How the exact p-value is computed

The test fixes the row and column totals and asks: if there were no association, how likely would each possible way of distributing the individuals within the table be? With fixed margins, knowing cell a is enough to know the whole table, and under \(H_0\) that cell follows a hypergeometric distribution:

\( P(X = k) = \dfrac{\dbinom{a+b}{k}\dbinom{c+d}{a+c-k}}{\dbinom{n}{a+c}} \)

The p-value adds up the probabilities of the tables at least as extreme as the observed one:

  • Right tail (\(OR > 1\)): \(p = \sum_{k \geq a} P(X = k)\).
  • Left tail (\(OR < 1\)): \(p = \sum_{k \leq a} P(X = k)\).
  • Two-tailed: sum of all the tables whose probability is less than or equal to that of the observed one.

The calculator's chart shows exactly that distribution: one bar for each possible value of a, with those included in the p-value highlighted.

Because the distribution is discrete, the test is conservative: the real risk of a false positive usually ends up somewhat below \(\alpha\). That is the price of not relying on approximations. With large samples, Fisher and chi-square give very similar results.

Decision rule

  • If the exact p-value is less than \(\alpha\), \(H_0\) is rejected: there is evidence of an association.
  • If it is greater than or equal to \(\alpha\), there is not enough evidence to reject independence.
  • The direction is given by the odds ratio: OR > 1 indicates that column A is more frequent in row 1.

Fisher, chi-square or z-test?

SituationRecommended test
2×2 table with an expected frequency < 5Fisher's exact test
2×2 table with large samplesChi-square or two-proportion z-test (Fisher also works)
Table larger than 2×2Chi-square test of independence (merging categories if needed)
Paired dataMcNemar's test

With today's computers, Fisher can also be used with large samples. The reason for preferring chi-square or the z-test in that case is practical: they make confidence intervals and power easy to compute, and the results agree.

Assumptions of the test

  • Independent observations, each classified into a single cell.
  • Two dichotomous variables.
  • Fixed margins: strictly speaking, the test assumes fixed row and column totals. In practice it is also used when they are not, and it is then somewhat conservative.

It requires no minimum sample size: it is valid even with zero cells, which give an odds ratio of 0 or infinity.

Worked example 1: a small trial

In a trial, 20 patients are randomly assigned to treatment or placebo, 10 in each group. With the treatment 7 of 10 recover; with placebo, 4 of 10. The table is a = 7, b = 3 (treatment) and c = 4, d = 6 (placebo). The margins are 10 and 10 per row, and 11 recovered and 9 not recovered in total.

The probability of the observed table under \(H_0\) is

\( P(X = 7) = \dfrac{\dbinom{10}{7}\dbinom{10}{4}}{\dbinom{20}{11}} \approx 0.150 \)

For the right tail (is the treatment better?) you add the tables with 7 or more recoveries in the treated group: \(P(X \geq 7) \approx 0.150 + 0.032 + 0.003 + 0.0001 \approx 0.185\). The two-tailed p-value adds the equally or less likely tables at both ends, \(p \approx 0.370\). \(H_0\) is not rejected.

The odds ratio is \((7 \times 6)/(3 \times 4) = 3.5\) and the difference in recovery is 30 percentage points: an effect that would be very relevant. The problem is the size. With 10 patients per group and 40% recovery on placebo, the test does not reach 80% power even for the largest possible difference: not even a treatment that cured everyone would be detected with that certainty. Here, a non-significant result says very little.

In the calculator: a = 7, b = 3, c = 4, d = 6.

Worked example 2: a large effect with little data

With the table the calculator opens with, a = 8, b = 2, c = 1, d = 9, column A appears in 80% of row 1 and 10% of row 2. The two-tailed exact p-value is \(p \approx 0.0055\): \(H_0\) is rejected. The odds ratio is \((8 \times 9)/(2 \times 1) = 36\), with a very wide confidence interval, as expected from such a small sample.

Pearson's chi-square would give \(p \approx 0.0017\) here, considerably smaller, but it is not reliable: the expected frequencies of the four cells are 4.5 and 5.5, at the limit of Cochran's rule. Fisher is the correct reference.

In the calculator: these are the values the page opens with.

How to interpret the result

Rejecting \(H_0\) (p-value < α) indicates that the two variables are associated. To judge the magnitude, look at the proportions in each row, their difference and the odds ratio with its confidence interval; in small samples, that interval tends to be very wide.

Not rejecting \(H_0\) does not prove independence. With little data, the power of the test is low, as in example 1.

In the calculator, the result shows the decision with the exact p-value, the odds ratio, the proportion of column A in each row and a sentence stating the conclusion. Under "Calculation details" you will find the difference between rows in percentage points and the probability of the observed table.

Power and minimum detectable difference

Below the conclusion, the calculator shows the minimum detectable difference: the smallest real difference from the proportion of row 2 that, with your row totals, the test would detect with 80% probability. With small samples it is usually huge, or even impossible to reach. The link opens the sample size calculator for Fisher's test with your proportions, your α, the ratio between the rows and the type of test.

Confidence interval for the odds ratio

The other link opens the odds ratio confidence interval calculator with your table. If a cell is zero, it applies the 0.5 correction automatically.

Common mistakes

  • Using chi-square with very small expected frequencies instead of the exact test.
  • Concluding that there is no effect in a small study because the test is not significant.
  • Choosing the tail after seeing the data.
  • Applying it to paired data: that is what McNemar's test is for.
  • Reporting only the p-value, without the proportions or the odds ratio.

Frequently asked questions

  • When is Fisher's exact test used? For 2×2 tables with small samples, especially when an expected frequency is below 5 and chi-square is not reliable.
  • Why is it called exact? Because it computes the p-value from the hypergeometric distribution, without normal or chi-square approximations.
  • Can I use it with large samples? Yes. It gives results very similar to chi-square; the latter is preferred then because it makes intervals and power calculations easier.
  • Does it work with zero cells? Yes. The odds ratio will be 0 or infinite, but the exact p-value is computed without problems.
  • Why is it conservative? Because the distribution is discrete: there is not always a rejection region with probability exactly α, and the real risk ends up somewhat below it.
  • Two-tailed or one-tailed? Two-tailed, unless before seeing the data you are interested in only one direction of the association.
  • What is the minimum detectable difference? The smallest real difference between the proportions of the two rows that the test would detect with 80% probability given your sizes.
  • Does it work for tables larger than 2×2? There is an extension (Fisher-Freeman-Halton), but this calculator is for 2×2 tables. For larger tables, use the chi-square test of independence.

Reference: Fisher's exact test — Wikipedia