Fisher's exact test calculator (2×2)

Explanation

Fisher's exact test lets you test whether there is an association between two binary categorical variables organized in a 2×2 table. Unlike Pearson's chi-square, it needs no asymptotic approximation: it calculates the exact probability of every possible table with the same fixed margins using the hypergeometric distribution.

It's especially suited to cases where expected frequencies are low (for example, below 5 in some cell), in small clinical trials, studies of rare events or subgroup analyses. When the sample size is large, Pearson's chi-square tends to agree with Fisher's test.

Under the null hypothesis of independence, the count in cell (1,1) follows a hypergeometric distribution conditional on the table's margins:

Hypotheses and test statistic

\(H_0\): independence between rows and columns (OR = 1)

\(H_1\): association (OR ≠ 1)

\[P(X = k) = \frac{\dbinom{a+b}{k}\dbinom{c+d}{a+c-k}}{\dbinom{n}{a+c}}\]

\[\text{OR} = \frac{a \cdot d}{b \cdot c}\]

Quick test

The exact p-value is obtained by summing the hypergeometric probabilities of every table with the same margins that is at least as extreme as the observed one. For the two-tailed test, every table whose probability is less than or equal to that of the observed table is summed.

Right tail: \(\displaystyle p = \sum_{k \geq a} P(X = k)\)

Left tail: \(\displaystyle p = \sum_{k \leq a} P(X = k)\)

Two-tailed: \(\displaystyle p = \sum_{P(X=k)\,\leq\,P(X=a)} P(X = k)\)

The odds ratio (OR) quantifies the strength of the association: OR = 1 indicates independence, OR > 1 indicates a higher frequency of the event in row 1, OR < 1 indicates the opposite. To complete the analysis, it's a good idea to accompany the p-value with a confidence interval for the OR.

Worked example

In a clinical trial, 20 patients are randomly assigned to two groups: 10 receive the treatment and 10 receive a placebo. In the treatment group, 7 patients recover and 3 don't recover. In the control group, 4 recover and 6 don't recover. The resulting \(2 \times 2\) contingency table is: treatment/recovered = 7, treatment/not recovered = 3, control/recovered = 4, control/not recovered = 6.

With small expected frequencies in some cell, the usual chi-square test can be imprecise, so Fisher's exact test is applied instead. The margins are: row 1 (treatment) = 10, row 2 (control) = 10, column 1 (recovered) = 11, column 2 (not recovered) = 9, total = 20.

The exact probability of obtaining the observed table under \(H_0\) (independence) is calculated with the hypergeometric distribution:

\( P = \frac{\binom{11}{7}\binom{9}{3}}{\binom{20}{10}} = \frac{330 \times 84}{184756} \approx 0.150 \)

The one-tailed p-value (is the treatment better than the placebo?) is obtained by summing the probabilities of every table that is just as extreme or more extreme in the direction favorable to the treatment: \(P(X \geq 7) = P(7)+P(8)+P(9)+P(10) \approx 0.150 + 0.032 + 0.003 + 0.0001 \approx 0.185\). The two-tailed p-value is \(p_{\text{two-tailed}} \approx 0.37\).

Since \(p_{\text{two-tailed}} = 0.37 > 0.05 = \alpha\), \(H_0\) is not rejected. The observed odds ratio is \(\widehat{OR} = (7 \times 6)/(3 \times 4) = 42/12 = 3.5\), suggesting a trend favorable to the treatment, but the sample (n = 20) isn't large enough to reach statistical significance.

Conclusion: at a 5% significance level, it can't be concluded that there is a statistically significant association between the treatment and recovery. It's recommended to replicate the study with a larger sample size to detect clinically relevant differences with adequate power.

How to interpret the result

If the exact p-value is less than \(\alpha\), \(H_0\) is rejected: there is statistical evidence of association between the rows and columns of the 2×2 table. If the p-value is greater than or equal to \(\alpha\), there isn't enough evidence to reject independence with the available data. Unlike chi-square, this p-value is exact: it doesn't depend on any asymptotic approximation and is reliable even when expected frequencies are very small or some cell contains few cases.

The odds ratio (OR) quantifies the magnitude and direction of the association. An OR = 1 indicates perfect independence between the two variables. An OR > 1 indicates that the event in row 1 is more likely in column A than in column B; OR < 1 indicates the opposite. For example, OR = 3.5 means the probability of the event is 3.5 times higher in the row 1 group. The exact direction and interpretation always depend on how the rows and columns were labeled: it's best to describe the result in substantive terms of the problem, not just numerically.

On choosing between Fisher and chi-square: Fisher's exact test is preferable when some expected (not observed) frequency is below 5, or when the total sample size is smaller than 20. With large samples, both tests converge and Pearson's chi-square is computationally more direct. One limitation of Fisher's test is that it conditions on the table's fixed margins, an assumption that isn't always realistic in prospective studies; in those cases the use of alternatives such as Barnard's test is discussed, although Fisher's test remains the standard in most biomedical and experimental contexts.

Frequently asked questions

• When should I choose Fisher's test over chi-square? When some expected frequency is below 5, or when the total sample size is smaller than 20.
• Is Fisher's test conservative? In some scenarios it can be, compared with alternatives such as Barnard's test, but it's the most widely accepted standard for 2×2 tables with small samples.
• Can I use Fisher's test with large samples? Yes, although with large samples Pearson's chi-square gives results that are practically identical and is computationally more direct.
• What does it mean if the OR is 0 or infinite? It happens when some cell equals 0. In that case the OR is undefined, although the exact p-value can still be calculated.

Reference: Fisher's exact test — Wikipedia