When should I use Fisher's exact test instead of chi-square?

Fisher's exact test is preferable when some expected frequency in the 2×2 table is below 5, or when the total sample size is small (typically N < 20-25). Chi-square with Yates' correction is an alternative, but Fisher's test is more exact since it doesn't rely on any asymptotic approximation.

Why is the Fleiss approximation used to calculate the sample size for Fisher's test?

Exact power calculation for Fisher's test requires simulation or full enumeration of the hypergeometric distributions, which is computationally intensive. The Fleiss (1981) approximation with the Casagrande and Pike continuity correction offers a closed-form formula that agrees well with simulation results and is the recommended standard in practice.

What does the continuity correction mean for the Fisher sample size?

The Casagrande and Pike continuity correction adjusts the Fleiss sample size upward to compensate for the discrete nature of the hypergeometric distribution. It produces more conservative estimates (a larger n) that more reliably reach the target power.

What's the difference between the uncorrected sample size and the continuity-corrected one?

The uncorrected n (basic Fleiss) can underestimate the size needed for Fisher's exact test because it doesn't account for discretization. The Casagrande and Pike correction increases n by a factor that depends on the difference between proportions and the ratio r. For large differences between proportions, the correction is small; for small differences, it can increase n by 10-20%.

Sample size for Fisher's exact test (2×2)

Calculator

Enter the expected proportions in each group, the significance level, the desired power, the group size ratio and the type of test.

Expected proportion group 1 (p₁)

Expected proportion group 2 (p₂)

Alpha (α)

Power (1-β)

Relative size of group 2 (r = n₂/n₁)

Test type

Result pending…

Explanation

Fisher's exact test is used to test independence between two binary variables in a 2×2 contingency table. It's the preferred method when the expected frequencies in some cell are below 5, or when the total sample size is small — situations where the asymptotic chi-square approximation may not be valid.

Since it's an exact test based on the hypergeometric distribution, calculating its power analytically would require enumerating the entire distribution of possible tables, which is computationally expensive for general use. The standard practical solution is to use the Fleiss (1981) approximation for two independent proportions, followed by the Casagrande and Pike (1978) continuity correction, which adjusts the sample size to compensate for the discrete nature of the exact test.

This approximation produces results very similar to those obtained by simulation and is the standard recommendation in reference texts such as Fleiss, Levin and Paik (2003).

Formula used

Let \(p_1\) be the proportion in group 1, \(p_2\) the proportion in group 2, \(r = n_2/n_1\) the size ratio, and \(\bar{p} = (p_1 + r\,p_2)/(1+r)\) the pooled weighted proportion.

Step 1 — Fleiss approximation (without correction):

\( n_1^{(0)} = \frac{\left(z_\alpha\,\sqrt{(1+1/r)\,\bar{p}(1-\bar{p})} + z_\beta\,\sqrt{p_1(1-p_1)+p_2(1-p_2)/r}\right)^2}{(p_1-p_2)^2} \)

where \(z_\alpha\) is the \(1-\alpha/2\) quantile (two-tailed) or \(1-\alpha\) quantile (one-tailed) of the standard normal, and \(z_\beta\) is the \(1-\beta\) quantile.

Step 2 — Continuity correction (Casagrande and Pike, 1978):

\( n_1 = \frac{n_1^{(0)}}{4}\left(1 + \sqrt{1 + \frac{2(1+1/r)}{n_1^{(0)}\,|p_1-p_2|}}\right)^2 \)

\( n_2 = \lceil r \cdot n_1 \rceil \quad;\quad n_1 = \lceil n_1 \rceil \quad;\quad N = n_1 + n_2 \)

\(p_1\), \(p_2\): proportions expected under the alternative \(H_1\) in each group.
\(r\): ratio \(n_2/n_1\). With \(r = 1\) the groups have the same size.
\(\bar{p}\): pooled weighted proportion; represents the proportion under \(H_0\) if the groups were equal.
Continuity correction: increases n to compensate for the discretization of the exact test.

Quick settings

p₁ and p₂: enter the proportions you expect to observe in each group. The difference \(|p_1 - p_2|\) is the effect to detect. Larger differences require a smaller sample.
α: significance level. 0.05 is the usual standard; use 0.01 if you want tighter control of the type I error.
Power: 0.80 (80%) is the common minimum; 0.90 or 0.95 for confirmatory studies or ones with high clinical relevance.
r: if the groups are naturally unequal (e.g., cases and controls), set r to the expected ratio of sizes. r = 1 is the most efficient per total participant.
One-tailed test: use it only if you have a justified a priori directional hypothesis; a one-tailed test requires a smaller sample but won't detect differences in the opposite direction.
Continuity correction: the calculator always applies the Casagrande and Pike correction for Fisher's exact test. If you prefer the uncorrected n (for example for a standard chi-square), use it as a lower-bound reference.

Worked example

A clinical trial compares the response rate to a treatment in two groups. A response rate of 40% is expected in the treated group (\(p_1 = 0.40\)) and 20% in the control group (\(p_2 = 0.20\)). A two-tailed \(\alpha = 0.05\) is chosen along with 80% power. The groups are the same size (\(r = 1\)).

Step 1 — pooled proportion: \(\bar{p} = (0.40 + 0.20)/2 = 0.30\).

With \(z_{\alpha/2} = 1.960\) and \(z_\beta = 0.842\):

\( n_1^{(0)} = \frac{\bigl(1.960\sqrt{2\times0.30\times0.70} + 0.842\sqrt{0.40\times0.60+0.20\times0.80}\bigr)^2}{(0.20)^2} \approx \frac{(1.960\times0.648 + 0.842\times0.566)^2}{0.04} \approx \frac{(1.270+0.477)^2}{0.04} \approx \frac{3.050}{0.04} \approx 76.3 \)

Step 2 — continuity correction:

\( n_1 = \frac{81.2}{4}\left(1 + \sqrt{1 + \frac{4}{81.2\times0.20}}\right)^2 \approx 20.3\times(1+\sqrt{1.246})^2 \approx 20.3\times(1+1.116)^2 \approx 20.3\times4.478 \approx 90.9 \)

Rounding up: n₁ = 91 and n₂ = 91 (total = 182). With this sample, Fisher's exact test will detect the 20-percentage-point difference 80% of the time.

Model assumptions

Two independent groups with a binary outcome (success/failure).
Independent observations; each unit contributes to a single cell of the 2×2 table.
Fisher's conditional framework (fixed marginal totals), appropriate when expected frequencies are low.
The sample size is obtained with the Fleiss approximation and rounded up.

How to interpret the result

The values \(n_1\) and \(n_2\) are the minimum sizes per group needed to detect the difference between proportions \(p_1\) and \(p_2\) using Fisher's exact test with the specified power and \(\alpha\) level. The total to recruit is \(N = n_1 + n_2\). Always round up and add a margin for attrition: divide each \(n_i\) by \((1 - \text{dropout rate})\). Fisher's exact test is appropriate precisely when some expected frequency falls below 5; if all expected frequencies are \(\geq 5\), Pearson's chi-square is equally valid and generally more familiar to reviewers.

The sample size calculation for Fisher's test is based on the hypergeometric distribution, which is discrete. This means the test's actual significance level can be lower than the nominal \(\alpha\) (the test is conservative), and the actual power may differ slightly from the planned value for the exact \(n\) calculated. In practice, it's recommended to add a couple of extra units to the \(n\) per group to compensate for the discretization. Run a sensitivity analysis by varying \(p_1\) and \(p_2\) by ±0.05: when both proportions are very extreme (\(<0.05\) or \(>0.95\)), the exact test becomes more important relative to chi-square and the \(n\) may differ from the estimate given by the approximate normal formula.

If the calculated \(n\) is so large that the expected frequencies in all cells would comfortably exceed 5, consider whether the chi-square test wouldn't be more appropriate (and plan the \(n\) with the sample size calculator for two proportions). When the \(n\) is unfeasible, check whether the minimum detectable difference \(|p_1 - p_2|\) is realistic or could be widened. Once the data has been collected, run the analysis with the hypothesis test calculator for two proportions, choosing Fisher's test if the expected frequencies are low.

References

Fleiss, J. L. (1981). Statistical Methods for Rates and Proportions (2nd ed.). Wiley.
Casagrande, J. T., & Pike, M. C. (1978). An improved approximate formula for calculating sample sizes for comparing two binomial distributions. Biometrics, 34(3), 483–486.
Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical Methods for Rates and Proportions (3rd ed.). Wiley.

Frequently asked questions

When should I use Fisher's exact test instead of chi-square? When some expected frequency in the 2×2 table is below 5, or when the total sample size is small (N < 20–25). In large samples, both tests converge to the same result.
Why does the continuity correction increase the sample size? Because Fisher's exact test operates on the discrete hypergeometric distribution. The correction adjusts n to guarantee that the actual power of the discrete test reaches the target, rather than the continuous asymptotic power.
Is the calculated sample size exact? No; it's an approximation. The actual value depends on the true parameters and on the discretization of the hypergeometric distribution. For confirmation in critical studies, supplement with Monte Carlo simulation.
Can I use this calculator for a case-control design? Yes. Enter the proportion of exposed cases as \(p_1\), the proportion of exposed controls as \(p_2\), and set \(r\) to the desired ratio of controls per case.