Use this tool to plan a chi-square goodness-of-fit test: calculate how many observations you need to detect, with the desired power, that your data depart from a theoretical distribution. The effect is expressed with Cohen's \(w\).
Calculator
Enter Cohen's w, the number of categories k, the significance level and the desired power.
Wizard: calculate w from observed and theoretical proportions
Enter the proportions under H1 (expected observed) and under H0 (theoretical), separated by commas. The number of values must equal k.
Explanation
The chi-square goodness-of-fit test checks whether the observed distribution of a categorical variable matches a specified theoretical distribution \(H_0\). With \(k\) categories, the statistic has \(df = k-1\) degrees of freedom under \(H_0\) (assuming the parameters of the theoretical distribution are known).
Under the alternative, the statistic follows a noncentral chi-square distribution with noncentrality parameter \(\lambda = N \cdot w^2\), where \(N\) is the sample size and \(w\) is Cohen's \(w\).
Noncentrality parameter and power
\( \lambda = N \cdot w^2 \)
\( w = \sqrt{\sum_{i=1}^k \frac{(p_{\text{obs},i} - p_{\text{teo},i})^2}{p_{\text{teo},i}}} \)
\( \text{Power} = 1 - F_{\chi^2_{df,\,\lambda}}\!\left(\chi^2_{1-\alpha,\,df}\right) \quad \text{with } df = k-1 \)
The noncentral chi-square distribution is evaluated exactly via the Poisson mixture of central chi-squares:
\( F_{\chi^2_{df,\lambda}}(x) = \sum_{k=0}^{\infty} \frac{e^{-\lambda/2}(\lambda/2)^k}{k!} \cdot F_{\chi^2_{df+2k}}(x) \)
Quick setup
- w: 0.1 (small), 0.3 (medium), 0.5 (large) according to Cohen. Use the wizard if you have the expected proportions.
- k: total number of categories. The degrees of freedom are df = k−1 (no estimated parameters).
- Expected frequencies: check that N/k ≥ 5 so the chi-square approximation is valid.
- Estimated parameters: if you estimate p parameters from the data, subtract p from df: df = k−1−p.
Worked example
A biologist wants to test whether the color of certain insects (5 categories: yellow, red, green, blue, black) follows a uniform distribution (\(p_0 = 0.20\) for each category). Based on prior data, they expect the real distribution to be (0.30, 0.25, 0.20, 0.15, 0.10).
Using the wizard: \(w = \sqrt{(0.30-0.20)^2/0.20 + (0.25-0.20)^2/0.20 + 0 + (0.15-0.20)^2/0.20 + (0.10-0.20)^2/0.20} = \sqrt{0.05+0.0125+0+0.0125+0.05} = \sqrt{0.125} \approx 0.354\).
With \(w = 0.354\), \(df = 4\), \(\alpha = 0.05\) and 80% power, approximately N = 69 insects are needed. With that sample, the test will detect the deviation from the uniform distribution 80% of the time.
Check: N/k = 69/5 = 13.8 ≥ 5. The chi-square approximation is adequate.
Model assumptions
- Random sample of independent observations.
- Categorical variable with \(k\) mutually exclusive and exhaustive categories.
- Sufficiently large expected frequencies (usual rule: \(E_i \ge 5\)).
- The effect size is expressed with Cohen's \(w\) and power is calculated using the noncentral chi-square distribution.
How to interpret the result
The value \(N\) is the total number of observations needed (sum across all categories) for the chi-square goodness-of-fit test to have the specified power and significance level. Always round up. If you expect some records to be discarded due to errors or missing values, divide \(N\) by \((1 - \text{dropout rate})\) to get the actual number to collect. The expected number of observations in each category \(i\) under \(H_0\) is \(N \times p_{0i}\); verify that all these expected frequencies are \(\geq 5\) so the chi-square approximation is valid.
Cohen's effect size \(w\) quantifies how far the real distribution (\(p_{1i}\)) departs from the theoretical distribution (\(p_{0i}\)): \(w = \sqrt{\sum_i (p_{1i} - p_{0i})^2 / p_{0i}}\). A value \(w = 0.10\) is small, \(w = 0.30\) is medium, and \(w = 0.50\) is large. The sensitivity of \(N\) to \(w\) is quadratic: if the real effect is 30% smaller than specified, \(N\) increases by roughly 100% \((1/0.7^2 \approx 2.04)\). If any theoretical proportion \(p_{0i}\) is very small, that category may have very few expected observations; in that case, merge categories with theoretical justification until expected frequencies of \(\geq 5\) are reached and recompute the degrees of freedom of the test.
When the calculated \(N\) turns out to be impractical, the available levers are: (1) accept a larger \(w\) (detect only bigger effects), (2) reduce the power, or (3) reduce the number of categories by merging the smallest ones. If any expected frequency falls below 5 even with the calculated \(N\), fall back on exact multinomial tests. Once the data has been collected, run the test with the chi-square goodness-of-fit test calculator and compare the standardized residuals per category to identify where the differences from the theoretical distribution are concentrated.
References
- Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum Associates.
- Agresti, A. (2013). Categorical Data Analysis (3rd ed.). Wiley.