What is the effect size d in the KS test?

The effect size d is the maximum absolute deviation between the true cumulative distribution function (CDF) F(x) and the theoretical CDF under H0, that is, d = max|F_true(x) − F_0(x)|. It measures how far the real distribution is from the hypothesized one: d = 0.10 is a small effect, d = 0.20 a moderate effect and d = 0.30 a large effect.

When should I use the KS test instead of the chi-square test?

The KS test is appropriate for continuous variables and does not require grouping data into categories, avoiding the loss of information that comes with the chi-square goodness-of-fit test. Chi-square is preferable for categorical or discrete variables. For continuous distributions with fully specified parameters, KS usually has higher power than chi-square.

Is the formula valid for small samples?

The power formula based on the shift-model approximation is conservative for small samples (n < 20-30), meaning the computed n may be slightly larger than strictly necessary. Exact tables of the Kolmogorov distribution exist for small n. The approximation improves as n grows and is very accurate for n ≥ 30.

Does this calculator work for the two-sample KS test?

Not directly. This calculator computes the sample size for the one-sample KS test (testing whether a sample comes from a fully specified theoretical distribution). The two-sample KS test compares whether two independent samples come from the same distribution and has a different asymptotic distribution; a dedicated calculator is needed for that case.

Sample size for Kolmogorov-Smirnov test calculator

Calculator

Enter the maximum CDF deviation (d), the significance level and the desired power to obtain the minimum sample size.

Maximum CDF deviation (d)

Alpha (α)

Power (1-β)

Result pending…

Explanation

The one-sample Kolmogorov–Smirnov test checks whether the data come from a fully specified theoretical distribution \(F_0\). The test statistic is the maximum absolute deviation between the empirical distribution function \(\hat{F}_n\) and the theoretical one:

\( D_n = \sup_x \left| \hat{F}_n(x) - F_0(x) \right| \)

The effect size is \(d = \max_x |F_{\text{true}}(x) - F_0(x)|\), that is, the maximum vertical distance between the real CDF and the hypothesized one. Under \(H_0\), the asymptotic distribution of \(\sqrt{n}\,D_n\) is the Kolmogorov distribution.

The power formula uses the shift-model approximation: under \(H_1\), the statistic \(\sqrt{n}\,D_n\) is shifted by approximately \(d\sqrt{n}\) relative to its distribution under \(H_0\). This approximation is conservative for small samples and very accurate for \(n \geq 30\).

Power formula

The CDF of the Kolmogorov distribution is:

\( P(K \leq x) = 1 - 2\sum_{k=1}^{\infty} (-1)^{k+1} e^{-2k^2 x^2}, \quad x > 0 \)

The critical quantile \(k_\alpha\) is defined by \(P(K > k_\alpha) = \alpha\), that is, \(1 - P(K \leq k_\alpha) = \alpha\). It is found by binary search.

The power of the test for sample size \(n\) is estimated as:

\( \text{Power}(n) = 1 - P\!\left(K \leq k_\alpha - d\sqrt{n}\right) \)

The calculator searches for the smallest integer \(n\) for which this expression reaches the target power. When \(k_\alpha - d\sqrt{n} \leq 0\), the power is 1.

Quick setup

d = 0.10: small effect. The real distribution deviates from the theoretical one by at most 0.10 in the CDF. Requires large samples.
d = 0.20: medium effect. Deviation clearly visible on a P-P or Q-Q plot. Moderate sample size.
d = 0.30: large effect. Substantial difference between distributions; detectable with relatively small samples.
α: 0.05 is the usual standard; use 0.01 if the cost of a false positive is high.
Power: 0.80 is the usual minimum; 0.90 if false negatives have important consequences.
Fully specified distribution: if you estimate parameters from the data (mean, variance…), the distribution of the statistic changes and the nominal level is no longer valid; in that case use the Lilliefors test.

Worked example

A process engineer wants to test whether the failure times of a component follow an exponential distribution with rate \(\lambda_0 = 0.1\) (fully specified). Based on historical data, they believe the real CDF deviates by at most \(d = 0.15\) from the theoretical exponential.

With \(d = 0.15\), \(\alpha = 0.05\) and 80% power, the calculator finds:

The critical quantile \(k_{0.05} \approx 1.358\) via binary search on the Kolmogorov distribution.
The smallest \(n\) such that \(1 - P(K \leq 1.358 - 0.15\sqrt{n}) \geq 0.80\).

The result according to the shift-model approximation is n = 23 observations. With that sample, if the real CDF deviates by 0.15 from the theoretical exponential, the test will detect it 80% of the time according to this asymptotic formula.

Note on the approximation: the asymptotic shift model tends to underestimate the actual sample size required, especially for small values of \(d\) and moderate \(n\). For studies where sample size is critical, it is recommended to complement this result with Monte Carlo simulations.

Sensitivity analysis: with \(d = 0.10\) (small effect) about n ≈ 51 observations are needed; with \(d = 0.30\) (large effect) n ≈ 6 suffices. The required sample grows approximately as \(1/d^2\).

Model assumptions

Random sample of a continuous variable, with independent and identically distributed observations.
The reference theoretical distribution is fully specified (its parameters are not estimated from the data).
The effect size is the maximum deviation \(d\) between the theoretical distribution and the alternative.
Power is evaluated using the exact Kolmogorov distribution.

How to interpret the result

The value \(n\) is the minimum number of observations required for the Kolmogorov-Smirnov (KS) test to detect the specified deviation from the theoretical distribution with the indicated power and \(\alpha\) level. Always round up. If you anticipate data loss or exclusions, divide \(n\) by \((1 - \text{dropout rate})\) to obtain the number of units to collect. A critical assumption is that the theoretical distribution under \(H_0\) must be fully specified (all its parameters known a priori): if instead the parameters are estimated from the data themselves (as in the Lilliefors test for normality), the standard KS critical tables are not valid and the p-value will be inflated.

The power of the KS test depends on the maximum distance \(D = \sup_x |F_n(x) - F_0(x)|\) you want to detect. The KS test is relatively powerful for detecting differences in the central region of the distribution, but has low power for differences in the tails. If the deviations you care about are mainly in the extremes (heavy tails, skewness), the Anderson-Darling test or the Cramér-von Mises test are more powerful and require a smaller \(n\) for the same effectiveness. Perform a sensitivity analysis by varying the minimum detectable distance \(D\) by ±0.05 to see how much \(n\) changes.

If the computed \(n\) exceeds a few hundred, keep in mind that with very large samples the KS test will detect trivial deviations from the theoretical distribution that have no practical relevance: a \(p < 0.05\) does not imply the distribution is "too different" for the purposes of the analysis. In that context, complement the test with Q-Q plots and assess whether the deviations are substantive. Once you have collected the data, run the test with the Kolmogorov-Smirnov calculator; if the goal is specifically to test normality, use the Shapiro-Wilk calculator for small or moderate samples.

References

Kolmogorov, A. N. (1933). Sulla determinazione empirica di una legge di distribuzione. Giornale dell'Istituto Italiano degli Attuari, 4, 83–91.
Smirnov, N. V. (1948). Table for estimating the goodness of fit of empirical distributions. The Annals of Mathematical Statistics, 19(2), 279–281.
Conover, W. J. (1999). Practical Nonparametric Statistics (3rd ed.). Wiley.
Massey, F. J. (1951). The Kolmogorov-Smirnov test for goodness of fit. Journal of the American Statistical Association, 46(253), 68–78.