When should I use the KS test instead of the chi-square goodness-of-fit test?

The KS test is preferable for continuous distributions and not-too-large samples, because it does not require grouping data into intervals and is more powerful in that context. The chi-square test is more suitable for categorical data or when working with large samples of continuous data that have already been grouped.

What happens if I estimate the parameters from the data itself?

If the parameters of the theoretical distribution are estimated from the same sample, the standard KS test becomes conservative: the p-values are larger than they should be, which reduces power. For normality with estimated parameters use the Lilliefors test; for other distributions analogous variants exist.

Does the KS test work for discrete distributions?

It is not designed for discrete distributions. With them, the distribution of the statistic \(D_n\) under \(H_0\) changes and the Kolmogorov approximation is inaccurate. For discrete data (counts by category), use the chi-square goodness-of-fit test.

How many observations are needed?

For the asymptotic p-value approximation to be reliable, \(n \geq 20\) is recommended. With smaller samples the p-value is approximate and the power to detect moderate deviations is low.

Is the test always two-sided?

Yes. The statistic \(D_n = \sup|F_n - F_0|\) measures the maximum discrepancy in either direction, so the test detects both excess and deficit of mass relative to the theoretical model in any region. There is no standard one-sided version of the one-sample KS test.

What is the difference with the two-sample KS test?

The one-sample test (goodness-of-fit) compares the data with a fully specified theoretical distribution. The two-sample test checks whether two independent samples come from the same distribution, without needing to specify what that distribution is.

Kolmogorov-Smirnov Goodness-of-Fit Test Calculator

Calculator

Enter the sample data separated by commas, select the theoretical distribution and enter its known parameters.

Sample data

Theoretical distribution

Mean (μ)

Standard deviation (σ)

Minimum (a)

Maximum (b)

Rate (λ)

Significance level (α)

Result pending…

Explanation

The Kolmogorov-Smirnov goodness-of-fit test (KS test) is used to assess whether a data sample comes from a previously specified continuous probability distribution. Unlike the chi-square goodness-of-fit test, it does not require grouping the data into intervals and works directly with the individual sample values, which makes it especially suitable for continuous distributions.

The core idea is to compare the empirical distribution function \(F_n(x)\) —the proportion of observations less than or equal to \(x\)— with the theoretical distribution function \(F_0(x)\) under the null hypothesis. The test statistic is the maximum absolute discrepancy between the two functions.

For example, if the response times of 25 servers are recorded and we want to know whether they follow an Exponential distribution with rate λ = 0.5, the KS test quantifies how far the empirical distribution departs from that Exponential and computes the probability that this difference is attributable to sampling chance.

The statistic \(D_n\) follows the Kolmogorov distribution under the null hypothesis when the parameters of the distribution are known and not estimated from the data itself. The test is always two-sided: it detects any type of deviation from the theoretical model, in any region of the distribution.

Hypotheses and statistic

\(H_0\): the sample comes from the theoretical distribution \(F_0\)

\(H_1\): the sample does not come from \(F_0\)

\(D_n = \sup_x \left| F_n(x) - F_0(x) \right|\)

Quick check

The data are sorted from smallest to largest: \(x_{(1)} \leq x_{(2)} \leq \cdots \leq x_{(n)}\). The empirical distribution function \(F_n\) is the step function that jumps by \(1/n\) at each observation. The statistic \(D_n\) measures the maximum distance between \(F_n\) and the theoretical distribution \(F_0\), considering both the value just before and just after each jump:

\(D_n = \max_{1 \leq i \leq n}\left\{ \left|\frac{i}{n} - F_0(x_{(i)})\right|,\, \left|\frac{i-1}{n} - F_0(x_{(i)})\right| \right\}\)

Under \(H_0\), the distribution of \(\sqrt{n}\,D_n\) converges to the Kolmogorov distribution. The p-value is obtained using the asymptotic formula:

\(p \approx 2\sum_{k=1}^{\infty}(-1)^{k-1}\exp\!\left(-2k^2 n D_n^2\right)\)

This approximation is reliable for \(n \geq 20\). With smaller samples, the p-value is approximate and results should be interpreted with caution. If the distribution parameters have been estimated from the data itself (rather than being known in advance), the p-values computed with this formula are too large: the test becomes conservative. In that case, the Lilliefors test is recommended for normality; for other distributions, analogous variants with specific corrections exist.

Worked example

The waiting times (in minutes) of 8 customers at an office are measured: 1.2, 2.1, 2.8, 3.4, 4.1, 5.0, 5.8, 6.5. We want to check, with \(\alpha = 0.05\), whether these data come from a Uniform distribution on the interval \([0,\, 8]\).

The hypotheses are \(H_0\): the sample follows a \(\text{Uniform}(0,8)\) distribution versus \(H_1\): the sample does not follow that distribution. The theoretical distribution function is \(F_0(x) = x/8\) for \(x \in [0,8]\).

With the data already sorted, the empirical distribution function \(F_n(x_i) = i/n\) is built and the maximum difference \(D = \max_i |F_n(x_i) - F_0(x_i)|\) is computed:

\( D = \max\!\left\{\left|\tfrac{i}{8} - \tfrac{x_i}{8}\right|,\; \left|\tfrac{i-1}{8} - \tfrac{x_i}{8}\right|\right\} \)

The tabulated critical value for \(n = 8\) and \(\alpha = 0.05\) is \(D_{0.05;\,8} \approx 0.457\). Since \(D = 0.1875 < 0.457\), \(H_0\) is not rejected.

Conclusion: at a 5% significance level, there is not enough evidence to claim that the waiting times differ from a Uniform distribution on \([0, 8]\). The maximum observed discrepancy between the empirical and theoretical distributions is small relative to the variability expected for samples of that size.

How to interpret the result

Rejecting \(H_0\) (p-value < \(\alpha\)) means that the empirical distribution function \(F_n(x)\) departs from the theoretical function \(F_0(x)\) by more than chance can explain. The data are not compatible with the proposed distributional model at the chosen significance level. The location of the maximum discrepancy —marked by the vertical amber line in the chart— reveals which region of the distribution the misfit is concentrated in: a discrepancy in the left tail indicates that the data have less mass than expected at low values; in the right tail, the opposite; in the center, a misfit around the median or the mode.

Failing to reject \(H_0\) (p-value ≥ \(\alpha\)) does not prove that the data "follow" the theoretical distribution; it only indicates that they are compatible with it at the chosen level. With small samples the power is low and moderate deviations can go unnoticed. With very large samples, the test may reject models that fit reasonably well in practice. It is therefore advisable to complement the p-value with a visual inspection of the empirical and theoretical CDF curves: if they overlap almost perfectly, the model is adequate even if \(D_n\) is nonzero.

The statistic \(D_n\) is the maximum vertical distance between the empirical and theoretical CDFs. The green zone in the chart represents the tolerance band around \(F_0(x)\) compatible with \(H_0\); if the empirical CDF exceeds that band, \(D_n\) falls into the critical region (red zone). One advantage of the KS test over the chi-square goodness-of-fit test is that it does not require grouping data into categories, making it especially useful for continuous distributions. However, if the parameters of the theoretical distribution have been estimated from the data itself, the critical values must be corrected (the Lilliefors test for normality, for example).

Frequently asked questions

When should I use the KS test instead of the chi-square goodness-of-fit test? The KS test is preferable for continuous distributions and not-too-large samples, because it does not require grouping data into intervals and is more powerful in that context. The chi-square test is more suitable for categorical data or when working with large samples of continuous data that have already been grouped.
What happens if I estimate the parameters from the data itself? If the parameters of the theoretical distribution are estimated from the same sample, the standard KS test becomes conservative: the p-values are larger than they should be, which reduces power. For normality with estimated parameters use the Lilliefors test; for other distributions analogous variants exist.
Does the KS test work for discrete distributions? It is not designed for discrete distributions. With them, the distribution of the statistic \(D_n\) under \(H_0\) changes and the Kolmogorov approximation is inaccurate. For discrete data (counts by category), use the chi-square goodness-of-fit test.
How many observations are needed? For the asymptotic p-value approximation to be reliable, \(n \geq 20\) is recommended. With smaller samples the p-value is approximate and the power to detect moderate deviations is low.
Is the test always two-sided? Yes. The statistic \(D_n = \sup|F_n - F_0|\) measures the maximum discrepancy in either direction, so the test detects both excess and deficit of mass relative to the theoretical model in any region. There is no standard one-sided version of the one-sample KS test.
What is the difference with the two-sample KS test? The one-sample test (goodness-of-fit) compares the data with a fully specified theoretical distribution. The two-sample test checks whether two independent samples come from the same distribution, without needing to specify what that distribution is.

Reference: Kolmogorov–Smirnov test — Wikipedia