The Kolmogorov-Smirnov Test: Checking Whether Your Data Follows a Known Distribution

Why Test for a Distribution?

Many statistical tests — such as the t-test, ANOVA, and linear regression — assume that your data follow a specific distribution, most commonly the normal distribution (the familiar bell-shaped curve). If your data violate this assumption, the results of those tests may be unreliable. Before running such tests, it is often a good idea to check whether your data actually fit the assumed distribution. The Kolmogorov-Smirnov test, often abbreviated as the K-S test, is one of the most widely used tools for this purpose.

What the Test Does

The Kolmogorov-Smirnov test compares your observed data to a theoretical distribution to see how well they match. It does this by looking at cumulative distribution functions (CDFs). A cumulative distribution function tells you, for any given value, what proportion of observations fall at or below that value. Your data produce an empirical CDF — a step function that climbs upward with each data point. The theoretical distribution you are testing against (for example, the normal distribution) has a smooth, mathematically defined CDF.

The K-S test measures the largest vertical distance between these two curves. This maximum distance is called the D statistic. If your data truly come from the theoretical distribution, the empirical CDF should closely track the theoretical one, and D will be small. If the data deviate substantially from the theoretical distribution, D will be large.

A Concrete Example

Imagine a researcher studying the test scores of 60 students in an introductory biology course. She wants to know whether the scores are normally distributed before running a t-test to compare two sections of the course. She applies the one-sample K-S test, comparing her 60 observed scores against a normal distribution with the same mean and standard deviation. The test produces a D statistic of 0.08 and a p-value of 0.82. Because the p-value is well above the conventional threshold of 0.05, she does not reject the null hypothesis — there is no strong evidence that the scores differ from a normal distribution, and she can proceed with her t-test with greater confidence.

One-Sample and Two-Sample Versions

The K-S test comes in two main versions. The one-sample version, described above, compares your data to a single theoretical distribution. You might use it to ask: "Do my data follow a normal distribution?" or "Do these arrival times follow an exponential distribution?"

The two-sample version compares two datasets to each other. Instead of testing whether your data match a theoretical curve, it tests whether two groups of observations come from the same (unspecified) distribution. For example, you might use the two-sample K-S test to check whether the distribution of income in two different cities has the same shape, without assuming what that shape is. The D statistic in the two-sample case is the maximum distance between the two empirical CDFs.

Sensitivity to Sample Size

An important characteristic of the K-S test is that its sensitivity depends heavily on sample size. With small samples, the test has relatively low statistical power, meaning it may fail to detect meaningful departures from the theoretical distribution. Your data could be noticeably non-normal, but if you only have 15 or 20 observations, the K-S test might not flag the problem.

Conversely, with very large samples, the test becomes extremely sensitive. It can detect tiny, trivial deviations from the theoretical distribution that have no practical importance. With 10,000 observations, even a near-perfect bell curve might produce a statistically significant K-S result. This means you should always interpret the K-S test in context. A significant result with a large sample does not necessarily mean your data are badly behaved — it might just mean the test is picking up on minor, irrelevant imperfections.

Alternatives: When to Consider Other Tests

If your primary goal is to test for normality specifically, the Shapiro-Wilk test is generally considered more powerful than the K-S test for that purpose, especially with smaller samples. The Shapiro-Wilk test was designed specifically for the normal distribution and tends to detect non-normality more reliably. The K-S test, on the other hand, is more versatile — it can be used with any continuous distribution, not just the normal. So if you are testing whether your data follow an exponential, uniform, or any other distribution, the K-S test is a natural choice.

Key Assumptions and Limitations

There are a few things to keep in mind when using the K-S test. First, it is designed for continuous data. If your data are discrete (whole numbers, categories), the test may not perform correctly. Second, for the one-sample version, the parameters of the theoretical distribution (such as the mean and standard deviation for a normal distribution) should ideally be specified in advance rather than estimated from the data. If you estimate the parameters from the same data you are testing, the test becomes conservative — it is less likely to reject the null hypothesis even when it should. Modified versions of the test, such as the Lilliefors test, are available to handle this situation.

Despite these limitations, the Kolmogorov-Smirnov test remains a valuable and widely used tool. It provides a straightforward, objective way to check whether your data match a theoretical distribution, helping you verify the assumptions that underpin many of the most common statistical analyses.