Law Of Large Numbers

TL;DR

Larger sample sizes make the sample mean more likely to match the true population mean.
Applies to random variables and probabilities (more data reduces sampling variability).
Does not hold for non-random or biased samples.

Definition

The law of large numbers states that as the sample size of a random variable increases, the average of the sample approaches the true mean of the population.

Explanation

As more observations are collected, the sample mean is increasingly likely to be close to the population mean because a larger sample can capture more of the population’s variation. This principle underlies why larger datasets typically produce more reliable estimates for quantities like averages and probabilities. The law applies to random variables sampled without systematic bias; it does not apply when samples are non-random or biased.

Examples

Average height of adult men in the United States

If we take a sample of 100 men and compute their average height, that sample mean may not accurately represent the true population mean because the sample is small. Increasing the sample size to 1,000 men will likely produce a sample mean much closer to the true population mean, since the larger sample accounts for more potential variations.

Coin flips

If a coin is flipped 100 times, the result may not be exactly 50 heads and 50 tails. If the coin is flipped 1,000 times, the ratio of heads to tails will likely be much closer to 1:1, because the larger number of trials gives a more accurate representation of the coin’s true probability of landing heads or tails.

Notes or pitfalls

The law of large numbers holds for random variables sampled randomly; it does not apply to non-random or biased samples.
A small sample may fail to represent the true population mean.

sample mean
population mean
random variable
probability
statistics