The law of large numbers

John Kerrich flipped a coin 10,000 times during the Second World War and got 5,067 heads, a proportion of 0.5067.

After five heads in a row, is tails due?

Five heads in a row. Surely the next one has to be tails? It is tempting to think so, and the feeling has a name: the gambler's fallacy. It sounds a lot like the law of large numbers, which is the trouble. The law is real. It says nothing about the next flip, though, and it never works by evening things out. So start with what the law does say.

If a coin has a 1/2 chance of heads, why does it so often land heads three or four times in a row? Because probability does not describe a handful of flips. It describes what happens over many repetitions. The fraction of times an event happens in a run of trials is its relative frequency: count ÷ number of trials. The law of large numbers says that as the number of trials grows, the relative frequency tends to get closer and closer to the theoretical probability.

Kerrich was 67 heads over, and only 0.0067 off

Kerrich's proportion was 5,067 ÷ 10,000 = 0.5067, only 0.0067 away from 0.5. Yet in raw numbers he was 67 heads above the 5,000 you might "expect". This is the part of the law that surprises people. As you keep flipping, the gap between heads and half the flips does not shrink to zero; it usually wanders and often grows. What shrinks is the gap divided by the number of flips. The law of large numbers works by swamping early luck with a huge number of later flips, not by paying it back.

That is also why the gambler's fallacy is a fallacy. After five heads in a row, the next flip of a fair coin is still heads with probability 1/2. The coin has no memory and owes nobody a tail. In the long run the early streak of five becomes an unnoticeable part of thousands of flips.

A hundred flips usually end between 0.40 and 0.60

There is a useful rule of thumb for how far a proportion usually strays. For n flips of a coin with probability p, about 95% of runs end with a proportion within p ± 2√(p(1 − p)/n). For a fair coin and 100 flips that is 0.5 ± 2√(0.25/100) = 0.5 ± 0.1, so anything from 0.40 to 0.60 is ordinary. For 10,000 flips it is 0.5 ± 0.01. Kerrich's 0.5067 sits comfortably inside that band, about 1.34 standard deviations above 0.5. Notice the square root: to make the band ten times narrower you need a hundred times as many flips.

The same idea lets OpenStax judge the Belgian euro. With 250 flips the 95% band for a fair coin is 0.5 ± 2√(0.25/250) = 0.5 ± 0.063, roughly 0.437 to 0.563. The observed 0.56 is inside it, close to the edge. That is why the honest conclusion is "possibly biased, flip it more", not "proven unfair".