Regression to the Mean

Chapter 01The Curse of the Cover

On January 31, 1955, a young Milwaukee Braves infielder named Eddie Mathews appeared on the cover of Sports Illustrated. The magazine was only five months old. Mathews had just come off a monstrous 1954 season — 40 home runs, a .306 batting average, an All-Star selection. He was 23 years old, and the cover photo seemed to anoint him as the future of baseball.

In 1955, his average dropped to .289. His home runs fell to 32. He was still good — but he wasn't that good. The whispers started almost immediately.

It happened again. And again. Pete Rose appeared on the cover in 1970 after hitting .348; the next year, .304. The 1987 Cleveland Indians made the cover as preseason favorites; they finished last. Skier Jill Kinmont, quarterback Lamar Jackson, golfer Karrie Webb — the list of athletes who "slumped" after appearing on the cover grew so long that by the 1960s it had a name: the Sports Illustrated cover jinx.1

Sports fans are a superstitious bunch, and the jinx became self-evidently real. The cover was bad luck. Some athletes reportedly refused the honor. Teams hid the magazine from their clubhouses. The explanation was obvious — the pressure of being on the cover, the distraction of fame, the weight of expectations.

Except none of that was the explanation. The explanation was much simpler, much more universal, and much more unsettling. It didn't require any curse at all.

Chapter 02Galton's Sweet Peas

In 1886, Francis Galton — Darwin's cousin, polymath, inventor of fingerprinting, and a man with an almost pathological need to measure things — conducted an experiment with sweet pea seeds.2 He sorted seeds by size: small, medium, large. He planted each group separately. Then he waited.

When the next generation of peas grew, he measured them too. And he found something he didn't expect.

The large seeds did produce larger-than-average offspring — but not as large as themselves. The small seeds produced smaller-than-average offspring — but not as small as themselves. The children of extremes were, on average, less extreme. They had regressed toward the center.

Galton turned to human height next. He measured 205 pairs of parents and their adult children. The pattern was identical. Tall parents had tall children, yes — but those children were, on average, a bit shorter than their parents. Short parents had short children — but those children were a bit taller. Everything was drifting back toward the average, generation after generation.3

This confused him. If tall parents always produce slightly shorter children, wouldn't the whole species converge to a single height over time? Wouldn't variation itself disappear?

No. And the reason it doesn't is the key to understanding everything that follows. Short parents also occasionally produce tall children. The regression works in both directions, from both tails, and the overall distribution stays stable. Galton called this phenomenon "regression toward mediocrity." We now call it regression to the mean.

Chapter 03The Israeli Air Force and the Illusion of Feedback

In the early 1960s, Daniel Kahneman was a young psychologist lecturing to Israeli Air Force flight instructors. His topic that day was the psychology of training — specifically, the well-established finding that positive reinforcement (praising good performance) is more effective at improving behavior than punishment (criticizing bad performance).4

One of the senior instructors raised his hand and disagreed.

Kahneman later described this as one of the most illuminating moments of his career. Because the instructor was completely right about the facts — and completely wrong about the explanation.

Think about it. When does a cadet get praised? After an exceptionally good landing. An extreme performance, well above their average ability. And what happens after an extreme performance? It regresses to the mean. The next landing will probably be closer to average — which is to say, worse. The praise didn't cause the decline. The decline was going to happen anyway.

When does a cadet get screamed at? After an exceptionally bad landing. An extreme performance, well below their average ability. And what happens next? Regression to the mean again. The next landing will probably be closer to average — which is to say, better. The punishment didn't cause the improvement.

The instructor had, through decades of experience, trained himself to believe that punishment works and praise backfires — when in reality, neither one was doing anything. He would have observed the exact same pattern if he'd flipped a coin to decide whether to praise or scold.5

This is the deep trap. Regression to the mean doesn't just fool sports fans. It fools teachers, doctors, coaches, parents, managers, and policy makers. It fools anyone who observes extreme behavior and then watches what happens next.

Chapter 04The Math: Talent, Luck, and Why Extremes Are Extreme

Here's the simplest way to think about it. Imagine that every performance — every batting season, every flight landing, every exam score — is the sum of two components:

Skill

The person's true, stable ability — consistent from trial to trial

Luck

Random variation — positive or negative, averaging to zero over many trials

Skill stays the same from season to season (roughly). Luck doesn't. Luck is freshly drawn each time, from a distribution centered at zero.

Now: who ends up at the top of the rankings after Season 1? People with high skill, obviously. But also people who happened to get lucky. To be the very best in any given season, you almost certainly needed both — genuine talent and a favorable roll of the dice.

In Season 2, those same people still have their skill. But the luck resets. Some of them get lucky again, sure. But on average, their luck will be closer to zero. Which means their Season 2 performance will be closer to their true skill — which is lower than their Season 1 performance, because Season 1 had that lucky boost baked in.

The same logic works in reverse. The worst performers in Season 1 were unlucky. In Season 2, they'll probably be less unlucky — so they'll improve. Not because of any intervention. Just because extreme bad luck, like extreme good luck, doesn't repeat reliably.

And the crucial variable is the ratio of luck to skill. In a domain where luck dominates — like a single coin flip, or one day's stock returns — regression to the mean is massive. The best performer will look nothing special next time. In a domain where skill dominates — like chess ratings over hundreds of games — regression is minimal. The best player will still be the best.6

Chapter 05The Curse of the Sequel

Once you see regression to the mean, you see it everywhere — and you see all the places where people have invented causal stories to explain a statistical artifact.

The Sports Illustrated jinx. Athletes appear on the cover because they just had an extreme performance. Their next season regresses. The cover didn't cause it.

The sophomore slump. A rookie has a breakout first season and wins Rookie of the Year. The second season is disappointing. We say the league "figured them out." Maybe. Or maybe their first season included some lucky bounces that didn't repeat.

The Madden curse. The cover athlete of Madden NFL allegedly gets injured or declines. Same logic — they made the cover because they peaked.7

The curse of the sequel. A movie is a smash hit. The studio rushes a sequel into production. The sequel underperforms. Everyone blames the director, the script, the studio's greed. But the original was probably an extreme positive outlier — a lucky combination of timing, marketing, cultural moment, and quality. The sequel merely regressed.

The "flight to mediocrity" in business. Jim Collins' Good to Great identified companies with extreme performance over a certain period and reverse-engineered what made them special. Many of those companies subsequently underperformed. The cynical explanation: they were never as special as they looked. Their extreme performance contained luck, and the luck ran out.8

Chapter 06Try It Yourself

This is one of those ideas that's easier to see than to explain. Below is a simulator that generates 100 random athletes. Each has a fixed "talent" level (drawn once) and a "luck" component (drawn fresh each season). You can adjust the slider to control how much of performance is luck versus skill — and watch regression to the mean appear before your eyes.

Play with the luck slider. When luck is high (say 80–90%), the top performers from Season 1 crater in Season 2 — often dropping all the way back to average. When luck is low (10–20%), the top performers barely budge. That's the whole story of regression to the mean in one slider.

Chapter 07The Dangerous Implications

If regression to the mean were merely an amusing statistical curiosity, we could note it and move on. But it's not. It contaminates our reasoning about some of the most important questions in medicine, education, and public policy.

The Placebo's Best Friend

When do people seek medical treatment? When they feel their worst. A patient goes to the doctor at the peak of their symptoms — an extreme observation. The doctor prescribes something. The patient improves. Was it the treatment, or was the patient going to improve anyway, because extreme symptoms naturally regress?9

This is exactly why we need randomized controlled trials with placebo groups. The placebo group also regresses to the mean — they also feel terrible when they enter the trial, and they also get better on average. A treatment only "works" if it outperforms this natural regression. Yet an alarming number of alternative medicine testimonials and even some published studies fail to account for this.

The Educational Intervention Trap

A school district identifies its worst-performing schools. It pours resources into them — new curricula, better teachers, after-school programs. Test scores improve. The intervention is declared a success.

But wait. Those schools were selected because they were at their worst. Some of that poor performance was bad luck — a tough cohort of students, a flu outbreak during testing week, a temporary administrative crisis. Even without the intervention, scores would have regressed toward the district average. The only way to know if the intervention actually helped is to compare against similar schools that didn't receive it.

Speed Cameras and Crime Crackdowns

A dangerous intersection has a spike in accidents. A speed camera is installed. Accidents decrease. Was it the camera? Maybe — but the camera was installed precisely because the intersection just had an extreme run of accidents. Some regression was guaranteed.10

The same logic applies to crime crackdowns, new management at struggling companies, and coaching changes in sports. We intervene at the moment of worst performance, observe improvement, and credit the intervention. The universe was going to provide some of that improvement for free.

Chapter 08Learning to See the Invisible

Kahneman went on to win the Nobel Prize in Economics, and he later wrote that the flight instructor lecture was the moment he first understood something deep about human cognition: we are wired to see causes where there are none.11 When we observe a pattern — praise followed by decline, punishment followed by improvement — our brains immediately construct a causal story. We can't help it. It's what brains do.

Regression to the mean is invisible. It has no mechanism, no agent, no intention. It's just what happens when you combine a stable signal with random noise and then select on the observed result. It's what happens when the world is partly predictable and partly not, which is to say: always.

The Sports Illustrated editors weren't cursing anyone. Eddie Mathews wasn't doomed by a photograph. The Israeli flight instructors weren't training their cadets through tough love. And the newest wonder drug doesn't always work as well as the first trial suggests.

The mean is patient. It always gets its due.