Simpson's Paradox

The University That Hated Women (Except It Didn't)

In the fall of 1973, the University of California at Berkeley had a problem. A big, embarrassing, front-page problem. Of the 8,442 men who applied to its graduate programs, about 44% were admitted. Of the 4,321 women who applied? Only 35%.1 The gap was so large that it looked, statistically speaking, like the kind of thing that doesn't happen by accident.

So naturally, people sued. The accusation: Berkeley was systematically discriminating against women in graduate admissions. And if you looked at the numbers — just the numbers — it was hard to argue otherwise. Forty-four percent versus thirty-five percent. That's not a rounding error. That's a policy.

Except it wasn't.

Read that again. Women were favored in most departments, roughly equal in the rest, and yet overall, they were admitted at a dramatically lower rate. If this doesn't make your brain hurt, you're not paying attention.

A trend that appears in every group of data can reverse when the groups are combined.

This is Simpson's Paradox, and it is, in my considered opinion, one of the most important ideas in all of mathematics — not because it's hard (it isn't, once you see it), but because it's easy to miss, and the consequences of missing it range from embarrassing to catastrophic.

How Berkeley Actually Worked

Here's the trick, and it's not really a trick at all — it's just arithmetic being sneaky. Women at Berkeley disproportionately applied to competitive departments: English, art history, the humanities programs that admitted maybe 20–30% of applicants. Men, meanwhile, flooded into engineering and physical science programs that admitted 60% or more.

So women were playing the game on hard mode and men were playing on easy mode, and then someone looked at the overall scoreboard and said, "Hey, the women aren't doing as well!" Well, no kidding.

The culprit wasn't sexist admissions committees. It was confounding — specifically, the confounding variable of which department people applied to. Women applied to harder-to-enter programs. When you naively aggregate across departments, that difficulty differential gets laundered into what looks like discrimination.

Kidney Stones and the Surgeon's Dilemma

You'd think this was just an academic curiosity — something that happens in admissions data but nowhere else. You'd be wrong. Simpson's Paradox shows up in medicine, and when it does, it can kill people.

In 1986, a study compared two treatments for kidney stones: Treatment A (open surgery) and Treatment B (percutaneous nephrolithotomy, a less invasive puncture procedure).2 Here were the results:

The lurking variable? Severity. Doctors tended to assign the more invasive Treatment A to the harder cases (large stones), while easier cases got the gentle Treatment B. So A was fighting with one hand tied behind its back in the aggregate numbers — it had 263 large-stone cases versus B's mere 80 — even though it was winning every individual fight.

Now imagine you're a patient. You Google "kidney stone treatment success rates." You see 78% versus 83%. You pick B. You just made a decision based on a number that is technically correct and completely misleading. This is not a thought experiment. This is how medical misinformation actually works.

The Math Behind the Magic

Enough stories. Let's understand why this happens, because once you see the mechanism, you'll start spotting it everywhere. And the mechanism is, honestly, disappointingly simple.

Imagine two groups. In Group 1, Treatment A works 90% of the time (9 out of 10 patients) and Treatment B works 85% (85 out of 100). In Group 2, Treatment A works 30% (30 out of 100) and Treatment B works 25% (2.5 out of 10 — round to 3). A is better in both groups. Now combine them:

See what happened? Treatment A had most of its patients (100 of 110) in Group 2, where success rates are low. Treatment B had most of its patients (100 of 110) in Group 1, where success rates are high. The combined rate is a weighted average, and the weights — the sample sizes — are exactly where the confounding variable hides.

This is the deep reason: weighted averages don't combine linearly. When the weights differ across the things being compared — and they almost always do when a confounding variable is at play — the aggregate can tell a completely different story than the parts. It's not a paradox in the logical sense. It's an arithmetic ambush.

You can win every battle and still lose the war — not because you fought badly, but because you fought in the wrong places.

Build Your Own Paradox

The best way to understand Simpson's Paradox is to create one yourself. The interactive below lets you set the success rates and group sizes for two treatments across two groups. Adjust the sliders until the paradox appears — you'll feel the moment it clicks.

Notice the pattern: the paradox appears when Treatment A's patients are concentrated in the hard group while Treatment B's patients are concentrated in the easy group. The aggregate is dominated by each treatment's largest sample — which comes from different difficulty levels. It's not magic. It's bookkeeping.

David Justice vs. Derek Jeter

Baseball, America's most statistically obsessive sport, gives us perhaps the cleanest example of all. In both 1995 and 1996, David Justice had a higher batting average than Derek Jeter. In both years. And yet, when you combined the two seasons, Jeter's average was higher.3

The confounding variable is playing time. Jeter's 1995 was a cup of coffee — 48 at-bats at a mediocre .250. His 1996 was a full season at a strong .314. Justice played a full season in 1995 (at .253) but only 140 at-bats in 1996. Each player's combined average is dominated by their full season — for Jeter, that was his better year; for Justice, his worse one.

It's Berkeley all over again. The "departments" are the seasons, the "acceptance rates" are the batting averages, and the "application counts" are the at-bats. Same paradox, different costume.

Enter Judea Pearl: Why "Just Look at the Data" Isn't Enough

Here's where Simpson's Paradox stops being a fun math puzzle and becomes something genuinely profound. Because the paradox poses a question that pure statistics cannot answer: which number should you believe — the aggregated one or the disaggregated one?

In the Berkeley case, the disaggregated data was clearly right. In a drug trial, disaggregating by severity makes sense. But is that always true? Should you always break the data into subgroups?

The computer scientist and philosopher Judea Pearl — whose work on causal inference won him the Turing Award — argued that the answer is: it depends on the causal structure.4 You can't resolve Simpson's Paradox with statistics alone. You need a causal model.

This is why Pearl insisted on drawing causal diagrams — directed acyclic graphs, or DAGs — and why he argued that you cannot do data science without them. The data alone cannot tell you which way the arrows point. For that, you need to think.

So What Do You Do About It?

Now I know what you're thinking: "Great, so I can never trust any statistic ever again." That's not quite right, but it's closer to right than you'd like. Here's what Simpson's Paradox actually teaches us:

First: Always ask what's being aggregated. Whenever someone shows you an overall number — a success rate, an average, a percentage — your first question should be: "What subgroups are hiding inside this number, and could they tell a different story?" Most of the time, they won't. But when they do, the difference is everything.

Second: Look for confounders. If two groups being compared weren't randomly assigned — and outside of controlled experiments, they almost never are — then there's probably a lurking variable distorting the comparison. Find it. In Berkeley, it was department choice. In kidney stones, it was severity. In baseball, it was playing time. The confounder is always there. Your job is to name it.

Third: Think causally. Don't just ask "what does the data show?" Ask "what is the data-generating process?" Draw the causal diagram, even if it's just on a napkin. Which variables cause which? Where are the arrows? This is Pearl's great contribution: the realization that no amount of data can substitute for a model of how the data came to be.

Fourth: Be humble. The world is full of confounders you haven't thought of. Every observational study is a potential Simpson's Paradox waiting to be revealed. This doesn't mean we should ignore data — it means we should treat it with the respect it deserves, which includes respecting its limitations.

The numbers never lie. But they never tell the whole truth, either.

Simpson's Paradox isn't really about paradoxes. It's about the gap between seeing and understanding. The data is a map, and like all maps, it leaves things out. The paradox is what happens when you forget that the map isn't the territory — when you trust the aggregate so completely that you forget to ask what it's made of.

Edward Hugh Simpson published his paper on this phenomenon in 1951.5 Udny Yule noticed it even earlier, in 1903.6 Karl Pearson and others observed related reversals around the same era. We've known about it for over a century. And yet it keeps catching us off guard — in medical research, in hiring data, in educational testing, in criminal justice statistics, in pandemic mortality rates. Every time we aggregate without thinking, we risk seeing a pattern that isn't there, or missing one that is.

The cure isn't to stop looking at data. The cure is to look harder.