The Paper That Should Have Changed Everything
In 1950, a sociologist named William S. Robinson published a nine-page paper that should have detonated like a bomb across the social sciences. Instead, it went off more like a firecracker in a library — people flinched, looked up, and then went right back to what they were doing.1
Robinson's observation was disarmingly simple. He looked at U.S. Census data from 1930 and calculated two correlations. The first was an ecological correlation — computed at the level of the 48 states. He asked: in states where a larger proportion of the population is foreign-born, is literacy higher or lower? The answer was clear. The correlation was +0.53. States with more immigrants had more literate populations.
If you stopped here — and many researchers did — you'd conclude that immigration was associated with literacy. Perhaps immigrants were especially motivated, bookish types? Perhaps the languages they brought enriched the educational atmosphere?
But Robinson didn't stop. He also computed the individual-level correlation: for actual people, is being foreign-born associated with being literate? The answer was −0.12. The sign flipped. At the individual level, immigrants were slightly less likely to be literate than native-born Americans (which makes sense — many were still learning English).
How is this possible? How can groups tell the exact opposite story from the people who make them up? Robinson had identified what we now call the ecological fallacy — the error of drawing conclusions about individuals from data about groups. And seventy-five years later, we're still making this mistake almost everywhere we look.
The Magic Trick Revealed
The explanation is beautiful once you see it. Immigrants in 1930 didn't spread themselves uniformly across America like butter on toast. They clustered. They moved to New York, to Illinois, to California, to Massachusetts — states that already had high literacy rates among their native-born populations. These were urbanized, industrialized states with extensive school systems.
So when you zoom out to the state level, you see: "New York has lots of immigrants AND high literacy." True! But the high literacy belongs mostly to the native-born New Yorkers. The immigrants are riding in a high-literacy vehicle, and the aggregate statistic gives them credit for the ride.
Meanwhile, states with very few immigrants — the rural South, primarily — had low literacy rates for entirely separate reasons. When you compute the state-level correlation, you're really picking up the geographic sorting of immigrants, not any causal relationship between immigration and literacy.
See It for Yourself
Robinson's insight is the kind of thing that's much easier to see than to explain. So let's see it. The scatter plot below shows individual data points — think of them as people — colored by which group they belong to. In the individual view, the overall trend runs one direction. Switch to the group-average view, and watch the correlation flip.
Aggregation Explorer
Individual data points colored by group. Toggle views and watch the correlation change sign.
Did you see it? The individual points slope gently downward — a negative correlation. But the group averages slope upward. Same data, opposite story. That's the ecological fallacy, live and in color.
The trick is that the groups differ from each other in both variables simultaneously. Groups on the right of the x-axis also tend to be higher on the y-axis — not because x causes y within any group, but because of how the groups are composed. The between-group pattern overwhelms the within-group pattern when you aggregate.
Chocolate, Nobel Prizes, and Other Nonsense
You'd think Robinson's 1950 paper would have vaccinated us against this error. It did not. Not even close.
In 2012, a physician named Franz Messerli published a paper in the New England Journal of Medicine — one of the most prestigious medical journals on Earth — with the title "Chocolate Consumption, Cognitive Function, and Nobel Laureates."2 He plotted per-capita chocolate consumption by country against Nobel prizes per capita and found a strikingly strong correlation: r = 0.79.
Switzerland, the chocolate capital of the world, also has the most Nobels per person. Sweden does well on both counts (though Messerli dryly noted that Sweden might be "slightly biased" given that the Nobel Committee sits in Stockholm). China and Japan eat less chocolate and win fewer Nobels.
Messerli's tongue was planted firmly in cheek, but the paper was real, the correlation was real, and the NEJM really published it. The point wasn't that chocolate makes you smart. The point was that country-level data can produce stunningly convincing correlations that are absolutely meaningless at the individual level. Nobody believes that if you personally eat more chocolate, you'll win a Nobel Prize.3
Red states vs. blue states: We talk about "red states" and "blue states" as though every person in Texas votes Republican and every person in California votes Democrat. In reality, about 46% of California voters chose Trump in some elections, and many Texas cities lean heavily Democratic. The "red/blue" frame is an ecological fallacy applied to an entire political culture.4
School district rankings: District A has higher average test scores than District B. Parents move to District A. But the individual student experience might be identical — District A simply has more affluent families, and affluence correlates with test scores for reasons that have nothing to do with the school itself.
National happiness rankings: Finland is the "happiest country." Does that mean every Finn is happier than every American? Of course not. It means the average Finn reports higher life satisfaction, which tells you very little about any particular Finn.
The Mathematics of Disappearing Variation
Let's be a bit more precise about why this happens. When you compute the average of a group, you destroy information. Specifically, you destroy within-group variation — the differences between individuals inside the same group. And it's precisely this within-group variation that might tell the true story.
Think about it with a simple formula. The total variation in any dataset can be decomposed into two parts:
When you aggregate to group means, the within-group variance vanishes entirely.
When you compute group averages, the within-group variance is annihilated. It's as if it never existed. All that remains is the between-group variance. If the between-group pattern and the within-group pattern point in different directions — which they absolutely can — then averaging will show you only one of the two stories, and it might be the wrong one.5
This is related to Simpson's paradox, but the ecological fallacy has its own flavor. Simpson's paradox is about a reversal when you condition on a lurking variable. The ecological fallacy is specifically about the level of analysis — aggregate versus individual. You can have an ecological fallacy without Simpson's paradox (the trend doesn't need to reverse when you condition on group, it just needs to differ), and you can have Simpson's paradox without ecological aggregation. They're cousins, not twins.6
Build Your Own Fallacy
The ecological fallacy isn't magic — it's mechanics. It happens when groups are arranged so that the between-group relationship differs from the within-group relationship. You can control exactly when and how strongly it occurs. Try it yourself:
Build Your Own Ecological Fallacy
Adjust the sliders to control how groups are arranged and how much variation exists within each group. Watch the two correlations diverge.
Notice how easy it is to create a situation where the two correlations have opposite signs. Whenever the between-group slope and within-group slope point in different directions — and the between-group spread is large enough relative to the within-group spread — you get an ecological fallacy. This isn't a rare edge case. It's the default in many real-world datasets.
Drawing Lines on Maps
There's a particularly insidious variant of the ecological fallacy that geographers call the Modifiable Areal Unit Problem, or MAUP.7 The idea is simple and devastating: when you aggregate spatial data, the correlations you find depend on how you drew the boundaries.
Imagine you're studying the relationship between income and crime in a city. If you aggregate by large districts, you might find a strong negative correlation — richer districts have less crime. But redraw the boundaries into smaller neighborhoods, and the correlation might weaken, strengthen, or even reverse, depending on exactly where you put the lines.
This isn't just an academic concern. It's the mathematical engine behind gerrymandering. By redrawing district boundaries, you can make the same voters produce wildly different political outcomes. The data doesn't change. The people don't move. Only the lines move, and the aggregated statistics move with them.
The Core Lesson
Whenever someone shows you a correlation based on grouped data — countries, states, districts, schools — ask yourself: would this still be true at the individual level? The answer is often no. The group is not the person. The map is not the territory. And the average is not the individual.
Learning to Zoom In
Robinson's paper has been cited thousands of times, and yet the ecological fallacy remains one of the most common errors in public discourse. Every election cycle, pundits talk about "what women think" or "how Hispanics vote" as though these groups are monoliths. Every international ranking — happiest countries, best education systems, healthiest diets — invites us to draw individual conclusions from aggregate data.
The defense isn't complicated. It's a habit of mind: whenever you encounter a claim based on grouped data, imagine the individuals within those groups. Picture the scatter plot with the actual dots, not just the group averages. Ask what's being hidden by the aggregation.
Robinson knew this in 1950. The rest of us are still catching up. His paper ended with a line that statisticians have been whispering ever since, like a prayer against the temptation of easy answers: "An ecological correlation is almost certainly not equal to its corresponding individual correlation."8
Almost certainly not. And often not even close.