July 30, 2014
Source: Shutterstock
You might think that humanity’s long failure until very recently to develop sophisticated statistics was because nobody had much data to play around with. Yet there were massive censuses thousands of years ago. The Gospel of Luke notes, “In those days a decree went out from Emperor Augustus that all the world should be registered.” A millennium or more earlier in the Book of Numbers, God tells Moses to enumerate all the Hebrew men old enough and able to fight in the military. Moses comes up with 603,550.
That’s a pretty big number. And yet, for most of human history, nobody seemed terribly motivated to innovate techniques for analyzing human quantities much beyond counting them. This apathy stands out when contrasted to the prodigious mathematical breakthroughs made in support of astronomy and physics. In fact, much of the foundational work in statistics, such as the development of the normal distribution by Gauss and Laplace in 1809-1810, was to help astronomers deal with the random errors in their observations.
A generation later, the Belgian astronomer Adolphe Quetelet applied the normal distribution for the first time to human data”a coat maker’s table of the chest sizes of 5,378 Scottish soldiers”and showed they formed a rough bell curve.
Stephen M. Stigler, a professor in the U. of Chicago department of statistics, argues that the “statistical enlightenment“ didn”t begin until 1885, when Francis Galton made sense of the fundamental stumbling block of “regression toward the mean.” That was 198 years after physicist Isaac Newton kicked off the Enlightenment by publishing his “Mathematical Principles of Natural Philosophy“ that explained “The System of the World.”
The lag between Newton and Galton is puzzling because the latter, an amateur mathematician in his 60s, was no Newton. To this day, the stereotype that calculus takes more brains than statistics has endured. For instance, the Advanced Placement test in statistics for high school students, which wasn”t introduced until 1996 but has become highly popular, is widely considered easier than the AP tests on Newton’s invention, calculus. And yet tremendously significant methods were still waiting to be discovered by Galton in the 1880s.
Galton had been struggling since 1859 with the quantitative puzzles about genetic diversity in plants, animals, and people implied by his cousin Charles Darwin’s book The Origin of Species. The mathematical sophistication of Galton’s 1869 book Hereditary Genius is nugatory compared to the supreme elegance of the famous equations in physicist James Clerk Maxwell’s paper from the same year, “A Dynamical Theory of the Electromagnetic Field.” Galton’s genius lay instead in asking a long series of questions”such as how to apportion the effects of nature and nurture”that now seem obvious but apparently seldom troubled anyone before him.
Galton kept at his self-appointed task and, aided by professional mathematicians, continued to make breakthroughs into the 20th century. His seminal “The Wisdom of Crowds paper was published in 1907 when he was 85.
But if an octogenarian is making theoretical breakthroughs in your field, that suggests he doesn”t have enough competition.
As best as I can understand this pattern over the millennia, celestial questions involving geometry were traditionally seen as higher, purer, cleaner than earthier matters, and thus they attracted the finest minds. Mucking around with human data must have seemed sublunary, déclassé.
To this day, statistical reasoning still strikes many as being in dubious taste. Larry Summers was let go as president of Harvard after pointing out that males tend to have larger variances in IQ than females. Similarly, Jason Richwine lost his job last year after it was revealed that his Harvard doctoral dissertation, IQ and Immigration Policy, was a sophisticated statistical analysis of matters of grave national import.
Here’s a way to test Dr. Gelman’s theory that the rise of statistics has changed the world less than the hype suggests: certain uses of statistics are banned by law. Does the world look much different where thinking statistically is punished?
For example, the federal government forbids lenders from using race or ethnicity in deciding whether or not to approve a mortgage. To enforce this prohibition, it collects a vast quantity of data under the Home Mortgage Disclosure Act to empower community NGOs like the late ACORN to more easily sue lenders for disparate impact discrimination.
In turn, the government nurtured a business culture in which anything you observed just by watching about racial differences in creditworthiness was something you couldn”t discuss publicly, or in emails that could be subpoenaed. Thus when ambitious corporate executives like Angelo Mozilo of Countrywide Financial and Kerry Killinger of Washington Mutual boasted that they would expand rapidly by loaning more to Hispanic victims of prejudicial redlining, no one, as far as I”ve been able to find in six years of looking, publicly criticized their assumption of irrational bigotry against minorities.
So, here’s a case where our culture laboriously emasculated itself of one obvious use of statistics. And what has been the result? Well, the world looked much the same. Countrywide and Washington Mutual flew even higher and faster. The only problem was that mortgage lending in the heavily Hispanic “Sand States” kicked off the Global Financial Crisis of 2008.
Without good statistics, the world would be quite similar, except for when it’s crashing and burning.
