In his impressive first book, Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society, entrepreneur/intellectual Jim Manzi has the makings of an airport best seller in the genre of Steven Levitt’s Freakonomics and Malcolm Gladwell’s Blink. Indeed, Uncontrolled is far more reliable than those two sometimes-dubious tomes.
Manzi certainly has a story worth telling. In 1999, at the peak of the tech start-up bubble, Manzi, an MIT math major-turned-business strategy consultant, founded Applied Predictive Technologies. After initially floundering, Manzi realized there could be a big market supplying corporate America with software for conducting experiments, or what he calls Randomized Field Trials (RFT): Will the main button on your home page get more clicks if it’s red or green? Will upgrading your motels’ lobbies pay off? Will issuing iPads to your sales force boost sales?
We all have opinions, but we seldom test them using control groups. The FDA forces drug companies to use rigorous methodologies, so why not employ the gold standard in other businesses? (And what about social policies such as education and crime? Why, for example, has letting the 50 state governments serve as “the laboratories of democracy” fallen out of fashion in favor of federal mandates such as No Child Left Behind?)
“What good social science has revealed—that parents matter, genes matter, race matters, sex matters, and IQ matters—is the opposite of what the vast majority of social scientists wanted to discover.”
Stumbling upon this potential gold mine forced Manzi into a major learning effort:
…I started to read and research the issues involved in experimental analysis of interventions, at first frantically, and then systematically over about ten years.
He discovered that great thinkers had long ago considered the problems he was now facing:
I iterated between our practical experiences trying to implement the approach and the rediscovery of historical academic and industrial research that had addressed what were really the same analytical issues.…I worked my way…from detailed study of the technical methods of clinical trials, back through [Ronald A.] Fisher, Donald Campbell, and other social scientists, ultimately to the pure philosophy of science.
This notion of a young software executive slowly realizing that his company’s survival depends upon answers to questions first wrestled with by giant intellects such as Thomas Kuhn, Karl Popper, David Hume, and even Shakespeare’s contemporary Francis Bacon is exciting. The hunt for old knowledge drives many popular thrillers such as The Da Vinci Code and National Treasure. Why not write a nonfiction book structured as a search for lost wisdom?
Unfortunately for Uncontrolled’s sales totals, Manzi has organized his book in the opposite fashion. Rather than begin in 1999 with Manzi dangling between ruin and riches, Uncontrolled’s first chapter, dauntingly entitled “Induction and the Problem of Induction” (presumably, the author didn’t use his own software to test how appealing his chapter names are), methodically starts in 1620 with Bacon’s Novum Organum.
Granted, Uncontrolled offers one of the most lucid and sensible historical overviews of the philosophy of science I’ve ever read. But how much demand is there at airport bookstores for philosophy of science?
Jim should have put his business war stories up front, his chapters on how to conduct experiments next, and his philosophy later. The frequent fliers who buy business books are happy to get a few enjoyable chapters out of their best sellers. In contrast, the handful of folks who like philosophy of science enjoy reading densely argued books like Uncontrolled all the way through.
In a few years, when confidentiality agreements with clients are less of a stumbling block for him, Manzi should revisit this material, aiming at the broader business market instead of Uncontrolled’s tiny target audience of Reihan Salam’s highbrow conservative friends such as David Brooks, Ross Douthat, and Tyler Cowen.
As a public policy book, Uncontrolled tries to answer the perennial question: What’s wrong with the social sciences? To Manzi, the problem is too many studies and not enough experiments.
For example, Manzi points out that the evidence for Levitt’s popular theory in Freakonomics that legalizing abortion in 1973 caused crime to fall in the later 1990s is doubtful. In fact, Manzi notes, it’s largely based on a programming blunder Levitt made.
But Manzi doesn’t have a suggestion for how his RFTs could have helped: randomized abortions? Moreover, as I pointed out to Levitt years before Uncontrolled in our debate at slate.com (apparently our names have since been redacted), he’d been negligent in checking the historical record. (Originally, Levitt plumb forgot the crack wars.) But many influential people seemed to wish Levitt’s story was true.
Manzi says the golden age of government-funded social-policy experiments was the late 1960s into the early 1980s. But he misses why social scientists lost enthusiasm for testing: Trials of progressive programs such as Head Start and the negative income tax turned out depressingly. What good social science has revealed—that parents matter, genes matter, race matters, sex matters, and IQ matters—is the opposite of what the vast majority of social scientists wanted to discover.
Ironically, social science has been quite successful at demonstrating the failures of social engineering.
SIGN UP
Daily updates with TM’s latest
Copyright 2017 TakiMag.com and the author. This copy is for your personal, noncommercial use only. You can order reprints for distribution by contacting us at editors@takimag.com.