Posted by Jay Livingston
“Big data has trouble with big problems,” says David Brooks (here).
we’ve had huge debates over the best economic stimulus, with mountains of data, and as far as I know not a single major player in this debate has been persuaded by data to switch sides.But it’s not the data that has trouble with big problems, it’s the “major players.” You can’t blame the data for the resistance of those players.
I’m not sure who he means by that phrase. Politicians? If Brooks thinks a politician will renounce a cherished policy just because the data show it to be unfounded, he is indeed naive.
But economists, too, cling to their theories, and for similar reasons. The theory has served them well in the past. It rests on evidence, and it has explained and solved many problems. The economists are like scientists in Thomas Kuhn’s Structure of Scientific Revolutions. They have been doing “normal science,” science framed by the dominant paradigm, and are now faced with an anomalous bit of evidence. Kuhn doesn’t really blame them for not jettisoning the paradigm that has been the basis of their life’s work. After all, the firm commitment to that paradigm, the belief that it can solve all its problems – “that same assurance is what makes normal or puzzle solving science possible.” And most science is normal science.
To abandon the old paradigm in favor of a new one, says Kuhn, is “a conversion experience.” Scientists “whose productive careers have committed them to an older tradition of normal science” are unlikely converts. He quotes Max Planck:
a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.Paul Krugman has a better quote from Planck. “Science progresses funeral by funeral.”