Justice Scalia Does the Math

March 25, 2008
Posted by Jay Livingston

Supreme Court Justice Antonin Scalia. Even those who disagree with him speak of his brilliance, his incisive intelligence, and his wit. Apparently, he does better on the verbal part than on the math. An article by Adam Liptak in the New York Times today nails it. In an opinion upholding a death penalty conviction, Scalia dismisses the problem of wrongful convictions because they constitute such a minuscule fraction of cases. For support, he cites the number: “Between 1989 and 2003, the authors identify 340 ‘exonerations’ nationwide—not just for capital cases, mind you, nor even just for murder convictions, but for various felonies.” (Note that Scalia puts exonerations in quotation marks. He still thinks the dudes are guilty.) Then he quotes approvingly from a prosecutor.

[L]et’s give the professor the benefit of the doubt: let’s assume that he understated the number of innocents by roughly a factor of 10, that of 340 there were 4,000 people in prison who weren’t involved in the crime in any way. During that same 15 years, there were more than 15 million felony convictions across the country. That would make the error rate .027 percent—or, to put it another way, a success rate of 99.973 percent.

Most students in the undergraduate methods course could tell you what’s wrong with this fraction. Exonerations are rare because they require extraordinary legal effort, efforts that prosecutors and often judges strongly resist. Claims of innocence in any but the most serious cases don’t get that kind of effort. And most of those 15 million felony convictions are not for the more serious degrees of murder and rape. So the 4,000 in the numerator of the fraction is almost certainly a severe undercount of all wrongful convictions. For the denominator, however, Scalia takes all felony convictions in the US. He’s using either a wrong numerator or a wrong denominator, or both. For an analogy, Liptak quotes Samuel Gross in a forthcoming article in Annual Review of Law and Social Science:

By this logic we could estimate the proportion of baseball players who’ve used steroids by dividing the number of major league players who’ve been caught by the total of all baseball players at all levels: major league, minor leagues, semipro, college, and little league – and maybe throwing in football and basketball players as well.

I’ll try to remember this example the next time I teach about stat and methods. Or courts. Or the next time I read about Scalia’s incisive intelligence.