Posted by Jay Livingston
Sunday, and no football. But we’ll always have Belichick and Brady.
I’m not saying that the Patriots are out-and-out liars. But they are outliers.
The advantage of an underinflated ball, like the eleven of the twelve footballs the Patriots used last Sunday, is that it’s easier to grip. Ball carriers will be less likely fumble if they’re gripping a ball they can sink their fingers into.
We can’t go back and measure the pressure of balls the Patriots were using before the Colts game, but Warren Sharp (here) went back and dug up the data on fumbles for all NFL games since 2010. Since a team that controls the ball and runs more plays has more chances to fumble, Sharp graphed the ratio of plays to fumbles (values in red squares in the chart below) along with the absolute number of fumbles (values in blue circles). The higher the ratio, the less fumble-prone the team was.
(Click on an image for a larger view.)
One of these things is not like the others. That’s what an outlier is. It’s off the charts. It’s nowhere near the trend line. Something about it is very different. The variables that might explain the differences among the other data points – better players, better weather or a domed stadium, a pass-centered offense – don’t apply. Something else is going on.
As the graph shows, when the teams are rank ordered on the plays/fumbles ratio, the difference between one team and the next higher is usually 0-2, there are only two gaps of 5 until the 9-point gap between #3 Atlanta and #2 Houston. From the second-best Texans and to the Patriots there’s a 47-point jump.
Sharp also graphed the data as a histogram.
The Patriots play in a cold-weather climate in a stadium exposed to the elements. Yet their plays/fumble ratio is 50% higher than that of the Packers, 80% higher than the Bears. They have good players, but those players fumble less often for the Patriots than they did when they played for other NFL teams.
Usually, the statistical anomaly comes first – someone notices that US healthcare costs are double those of other nations – and then people try to come up with explanations. In this case, it wasn’t until we had a possible explanatory variable that researchers went back and found the outlier. As Peter Sagal of “Wait, Wait, Don’t Tell Me” said, “The League became suspicious when a Patriots player scored a touchdown and instead of spiking the ball he just folded it and put it in his pocket.”
UPDATE, Jan. 28: Since I posted this, there has been some discussion of Sharp’s data (“discussion” is a euphemism – this is sports and the Internet after all). If you’re really interested in pursuing this, try Advanced Football Analytics or this piece at Deadspin “Why Those Statistics About The Patriots’ Fumbles Are Mostly Junk,” (to repeat, “discussion” is a euphemism, and if you want more strongly voiced views, read the comments). Neil Paine at FiveThirtyEight links to some others. In sum, the evidence is not as strong as what Sharp’s version suggests. (One of the difficulties I suspect is that a fumble is a rare event. The difference between the teams with the surest grip and the most butterfingered is about one fumble every couple of games.
You got punk'd by an amateur statistician.
ReplyDelete“Punk'd” implies that Sharp was trying to deceive people. Was he? Or did he just grab some quick data?
ReplyDeleteFrom what I understand, the complaints about his numbers are that they included fumbles on returns and that they didn’t go back far enough. (Someone complained that he left out teams that play indoors, but I think he corrected that.) I haven’t followed this closely, but do you know if someone has compiled that data and computed the ratio of plays to fumbles on plays? What other numbers should be considered?
I recall seeing a recalculation of that play/fumble figure for individual Patriot players who had played with other teams. The difference still held for some of the players, though not all. As for the data going back well before 2010, I think I saw something showing that the Patriots’ performance relative to other teams improves in 2007 and subsequent years. That was the year that visiting teams were allowed to supply their own balls.
It still may turn out that the Pats have been deflating balls for a long time, not just the Colts game. If so, all that the revised data means is that the advantage Belichick got by cheating isn’t as great as Sharp thought. Or maybe it will turn out that the underinflation in the Colts game was a one-off (or an eleven-off), and that the Patriots really are more sure-handed.
Unfortunately, it’s unlikely that Goodell is going to give us real investigation, so we’ll never know.
Yes, he was trying to deceive people. He knew it was a headline that the haters would uncritically accept and then propagate - which is exactly what you did. If you had spent even 20 minutes looking at the tendentious data and interpretation, you would have known better. You drank the Haterade. You got punk'd. Own it, bro.
ReplyDeleteLet me ask again: can you point me to some numbers that you find less tendentious and more accurate? Thanks.
ReplyDeleteNate Silver published a post that linked to five different analyses that, he said, "Collectively...did a great job of breaking down the Statistics 101 problems with Sharp’s original analyses. But even if Sharp had been less sloppy, it would have been right to take issue with the larger implication of his work — that any major outlier, if shown to be statistically significant, should be seen as evidence of rule-breaking." Link: http://fivethirtyeight.com/datalab/your-guide-to-deflate-gateballghazi-related-statistical-analyses/
ReplyDeleteThe best post he didn't include is here: http://forums.realgm.com/boards/viewtopic.php?f=139&t=1368741&p=42486161#p42486161
The Patriots weren't even the biggest outlier in his own study. He excluded dome teams (rather than the more accurate way, which would be to exclude games that happen indoors...even though there's no significant difference in fumble rate between indoor/outdoor games). If you include all the teams, the Falcons, Saints, Colts, and Pats are all bunched up at the top. Just go and look at the original analysis done by Sharp, it's in his own chart.
Further: he counts special-teams fumbles but not special team plays. That obviously distorts the odds ratios, even setting aside the fact that that the special-teams footballs are different from the game footballs, not under team control at all.
Finally, the more accurate analysis shows that comparing Pats players fumbles vs same players fumbles when on other teams, the players fumbled 23% less while on the Pats (rather than the 88% difference Sharp finds). But it's such a small sample size, that difference turns out to be...wait for it...four fumbles over a seven year period. http://soshcentral.com/football-science/football-statistics/2015/01/27/fumbling-data-truth-patriots-fumble-rate/
You wrote a post that called the Patriots liars (except you put a question mark at the end...a nice little rhetorical trick) without doing your homework.
Another issue worth noting is that a more sociological take on this might take issue with the fact that the NFL interviewed more than 40 people about allegedly underinflated footballs, while doing everything it possibly could to not learn about the Ray Rice spousal abuse incident, the Greg Hardy alleged domestic abuse, and the enormous issue of concussions. Perhaps there is an institutional analysis worth doing, but Dr. Livingston took the bait - hook, line, and stinker. The unfortunate thing is that, in doing so, he reproduced an argument that was debunked, redebunked, and even further debunked, without bothering to look for evidence that disconfirmed what he wanted to believe.
ReplyDeleteAnon: I have no idea why the animus, but in my Jan. 28 update to the post, I said that the evidence was shaky, and I put in links to other sources, including the very same FiveThirtyEight link that you cited in your comment ten days later. (I'm assuming that the Anonymous of all these comments is the same person.)
ReplyDelete