Margin of Error – Mostly Error

February 14, 2016
Posted by Jay Livingston

It’s the sort of social “science” I’d expect from Fox, not Vox. But today, Valentine’s Day, Vox (here) posted this map purporting to show the average amount people in each state spent on Valentine’s Day.

(Click on the image for a larger view.)


“What’s with North Dakota spending $108 on average, but South Dakota spending just $36?” asks Vox. The answer is almost surely: Error.

The sample size was 3,121. If they sampled each state in its proportion of the US population, the sample in the each Dakota would be about n = 80 n = 8. The source of the data, Finder, does not report any margins of error or standard deviations, so we can’t know. Possibly, a couple of guys in North Dakota who’d saved their oil-boom money and spent it on chocolates are responsible for that average. Idaho, Nevada, and Kansas – the only other states over the $100 mark – are also small-n. So are the states at the other other end, the supposedly low-spending states (SD, WY, VT, NH, ME, etc.). So we can’t trust these numbers.

The sample in the states with large populations (NY, CA, TX, etc.) might have been as high as 300-400, possibly enough to make legitimate comparisons, but the differences among them are small – less than $20.

My consultant on this matter, Dan Cassino (he does a lot of serious polling), confirmed my own suspicions. “The study is complete bullshit.”

UPDATE February 24, 2016: Andrew Gelman (here) downloaded the data did a far more thorough analysis, estimating the variation for each state. His graph of the states shows that even between the state with the highest mean and the state with the lowest, the uncertainty is too great to allow for any conclusions: “Soooo . . . we got nuthin’.”

Andrew explains why it’s worthwhile to do a serious analysis even on frivolous data like this Valentine-spending survey. He also corrects my order-of-magnitude overestimation of the North Dakota sample size. 

More Good News About Kids

February 12, 2016
Posted by Jay Livingston

Five weeks ago I asked

When’s the last time you read an op-ed or magazine article that began, “Kids today are just so much better than kids of a generation or two ago.” 
 
That post (here) had some data showing that in crime, drug use, unwanted pregnancy, and other categories, today’s youth were doing much better than their counterparts of earlier generations.
           
Now, the answer to that question (“When was the last time you read . . ?) is “Today.”

Today, Vox has an article called “Today’s Teens Are Better Than You, and We Can Prove It” (here). It has data on the variables I mentioned plus meth and other drugs, carrying guns to school, fighting, and other things most of us are glad to see less of. There’s even an interactive function where you can compare kids today against your own cohort – if you are under 45.

The article begins, “The kids are all right,” an obvious line that I had to try very hard to avoid in my post. But take a look at the data.

The article makes no attempt to pinpoint the causes of these changes, so feel free to attribute the good news to whatever factors you favor.

Fairway Farewell?

February 10, 2016
Posted by Jay Livingston

Update: Three months ago, I blogged (here) about Fairway, a New York food market. Things had not been going well for Fairway since its buyout by a private equity firm, and especially since the IPO in 2013. “That’s private equity for you,” said one of their former managers.

Now Fairway is looking at bankruptcy. The New York Post reports that Fairway has lost over $300 million in the last five years. The Wall Street Journal  says, “Fairway—a high-end chain with 15 stores in New York, New Jersey and Connecticut—said that there is substantial doubt about its ability to continue as a going concern.”

Auteur, Schmauteur

February 10, 2016
Posted by Jay Livingston

Despite the maxim about familiarity breeding contempt, we usually like what’s familiar.  With music for example, familiarity breeds hits in the short run and nostalgia in the long run. The trouble is that it’s tempting to attribute our liking to the inherent quality of the thing rather than its familiarity.  With movies, film buffs may make this same conflation between what they like and what they easily recognize.

That’s one of the points of Scott Lemieux’s takedown (here) of Peter Suderman’s Vox article about Michael Bay.


Suderman hails Bay as “an auteur — the author of a film — whose movies reflect a distinctive, personal sensibility. Few filmmakers are as stylistically consistent as Bay, who recycles many of the same shots, editing patterns, and color schemes in nearly all of his films.”

But what’s so great about being an auteur with a recognizable style? For Lemieux, Michael Bay is a hack. His movies aren’t good, they’re just familiar. Bay’s supporters like them because of that familiarity but then attribute their liking to some imagined cinematic quality of the films.

My students, I discovered last week,  harbor no such delusions about themselves and the songs they like. As a prologue to my summary of the Salganik-Watts MusicLab studies, I asked them to discuss what it is about a song that makes it a hit. “Think about hit songs you like and about other hit songs that make you wonder, ‘How did that song get to be #1?’” The most frequent answers were all about familiarity and social influence. “You hear the song a lot, and everyone you know likes it, and you sort of just go along, and then you like it too.” I had to probe in order to come up with anything about the songs themselves – the beat, the rhymes, even the performer.

Lemieux cites Pauline Kael’s famous essay “Circles and Squares” (1963), a response to auteur-loving critics like Andrew Sarris. She makes the same point – that these critics conflate quality with familiarity, or as she terms it “distinguishability.”

That the distinguishability of personality should in itself be a criterion of value completely confuses normal judgment. The smell of a skunk is more distinguishable than the perfume of a rose; does that make it better?

Often the works in which we are most aware of the personality of the director are his worst films—when he falls back on the devices he has already done to death. When a famous director makes a good movie, we look at the movie, we don't think about the director's personality; when he makes a stinker we notice his familiar touches because there’s not much else to watch.

Assessing quality in art is difficult if not impossible. Maybe it’s a hopeless task, one that my students, in their wisdom, refused to be drawn into. They said nothing about why one song was better than another. They readily acknowledged that they liked songs because they were familiar and popular, criteria that producers, promoters, and payola-people have long been well aware of.

“In the summer of 1957,” an older friend once told me, “My family was on vacation at Lake Erie. There was this recreation hall – a big open room where teenagers hung out. You could get ice cream and snacks, and there was music, and some of the kids danced. One afternoon, they played the same song – “Honeycomb” by Jimmie Rodgers – about twenty times in a row, maybe more. They just kept playing that song over and over again. Maybe it was the only song they played the whole afternoon.”

It wasn’t just that one rec hall. The people at Roulette Records must have been doing similar promotions all around the country and doing whatever they had to do to get air play for the record. By the end of September, “Honeycomb” was at the top of the Billboard charts. Was it a great song? Assessment of quality was irrelevant, or it was limited to the stereotypical critique offered by the kids on American Bandstand: “It’s got a good beat. You can dance to it.” Of course, this was before the 1960s and the rise of the auteur, a.k.a. the singer-songwriter.

Hollywood uses the same principle when it churns out sequels and prequels – Rocky, Saw, Batman. They call it a “franchise,” acknowledging the films have the similarity and predicatability of Burger Kings. The audience fills the theaters not because the movie is good but because it’s Star Wars. Kael and the other anti-auteurists argue that auteur exponents are no different in their admiration for all Hitchcock. Or all Michael Bay. It’s just that their cinema sophistication allows them to fool themselves.


(Big hat tip to Mark at West Coast Stat Views.)