Overcoming Social Desirability Bias – He’s Got a Little List

April 19, 2011
Posted by Jay Livingston

As some day it may happen that a survey must be done, you need a little list, a quick five-item list – for sex or race or crime or things quite non-PC but fun, where pollsters all have missed, despite what they insist. There’s the guy who says he’d vote for blacks if they are qualified; he’d vote for women too, but are we sure he hasn’t lied? “How many partners have you had?” Or “Did you ever stray?” With things like this you can’t always believe what people say. You tell them it’s anonymous, but still their doubts persist, and so your methodology can use this little twist.

It’s called the List Experiment (also the Unmatched Count Technique). It’s been around for a few years, though I confess I wasn’t aware of it until I came across this recent Monkey Cage post by John Sides that linked to another post from the presidential year of 2008. Most surveys then were finding that fewer than 10% of the electorate were unwilling to vote for a woman (Hillary was not mentioned by name). But skeptical researchers (Matthew Streb et al., here gated), instead of asking the question directly, split the sample in half. They asked one half

How many of the following things make you angry or upset?

• The way gasoline prices keep going up.
• Professional athletes getting million dollar-plus salaries.
• Requiring seat belts to be used when driving.
• Large corporations polluting the environment.

Respondents were told not to say which ones pissed them off, merely how many. Researchers calculated the average number of items people found irritating. The second half got the same list but with one addition:

• A woman serving as president.

If the other surveys are correct, adding this one item should increase the mean by no more than 10%. As it turned out, 26% of the electorate would be upset or angry about a woman president, considerably more than the 6% in the GSS sample who said they wouldn’t vote for a woman.

The technique reminds me of a mentalist act: “Look at this list, sir, and while my back is turned tell me how many of those things you have done. Don’t tell me which ones, just the total number. Now I want you to concentrate very hard . . . .” But I can certainly see its usefulness as a way to check for social desirability bias.

Iyengar Management

April 14, 2011
Posted by Jay Livingston

I think it came up in a discussion of culture and the observation that American culture generally values rationalism over traditionalism.* I was reminded of this anecdote that Sheena Iyengar tells in her TED talk (it’s also in her recent book The Art of Choosing).

That in turn reminded me of the famous** diner scene from “Five Easy Pieces.” The conflict is similar – individual goals in conflict with rules, though in this case the rules are bureaucratic regulations rather that cultural norms.

These clips relate to other issues besides culture and bureaucracy – social class comes quickly to mind – but also occupational roles , the self and presentation of self, and of course, conflict resolution (I can’t imagine Prof. Iyengar sweeping the crockery off the table).

-------------
* From Robin Williams (no, not that Robin Williams, not the one of “Mrs. Doubtfire” and “Flubber”) American Society, first published sixty years ago.

** Or maybe not so famous. None of my students had heard of it (the movie was made twenty years before they were born). They did, however, recognize a very young Jack Nicholson.

AKD 2011

April 13, 2011
Posted by Jay Livingston

Montclair had its annual AKD induction ceremony a week ago. This year, fourteen students joined – a good number, especially considering that this year we raised the minimum GPA.

(Click on the photo for a larger view.)

From left to right:

• Malgorzata Slusarek
• Courtney Artz
• Lisa M. Applegate
• Jesenia Rivera
• Irina Gavdanovich
• Anthony DeLello
• David Stever

Seven students weren’t able to attend (or couldn’t get there till after the photo-op)

• Lauren Breem
• Concetta Cardellicchio
• Staycee Marshall
• Seth Mendez
• Cassandra Moran
• Jenna Pariso
• Gabrielle Walker

Our speaker was Peter Moskos, author of Cop in the Hood, and (just out today) In Defense of Flogging. His talk was “The Wire, for Real: My Year as a Cop Baltimore's Eastern District,” but the real theme, not quite explicitly stated, was the wrongheadedness of the war on drugs. Peter makes his point with macro data (his slides included graphs of crime rates and incarceration rates) and ethnographic data (photos of the hood with its boarded-up buildings, desolate streets, grafitti (some of them very amusing) and the kids who sell drugs.

You can get more of Peter’s take on all this in his book, or at his Website.

What’s Wrong With (Percentages in) Mississippi

April 10, 2011
Posted by Jay Livingston

A Public Policy Polling survey asked Mississippi Republicans about their opinion on interracial marriage. It also asked how they felt about various politicians. The report concludes, “Tells you something about the kinds of folks who like each of those candidates.”

Not quite.

What’s been getting the most attention is the finding that Mississippi Republicans think interracial marriage should be illegal. Not all Mississippi Republicans. Just 46% of them (40% think it should be legal).* Does their position on intermarriage tell us anything about who they might like as a candidate? Does a Klaxon wear a sheet?

(Click on the chart for a larger view.)

It’s no surprise that Sarah Palin is much preferred to Romney. But as PPP points out racial attitudes figure differently depending on the candidate. When you go from racists to nonracists,** Palin’s favorable/unfavorable ratio takes a hit. But Romney’s gets a boost.

But does this tells us something about “the kinds of folks who like each of those candidates”? The trouble is that statement is percentaging on the dependent variable, implicitly comparing Romney supporters with Palin supporters. But the percentages actually given by PPP compare racists with nonracists** The statement is implying that candidate preferences tell us about racial attitudes. But what the data show is that racial attitudes tell us about candidate preferences. The two are not the same. From the data PPP gives, we don’t actually know what percent of Palin supporters favor laws against intermarriage. Ditto for Romney supporters.

In any case, neither Palin nor Romney is the top choice of Mississippi Republicans (especially the racists), who may be thinking racially but are acting locally and going with their own governor first and the former governor of neighboring Arkansas second.


* The sample was only 400. But the results aren’t too different from what the GSS has found. The most recent GSS I could find that included RACMAR was from 2002. In the “East South Central” region, the percent favoring laws against interracial marriage was 36%. So among Republicans, it might have been ten points higher.

**I realize that neither of these terms “racist” and “nonracist” is necessarily accurate. I use them as shorthand for, respectively, “people who think interracial marriage should be illegal” and “people who think interracial marriage should be legal.”