Do You Hear What I Hear? Maybe Not.

December 18, 2011
Posted by Jay Livingston

As I’ve said before (here), the question the researcher asks is not always the question people hear.   That’s especially true when the question is about probabilities.

Here, for example, is the ending of a fictional vignette from a recent study in the Journal of Personality and Social Psychology.

Richard found a wallet on the sidewalk. Nobody was looking, so he took all of the money out of the wallet. He then threw the wallet in a trash can.

Is it more probable that Richard is
    a. a teacher
    b. a teacher and a rapist

Since the category “a teacher” necessarily includes teacher/rapists as well, the correct answer is “a.” But many people choose “b.”  The study used this “conjunction fallacy”* to probe for prejudices by switching out the rapist for various other categories.  Some subjects were asked about atheist/teachers, others about Muslim/teachers, and so on.  The finding:

A description of a criminally untrustworthy individual was seen as comparably representative of atheists and rapists but not representative of Christians, Muslims, Jewish people, feminists, or homosexuals.

Andrew Gelman, a usually mild-mannered reporter on things methodological, had a post on this with the subject line, “This one is so dumb it makes me want to barf.”

What’s really disturbing about the study is that many people thought it was “more probable” that the dude is a rapist than that he is a Christian! Talk about the base-rate fallacy.

Maybe it would settle Andrew’s stomach to remember that the question the researchers asked was almost certainly not the question people heard.   What the researchers pretend to be asking is this:

Of all thieves, which are there more of – teachers or rapist/teachers? 

After all, that is indeed the literal meaning.  But it’s pretty obvious that the question people are answering is something different:

Which group has a higher proportion of thieves among them – all teachers or the subset rapist/teachers?

The researchers say they weren’t at all interested in demonstrating the conjunction fallacy.  They were just using it to uncover the distrust people feel towards atheists.  What they found was that when it comes to dishonesty, people (specifically, 75 female and 30 male undergrads at the University of British Columbia) rank atheists at about the same level as rapists.

But why resort to such roundabout tricks?  Why not ask the question directly?**

Who is more likely to steal a wallet when nobody is looking?
    a.  an atheist
    b. a rapist
    c.  neither; they are equally larcenous

Or:

On a seven-point scale, rank each of the following on how likely they would be to steal a wallet when nobody is looking:

•     an atheist: 1   2   3   4   5   6   7
•     a Christian: 1   2   3   4   5   6   7
•     a rapist: 1   2   3   4   5   6   7
•     etc. 

Instead, they asked questions that they knew would confuse nearly anyone not fluent in the language of statistics and probability.  I wonder what would happen if in their “who do you distrust” study they had included a category for experimental social psychologists.***

---------------
*  Daniel Kahneman and Amos Tversky pretty much invented the conjunction fallacy thirty years ago with their “Linda problem,” and Kahneman discusses it in his recent book Thinking Fast and Slow.  To get the right answer, you have to ignore intuition and make your thinking very, very slow.  Even then, people with no background in statistics and logic may still get it wrong.

** The authors’ presentation of their results is also designed to frustrate the ordinary reader. Each condition (rapist/teacher, atheist/teacher, homosexual/teacher, etc.) had 26 (or in one case 27) subjects.  The payoff was the number of errors in each group.  But the authors don’t say what that number was.  They give the chi-square, the odds ratios, the p’s and the b’s.  But they don’t tell us how many of the 26 subjects thought that the wallet snatcher was more likely to be an atheist/teacher or a Christian/teacher than to be merely a teacher.

*** The JPSP is one of the most respected journals in the field, maybe the most respected, influential, and frequently cited, as I pointed out here.

Graphic Design the Fox News Way

December 13, 2011
Posted by Jay Livingston

The best way to lie with statistics, says Andrew Gelman, is just lie.  This graph from Fox news is a visual version of that.  It’s published at Flowingdata.com via Media Matters.

The numbers are correct, but the Foxy graphmongers are making up the Y-axis as they go along.  The 8.6% of November is higher than than 8.8%, 8.9%, and maybe even the 9.0% of the first three months of the year.

Or maybe it’s an optical illusion.


[HT:  Max Livingston]

The Descendants

December 11, 2011

Posted by Jay Livingston

Children in American movies are typically superior to adults. The kids are not only all right, they are wiser, less corrupt, and more competent. “Home Alone” is a classic example, where the plucky, resourceful kid triumphs over both the vindictiveness of the burglars and the mindlessness of his parents. (An earlier post on children in films is here.)

 “The Descendants,” the recent film with George Clooney (I saw it last night), starts more like a French film, where children are, well, children, and it’s the parents who must endure and learn to cope with the kids’ immaturity and thoughtlessness.

Clooney is Matt King, and the name is a deliberate irony. Kinglike, he must decide the fate of a huge tract of pristine Kauai land that his family has owned for many generations. The money from the sale will make him and his many cousins and their families rich. Which developer will he sell the land to?

But as a husband and father he is far being monarch of all he surveys. His wife has been in an accident and lies in a coma. His two daughters are unapologetically impudent and insufferable. As the film starts, Scottie, age ten, has sent a nasty, obscene text to a classmate. Alex, seventeen, now at an expensive private rehab/therapeutic school, first appears on screen drunk, having sneaked out of her room at night with another girl. Then there’s Sid, Alex’s friend, a slightly older boy, all stupidity and insensitivity, a chubby incarnation of Beavis and Butthead.

Then the film magically transforms the kids. Each has been introduced as obtuse, obscene, or obnoxious. But now Alex, it turns out, knows more than her father does, at least in one crucial area – that his wife, now on life support, had been cheating on him.

The kids change from being French, a burden for the grown-up, to becoming almost classically American, not superior but equal. They are now his partners. Teens and adult are a team trying to discover the identity and location of the seducer so that King can confront him. The teenagers are suddenly much less difficult and much more helpful, while King sometimes appears uncertain and even silly, peering over hedges to spy on his wife’s lover. He asks his daughter for advice. He even asks Sid what he should do.

(You can get some sense of this transformation in the trailers here and here, which also outline the rest of the story.) Still, the movie doesn’t go pure Hollywood. It does not present the world as a character contest where good faces evil, where the right action is clear and the only question is how the hero will come to make it. Instead, it shows a grown-up trying to understand and cope with problems and people he cannot really control.

And nobody blows up a helicopter.