Posted by Jay Livingston
Survey questions, even those that seem simple and straightforward, can be tricky and yield incorrect answers. Social desirability can skew the answers to questions about what you would do – “Would you vote for a woman for president. . . .?” and even factual questions about what you did do. “Don’t ask, ‘How many books did you read last year?’‘ said the professor in my undergraduate methods course. “Ask ‘Did you read a book last week?’” There’s no shame in having been too busy to read a book in a seven-day period. Besides, people’s recall will be more accurate. Or will it? Is even a week’s time enough to distort memory?
Leif Nelson (Berkeley, Business School) asked shoppers, “Did you buy laundry detergent the last time you went to the store?” Forty-two percent said yes.
Nelson doesn’t question the 42% figure. He’s interested in something else: the “false consensus effect” – the tendency to think that others are more like us than they really are.
So he asks, “What percentage of shoppers do you think will buy laundry detergent?” and he asks “Did you buy laundry detergent.” Sure enough, those who said they bought detergent give higher estimates of detergent buying by others. (Nelson’s blog post, with other interesting findings, is here.)
But did 42% of those shoppers really buy detergent last time they were in the store? Andrew Gelman is “stunned” and skeptical. So am I.
The average family does 7-8 washes a week. Let’s round that up to 10. They typically do serious shopping once a week with a few other quick express-lane trips during the week. This 50 oz. jug of Tide will do 32 loads – three week’s of washing.
That means only 33% of customers should have said yes. And that 33% is a very high estimate since most families by in bulk, especially with items like detergent. Tide also comes in 100-oz. and 150-oz. jugs.
If you prefer powder, how about this 10-lb. box of Cheer? It’s good for 120 loads.
Why then do people think they buy detergent so much more frequently? I’m puzzled. Maybe if washing clothes is part of the daily routine, something you’re always doing, buying detergent seems like part of the weekly shopping trip. Still, if we can’t rely on people’s answers about whether they bought detergent, what does that mean for other seemingly innocuous suvey questions?