Tide and Time

June 4, 2014
Posted by Jay Livingston

Survey questions, even those that seem simple and straightforward, can be tricky and yield incorrect answers.  Social desirability can skew the answers to questions about what you would do – “Would you vote for a woman for president. . . .?” and even factual questions about what you did do.  “Don’t ask, ‘How many books did you read last year?’‘ said the professor in my undergraduate methods course. “Ask ‘Did you read a book last week?’” There’s no shame in having been too busy to read a book in a seven-day period. Besides, people’s recall will be more accurate.  Or will it? Is even a week’s time enough to distort memory?

Leif Nelson (Berkeley, Business School) asked shoppers, “Did you buy laundry detergent the last time you went to the store?” Forty-two percent said yes.

Nelson doesn’t question the 42% figure. He’s interested in something else:  the “false consensus effect” – the tendency to think that others are more like us than they really are.

So he asks, “What percentage of shoppers do you think will buy laundry detergent?” and he also asks “Did you buy laundry detergent.” Sure enough, those who said they bought detergent give higher estimates of detergent buying by others. (Nelson’s blog post, with other interesting findings, is here.)

But did 42% of those shoppers really buy detergent last time they were in the store? Andrew Gelman is “stunned” and skeptical. So am I.

The average family does 7-8 washes a week. Let’s round that up to 10.  They typically do serious shopping once a week with a few other quick express-lane trips during the week.  This 50 oz. jug of Tide will do 32 loads – three week’s of washing.

That means only 33% of customers should have said yes.  And that 33% is a very high estimate since most families buy in bulk, especially with items like detergent. Tide also comes in 100-oz. and 150-oz. jugs.

If you prefer powder, how about this 10-lb. box of Cheer? It’s good for 120 loads. 

A family should need to buy this one in only one out of 12 trips. Even at double the average washing, that’s six weeks of detergent. The true proportion of shoppers buying detergent should be well below 20%.

Why then do people think they buy detergent so much more frequently?  I’m puzzled.  Maybe if washing clothes is part of the daily routine, something you’re always doing, buying detergent seems like part of the weekly shopping trip. Still, if we can’t rely on people’s answers about whether they bought detergent, what does that mean for other seemingly innocuous survey questions?