Survey This

October 4, 2007
Posted by Jay Livingston

Often in class when I ask students how they might find out about some variable, their response is, “Do a survey.” It’s almost as though a survey were a magical rite with mystical powers able to reveal the unknowable. Maybe some of them are, but the reality – the way many surveys are actually done – has made me a bit skeptical.

I was polled twice yesterday. My phone number must be on a “do-call” list for pollsters. I don’t mind. In fact, I find it interesting to be on the other side of the questionnaire. When the interviewer asks if I’d be willing to participate, I say, “Yes, if you’d be willing to answer some of my questions when we’re done.”

I usually ask the same questions. Last night for example, I discovered that my interviewer was in New Mexico, though he was asking me about a court case – a complicated civil suit – in New York. He was getting $6 an hour, which is maybe why the polling company hires people in Las Cruces rather than in Las Bronx. In three nights of calling, he’d completed five interviews and had a lot of refusals. So we respondents were not exactly a random sample. He didn’t know whether it was the defendant or one of the plaintiffs who was footing the bill for this research, which we agreed was probably good methodology.

The case was complex – it involved at least four “parties,” verbal agreements vs. written ones, and multiple deals that were contingent on other deals. I’d tell you more but I promised I’d keep mum till after the trial. Besides, I’m still not sure I understand it. The questions setting up the case were long and involved, and my interviewer (a college kid studying athletic management) read through them at verbal warp speed. I was surprised that anyone would respond. Or if they did respond, whether they knew what they were responding to. It wasn’t until he’d gone through three or four questions that I was able to form even a murky picture of the case. I'm still not sure I have the names straight.

On the basis of this, I thought, they’re going to decide who to select for the jury and how to present the case. And then I wondered: if they lose, are they going to sue the company that sold them on this survey?