Sampling — the General Idea

March 17, 2020
Posted by Jay Livingston

Need a current example for the unit on sampling, day one? Read on.

Today, NPR tweeted the results of a NPR/PBS NewsHour/Marist poll on perceptions of the coronavirus.

“This survey of 835 adults was conducted Friday and Saturday using live telephone callers via landline and cellphone. It has a margin of error of +/- 4.8 percentage points.”

It found that since last month, the percent of Americans who thought that the virus was a “real threat” had fallen from 66% to 56%, barely more than half. The decline was especially steep among Republicans – from 72% to 40%

Most of the Twitter comments critical of the NPR tweet were political, echoing Trump’s “media hoax” claims of last week. But one of them was methodological.  

(I wouldn't trust this poll.
Looking at the methods used it isn't like they

 asked the same 835 adults for the poll that 

they asked last month. Not terribly reliable.)

Yes, asking the question of the same sample would be ideal. But is it really necessary?

Apparently, the concept of “representative sample” is not intuitively obvious. My favorite illustration is the army general in the Pentagon who was presented with the results of a survey and informed that these were based on the responses of 1500 soldiers. He was incredulous. After all, there were 300,000 in uniform. How could this sociologist know what they were thinking and doing by asking not even one percent?

“How many should we survey?” asked the sociologist.

“You gotta ask ’em all.”

The conversation then proceeded something like this:

“General, do you ever go to the doctor for a physical?”

“Yep, every year.”

“And to find out your cholesterol levels and other things, does he take blood?”

“Sure.”

“Well, how much of your blood does he take?”

“You know, just that little tube, maybe an ounce or two?”

“So, do you tell him that if he really wants to know the percent of cholesterol in your blood, he’s going to have to take more than that little tube; he’s going to have to take it all?”

I remembered this anecdote from a small book on sampling that some publisher sent decades ago. I can’t recall the author’s name, and I may have gotten some of the details wrong. Nor do I remember if he said what the general’s response was. My guess is that the general sort of got the idea of sampling but still suspected that there was something fishy about it.