If Your Survey Doesn’t Find What You Want It to Find . . .

October 19, 2011
Posted by Jay Livingston(Cross posted at Sociological Images)


. . . say that it did.

Doug Schoen is a pollster who wants the Democrats to distance themselves from the Occupy Wall Street protesters.   (Schoen is Mayor Bloomberg’s pollster.  He has also worked for Bill Clinton.)  In The Wall Street Journal yesterday (here),  he reported on a survey done by a researcher at his firm.  She interviewed 200 of the protesters in Zucotti Park.

Here is Schoen’s overall take:

What binds a large majority of the protesters together—regardless of age, socioeconomic status or education—is a deep commitment to left-wing policies: opposition to free-market capitalism and support for radical redistribution of wealth, intense regulation of the private sector, and protectionist policies to keep American jobs from going overseas.

I suppose it’s nitpicking to point out that the survey did not ask about SES or education.  Even if it had, breaking the 200 respondents down into these categories would give numbers too small for comparison. 

More to the point, that “large majority” opposed to free-market capitalism is 4% – eight of the people interviewed.  Another eight said they wanted “radical redistribution of wealth.”  So at most, 16 people, 8%, mentioned these goals.  (The full results of the survey are available here.)

What would you like to see the Occupy Wall Street movement achieve? {Open Ended}

35% Influence the Democratic Party the way the Tea Party has influenced the GOP
4% Radical redistribution of wealth 5% Overhaul of tax system: replace income tax with flat tax
7% Direct Democracy
9% Engage & mobilize Progressives 
9% Promote a national conversation
11% Break the two-party duopoly
4% Dissolution of our representative democracy/capitalist system  4% Single payer health care
4% Pull out of Afghanistan immediately 
8% Not sure

Schoen’s distortion reminded me of this photo that I took on Saturday (it was our semi-annual Sociology New York Walk, and Zucotti Park was our first stop).

The big poster in the foreground, the one that captures your attention, is radical militance – the waif from the “Les Mis” poster turned revolutionary.  But the specific points on the sign at the right are conventional liberal policies – the policies of the current Administration.*

There are other ways to misinterpret survey results.  Here is Schoen in the WSJ:

Sixty-five percent say that government has a moral responsibility to guarantee all citizens access to affordable health care, a college education, and a secure retirement—no matter the cost.

Here is the actual question:

Do you agree or disagree with the following statement: Government has a moral responsibility to guarantee healthcare, college education, and a secure retirement for all.

“No matter the cost” is not in the question.  As careful survey researchers know, even slight changes in wording can affect responses.  And including or omitting “no matter the cost” is hardly a slight change.

As evidence for the extreme radicalism of the protestors, Schoen says,

By a large margin (77%-22%), they support raising taxes on the wealthiest Americans,

Schoen doesn’t bother to mention that this isn’t much different from what you’d find outside Zucotti Park.  Recent polls by Pew and Gallup find support for increased taxes on the wealthy ($250,000 or more) at 67%.  (Given the small sample size of the Zucotti poll, 67% may be within the margin of error.)  Gallup also finds the majorities of two-thirds or more think that banks, large corporations, and lobbyists have too much power. 

Thus Occupy Wall Street is a group of engaged progressives who are disillusioned with the capitalist system and have a distinct activist orientation. . . . .Half (52%) have participated in a political movement before.

That means that half the protesters were never politically active until Occupy Wall Street inspired them.

Reading Schoen, you get the impression that these are hard-core activists, old hands at political demonstrations, with Phil Ochs on their iPods and a well-thumbed copy of “The Manifesto” in their pockets.  In fact, the protesters were mostly young people with not much political experience who wanted to work within the system (i.e., with the Democratic party) to achieve fairly conventional goals, like keeping the financial industry from driving the economy into a ditch again.

And according to a recent Time survey, more than half of America views them favorably.

------------------------------
* There were other signs with other messages.  In fact, sign-making seemed to be one of the major activities in Zucotti Park.  Some of them. like these, did not seem designed to get much play in the media.