Posted by Jay Livingston
1. T / F ____ Most universities are now in the final exam period.To answer Question #2 correctly, to say that negatively-phrased questions are more confusing, you have to go through the mental contortion of negating the negative.*
2. T / F ____ A negatively-phrased question is rarely less confusing than a positively-phrased question.
Take a look at the test-bank that accompanies a textbook, and you’ll see at least a few items like this. Those questions are not written by professional test-makers. Sociology textbook test banks are written by sociology instructors, history test banks by history instructors, and so on. Neither they nor the authors of the books themselves are schooled in writing test questions.
But what about this item?
Agree or Disagree: My home life is rarely stressful.
Maybe you recognized it. It’s from the GSS (STRSSHME). A student in my class had used it in her cross-tab exercise. She had thought that women would be much more likely than men to experience stress at home. But, she said showing me her table, 43% of women disagreed; only 28% of men.
I had to look twice at the item and think it through carefully. The item is about stress, I explained, but if you want to say that you agree that your home life is stressful, you have to disagree with the question.
I assume that the GSS questions are written by people who know what they are doing, not instructors who need to supplement their income by writing textbook supplements. I also assume that the survey experts at the GSS test drive each item before including it in the interview schedule. But STRSSHME makes me less confident about the way the GSS develops questionnaire items.
Did the GSS compare this item against the same idea phrased positively:
My home life is often stressful.
No. STRSSHME seems to be part of a 2002 module that was given only once. I wonder if the GSS will use this question again.
*Another post on negativity is here.
The GSS should know better.
ReplyDeleteI take questions from test banks for my intro classes. It's amazing how poorly phrased many of the questions are. And often it is simple to rephrase those question into positives. So I do so. But would it really be too hard for the question writers to figure that out?
That question is in the middle of a series of questions that alternate between agreement indicating more and less stress in particular settings. I believe this is done intentionally to prevent distortions from people who tend to simply agree with questions no matter what their content; this problem is discussed many places, but for example in Ross and Mirowski's 1984 paper in the Journal of Health and Social Behavior.
ReplyDeleteGeoffrey, Thanks for the info. That's the great thing about blogging -- there are all these people so much better informed than I am so I find out just how ignorant I am. I do wonder if they test the questions in split sample tryouts with half getting the positive phrasing.
ReplyDelete