Wanted – Bad Research

April 22, 2013
Posted by Jay Livingston

I’m not a research director.  But if I were, I hope I wouldn’t write questions that are obviously designed to bias the results.*  And if I did ask such questions, I wouldn’t boast about it in the newspaper, especially if my stacking of the deck got barely a majority to give the answer I wanted. 

But then, I’m not Michael Saltsman, research director for the Employment Policies Institute, whose letter to the Record (formerly known as The Bergen Record) was published today.

Regarding "Most favor minimum wage hike" (Page L-7, April 18):

The recent Rutgers-Eagleton poll finding that 76 percent of New Jerseyans support a minimum wage increase only proves that incomplete poll questions yield misleading results.

My organization commissioned ORC International to conduct a similar poll regarding an increase in the minimum wage. When respondents were informed of the unintended consequences of minimum wage hikes — particularly how such hikes make it more difficult for the least-skilled to find work— 70 percent support flipped to 56 percent opposition. [emphasis added]

This consequence isn't a hypothetical: Fully 85 percent of the most credible economic studies from the past two decades indicate a loss of job opportunities following a wage hike.

Michael Saltsman
Washington, D.C. , April 18

As for the facts on the effects of an increase in the minimum wage, Saltsman’s literature review is on a par with his questionnaire construction.  Apparently he missed John Schmitt’s CEPR article from two months ago (here).    The title pretty much sums it up:

Why Does the Minimum Wage Have No Discernible Effect on Employment?

Schmitt includes this graph of minimum-wage effects from a meta-analysis.

Hristos Doucouliagos and T. D. Stanley (2009) conducted a meta-study of 64 minimum-wage studies published between 1972 and 2007 measuring the impact of minimum wages on teenage employment in the United States. When they graphed every employment estimate contained in these studies (over 1,000 in total), weighing each estimate by its statistical precision, they found that the most precise estimates were heavily clustered at or near zero employment effects.

Schmitt offers several guesses as to why employers don’t cut jobs when the minimum wage rises – maybe they raise prices, or accept a lower profit margin, or reduce the wages of better-paid employees; or maybe the increased minimum wage brings more customers, and so on.**

But regardless of the findings on minimum wage, Saltsman’s letter carries a more important if depressing message.  We try to teach our students to design good research.  We tell them that good research skills might help them get jobs.  Yet here is an example of a research-director job that depends on designing bad surveys and doing bad research. 
                                                           
------------------------------
*In his methods course, my colleague Chris Donoghue uses a made-up abortion item for teaching items that introduce bias:
“Every year in the US, over a million babies are killed by abortion. Do you agree that laws should make it more difficult to get an abortion?”

** Brad Plumer at WaPo’s WonkBlog has more on this, including a fuller discussion of Schmitt’s paper (here).