But Is That True?

December 26, 2017
Posted by Jay Livingston

A New York Times op-ed this weekend by Margaret Renki (here) reminded me of an assignment I regularly give. Here’s the relevant excerpt from the Times.

Eighty percent of white born-again Christians voting in Alabama backed Roy Moore, and there is no skirting the damage they’ve done to their own moral standing.

The day of the election, the editor in chief of Christianity Today, Mark Galli, identified the biggest loser in Alabama: Christian faith itself. From now on, Mr. Galli wrote, “When it comes to either matters of life and death or personal commitments of the human heart, no one will believe a word we say, perhaps for a generation.”

Is this true? 

For the assignment, I ask students to convert a claim like this into a testable hypothesis. Students are welcome to find such statements on their own, but to make life easier, I offer several that I have collected over the years.* I may add this one to the folder. The point of the assignment is that evidence, good evidence, is always necessary, but figuring out what that evidence should be is often difficult.

Renki’s statement sounds reasonable. But what if someone disagrees? That’s why we need evidence. What if someone says, “No, the Roy Moore thing didn’t damage anyone’s moral standing except perhaps Roy Moore’s. The credibility of born-again Christians won’t change.”

For the assignment, students have to be very clear about what the variables are, and how to measure them. They have to specify the categories or values for each variable and the information they would use to decide the category or score for each unit. Don’t be be constrained by technical or realistic limits, I tell them.  You have unlimited capacity to get the information you need. You can even have a time machine if you want to get comparative data from the past or future. 

In this case, they can travel forward several years to see what has happened to the moral standing of evangelicals. The trick is to turn “moral standing” into something that can actually be measured. That’s the other point of the assignment: operationalizing a vague concept like “moral standing” is difficult. No matter how you do it, someone will be able to point out the flaws. And they’ll be right. At some point, a researcher just has to say, “Yes, I know it’s not perfect, but it’s the best I could come up with.”

* This is also an exercise in reading. Rarely do op-ed writers make a direct cause-effect statement.  Instead, that idea is often implied, mixed in among statements about what should or should not be done. A student has to be able to extract the hypothesis. Here are some examples of ideas distilled from these articles.
  • NFL officiating is bad because the refs are older and because they do this only as a part-time job. Refs should also have to face the media post-game.
  • Allowing women to serve in combat units in the Army will lead to sexual tension and lower unit cohesion.
  • Replacing tipping with a living wage and European-style service charge will be better for waiters and not reduce the quality of service.
  • Cinderella movies give girls a distorted idea of love and relationships and endangers their emotional health and self-image.
  • Banning on-field use of smokeless tobacco will improve public health because kids imitate their sports heroes.
  • “Bathroom laws” like the one in North Carolina are bad for the mental health of transgender kids.

No comments: