Evidence at the Upshot

March 31, 2018
Posted by Jay Livingston

“Common sense” is not evidence. Neither is “what everyone knows” or, to use a source of data favored by our president, what “people say.”  That’s one of the first things students hear in the intro sociology course. Our discipline is empirical, we insist. It is evidence-based, and evidence is something that really happened. Often you have to actually count those things.

The Upshot is the “data-driven” site that the New York Times created to compete with FiveThirtyEight. Friday, an Upshot article about marriage, social class, and college had this lede,* a six-word graf.*

Princetonians like to marry one another.

The article, by Kevin Carey, showed that students from wealthier families are more likely to be married by their early thirties than are students from the bottom fifth of the income ladder. Carey argued that the cause was “assortative mating” – like marries like – and that the pattern holds even for graduates of the same elite school – Princeton, for example. Rich Princetonians marry other rich Princetonians, says Carey. Poor Princetonians remain unmarried. In their early thirties, only a third of them were married.

(Click on the image for a larger view.)

According to Carey, the sorting that leads to mating takes place in the “eating clubs” – Princeton’s version of fraternities and sororities. Acceptance into this or that club depends in part on social class, so as Carey sees it, “Eating clubs are where many upper-income marriages begin.”

It’s logical and it makes sense. The only trouble is that Carey provides no evidence for Tiger intermarriage. That 56% of rich Princeton alums who were married by age 32-34 – we don’t know who they married. Another rich Princetonian? Maybe, maybe not. We know only that they were married, not to whom.

Oh, wait. I said Carey provided no evidence. I take that back. Here’s the second graf.

Although the university is coy about the exact number of Tiger-Tiger marriages, Princeton tour guides are often asked about matrimonial prospects, and sometimes include apocryphal statistics — 50 percent! Maybe 75! — in their patter. With an insular campus social scene, annual reunions and a network of alumni organizations in most major cities, opportunities to find a special someone wearing orange and black are many.

You don’t have to have taken a methods course to know that this is not good evidence, or even evidence at all. What people say, and even logical reasons that something should happen, are not evidence that it does happen. Carey all but admits that he has no real data on Princeton intermarriage, but that doesn’t stop him from writing about it as though it’s a solid fact.

Is it? Five years ago, a Princeton alumna, president of the class of ’77, published a letter in The Daily Princetonian giving her 21st-century counterparts this bit of advice: “Find a husband on campus before you graduate.”

The reaction was swift and predictable. Some even thought that the Princetonian had run the piece as an April Fool’s joke. Besides, people these days typically do not get married till their late twenties – at least five years after they graduate. A lot can happen to that eating-club romance in those five years.

Let me clear: the negative reaction to the letter and the median marriage age of the US population are not evidence that Princetonians are not marrying one another. But it’s just as good (or bad) as Carey’s evidence that they are.

---------------
*Using journalism jargon when I’m writing about journalism is one of my favorite affectations.

Happy Birthday, Jay Livingston (1915 - 2001)

March 28, 2018
Posted by Jay Livingston

It’s strange to go ego-surfing and the only person who turns up is some other guy. It’s only fair. He’s the one who wrote all those songs that were hits a half-century ago – “Mona Lisa,” “Tammy,” “The Mr. Ed” theme song, and of course of course, “Silver Bells.” Simple songs with simple chords.

That’s what he became known for. That’s what Hitchcock was looking for when he hired him to write a song for “The Man Who Knew Too Much.” What Livingston (and his songwriting partner Ray Evans) came up with was “Que Sera Sera.” Said Hitchcock when he heard it, “Gentlemen, I told you I didn't know what kind of song I wanted, but that's the kind of song I want.”

His only tune in the jazz standards repertoire is “Never Let Me Go,” so different from those other songs – darker, in a minor key, and with complex and unexpected chord changes. It was also his own favorite. He tells the story that after he finished it, he took it to another song writer (I can’t remember who) to ask his opinion. The other composer sat down at the piano, played through it, and said, “You didn’t write this.”

He did write one other song that a few jazzers and singers do. “Maybe September.” Here is the version from the second Bill Evans - Tony Bennett album, 1977. There may be other recordings with a better vocal, but this one has Bill Evans.

“Concerted Cultivation” and the March For Our Lives

March 25, 2018
Posted by Jay Livingston

A question that few people seem to be asking about Enough Is Enough and the March for Our Lives is: Why now? Or to paraphrase a question that some people soon will be asking: How is this school shooting different from all other school shootings?

There’s #MeToo and #Time’sUp of course. These may have inspired advocates of other liberal causes like gun control. But just three weeks earlier, a 15-year old in Benton, Kentucky brought a handgun to school and started shooting – 2 dead, 18 injured. The incident evoked only the usual responses, nothing more.

Here’s my hunch. When I first saw the kids in Parkland speaking out, organizing, demanding that adults do something, I immediately thought of a sociology book that had nothing to do with guns – Unequal Childhoods by Annette Lareau published in 2003.

These high-schoolers, I thought, are the children of “concerted cultivation.” That was the term Lareau used for the middle-class approach to raising kids. It’s not just that middle-class parents cultivate the child’s talents, providing them with private coaches and organized activities. There is less separation of child worlds and adult worlds. Parents pay attention to children and take them seriously, and the children learn how to deal with adults and with institutions run by adults.

One consequence is the notorious sense of “entitlement” that older people find so distressing in millennials. Here is how Lareau put it:

This kind of training developed in Alexander and other middle-class children a sense of entitlement. They felt they had a right to weigh in with an opinion, to make special requests, to pass judgment on others, and to offer advice to adults. They expected to receive attention and to be taken very seriously.

It is this sense of entitlement – the teenager’s sense that she is entitled to have some effect on the forces that affect her life – that made possible the initial protests by the students at Marjory Stoneman Douglas High School. And once word of that protest spread, it was this same sense of entitlement, these same assumptions about their place in the world, that made so many other high school students join the movement.

At Fox and elsewhere, conservatives had to denigrate the students. That goes without saying. But their choice of lies and slurs is instructive. Conservatives just could not believe that kids could or should be so adept at mounting an effective movement or that they could or should speak intelligently about political issues. So the right-wingers insisted that the students were paid “crisis actors” or pawns of various forces of evil – adult anti-gun activists, the media, the deep state, etc. They also claimed that the students were “rude” and that they did not have standing to raise the issue of gun control.

“[the students] say that they shouldn’t be able to own guns even though they can go to war, but they think that they should be able to make laws. None of this makes any sense at all.

(For this and more really stupid Fox commentary the day before the march, see the excerpts in the transcript here.)

In a way, Fox and their friends are hauling out the old notion that children should know their place. But it’s not necessarily do-as-you’re-told authoritarianism. The issue isn’t the child’s independence. As Lareau says, concerted cultivation makes children far more dependent on parents than does the “natural growth” parenting more common in working-class families. Besides, foreign visitors since the early days of the republic have remarked on the independence of American children. What’s new, and what is so upsetting to exponents of older ideas, is the demand by teenagers to be involved in the decisions that shape their lives.

Sondheim – “The Glamorous Life”

March 22, 2018
Posted by Jay Livingston

It’s Stephen Sondheim’s birthday – he’s a pianistic 88.

It’s hard to write about Sondheim without repeating what are by now cliches. But cliches (which Sondheim scrupulously avoids) usually contain some truth, and the one that sticks in my mind is this: Sondheim brings ambivalence to center stage. Sometimes it’s right there in the title of a song. In “Company,” in answer to the question, “Are you ever sorry you got married?” a character sings “Sorry Grateful” (“You’re always sorry, you’re always grateful, you’re always wondering what might have been.”)

In other songs, the ambivalence is subtext, as in “The Glamorous Life” from “A Little Night Music.” Frederika, young teenage girl, is singing about the glamorous life of her actress mother compared with the lives of “ordinary mothers.” It’s no contest, and yet, under the protestations of how wonderful it is to have a glamorous mother, Frederika senses that she’s missing something.

Ordinary daughters may think life is better
With ordinary mothers near them when they choose. . .

No, ordinary mothers merely see their children all year,
Which is nothing, I hear…
But it does interfere
With the glamorous—

Here is the great Audra McDonald performing the song at a celebration for Sondheim’s 80th birthday. Yes, there’s her powerful voice, but she also perfectly reveals the ambivalence. (I don’t see how an actual teenage girl could sing this song – the version in the movie of “Night Music” is not good – especially if she has ever heard this performance by McDonald.)

(The original Broadway version of the song was far different, an ensemble piece combining other songs performed by other characters. Sondheim rewrote the song for the movie so that it presented only Frederika's perspective.)

Connecting the Dots

March 22, 2018
Posted by Jay Livingston

Brilliance in science is sometimes a matter of simplifying – paring away complicated scientific techniques and seeing what non-scientists would see if they looked in the right place. That’s what Richard Feynman did when he dropped a rubber ring into a glass of ice water – a flash of brilliance that allowed everyone to understand what caused the space shuttle Challenger disaster.

Andrew Gelman isn’t Richard Feynman, but he did something similar in his blog post about an article that’s been getting much buzz, including at Buzzfeed, since it was posted at SSRN two weeks ago. The article is about Naloxone, the drug administered to people who have overdosed on heroin or opoiods. It keeps them from dying.

The authors of the article, Jennifer Doleac and Anita Mukherjee, argue that while the drug may save lives in the immediate situation, it does not reduce overall drug deaths. Worse, the unintended consequences of the drug outweigh its short-run benefits. Those whose lives are saved go back to using drugs, committing crimes, and winding up in emergency rooms. In addition, a drug that will prevent overdoes death “[makes] riskier opioid use more appealing.” 

The title is “The Moral Hazard of Lifesaving Innovations: Naloxone Access, Opioid Abuse, and Crime.” (A moral hazard is something that encourages people to do bad things by protecting them from negative consequences.)

Naloxone didn’t happen all at once. In 2013 fewer than ten states allowed it; the next year the number had doubled. In 2015, only nine states still did not allow its use. Doleac and Mukherjee used these time differences to look at bad outcomes (theft, death, ER admissions) before and after the introduction Naloxone in the different states.  Here are some of their graphs.

(Click on an image for a larger view.)

They conclude that “broadening Naloxone access led to more opioid-related ER visits.” As for deaths, “in some areas, particularly the Midwest, expanding Naloxone access has increased opioid-related mortality.”

There are reasons to be skeptical of the data, but let’s assume that the numbers – the points in the graph – are accurate. Even so, says Andrew Gelman (here), there’s still the question of how to interpret that array of points. Doleac and Mukherjee add lines and what I assume are confidence bands to clarify the trends. But do these added techniques clarify, or do they create a picture that is different from the underlying reality? Here’s Gelman:

The weird curvy lines are clearly the result of overfitting some sort of non-regularized curves. More to the point, if you take away the lines and the gray bands, I don’t see any patterns at all! Figure 4 just looks like a general positive trend, and figure 8 doesn’t look like anything at all. The discontinuity in the midwest is the big thing—this is the 14% increase mentioned in the abstract to the paper—but, just looking at the dots, I don’t see it.

Are these graphs really an optical illusion, with the lines and shadings getting me to see something that isn’t really there? My powers of visualization are not so acute, so to see what Gelman meant about looking only at the dots, I erased the added lines and bands. Here is what the graphs looked like.

Like Gelman, I can’t see any clear patterns showing the effect of Naloxone. And as I read the reactions to the paper, I sense that its results are ambiguous enough to provide rich material for motivated perception. Conservatives and libertarians often start from the assumption that government attempts to help people only make things worse. The unintended-consequences crowd – Megan McCardle, for example (here) – take the paper at face value. Liberals Richard G. Frank, Keith Humphreys, and Harold A. Pollack (here), who have done their own research on Naloxone – are more skeptical about the accuracy of the data.*

----------------

* This reminded me of a post I did in the first year of this blog.  It was about an editorial in the WSJ that included an utterly dishonest, ideologically motivated connect-the-dots line imposed on an array of points. The post is here.

Le Bron and Grades

March 21, 2018
Posted by Jay Livingston

I’ve never used those percentages in grading – you know, 94% is an A, 82% is a B– and so on. Eighty-two percent of what, I wonder, especially on essays? The percent may be just the same subjective judgment of an essay, but one that makes it seem more precise and objective. You can’t argue with numbers. 

I do use points – a point for each multiple-choice or short answer, so many points for a short essay, more for a long essay – to weight different parts of the exam. But I don’t use a standard formula for converting exam scores to letter grades. Instead, I adjust my own idea of what the grades should be to the actual distribution of scores in the class.

I explain this on the first day of class, and I post a document (“How I Grade”) on Canvas at the start of the semester, but apparently neither of those is very convincing. Students see their scores on Canvas, which also converts these to percents. The student who sees 55% on Canvas comes to class convinced it’s a failure. Even when I give them my own number-to-letters conversion for the exam, some remain skeptical. It’s as though I were singing my own made-up lyrics to some well-known song. They hear, but deep down they know, “Those aren’t the real words.”
                                                   
I came up with a new strategy this week when I gave back the midterms. “Who’s the best basketball player in the NBA?” I asked. Everyone knew: LeBron James.

“What’s LeBron’s field-goal percentage?” The few students who had some idea lowballed it, guessing 40-45%. “Actually, he’s having a good year,” I said. “He’s hitting about 55%.” Then. “So I guess that means LeBron gets an F in basketball.”

That analogy may have done the trick – that, plus a letter grade that was higher than what they had expected.

Drugs and Death Penalties

March 13, 2018
Posted by Jay Livingston


It’s a sure sign of desperation when a politician calls for the death penalty for drug dealing.

Trump is basically admitting that his administration has no idea how to solve the drug problem.

Two hundred fifty years ago, Cesare Beccaria argued that of the three elements of punishment – certainty, swiftness, and severity – severity is the least effective. All the subsequent research has proven him right. But certainty and swiftness are hard to increase; severity is easy. Just pass some laws imposing long sentences, mandatory sentences, life sentences, and of course, death. When politicians call for the death penalty what they are saying is, “We don’t know how to catch very many of these guys, and it takes a long while before they are actually sentenced, so when we do catch one of them, we’re going to show him how pissed off we are.”

Draconian punishments may be very good for expressing the frustration and anger of law-abiding people, and Trump is very good at playing to those emotions. But as for the practical effects, executions are unlikely to have much of an impact on crime. What was true in in the crack crisis of the 1980s is still true: there already is in fact a death penalty for drug dealers. It’s just not administered by the state. It’s administered by rival drug dealers. And compared with any death penalty the state might impose, it is carried out with far greater frequency and swiftness.

President Reagan was fond of saying that in the sixties we fought a war on poverty, and poverty won. He was factually wrong. But if he had made the statement about the war on drugs that the government waged in the following decades, he’d have been closer to the truth. Those years when drugs were winning the war also gave us the spectacle of politicians falling all over themselves to pass harsher and harsher drug laws. Conservative politicians them sounded very much like Trump today. And like Trump today, many of them, perhaps, thought that in this way they were “doing something” about drugs. Of course, what they were more certain of was that they were doing something about getting re-elected.

Ass-Backwards Through the Gateway

March 11, 2018
Posted by Jay Livingston

Imagine that you’re a US Attorney on the drug beat. Your boss is Jeff Sessions, who has announced that he’s going to vigorously enforce laws against marijuana and use the federal law when state laws are more lax. Maybe you also think that weed is a dangerous drug. You do a little “research” and tweet out your findings.

This brief tweet might serve as an example of how not to do real research. The sample, which excludes people who have not gone to treatment centers, is hardly representative of all users. There’s researcher bias since the guy with the ax to grind is the one asking the questions. The respondents too (the drug counselors) no doubt feel some pressure to give the Sessions-politically-correct answer. They may also be selectively remembering their patients. 

But even without the obvious bias, this tweet makes an error that mars research on less contentious issues. It samples on the dependent variable. The use of heavier drugs (opioids, heroin, meth, etc.) is the dependent variable – the outcome you are trying to predict. Marijuana use is the independent variable – the one you use to make that prediction. Taking your sample from confirmed heroin/opioid addicts gets things backwards. To see if weed makes a difference, you have to compare weed users with those who do not use and then see how many in each group take up more serious drugs.

Here’s an analogy – back pain. Suppose that, thanks to advances in imaging (MRIs and the like) doctors find that many of the people who show up with back pain have spinal abnormalities, especially disk bulges and protrusions. These bugles must be the gateway to back pain. So the doctors start doing more surgeries to correct these bulges. These surgeries often fail to improve things.

The doctors were sampling on the dependent variable (back pain), not on the independent variable (disk bulges). The right way to find out if spinal abnormalities cause back pain is to take MRIs of all people, not just those who show up in the doctor’s office. This is pretty much the way it happene in the real world. Eventually, researchers started doing the research the right way and found that lots of people with spinal abnormalities did not pass through the gateway and on to back pain.

The same problem often plagues explanations that try to reverse-engineer success. Find a bunch of highly effective people, then see what habits they share. Or look at some highly successful people (The Beatles, Bill Gates) and discover that early on in their careers they spent 10,000 hours working on their trade.

US Atty. Stuart’s tweet tells a good story, and it’s persuasive. But like other anecdotal evidence and eyewitness testimony, it is frequently misleading or wrong. The systematic research – many studies over many years – shows little or no gateway effect of marijuana. No wonder US Attorney Stuart chose to ignore that research.*

----------------------------
* As Mark Kleiman has argued, even when a marijuana user does add harder drugs to his repertoire, the causes may have less to do with the drug itself than with the marketplace. The dealer you go to for your weed probably also carries heavier drugs and would be only too happy to sell them to you.  Legalizing weed so that it’s sold openly by specialty shops rather than by criminals may break that link to other drugs.

This Is Not a Pipe . . .

March 7, 2018
Posted by Jay Livingston

. . .  But what is it? Is there a word for a statement that contradicts the assertion in the statement?  For example, one of Trump’s better lines at the Gridiron Club was this.

My staff was concerned that I couldn’t do self-deprecating humor, and I told them not to worry, nobody does self-deprecating humor better than me. It’s not even close.

Not a new joke, but a good one, especially for Trump.

Calling it self-contradiction does not really convey the delightful irony. Trump contradicts himself all the time, but usually the contradiction is not inherent in the statement itself as it is in this instance.

It’s kinda sorta like apophasis (my favorite rhetorical term), but it’s different. I posted about it last summer (here) with Magritte’s painting and this unwitting example that was going around the Internet.

I still haven’t found le mot juste for it.