Confidence Games

January 19, 2019
Posted by Jay Livingston

Timing is crucial in comedy. In can be important in survey research as well. If you ask about satisfaction with government, and you take your survey at a historical moment when the Republican party controls the government, don’t be surprised if Republicans are more satisfied than Democrats. But also don’t write up your findings to imply that this means that Republicans have a deep and abiding faith in American institutions.

We’ve been here before, not with “satisfaction,” but with something similar — happiness. People who make claims about the relation between happiness and political views — people like Arthur Brooks, for example — often don’t bother to look at which party was holding sway at the time the survey they’re using was done. But that context matters a lot, especially now that the country has become so partisan and polarized, with people remaining loyal to their party the way sports fans are loyal to their team. In a post two years ago inspired by a Brooks column, I put it this way

When you’re talking about the relation between political views and happiness, you ought to consider who is in power. Otherwise, it’s like asking whether Yankee fans are happier than RedSox fans without checking the AL East standings. [the full post is here.]

I had a similar reaction to a recent thread on Twitter about who has lost confidence in American institutions. The answer is: everybody. But some more than others.  Patrick Egan of NYU looked at the “confidence” items in the General Social Survey and created these graphs showing the average confidence in twelve different institutions.

(Click on an image for a larger view.)

Confidence has dropped among all categories. But the steepest decline has come among non-college Whites. Their overall level of confidence is the lowest of any of these groups. They are also the strongest supporters of Donald Trump. This reinforces the image of the core Republican constituency — Trump’s staunchest supporters — as dissatisfied, even resentful. They have lost confidence in traditional American institutions, and they acclaim the strong outsider who could bring sweeping changes.

In response, Joshua Tucker posted a link to a report he was co-author on — the American Institutional Confidence Poll (AICP) from the Baker Center for Leadership & Governance at Georgetown University. The AICP found that demographic characteristics didn’t make much difference. Politics did. Here is AICP’s Number One Key Finding:


Why the discrepancy between the GSS data the AICP conclusions? I wondered if it might be the sample. It wasn’t.

The interviews were conducted online from June 12 to July 19, 2018, by the survey firm YouGov. The sample includes 3,000 respondents from the U.S. general population. Additionally, the poll includes samples of 800 African-Americans, 800 Latinx Americans, and 800 Asian Americans.

Their sample, as they note elsewhere, is larger than that of most political surveys, plus the  oversampling of smaller populations they want to have good data about. No problem there.

But what about the timing? We know that on November 1, 2016, Democrats were much more likely than were Republicans to say that the economy looked good. Two weeks later, those positions were reversed. The economy did not change in those two weeks. The occupancy of the White House did.

The AICP survey was done last summer, months before the midterm elections, when the GOP controlled the White House, the Senate, the House, and the Supreme Court. That seems like kind of an important fact, but to find it, you have to scroll down to the methodology notes at the end of the report. 

Even in the GSS graphs, Egan has drawn a trend line that smooths out these shifts that are possibly caused by electoral changes. Egan also has lumped together twelve institutions. Separating them in to categories (e.g,. government, non-government) might allow us to see even sharper demographic differences.

The AICP, on the other hand, does report about confidence in specific institutions, twenty in all. The authors conclude that “confidence in institutions is largely driven by party affiliation.” They neglect the corollary: who has confidence in which institution can shift quickly when an election changes the party in power. This volatility makes it a bit misleading to talk about confidence in “institutions” as though people were thinking about them in the abstract. For example, the authors say, “The executive branch is the institution in which Democrats have the least confidence, while Republicans rank it the fourth highest.” Surely this difference is not about what people think of “the executive branch.” It’s about Donald Trump. These days, isn’t everything?

No comments: