Today’s Big Match-Up

February 3, 2019
Posted by Jay Livingston

It’s Superbowl Sunday, and this year we’re about to see a contest between two rivals that have met several times previously on this blog. No, not the Rams and the Patriots, not exactly. It’s The Wisdom of Crowds versus The Smart Money.

The theory of the wisdom of crowds says that the average guess of all the interested participants is better than the guesses of the experts. The full title of James Surowiecki’s 2004 book on the topic is The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations. He begins with the famous anecdote of Galton at the fair. Here’s a summary from an earlier blog post on the topic.

Plymouth, England, 1906. On display is an ox, slaughtered and dressed. How much does it weigh? Fairgoers submitted their guesses. A statistician, Francis Galton, happened to be there and recorded the data. Galton was also a eugenicist, so he was certain that the guesses of the masses would be less accurate than those of the experts. But it turned out that the crowd, as a group, was far more accurate. The average of all the guesses (n=787) was within one pound of the actual weight (1,198 lbs). No individual guess came that close.

Surowiecki doesn’t say much about sports betting, unless you consider ox-weight estimation to be a sport. But my immediate reaction was that if Surowiecki is right, then bookmakers should be an endangered species, constantly paying out on many bets and collecting few. Not a good business model.

Sports books are experts. They set a line that they think will bring in an equal amount on both sides.* They often assume that the public will share their views on the abilities of the teams, and often they are right. But sometimes, the public thinks that the bookmakers are wrong and bet so much on one team that the books have to adjust the point spread to bring in more action on the other side.

This year, bookmakers judged the Rams and Patriots as evenly matched. The opening line on the Superbowl was pick-’em. Neither team was favored. (A small number of books had the Rams as a 1-point favorite, a few others had the Patriots by one.) The crowd roared in on the Patriots, and the books quickly raised the line to New England minus 2½. Bet the Rams, and you start the game ahead by that many points. Or bet them without points and get $120 for a $100.

Even that couldn’t attract enough money on LA.  Bookies are holding three times as much money on the Pats as on the Rams. On Thursday, a high roller bet $2 million on the Rams at the MGM, and that still didn’t offset the New England money. If the Rams win and MGM has to pay out that $2M, it will still finish well in the black from all the losing bets from Patriots backers.

It’s not just the oddsmakers who think the crowd is wrong. The “sharps,” professional gamblers who make a living from sports betting,** are also hitting the Rams — just not in large enough amounts to balance the millions of dollars coming in on the Pats.

I am posting this four hours before kickoff, and perhaps the crowd will move in with lots of money on the Rams, but I doubt it. If things stay as they are, today’s game is a good example of The Wisdom of Crowds vs. The Smart Money. (Of course, it is only  a single data point and by itself will prove nothing.)
 
UPDATE: The crowd was wise. The Patriots won 13 - 3. The crowd was also wise on the over/under which started at about 58, but the crowd, betting heavily on the under, brought it down a couple of points.




------------------------------
* Most point-spread bets are 11-10 — the bettor wagers $110 to win $100. If the action is evenly divided, the book makes money no matter who wins, paying out $100 to each winner and collecting $110 from each loser.

** The guy who made the $2M bet is not considered a sharp, even though he won a very large bet last year when he took the Eagles over the Patriots in last year’s Superbowl.

The First Derivative of the Wisdom of Crowds

February 2, 2019
Posted by Jay Livingston

If this is Superbowl weekend, then the Socioblog’s fancy must be turning to thoughts of the wisdom of crowds vs. the smart money.  It’s a question I have returned to several times since the first year this blog was on the field. (See. for example, this post about the 2010 Superbowl.)

The “wisdom of crowds” is like the Ask-the-Audience option in “Who Wants to Be a Millionaire.” The “smart money” is like Phone-a-Friend — a friend who knows a lot about the subject.

The trouble with the wisdom of crowds is that sometimes the crowd is wrong, as it was in the 2007 NFC championship game between the Bears and the Saints that I blogged about at the time (here.)

Now, a trio of academics — John McCoy (marketing), Dražen Prelec (management), and  H. Sebastian Seung (neuroscience) — has a variation that allows you to derive the right answer from the crowd even when the crowd is wrong. You might call it the first derivative of crowd wisdom.

Is Philadelphia the capital of Pennsylvania? 

Suppose you don’t know, and you ask the crowd.

The correct answer is no. The capital is Harrisburg. But many people think it is, because Philadelphia is a large, populous city. Most people know about Philadelphia. When you ask that question to a crowd of people, as we did with MIT students, only about a third of the crowd gets the correct answer.*

Yes is the popular answer. The crowd, by two-to-one, says Yes, Philadelphia is the capital. The crowd is wrong. The capital of Pennsylvania is Harrisburg. So much for the wisdom of crowds.

Wait, not so fast, say McCoy and his colleagues. Let’s also ask another question: “What percent of people do you think will answer No to this question?” The average estimate is 23%. But in fact, 33% answer No. This makes No a “surprisingly popular” answer, surprising in that more people than expected say No. It’s as though you are taking the first derivative of crowd wisdom rather than the wisdom function itself.

If you went with the popular answer, you’d say Yes and be wrong. But if you go with the derivative — the “surprisingly popular” answer —  you’ll get it right.

McCoy sees applications of this to all kinds of forecasts — the market for some product, the price of gold, voting, He doesn’t mention the Superbowl. Right now, about 25% of bettors think that the Rams will win or that they will lose by 2 points or less. But suppose we asked all bettors, “What fraction of people do you think are betting the Rams?” If they guessed that only 10% of them are backing the Rams, then the Rams would be the “surprisingly popular” choice, and you would be a fool not to put down a grand to win $1250. Alas, I know of no such surveys. Besides, I don’t trust Belichick.

Two other thoughts:
First, McCoy’s makes the concept harder to understand by choosing an example where No is right. “Is No the correct answer?” “Yes, No is right.”

Second, I was stunned that two-thirds of MIT students did not know the capital of fifth most populous state in the country. Look, people, we’re not asking about Pierre or Carson City. This is not rocket science. And now I get the feeling that at MIT a question about rocket science might have gotten a higher proportion of correct answers.

-----------------------------
*From an interview with McCoy on a Wharton School podcast. An article by Prelec, Seung, and McCoy in Nature is here behind a paywall.

Michel Legrand — 1932 - 2019

January 28, 2019
Posted by Jay Livingston

As a callow youth, I dismissed Michel Legrand as a guy who wrote pretty tunes and scores. In fact, he did write 200 or more scores for movies and TV. Then I heard Legrand Jazz (1958) with his arrangements for three different groups — 1. a big band, 2. a group centered on Ben Webster and four trombones, and 3. the core of Miles Davis’s 1958 sextet (Miles, Coltrane, Bill Evans, Paul Chambers), Phil Woods instead of Cannonball, plus a few other instruments including flute and harp.

The tunes were  eleven jazz classics, from Dixieland (Louis Armstrong’s “Wild Man Blues” ) through swing (Benny Goodman, “Stompin’ at the Savoy”) to bebop (Dizzy’s “Night in Tunisia”). The arrangements were “far out” for the time, and even 60 years later, they hold up well.

Legrand later described the recording session with the Miles.

Everyone said to me: “Miles will come to the meeting and stand near the door, keeping his trumpet in his closed case. He will listen for five minutes, and if he likes music, he will sit down, open his case, and play. If he does not like, he will leave and he will never again contact you.” I was so afraid that I had flare-ups of sweat! I started rehearsing with the orchestra. The door opened, and Miles listened by the door for five minutes. Then he sat down, opened his case and began to play. After the first catch, he asked me, “Michel, is my game [playing] suitable?” That is how it all began.

Here is that group playing Fats Waller’s “Jitterbug Waltz.”

The record went out of print and out of sight. I rarely met anyone, jazzers included, who knew of it. It was something of a collector’s item. Somewhere along the line, I lost my copy and wound up paying the equivalent today of about $75 for a used copy. Eventually, the album was re-issued as a CD, and now you can stream it anytime, anywhere.

Flashback Tuesday — MLK - AOC Edition

January 22, 2019
Posted by Jay Livingston

Alexandria Ocasio-Cortez was on Colbert last night. Colbert asked her about accusations that she was divisive. Said AOC:

Today is Martin Luther King day, and people called Martin Luther King divisive in his time. We forget that he was wildly unpopular when he was advocating for the Civil Rights Act.

She was right, sort of. King’s “unfavorables” were in the 40-45% range in the years AOC is talking about, much lower than Trump’s “unfavorables” today. But the favorable/unfavorable ratio did not turn positive until after he had been assassinated.

Here is part of a blog post I did seven years ago. The latter part of the post, about Reagan and Buckley, is not exactly relevant, but I’m leaving it in as a corrective to possible inaccuracies about them too.

*     *     *     *     *     *     *    

In August, Gallup (here) published some of their polling from the 1960s. The contrast with opinions today, when only 4% are unfavorable, is remarkable.

(Click on the chart for a larger view.)

(Note: these results include all races. The data for Whites only would surely show a higher percent unfavorable and a lower percent favorable.)

Except for 1966, the total favorable and unfavorable are fairly close.  (The change in 1966 is a result of King’s opposition to the Vietnam war.  He was right about that too.)  But of those with strong opinions, the “highly unfavorables” always outnumber the “highly favorables.” 

The unfavorables weren’t just those rabid Southern whites so familiar from the historical news footage. The same ideas could be found among seemingly temperate, sophisticated, and intellectual conservatives. Affable Ronald Reagan opposed the Civil Rights Act of 1964 and the Voting Rights Act of 1965. 

In 1957, William F. Buckley, Jr. supported the suppression of black votes in the South

The central question that emerges . . . is whether the White community in the South is entitled to take such measures as are necessary to prevail, politically and culturally, in areas in which it does not predominate numerically? The sobering answer is Yes – the White community is so entitled because, for the time being, it is the advanced race.  (The full article is excerpted by Brad DeLong here.)

That was before the rise of the Civil Rights movement.  Six years later, when Dr. King had come to prominence, a black church in Birmingham was firebombed.  Four young girls died.  Here is how Buckley’s National Review responded.

The fiend who set off the bomb does not have the sympathy of the white population in the South; in fact, he set back the cause of the white people there so dramatically as to raise the question whether in fact the explosion was the act of a provocateur – of a Communist, or of a crazed Negro. Some circumstantial evidence lends a hint of plausibility to that notion, especially the ten-minute fuse (surely a white man walking away from the church basement ten minutes earlier would have been noticed?). And let it be said that the convulsions that go on, and are bound to continue, have resulted from revolutionary assaults on the status quo, and a contempt for the law, which are traceable to the Supreme Court’s manifest contempt for the settled traditions of Constitutional practice. [emphasis added]

The suggestion that the firebombing was committed by “a communist or a crazed Negro” is a fantasy of pure desperation and wish-fulfillment.  Note also NR’s concern for “the cause of white people.”  As for the church bombing, the beatings, the tortures, the murders, and other acts of terrorism (“convulsions” as the NR calls them), committed against blacks and civil rights workers, just blame it all on the Supreme Court. 

All this would be laughable if the events were not of such grave importance and if the commentary were from some obscure, racist corner.  But National Review, then as now, was the main voice of intellectual conservatism. 

Eugene Volokh, in an appreciation of Buckley (here), notes that it wasn’t until the late 1960s, after the passage of the major civil rights laws and probably after the King and RFK assassinations, that Buckley and NR finally gave up defending segregation.  Volokh also says, approvingly,

Buckley tried very hard to create a genial and friendly image for conservatism as opposed to one that projected anger, intolerance, and rage.

Michael Harrington put it somewhat differently:

William Buckley is an urbane front man for some of the most vicious emotions in this country.