Posted by Jay Livingston
The Rangers crushed the Canadiens convincingly in game one: 7-2. The question was whether that result could be replicated . . . three more times.
Replication is hard (as the Rangers and their fans discovered in overtime at the Garden last night). That’s true in social science too. The difference is that the results of the Rangers’ failure to replicate were published. Social psychologists are now paying more attention to the replication question. In the Reproducibility Project, Brian Nosek and others have set about trying to replicate most of the studies published in three top journals since 2008. The first round of results was encouraging – of thirteen attempts, ten were consistent with the original findings. In one case, an “anchoring” study by Daniel Kahneman, the effect was stronger than in the original.
What failed to replicate? Mostly, experiments involving “priming,” where subliminal cues affect people’s ideas or behavior. In the best known and now most controversial of these, participants were primed by words suggesting old age (wrinkles, bingo, alone, Florida). They were then surreptitiously timed as they walked down the hall. In the original study by John Bargh (the priming primus inter pares), participants who were primed walked more slowly than did the controls.*
Many people have tried to replicate this study, and the results are mixed. One problem might be a “Rosenthal” effect, where the experimenters unintentionally and unknowingly influence the participants’ behavior so that it conforms with their expectations. Double-blind experiments, where the experimenters don’t know which participants have been primed, do not produce significant differences. (More here.)
I had a different explanation: some guys can prime; some can’t.
Maybe John Bargh and his assistants are really good at priming. Somehow, when they give participants those words mixed in among others, the subjects get a strong but still subliminal mental image of wrinkled retirees in Miami. But other psychologists at other labs haven’t got the same touch. Unfortunately, the researchers did not use an independent measure of how effective the priming was, so we can’t know.
I was delighted to see that Daniel Kahneman (quoted here ) had the same idea.
| The conduct of subtle experiments has much in common with the direction of a theatre performance . . . you must tweak the situation just so, to make the manipulation strong enough to work, but not salient enough to attract even a little attention . . . .Bargh has a knack that not all of us have. |
Many social psychology experiments involve a manipulation that the participant must be unaware of. If the person catches on to the priming (“Hey, all these sentences have words with a geezer theme,”), it blows the con. Some experiments require more blatant deceptions (think Milgram), and not all psychologists are good deceivers.
What reminded me of this was Eliot Aronson’s memoir Not by Chance Alone. Aronson is one of the godfathers of social psychology experiments, and one of his most famous is the one-dollar-twenty-dollar lie, more widely known as Aronson and Carlsmith, 1963. Carlsmith was J. Merrill Carlsmith. The name seems like something from central casting, and so did the man – a polite WASP who prepped at Andover, etc.
In the experiment, the subject was given a boring task to do – taking spools out of a rack and then putting them back, again and again, while Carlsmith as experimenter stood there with a stopwatch pretending to time him. The next step was to convince the subject to help the experimenter.
| [Merrill] would explain that he was testing the hypothesis that people work faster if they are told in advance that the task is incredibly interesting than if they are told nothing and informed, “You were in the control condition. That is why you were told nothing.” At this point Merrill would say that the guy who was supposed to give the ecstatic description to the next subject had just phoned in to say he couldn’t make it. Merrill would beg the “control” subject to do him a favor and play the role, offering him a dollar (or twenty dollars) to do it. Once the subject agreed, Merrill was to give him the money and a sheet listing the main things to say praising the experiment and leave him alone for a few minutes to prepare. |
But Carlsmith could not do a credible job. Subjects immediately became suspicious.
| It was crystal clear why the subjects weren’t buying it: He wasn’t selling it. Leon [Festinger] said to me, “Train him.” |
Sell it. If you’ve seen “American Hustle,” you might remember the scene where Irving Rosenfeld (Christian Bale) is trying to show the FBI agent disguised as an Arab prince how to give a gift to the politician they are setting up. (The relevant part starts at 0:12 and ends at about 0:38)
Here is the script:
Here is the script:
Aronson had to do something similar, and he had the qualifications. As a teenager, he had worked at a Fascination booth on the boardwalk in Revere, Massachusetts, reeling off a spiel to draw strollers in to try their luck.
| Walk right in, sit in, get a seat, get a ball. Play poker for a nickel. . . You get five rubber balls. You roll them nice and easy . . . Any three of a kind or better poker hand, and you are a winner. So walk in, sit in, play poker for a nickel. Five cents. Hey! There's three jacks on table number 27. Payoff that lucky winner! |
Twenty years later, Aronson still had the knack, and he could impart it to others. Like Kahneman, he thinks of the experiment as theater.
| I gave Merrill a crash course in acting. “You don’t simply say that the assistant hasn’t shown up,” I said. “You fidget, you sweat, you pace up and down, you wring your hands, you convey to the subject that you are in real trouble here. And then, you act as if you just now got an idea. You look at the subject, and you brighten up. ‘You! You can do this for me. I can even pay you.’” |
The deception worked, and the experiment worked. When asked to say how interesting the task was, the $1 subjects give it higher ratings than did the $20 subjects. Less pay for lying, more attitude shift. The experiment is now part of the cognitive dissonance canon. Surely, others have tried to replicate it. I just don’t know what the results have been.
--------------------
* An earlier post on Bargh and replication is here
.
No comments:
Post a Comment