Posted by Jay Livingston
Imagine that you’re a US Attorney on the drug beat. Your boss is Jeff Sessions, who has announced that he’s going to vigorously enforce laws against marijuana and use the federal law when state laws are more lax. Maybe you also think that weed is a dangerous drug. You do a little “research” and tweet out your findings.
This brief tweet might serve as an example of how not to do real research. The sample, which excludes people who have not gone to treatment centers, is hardly representative of all users. There’s researcher bias since the guy with the ax to grind is the one asking the questions. The respondents too (the drug counselors) no doubt feel some pressure to give the Sessions-politically-correct answer. They may also be selectively remembering their patients.
But even without the obvious bias, this tweet makes an error that mars research on less contentious issues. It samples on the dependent variable. The use of heavier drugs (opioids, heroin, meth, etc.) is the dependent variable – the outcome you are trying to predict. Marijuana use is the independent variable – the one you use to make that prediction. Taking your sample from confirmed heroin/opioid addicts gets things backwards. To see if weed makes a difference, you have to compare weed users with those who do not use and then see how many in each group take up more serious drugs.
Here’s an analogy – back pain. Suppose that, thanks to advances in imaging (MRIs and the like) doctors find that many of the people who show up with back pain have spinal abnormalities, especially disk bulges and protrusions. These bugles must be the gateway to back pain. So the doctors start doing more surgeries to correct these bulges. These surgeries often fail to improve things.
The doctors were sampling on the dependent variable (back pain), not on the independent variable (disk bulges). The right way to find out if spinal abnormalities cause back pain is to take MRIs of all people, not just those who show up in the doctor’s office. This is pretty much the way it happene in the real world. Eventually, researchers started doing the research the right way and found that lots of people with spinal abnormalities did not pass through the gateway and on to back pain.
The same problem often plagues explanations that try to reverse-engineer success. Find a bunch of highly effective people, then see what habits they share. Or look at some highly successful people (The Beatles, Bill Gates) and discover that early on in their careers they spent 10,000 hours working on their trade.
US Atty. Stuart’s tweet tells a good story, and it’s persuasive. But like other anecdotal evidence and eyewitness testimony, it is frequently misleading or wrong. The systematic research – many studies over many years – shows little or no gateway effect of marijuana. No wonder US Attorney Stuart chose to ignore that research.*
----------------------------
* As Mark Kleiman has argued, even when a marijuana user does add harder drugs to his repertoire, the causes may have less to do with the drug itself than with the marketplace. The dealer you go to for your weed probably also carries heavier drugs and would be only too happy to sell them to you. Legalizing weed so that it’s sold openly by specialty shops rather than by criminals may break that link to other drugs.
No comments:
Post a Comment