Posted by Jay Livingston
Getting data is hard. It’s time consuming and laborious and often, truth be told, not all that interesting. On top of that, you worry about validity – does the data set really tap what I say I’m studying? And in the end, it may turn out that the results are disappointing; you wind up with something reviewers won’t think is worth reporting.
It’s not like medical science, with its strict and precise definitions and measurements – those doctors in white lab coats carefully testing the effects of drugs and coming up with results that help humanity.
But now medical science shows us the way to get convincing data, data that shows results: make the stuff up.
Concocted. That’s the word they use (in case you have trouble reading the print in the boxes – the full story is here). Dr. Reuben concocted data. Why didn’t I think of that? Maybe because no huge drug company like Pfizer is underwriting my “research” that shows their pain drugs to be so highly effective.
I’m going to repeat a quote I posted a couple of months ago. It’s from a former editor of the New England Journal of Medicine written well before this latest bit of news about Dr. Reuben:
It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine.
Yeah, but they had p-values < .05, on the correlation matrices, right? I mean, while the data might be faked, they can't fake their stats... Right?
ReplyDeleteYou've hit on the scary part -- p < .05 --scary even if the data weren't faked. It's one thing if you're just testing some hypothesis about sociological data. But when it comes to my health, if I'm taking a drug, I'd like to think that the standard for deciding that it will work and that it won't harm me is a little better than 19-1. Hell, I've won on horses at Waterford that whose odds against winning were higher than 20-1. And I've seen 1-20 favorites lose.
ReplyDelete