May 1, 2023
Posted by Jay Livingston
| Help. A couple of hours ago, I swallowed a wooden toothpick that the cook had missed when removing them from the stew he’d made. It wasn’t that long, an inch or so, and it couldn’t have been very sharp since I didn’t feel it much when I swallowed it. But can it be serious or even fatal? |
A question similar to this was posted on Reddit’s r/AskDocs forum, where doctors or other healthcare professionals post answers. Here are two responses.
| If you’ve surpassed 2-6 h, chances are they’ve passed into your intestines. Which means it can’t be retrieved easily.
Many people can swallow razorblades and toothpicks without issue. In case you develop stomach ache, then don’t hesitate seeking out an emergency room, and remember to point out the swallowed tooth pick. For now, they won’t be doing much. |
| It’s natural to be concerned if you have ingested a foreign object, but in this case, it is highly unlikely that the toothpick you swallowed will cause you any serious harm.
Toothpicks are made of wood, which is not toxic and will not cause poisoning. However, it is possible for a toothpick to cause injury if it becomes stuck in the throat or if it punctures the digestive tract. However, this is unlikely to happen with a dull, cooked toothpick that is only 2 cm long. If you are experiencing any discomfort or symptoms such as abdominal pain, difficulty swallowing, or vomiting, it is important to contact a health care provider for further evaluation. However, if you are not experiencing any symptoms, it is safe assume that the toothpick has passed through your digestive system and you do not need to concerned. It’s always a good idea to be cautious when consuming food and to remove any foreign objects before eating. It's understandable that you may be feeling paranoid, but try not to worry too much. It is highly unlikely that the toothpick will cause you any serious harm. |
How would you rate the quality of each response on a 5-point Lkert scale (very poor, poor, acceptable, good, or very good)?
How would you rate the empathy or “bedside manner” of each response (not empathetic, slightly empathetic, moderately empathetic, empathetic, and very empathetic)?
The first response is from an actual doctor. The second is from ChatGPT. Which did you rate more highly?
Chances are that your evaluation was no different from those of a team of three licensed healthcare professionals who reviewed 200 sets of questions and answers. On measures of both quality and empathy, ChatGPT won hands down. (The JAMA article reporting these findings is here.)
On a five-point scale of overall quality, the ChatGPT average was 4.13, Doctors 3.26. (On the graph below, I have multiplied these by 10 so that all the results fit on the same axis.) On both Quality and Empathy, Doctors got far more low (1-2) ratings (very poor, poor; not empathetic, slightly empathetic), far fewer high (4-5) ratings.
The great irony is that the doctors tended to be impersonal while the machine (ChatGPT) responded to the patient as a person, not just a symptom reporter.
People who ask medical questions are worried. If you have something going on with your body that seems wrong, and you don’t know what it is, you probably are going to have some anxiety about it. So ChatGPT might begin with a general statement (“It’s always best to err on the side of caution when it comes to head injuries,” “It’s not normal to have persistent pain, swelling, and bleeding. . . “) or an expression of concern (“I’m sorry to hear that you got bleach splashed in your eye”). The doctors generally focused on the symptom, its causes and treatment.
Doctor responses were considerably more brief than those of ChatGPT (on average, 50 words compared with 200). That’s partly because of time. If doctors were at all concerned with an efficient use of their time, they couldn’t turn out the longer responses that ChatGPT generated in a few seconds.
But I think there’s something else. For patients, the symptom is new and unusual. They feel worried and anxious because they don’t know what it is. But the doctor has seen it a thousand times. It’s routine, not the sort of thing that requires a lot of thought. Here’s the diagnosis, here’s the recommended treatment, and maybe here are some other options. Next.
People who ask medical questions are worried. If you have something going on with your body that seems wrong, and you don’t know what it is, you probably are going to have some anxiety about it. So ChatGPT might begin with a general statement (“It’s always best to err on the side of caution when it comes to head injuries,” “It’s not normal to have persistent pain, swelling, and bleeding. . . “) or an expression of concern (“I’m sorry to hear that you got bleach splashed in your eye”). The doctors generally focused on the symptom, its causes and treatment.
Doctor responses were considerably more brief than those of ChatGPT (on average, 50 words compared with 200). That’s partly because of time. If doctors were at all concerned with an efficient use of their time, they couldn’t turn out the longer responses that ChatGPT generated in a few seconds.
But I think there’s something else. For patients, the symptom is new and unusual. They feel worried and anxious because they don’t know what it is. But the doctor has seen it a thousand times. It’s routine, not the sort of thing that requires a lot of thought. Here’s the diagnosis, here’s the recommended treatment, and maybe here are some other options. Next.
Interestingly, I far prefer the GP's response.
ReplyDeleteAt least it's honest.
ChatGPT sounds like it's frantically covering its arse.
Part of that is that I've already seen way too much of the slish ChatGPT spews out, and part of that is that I know how ChatGPT works (hint: it doesn't do any actual reasoning; it exactly and only randomly recombines stuff (it's "words" are undefined tokens with no connection to physical (or human) reality)) in it's database).
As someone with an all-but-thesis in AI (albeit from an earlier century), I've been aghast at what the AI field has been doing of late. It's all lies, from "neural nets" (which are nothing whatsoever like actual neurons) to "machine learning" (statistics done badly) to LLMs, which really are nothing more than stochastic parrots. Sigh. The original definition of the field (1956, John McCarthy) was pretty kewl: use computation (the mathematical abstraction) to think about what the human brain might be doing and use programming to test the resultant ideas. Where it's gone of late is pretty ridiculous. Sigh.
Note that there's an interesting common theme to these three areas: an absolute refusal to actually think about how real humans might be doing the kewl things they do, and a naive hope that "intelligence" will somehow magically emerge from doing a large enough number of stupid things fast enough with a large enough amount of data.
YMMV, of course. But, really. It looks like BS to me...
David: True, ChatGPT composes its answers via method that’s very different from that of the doctors. But my point was that the doctors’ responses didn’t take into account how the patients my be feeling about their medical problem. Maybe that’s because the doctors were basing their responses on medical resources while the ChatGPT responses were based on a much broader range of sources for those words that wound up in the message.
ReplyDeleteI’ve had similar experiences when seeing a doctor in person — the same narrow focus on the symptom, its possible causes and the treatment, and apparently no realization that I might be seeing it from a different perspective, one marked by ignorance, uncertainty, and anxiety. No artificial intelligence, but no emotional intelligence either.
Just before seeing your comment, I came across this article that might interest you on why ChatGPT doesn’t write very good college admissions essays.