Can ChatGPT solve medical mysteries? (Featuring Adam Rodman)

Рет қаралды 5,424

11 ай бұрын

I talk about a new paper in JAMA in which ChatGPT was tasked with solving 70 CPCs (i.e. real-life diagnostic dilemmas) from the new England Journal of Medicine, followed by an interview with one of the authors. (spoiler alert: it did surprisingly well!)
Link to the paper: jamanetwork.com/journals/jama...
Dr. Rodman's podcast, Bedside Rounds: bedside-rounds.org/
My previous video on ChatGPT solving clinical cases from my medical school final: • Can ChatGPT Pass a Med...
#artificialintelligence #clinicalreasoning

Пікірлер: 27

@kmd6992 11 ай бұрын

Hello Dr. Strong! I've been following your page and watching your videos ever since I was in medical school. Now that I'm already a consultant (Family Medicine), I still watch your videos whenever I feel the need to refresh my knowledge about certain topics. You've been a great part of my medical career. Thank you. Always keep safe!

@StrongMed 11 ай бұрын

You're very welcome! I'm glad you've found the videos to be helpful!

@VedTopkar 11 ай бұрын

Strong x Rodman = greatest med-ed crossover of all time

@StrongMed 11 ай бұрын

Lol. Thanks Ved! Hoping to do more crossovers with other folks in the future.

@dizzyboy92 10 ай бұрын

You took my words one by one, a month before I thought them.

@fiftygrapes 11 ай бұрын

I feel like people still skeptical about generative LLMs being able to impact medicine, and even all facets of our life where knowledge recall is important, fail to be able to see and extrapolate how these things advance. Ive used open ais gpt models for my own further medical learning and psuedo practice, and whilst 39% may not seem like an impressive number right now, it feels like being skeptical on this point is 1. Not understanding how fast these things improve and 2. Literally not realising its electricity running algorithms through wires and chips generating both logical and semantically understandable responses to complex inputs on the level of advanced professionals. My first feeling when I tried chatgpt on release last year was apprehensive, a little bit of me hoped it wouldnt be good and instead an overhyped fad. The fear that your everything youve studied could be replaced is always something that can spawn feelings of denial. However using it more in my opinion has led me to feel its going to be very hard for it to be anything other than world changing.

@piotr5349 11 ай бұрын

I love the interview format and am looking forward to more content like this!

@andeggbreaks 3 ай бұрын

Very interesting, I was not expecting GPT to perform this well considering how many controversies have followed its use in law. Hopefully it can eventually aid doctors with diagnoses. I doubt it'll ever replace medical professionals, even in the far future, but people will definitely be more inclined to chat GPT their symptoms in the first instance instead of seeing a doctor. That seems like a good thing right now with how understaffed and strained the medical system is, so hopefully it does more good than harm.

@kvasirofold4784 11 ай бұрын

They are already doing it in ER for PE's. Answer some questions, it figures out if they have a PE, then notifies specialists, orders new labs if they haven't been ordered, and orders the CT (it is done before the CT) and notifies the radiologist of the stat order.

@aavosooghi 11 ай бұрын

Hi Dr. Strong, I loved this new video format and found your discussion with Dr. Rodman so captivating! As a current student, I’m very curious about how we’ll ultimately decide the most appropriate time in a learner’s trajectory to begin incorporating LLMs into MedEd (I believe this integration to be inevitable). Do we wait for early learners to establish some arbitrarily defined baseline fund of knowledge (defined by passing Step 1/Step 2/etc)? Do we incorporate LLM tools from the start of medical education?

@fiftygrapes 11 ай бұрын

About your point about the variability of responses given the same prompt, this is a design feature of the LLM of Chatgpt4 presented through the web client, and actually "fixable" or easily changed when you use an interface which can adjust the models parameters (like the api). LLMs are by definition deterministic. At a simplistic level they generate a list of next appropriate words or tokens given a previous input of tokens. You can make the LLM deviate from always choosing the top most likely next best word or token, which is what the web interface does, by adjusting what's known as its temperature parameter, where high temperatures allow for more deviation. If you are doing psuedo research testing on chatgpts outputs and are worried about reliability of responses due to this, you can simply just use an api interface and adjust that parameter.

@StrongMed 11 ай бұрын

I appreciate your points here. To clarify, the comment about ChatGPT's variability being a "limitation" was referring to a limitation of the paper (albeit a minor one), not a limitation of the technology itself. As I briefly mentioned, at least as far as the bot providing possible diagnoses in response to long case presentations, GPT4 appears to demonstrate less variability than GPT3.5. Whether that's due to a true difference in variability, or is an artifact caused by GPT4 just being more accurate with this task, I don't know. I do wish that the temperature parameter was directly adjustable in Open AI's own interface; it would only make the tool more interesting.

@fiftygrapes 10 ай бұрын

@@StrongMed I agree with everything you've pointed out. I really like your videos. As an Australian medical student studying in a non-US system it really makes me appreciate and also understand little bit more of what we've been told but never really experienced about the level of higher education in the US. I watch a lot of your videos on release and really enjoy them. And I really like that it comes from someone who teaches. Thanks for replying to my message.

@MrZigan4ik 11 ай бұрын

Really interesting, thanks for the talk!

@ertwro 11 ай бұрын

Insurance companies will be like: “We can audit doctors with this 🤑”

@eduardogimenezdepaz4391 4 ай бұрын

EXCELLENT VIDEO CONGRATULATIONS ERIC ,Edward from Argentina

@dr.luisazpurua1656 10 ай бұрын

Greeetings from South America! As Dr. Rodman said, we are seen a glimpse of the future. Not only imagine the 2040's doctors, but also the patient! They won't come to the ER, they will chat with the avatar in home or wherever they are! Also, if we're doing this with "normal" computing power right now, can you imagine in the quantum computer era? It will come a lot of changes.

@molyosistube4857 11 ай бұрын

Where could i watch CPCs !!

@racoon96 11 ай бұрын

Hey Eric, i enjoyed this video very much, but i hesitated clicking on the video. For me this whole ai thing is a tough pill to swallow. The most disappointing thing for me might be the black box aspect. But if we test it and it performs better, who are we not to use it and refuse to give the patients the best care possible? At this point, it really demotivates me. I always like the art of the differential diagnosis and the "detective" part. The more the technology adavances the more we get to know our own cognitive limitations. That´s a hard pill to swallow for our egos. Anyway: Thanks for making these videos!

@StrongMed 11 ай бұрын

Thanks for the comment! I agree that the black box aspect of ChatGPT and other LLMs is a little unnerving. However, when it comes to making medical diagnoses, the process by which clinicians reach a diagnosis has a little bit of a black box nature to it as well (i.e. hueristics and automatic "system I" thinking) - particularly with "simple" cases and common diagnoses. We can explain our thought process afterwards, but that's sometimes us seeking to justify our diagnosis after the fact rather than elaborating on how we arrived at a diagnosis in the first place.

@ashkanmohajer2223 11 ай бұрын

Hi Doc, I hope your'e doing well. Why don't we put patients with predictable/iatrogenic QT prolongation, on a prophylactic low dose/daily PO Mg+ and K+ supplementation to prevent Torsades de Pointes? Because it does look like that vast majority of the (North American +/-) population maybe subclinical/borderline deficient in Mg+. Because if it actually prevents the Torsades from developing, it can prevent 10% of sudden cardiac death from this arrhythmia alone ...Thanks.