DoctorGPT: Offline & Passes Medical Exams!

[@DionKenney-f1k]

That was amazing - both the DoctorGPT app and the high-level walkthrough from the developer perspective. Rock On!

[@TomCourtney]

This is impressive, but it is also a dangerous to suggest that someone rely in a LLM for medical advice.

[@quasarsupernova9643]

Would you second guess a human doctor who shows you her medical license certificate?

[@Ceelvain]

@@quasarsupernova9643 The issue is that LLMs have a *very* limited "understanding" of... well anything. Their design prevent them almost completely to produce logic reasoning that goes beyond "trivial" stuff. And they've shown this limitation times and times and times again. I personally wouldn't even ask for a medical license. (Who does?) But in front of any medical professional doing as many mistakes as an LLM, I'd just go away and find someone else. Like any sane person should. IA as a whole is not ready to take over this kind of task. We don't even have the right tools yet for it.

[@quasarsupernova9643]

@@Ceelvain Your statements contradict the claim that it passed exams with flying colors. If it makes a lot of mistakes, it would fail by definition. Obviously you should test it on many different question papers not found on the internet. Maybe create you own tests.

[@glenneric1]

It's even more dangerous when people get NO medical advice.

[@dreamson280]

It passes medical tests so

[@MHTHINK]

I've been waiting for someone to do this since I saw Med-PaLM-2. Great video and I look forward to following more of your content.

[@TheJay6621]

Siraj Goes to see a Doctor, Siraj : Doctor, My stomach Hurts.. Doctor: Huuh...

[@jayhu6075]

This is the way to make AI useful for the humanity. Hopefully more for this stuff in the future.

[@abhilashjoy]

This is truly amazing man! This video should be trending globally!

[@thearchemists3940]

I came here expecting more AI clickbait, but this is something really remarkable. Thanks for making this with such precision. Deep content made accessible and approachable. I hope this goes viral.

[@thearchemists3940]

@Siraj, How did you end up testing the SoftPrompt against the medical dataset? I'm tracking everywhere but here. The code in the Jupyter notebook doesn't work, and once fixed this Jailbroken Llama doesn't like to answer A/B/C/D consistently (it prefers emojis half the time). Do you have your test code that you could share or commit to the notebook?

[@thearchemists3940]

You just dropped a whole APP in the github? You're the man!

[@juda-marto]

Wow. This is amazing, a very thorough tutorial with all the techniques on how to make Llama 2 useful for any domain. If I could give a million likes I definitely do that! Keep up your good work!

[@mdmusaddique_cse7458]

This was super insightful! Thanks for sharing content like this.

[@Raffa064]

I can't compile the last version of the app... can someone give a link to download it? the available apk of the app isn't working on Android 10

[@medec021]

Hey Siray, this is absolutely top-notch stuff and as a surgeon, I am looking for ways to implement autonomous LLMs in our hospital workflow. This video really inspired me.

[@Letsgetitdone]

If you need some help setting up the platform let me know :-) we just finished a remote patient monitoring system, im sure we can build you your next platform

Thanks. I like this kind of videos. Its been a while since I saw a video from you.

[@jskanenh]

Really liked it and looking forward to going through the process. This was an excting application to dig into to learn more about LLM.

[@caiyu538]

great explanation in technical details. But I want to make sure that the model we download can do inference directly, it is already trained from medical data, or do we still need to repeat what you show to train the model by ourself after we download it?

[@nayanjitsarkar9281]

this is 💎!!! really enjoyed it

[@snipestream1322]

great video, we have a bright future ahead of us.

[@solaxun]

@13:21What exactly are you concatenating? You say "instruction column and input column into a single input" but the code references only the "question" column from the "GBaker/MedQA-USMLE-4-options" dataset. The question is then submitted for inference as-is, without being combined with anything as far as I can tell. Also - are the options (answer choices) and correct_answer_idx (multiple choice answer) used anywhere? @13:12 you are showing what appears to be yet another medical dataset "medalpaca/medical_meadow_mediqa" but it is unclear how that is used. @18:13 you mention SFT with the base model, but the code appears to be using the chat model

[@BrutalStrike2]

Awesome video almost like watching Karpathy!

[@origamitraveler7425]

Can't wait until it hallucinates a wrong dosage number and someone dies. Epic!

[@lukeskywardai]

This is just incredible! Wish more people were doing cool stuff like this!

[@salesgurupro]

Woww! This is one of your best videos Siraj. Thanks for sharing with the community :)

[@bearwater8747]

Siraj Thank you for being such a beautiful person and sharing this knowledge!

[@jonclement]

Nice! I always get uneasy when the tokenizer is handled apart from the model. But it appears (?) TVM bundles these.

[@BibleBuddyAi]

Hey great video! Has the model been evaluated against the Medical Exam after all the compression techniques?

[@dtvonline100]

thanks siraj for the uploaod !

[@nauman187]

amazing stuff as always. can't stop watching. Thanks for this huge thing

[@carlosmastrangelo5398]

This is probably 1000x better than the stupid nurse/family practice interrogation drill I get when going to the clinic. Watch out family practicioners. You and your expensive crappy medical and insurance system will be obsolete

[@TonyManso]

I love how passionate you are about this stuff. It makes your videos much more engaging.

[@Giuliano33]

Medical student here this is truly amazing! I really want to get into this AI boom early, this is hands down the future!!

[@dedenramadhan7671]

This is gem!!!

[@AriL700]

Great stuff Siraj

[@fabio2002]

I watched the entire video, but I have a very diffuse feeling at the end of the video. Different alternatives are shown without giving clarity, in the github repository the link with the colab notebook is broken. As an idea, the doctorGPT project is interesting, but I was very confused with the tutorial on how to do the training. For example, the function"constitutional_evaluation" was not implemented.

[@MrTk3435]

Siraj, Nice one Thank you. ✨🤟✨

[@jean-lucamitousamankoy2937]

Thanks for the tutorial ! I've noted that it's important to finish all prompts with ' ' otherwise the assistant does not give any answer. Looks like the assistant assume we did not finish out sentence. (This is certainly related to some updates in the model). Not working : input_text = conversation_history + question + " " + " ".join([f"{k}: {v}" for k, v in options.items()]) + 'only respond with a single alphabetical character.' Working: input_text = conversation_history + question + " " + " ".join([f"{k}: {v} " for k, v in options.items()]) + 'only respond with a single alphabetical character. '

[@asithakoralage628]

Hi Siraj, this is fantastic and high profile. Well done. I would like to talk to you about you remote patient project. Keep up yhe good work.

[@juanjesusligero391]

Given the sensitive nature of medical applications, it would be crucial for "DoctorGPT" to be approached with extra caution and clarity. Has there been any explicit discussion on potential risks and limitations associated with using this model? You don't speak about that at all on the video.

[@VenkateshPorika]

where can we download the pre trained model?

[@Adhil_parammel]

Android app not available

[@1111quarn]

Not working in ipad= Sorry, the system cannot provide 3814.7MB VRAM as requested to the app, so we cannot initialize this model on this device.

[@ProfitScalpKing1]

This is AMAZING!!!

[@alabiobin128]

Hi, I'm trying to get the colab notebook to this project from your github but it's saying "notebook not found", is there a way I can get the code to this please?

[@joegadget670]

Don’t ask to see a Doctor’s diploma. Ask to see their USMLE score! Only 60% score needed to pass?

[@Tstanek]

Hey bro, awesome job! this is just incredible really. Im preparing for my final exam to become fully trained oncosurgeon and was thinking this could be used as very effective tool not just for studying. Im very interested in AI, tho Im a noob at coding etc and this stuff is just my kinda hobby. But i was thinking if you maybe have some hints, tips what to study etc to be able to train the model with my own clinical data and findings. Sorry for asking general and lame question, but I dont really know where to start. Any response would be super helpful! Cheers from czech republic

[@Letsgetitdone]

Nice, tried it out but your huggingface link is not accepted by the app.. Hopefully a fix comes in soon :)

[@Gameboygenius]

But why does DoctorGPT's mascot look like Crazy Frog?

[@dosapatisuresh]

Awesome Tutorial. You are cool.

[@nickinhai]

Really amazing!! DoctorGPT supports others medical languages?

[@brunob.7792]

Try to imagine global medicine feeling itself fighting a trench war with patients with 300mb models installed on its phones discussing and complainning about their diagnosis. Jeez, google will be small near that

[@maizizhamdo]

hi bro , it support french language?

[@BrutalStrike2]

Hey Siraj, could you possibly make a video all about Hivemind Petals? I find it super intriguing, but honestly, I have zero clue how it actually works!

[@phils744]

Hi, can "people explain what the values of / is [7B, 13B, 34B] Equates too. I'm still a little confused. How many Bytes is 7B params. What is 34B. Be safe everyone

[@nidalidais9999]

You are amazing

[@NagibDev]

You're the best!

[@bobayoub6024]

this is good content, and a valid recipe to build you own chatgpt, but the hand gestures though, it's so funny

[@pavanlulla]

I wish doctors could be retrained like llama 2 😅

[@ScottzPlaylists]

👏Very nice video Siraj. Can you make a version optimized for a PC Running an NVIDIA RTX 3090. And instructions on training it until it's optimized fully? 👏 ❤I'd love like to see more Stock / Crypo trading Bots creations, but get it very profitable (unlike the previous ones you did). ❤

[@hod4378]

Hi Siraj Raval , can you explain for beginners how I can execute the DoctorGPT to check it on web? it give me some errors ...

[@hod4378]

Someone maybe know how to run the app in Flask - give me errors ? thanks in advanced ..

[@emilsonkafintu]

Aawwooo....God Himself should bless you!!

[@AdityaSingh-xy9bk]

Is your chanel has monetize or not bro

[@rachitxgames]

bro how can Google has more data to set quantum company

[@VibudhSingh]

As you said, this is a very packed video. Thanks for sharing a lot of tricks! on a side note, are you sing your ipad as your mouse?

[@rochtm]

You Sir are awesome

[@vincentkubicki1626]

Asserting that it is better than a human doctor relies on the assumption that the US Medical Licensing Exam is the only way to assess the level of a doctor. I have no idea if that is the case, but exams focus on series of small problems while the real job might differ. To make a parallel in CS (which I know a little more about), you have candidates which ace technical tests (even AI system do this now), but are incapable of working on complex problems. I do not want to diminish your efforts and results, just to point at some general questions around the topic.

[@Isaacrl67]

Also, consider that the doctors who take those exams are not likely scoring 100% across the board. As with any exam, there is a 'good enough' threshold to it, yet we concern over the possibility that a computer could make a mistake. Self-driving cars are a great example. Currently, the have less accidents than humans do, but until they can prove the AI is infallible, people will not trust it. The AI car might have 1/100th the accident rate of an average human and the average person would still choose to trust another human over a self-driving machine.

[@muzammalrajpot3791]

❤❤😢😂😂 give me timing medicine 😅😂

[@viruldewnaka1193]

Great stuff

[@dreamson280]

32:38 😂😂

[@ralfg7708]

No, you did not build an app. You are unable to do such a thing.

[@michaelcook3168]

Don't ever show your doctor this !!

[@weluvtech]

Why? It might improve their diagnosis. Many are pathetically incompetent! 😂

[@michaelcook3168]

@@weluvtech No, not at all. They will get pissed of at you. Many are arrogant and incompetent as yo say.

[@soraygoularssm8669]

Awesome

[@tomoki-v6o]

Jailbreaking sounds like a shrink doing hypnosis

[@benoitranque7791]

Shame on you. You should know better. These bots pass exams because the questions and answers are in their training dataset. They are no substitutes for humans and do not know truth from lie, or anything really. Get a database and a search engine.

[@magicliuliu745]

not really, for chinese, it should be chatglm

[@michaelparis6039]

Make it multimodal in the next iteration. Like taking a picture or a video and done text.

[@michaelparis6039]

Hope you recover quickly from your cold.

[@KutluMizrak]

You are ducking insane with that " it answers any medical question better than your doctor ever could !" bit. You'll be sued within an inch of your life.

[@glenneric1]

money back guarantee

[@latestAiHacks]

Man, your idea is really good but this AI you created is not usefull yet. Keep going

[@eddieh5356]

Promo SM 🤤

[@dabunnisher29]

WAIT..... DIDNT YOU GET CAUGHT BEING A BIG TIME LIAR?

[@bmoore813]

This is awesome

[@CitydentBg]

Trying to load it in oobabooga but gives me an error