The medical test paradox, and redesigning Bayes' rule

About Likelihood Ratios, also sometimes called Bayes Factors*.
Help fund future projects: / 3blue1brown
An equally valuable form of support is to simply share some of the videos.
Special thanks to these supporters: 3b1b.co/bayes-factor-thanks
Home page: www.3blue1brown.com
The book by my friend Matt Cook about paradoxes mentioned at the end:
amzn.to/3aBrEzg
On the topic, I can't help also mentioning another paradox book I'm rather fond of by Bunch:
amzn.to/3mBDSKE
Another video on Bayes' theorem:
• Bayes theorem, the geo...
*As mentioned in the on-screen note at the end, while the terms "Bayes Factor" and "Likelihood Ratio" refer to the same ratio in this setting, where Bayes rule is used on the probability of an event with only two possible outcomes (you either have the disease or you don't), they do take on divergent meanings in more general contexts. Namely, if you have a continuous parameter you are trying to estimate, the two terms reflect two alternate approaches you can use in comparing hypotheses. In fact, some people take the phrase "Bayes factor" to specifically refer to its use in this more continuous context.
If you want more details, Wikipedia actually has a really nice example discussing the difference:
en.wikipedia.org/wiki/Bayes_f...
This post has some nice discussion of the distinction:
stats.stackexchange.com/quest...
Timetable:
0:00 - What is the paradox?
5:53 - The Bayes factor
11:00 - The snazzy Bayes rule
14:42 - Contrast with the usual formula
Thanks to these viewers for their contributions to translations
French: Raphaël Ventura
Russian: Ilia Konnov
Vietnamese: duyhub
------------------
These animations are largely made using manim, a scrappy open source python library: github.com/3b1b/manim
If you want to check it out, I feel compelled to warn you that it's not the most well-documented tool, and it has many other quirks you might expect in a library someone wrote with only their own use in mind.
Music by Vincent Rubinetti.
Download the music on Bandcamp:
vincerubinetti.bandcamp.com/a...
Stream the music on Spotify:
open.spotify.com/album/1dVyjw...
If you want to contribute translated subtitles or to help review those that have already been made by others and need approval, you can click the gear icon in the video and go to subtitles/cc, then "add subtitles/cc". I really appreciate those who do this, as it helps make the lessons accessible to more people.
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with KZbin, if you want to stay posted on new videos, subscribe: 3b1b.co/subscribe
Various social media stuffs:
Website: www.3blue1brown.com
Twitter: / 3blue1brown
Reddit: / 3blue1brown
Instagram: / 3blue1brown_animations
Patreon: / 3blue1brown
Facebook: / 3blue1brown

[@johnchessant3012]

This is awesome. I have now updated my odds of correctly answering a Bayesian probability question.

[@ivocanevo]

Meta.

[@NStripleseven]

Whoa

[@madlad255]

@@NStripleseven I'd be surprised if that wasn't a Baba is You reference. (If it wasn't: Whoa is a level in a world called Meta, in that game XD) I don't have a prior though, (your knowledge about that game) so the odds of it not being a reference, and me being surprised, are unknown.

[@myrus5722]

ML255TV Well now you need to rewatch the video to help update your odds of knowing

[@TheSummoner]

That’s the spirit!

[@connermcd]

As a doctor I'm so happy you're using your platform to get this information out. Let me tell you though... it gets way more complicated! Unfortunately prevalence estimates aren't always known and are constantly changing (especially in pandemics). Another thing to consider is the gold standard. If your test looks for breast cancer you can cut out the lump and look at it under a microscope. Some diseases aren't as easily clarified. For instance, since we don't have a highly accurate, easy test for pancreatic cancer we rely on imaging, demographics, blood markers, symptoms (or lack thereof) as multiple things that form a conglomerate test to increase our Bayes factor. Despite all these things we can't always get a great prediction on whether that scar in your bile duct is cancer or just a residual scar from pancreatitis you had 10 years ago. So we offer the patient a huge surgery to remove the head of their pancreas and duodenum only to find that it wasn't cancer. You can imagine the patient is happy it's not cancer but not so happy they don't have half their pancreas and have abdominal pain and maybe diabetes. Medicine is a tricky thing. Another tricky thing is operator error. Some tests depend on the skill of the lab tech, radiologist, or surgeon. The complexity of the human body and the uniqueness of each individual also plays a role. Your test may be false positive in a particular patient 100% of the time because they have some strange protein mutation. It's tough!

[@danielf9110]

This needs to be pinned (=

[@descuddlebat]

"Your test may be false positive in a particular patient 100% of the time because they have some strange protein mutation." This one bugged me - When you take one test then this isn't an issue, but surely when you take two, just multiplying the probabilities together won't be accurate, right? Just multiplying them works for independent probabilities, but there is a factor linking them, the person being examined. I didn't think of the rest, and I don't work in med, but in situations like covid, I can see how prevalence rate is hard to determine - seems like it would be heavily skewed by people just staying home with a cough for one reason or another.

[@dagwould]

All too true. We deceive ourselves that medicine is as simple as motor mechanics, and even that is not that simple. Diagnosis is fraught with challenges and it is always 'best opinion at the time with the evidence available and that not present' to give the differential result, which could still be wrong, particularly with rare diseases.

[@farazalam3325]

Yeah just like we go for triple test for ca breast/ quadruple test for thyroid nodule or as in the case of whipple's described by you.. We still have Diagnostic uncertainties

[@WakeEternal]

Regarding having a mutation that rules out any chances of having a disease, that would in itself would be tested for, i.e. essentially applying a Bayes factor based on the genetic test for that mutation. The odds is based on the information we have on hand until new information is known for that patient to update the odds estimate. However, you do hit on one factor not addressed in this video which is some medical assessments are not hindered by the uncertainty inherent in many tests when the assessment directly shows-often by imaging or pathology-that an individual definitively has or does not have a condition. To add to your examples, angiograms can identify the exact location of a clot or genetic sequencing can specify whether an agent will work against a patient's cancer. (Then again, the angiogram may fail to find the clot and the tissue biopsy may miss the neoplastic cells.) I see one promise of medical advancement is to push this boundary between certainty and uncertainty.

[@Jebusankel]

I'm going to apply this to the world of dating. Everything I learn about a potential match updates my prior about our compatibility. I call this Bae's rule.

[@rishiagarwal8864]

Severely underrated comment

[@mathwithjanine]

haha this is so good!

[@jamfold2956]

Dear Grant, Plz pin this comment. Please.... please..... please.

[@kevins6732]

lmfaooo

[@I4MGUCCI]

you’re the real MVP

[@Asdfgfdmn]

I hope Grant read this 😇 I am an MD and Associate Professor of internal medicine. I teach medical students, residents, and fellows. I used to be a program director for a fellowship at a prestigious American university. This is a recurring lesson I teach. The example I usually use is the DNAJB9 kidney biopsy stain sensitivity and specificity for a disease called Fibrillary GN., and I do the exact walkthrough with my students. I never get bored when I see how surprised they are with the final conclusion. Which is, by the way, is: you can't use a test willy-nilly without considering the pre-probability (you are referring to it here as ”prior.” And I also tell my students that you can increase the prevalence of the disease is by applying it to the right population (signs and symptoms). I am thrilled that Grant validated this with this awesome video

[@kingbradley3402]

This is actual gold content being uploaded for free. It's like I'm unlearning what I learnt in all my classes and seeing Maths in a whole new way. I was asked in an interview one concept that is often confused but makes sense in general. I spoke about Bayes' Theorem. And this has given me something more to talk about. Quite possibly the best educational channel on KZbin.

[@brightsideofmaths]

I totally agree: Educational content should be free!

[@pewien_internauta]

@@brightsideofmaths this episode is sponsored by brilliant

[@brightsideofmaths]

@@pewien_internauta Don't get me started...

[@12Rman21]

@@brightsideofmaths something needs to pay for it, somehow. Unless we are going to value skilled educators time at 0

[@oliverwilson11]

@@12Rman21 Thank you captain obvious

[@KX36]

As a scientist who does medical tests, I'm amazed that any of the doctors asked got the right answer. Every doctor I work with assumes tests are 100% accurate.

[@compuholic82]

I work in the medtech industry. In my experience, it depends on who you are working with. The more research-focused physicians (e.g. in university hospitals) tend to be better. But generally, I agree: The vast majority of physicians is shockingly ignorant about the limitations of the technology they use everyday.

[@TheOneWhoHasABadName]

as a medical student this is very instructive. if I were in that seminar room I’d probably go with the “more intuitive” answer without thinking first then act surprised when they explain that this is wrong.

[@ivocanevo]

Thanks for this thread @KX36

[@SKyrim190]

@@Roman_4x5 isn't it H1N1?

[@guest_of_randomness]

@@Roman_4x5 hey!😂 i haven't seen it!

[@D4n21]

Medical Student here, THIS IS GOLD. THANK YOU, this is going to help with my boards and future patients

[@hjfreyer]

This is the first presentation of Bayes' theorem that didn't leave me feeling both like it was trivial and like it was inscrutable magic.

[@davidholle6569]

what really helped me was not using words like "sensitivity" but instead "true positive"

[@jenniferpearce1052]

I had to go back to the definitions a few times to follow. I get lost by the words more than the math. :/

[@gabrielbn]

Strongly agreed. For a channel so focused on simplicity and clarity, this is a flaw in an otherwise excellent video.

[@dodg3r123]

Be aware that “sensitivity” is not the same as “true positive”

[@jinjunliu2401]

@@gabrielbn 3b1b translated those terms into the abbreviations for True Positive Percentage, False Positive Percentage, in most of the animations. I'm not sure on the script anymore though

[@jenniferpearce1052]

@@gabrielbn I don't think of it as a flaw in the video. The disconnect in my brain between language processing and math processing is stronger than I would like! And it takes a while to just roll with new vocab, especially when you have a "which s word is which concept" situation!

[@RasperHelpdesk]

Certainly drives home the point of why running a test twice after getting a positive result is so important when possible.

[@Yakushii]

This is such an important takeaway from this lesson!

[@michaelguenot6177]

Even then, be careful. The results of multiple tests on the same person are likely correlated.

[@okuno54]

@@michaelguenot6177 And that really drives home why you want multiple distinct methods to verify a claim

[@paolobassi544]

It also depends on the priors and accuracy of test...it can be an overkill to repeat the test

[@leonardonakatanimoretti6516]

@@paolobassi544 how much is a overkill?

[@tejing2001]

This perspective makes the 'update' concept so much cleaner. I've long believed that until something is utterly obvious to you, you still don't truly understand it. I just got much closer to understanding bayesian updating. I could already do it, and even explain it, but it wasn't the same. True understanding is precious. Thank you for what you do, and as always, I look forward to the next video.

[@DJNHmusic]

As a young doctor, thank you so much. I understood the distinction between the different accuracy parameters and PPV beforehand, but this has fundamentally changed how I view testing. This is a very useful thing to understand as a medical professional.

[@goncalodvd]

And the ones who dont get it helped create the pandemic ...

[@sarvagyagupta1744]

You should definitely checkout precision and recall. They are machine learning terms but they essentially mean the same things as mentioned because the formulae are the same as well. One technique that we ML practitioners use is F1 score. As you would've seen that there an inverse relationship between PPV (what we call precision) and sensitivity (what we call recall). If you plot them for different threshold (some min value for which you classify something), you'll clearly see the inverse relationship. If you use TP, FN, and FP and put them in F1 score (TP/ (TP + 0.5(FP+FN))), you'll be able to find an optimal threshold value. This threshold we refer to as probability of something being detected as positive. And this takes into consideration both FP and FN which are crucial for medical examinations.

[@wernerviehhauser94]

This topic always breaks the brains of my students. Every year. Every time. This IS pretty hard stuff to wrap your mind around.

[@asdfghyter]

I think phrasing it the way 3b1b did in this video helps a lot in making it more intuitive. When you think of concrete sample populations instead of probabilities, it makes a lot more sense.

[@MrAntifascista23]

I remember having been presented a similar problem with concrete examples in a admission test and without knowing anything I just figured out a formula that would work for the example and gave the correct answer, however when I came to study this in statistics it became so confusing and failed so many exercises by applying a formula I didn't understood

[@wernerviehhauser94]

@@asdfghyter one major problem the students have is that they are careless with the meanings of the terms and mix them. You can be somewhat sloppy in algebra and analysis, but if you do this in probability, you're setting yourself up for a lot of pain.

[@bertjesklotepino]

@@MrAntifascista23 "i didnt understood" ?

[@jessstuart7495]

If 1% of the population knows how to correctly use Bayes Theorem, and 80% of your students get the correct answer on a Bayes Theorem problem on the final exam, what percentage of your students will be able to recall Bayes Theorem, three semesters after having taken your class?

[@egillandersson1780]

As doctors, we use this every day, often without thinking about the mathematical foundations. Unfortunately, very few diagnostic tests ou exams are indeed both sensitive AND specific. So, if we think a diagnostic unlikely (based on prevalence, physical exam, previous tests, ...), we choose first the more sensitive test in order to exclude this diagnostic. On the contrary, if a diagnostic is very probable, we choose first a specific test to confirm. It is not always easy for technical exams, as we can often only choose between them (if there are several !), without changing their sensitivity and specificity. But for biological tests, we can adjust our cutting values to improve either sensitivity or specificity.

[@aitotem]

So basically the video is moot and based off a trick question given to some tired doctors. So glad I wasted my time.

[@marcevanstein]

Thanks for this -- adds some very interesting context!

[@1994mrmysteryman]

@@aitotem the video isn't moot at all. You just didn't understand the video nor the comment above. The video makes a very legit point about how a lot of people and doctors can have an absurd amount of faith in the accuracy of a test which can lead to a lot of problems. And panic. While the comment above is simply describing how doctors tread around this problem.

[@rainbowevil]

@@1994mrmysteryman I would ignore that guy, he seems to have an issue with the channel and is commenting lots of unnecessarily hostile and useless replies to comments on 3b1b’s videos. Haven’t a clue why, but some people are just broken wastes of space.

[@qazwerty41339]

The mnemonic I learned is SNOUT (SeNsitive tests rule OUT) and SPIN (SPecific tests rule IN)

[@tk27a]

Grant, I am rewatching your videos about Bayes’s a theorem and I want to underline the quality and details (and the amount of work!) you put in your videos. We are blessed to have people like you to do such exceptional work

[@LoscoX]

I started watch your channel a month ago, and now you're my favourite math channel! You talk about math in a not too formal and technical way, but at the same time, speaking about technical math things which encourage and help me to go on study math at the university. Thanks for each your videos!

[@matheussaldanharodriguesdu1850]

I was just feeling bad seeing your videos when I`ve should be studying medicine. Now you`ve done the best of the both worlds.

[@aytide5179]

Grant never misses. He's always brilliant

[@absence9443]

You forgot a g

[@omarabdelkadereldarir7458]

@@absence9443 what does this mean?

[@SKAOG21]

He don't miss

[@DennisMathgod]

This is so much better than the Veritasium video on this topic. You went into so much more detail and presented it in a very clear, easy to understand way. Nice job!

[@eyabs]

Thank you so much for this video. This concept of updating odds is what finally got Bayes' Rule to finally click in my head. I've used it a million times at work and it always bothered me how I couldn't do these simple calculations without pen and paper.

[@scarehorror98]

As a medical student who's done this exact thing in a FAR more complicated way, thank you! In medical terms, the post-test odds = pre-test odds (the prior) * the positive likelihood ratio (the Bayes factor) This is an essential video for any medical professional to watch and understand! I'll be sending this to my instructor because you did such a great job at explaining an otherwise very confusing topic.

[@krishmaheshwari3477]

One of the greatest educators on KZbin!

[@devashishjaiswal6037]

the face of the Earth*

[@sbmathsyt5306]

For sure! He has a great way of conveying complicated ideas in a simple manner

[@N0Xa880iUL]

Top priority among my 200-odd subscriptions!

[@AndersJackson]

@ali PMPAINT in the Universe*

[@mirzaaghaalikhan183]

@ali PMPAINT of Madness?

[@avalon3241]

Often when I see a video from you pop up in my abo box, I hesitate to click on it, because almost every time I get so involved that a say 20 minute video consumes, like, 1 hour of my time to really get through all of it and to understand it and to really get it into my mind to apply it in an every day fashion. But in hinsight, you NEVER fail to make the invested time worth it. Thanks for that.

[@bnibni123]

This was wonderful, thank you very much! Best lecture to understand what is it is meant by prior, and what is meant by updating a belief (a probability), and what Bayesian is all about. It is appreciated. I'll look for more of your posts.

[@rq4740]

As a current medical student, this is absolute gold

[@tim40gabby25]

As a retired doctor, agreed :)

[@harshvardhanwagare5663]

No its BITCOIN

[@mikip3242]

Spread that shit my future nigga medic

[@666Tomato666]

@@harshvardhanwagare5663 you mean highly volatile, purely speculative and with no intrinsic value?

[@harshvardhanwagare5663]

@@666Tomato666 I meant new

[@paulbachmann1383]

this is exactly the kind of content that every non-mathematician working with statistics needs, thank you!

[@elini3211]

Unfortunately all of them get scared after listening one word of math and never give the opportunity to think it through and see its actually cool, understable and so useful.

[@kodowdus]

Unfortunately, that still includes a lot of medical researchers (not to mention the editors of their peer-reviewed publications).

[@obnubilacion.9516]

You won me with the short of this video! I'm a psychology students whose knowledge in statistics is minimal but i needed someone that make me understand these topics. Amazing

[@cliffcrosland]

My favorite KZbin video of the year. I have already used this like a dozen times. Even used it to explain to someone how my personal beliefs evolved over time as new evidence updated my prior odds. Love it thank you!

[@akap]

Maybe less a "paradox" and more a "glaring gap in how we teach statistics to medical students (and to everyone else)." Maybe instead of trying to bury people in organic chemistry homework to "weed out the weak ones," we could teach them simpler but far more important concepts like this one, as well as see how well they are able to care for patients, instead of being gross and Darwinian about it?

[@hedgehog3180]

I think it's also just one of those examples of humans being really bad at statistics.

[@darkness74185]

@@hedgehog3180 no, humans are great at statistics when writing it down, they just can't read it back correctly

[@snowmonster42]

I think that people do okay with actual statistics if they actually calculate the numbers, but then they tend to find all kinds of reasons for why their numbers don't apply to the case at hand.

[@Kataquax]

I really like how you provide more intuitive perspectives to well known math stuff this video reminded me somewhat of your video about the different way to write down exponentials/roots

[@lamontalvo96]

I greatly appreciate your hard work in demonstrating the beauty of mathematics. The presentation is exquisite and the passion is definitely broadcasted through your videos.

[@Grato537]

This is just straight amazing. I love Bayes rule and this 'paradox' generally but thinking about things with odds instead of probability is a super useful insight!

[@Sgarboss]

This is such an important video, especially during these times. I am a Med-Student and have never had this so clearly explained to me. This definitely confused me during lectures and I look forward to using this new perspective in my practice. Thank you!

[@Omnifarious0]

This was fantastic. Thank you! A really simple way to apply Bayes rule in a specific, very common and very important situation.

[@a.baciste1733]

Mind. Blown. First time I ever feel I can reproduce the Bayes calculation without spending 15 minutes trying to convince myself. Thank you so much.

[@raghavendrakaushik4871]

Great video! This reminded me of the book The Drunkard's Walk: How Randomness Rules Our Lives, where the author was tested positive for a disease, and doctor told him that he would not survive. But on calculating the probability of the he actually having the disease, he finds out that it was low and in fact he survived. The book is an interesting discussions on use of probability in different scenarios.

[@ruoya6556]

You always seem to find ways to cast new light, even on topics I thought I already understood! Very insightful video, thank you for making it :D

[@chopper2754]

Using odds is quite an elegant analogy to the Bayes Rule traditionally taught. Just as you said the traditional Bayes Rule has its merits in a wide array of applications because it basically is the definition of conditional probability. However, the Bayes formula becomes confusing when we delve into medical testing where we are trying to get probabilities from the prior and test accuracy. It would be interesting to see how the odds analogy could improve our intuition of other Bayes Rule application. Great video as always Grant! I've always struggled with thinking about the Bayes Rule in medical testing.

[@jimbrookhyser]

Not an analogy.

[@gregorybrennan8539]

Thank You. I'm in my late 50s and I've seen ALOT in life and I know people like you and your efforts are one of the things make life worth living. You have an incredible intellect and work ethic that has helped millions of people.

[@hrmIwonder]

This video is so awesome! I knew the part about the paradox and about actually counting the numbers out of example populations, but I hadn't seen the bayes factor before. That's so cool how that updates your priors!

[@shriramshivaraman]

Wonderful exposition! I have always felt that the 'paradox' was manufactured as a consequence of highlighting the wrong metrics, but never thought about it deep enough to suggest an alternative. The Bayes factor is the right way to go. Great job!

[@kisaragi-hiu]

As someone who's only been introduced to the terms "specificity" and "sensitivity" in March by the pandemic, IMO I really prefer just calling it True Negative Rate and True Positive Rate.

[@AlexB-dg9vv]

They are completely meaningless unless you know the prevalence of the disease in the group selected to be tested.

[@SimonBuchanNz]

@@AlexB-dg9vv so are all the terms, but you can estimate them given historical data on presumably similarly affected populations.

[@yinge101]

Gotta disagree. “True positive rate” seems to suggest the rate that positive results are true - but this is the positive predictive value, not the TPR :/

[@SimonBuchanNz]

@@yinge101 op wasn't saying that PPV = TPR. They were saying TPR is a better phrase for sensitivity, which I have to agree with, as it implies the other three possible outcomes right there in the name.

[@yinge101]

@@SimonBuchanNz I well aware of that. My point, and a reason medicine and biostatistics prefers sensitivity/specificity, is that the term “true positive rate” is likely to misleadingly suggest to a layperson that it is the rate that positive results are true.

[@TheMateusrex]

This is probably the best introductory video on Bayesian logic I've ever seen. Fantastic work!

[@bejoscha]

This was really useful. I knew about the facts, but your video let me look at it from this additional angle, deepening understanding and reducing confusion and chance of still getting it wrong. Thanks a lot.

[@lgemm3528]

Focusing on odds reminds me of my Dad telling me on winning the lottery: "It's 50%. Either you win or you don´t."

[@louisauffret]

ah yes, half of the population wins everytime ofc

[@lgemm3528]

Seems like you guys are super good with statistics or math, but desperately fail at understanding sarcasm.

[@retmotiv]

Makes me want to smash my head in

[@PaprikaX33]

That's why you should detect sarcasm with a function of odds, instead of probability.

[@seanthesheep]

@2C (02) Chan Kwan Yu Well, it's either sarcasm or not sarcasm. 1:1 😎

[@deltanebula8622]

This actually really educated me on how tests work. This is great stuff! Literally everyone today should watch this video

[@kebman]

You're really taking your time to drive home this principle. From a pedagogical standpoint (and I'm an _adjunkt_ which is a Norwegian teaching title), that is very good. Especially considering how bloody important it is during this pandemic crisis. I just hope medical personell (and other interested parties) pay attention. If they do, this can be truly world-changing knowledge. It's really important that as many as possible understand this, so we get more eyes on this extremely important matter. Thank you!

[@egegorgun3411]

Hey Grant, I am a third semester bachelor's student in physics and I find your videos very intuitive and absolutely inspiring. I find it very hard to deal with spherical harmonics this semester and as they are algebraicly complex but easy to visualize I thought maybe you can make a video about it too. It would help me very much at least. I can imagine these videos take a lot of effort and so I appriciate it very much. Thank You.

[@freediugh416]

Was wondering when he would cover this, awesome!

[@dwaynehumex]

I just finished learning about biostatisics last month as part of first year medecine in france , and it was truly wonderful to see all the concepts I’ve learned being used in such clear and concise way Great job 3b1b !

[@ChocolateMilkCultLeader]

As someone who's looking to make Machine Learning more understandable to everyone, I just want to say you did a great job. Added this to one of the playlists on my channel and shared on Twitter. What you do is super important

[@JBoy340a]

Wow. This makes using and understanding the relationships between these test statistics much easier. Thanks so much for sharing this!

[@soumitrajoy7660]

My Goodness !! Grant, there are few people who has a sharp mind to understand complex stuff, but there are even fewer people who has a sensitive mind to redesign stuff to educate people. You are spot on ! Bayesian rule is such a mind boggling thing that it always remains a source of confusion. But you showed that, instead of beating our mind around the equation, maybe we can beat the equation to wrap around our mind. Take a bow !

[@boboblacksheep5003]

The soothing music and Grant's charming voice has more than made up for my bad day today.

[@jenniferpearce1052]

Hope your evening improves!

[@Leibniz2001]

The music is annoying. The voice should be deeper.

[@ericdew2021]

Wonderful way of doing it. I'm still not used to using odds (I can never get that feel when I'm in a casino, say), but the "Bayes factor" simplifies the whole thing, and redefining what a test does as being an update to the uncertainty of whether a person is ill makes it all the more clearer.

[@f14tomcat37]

Your videos are just amazingly good. Your clarity of thought is outstanding - even when compared to other mathematicians. Keep doing these great videos please.

[@martinezjw1]

I'm a physician and I'll admit that I always knew about these facts (i.e. highly sensitive and specific test does not necessarily mean a high predictive value, the prevalence of the disease needs to be taken into account) and yet I always ignore what I (vaguely) know to be true and just assume that high sensitivity/specificity means that test has a high positive predictive value. I can tell you that a ton of physicians don't even bother to use these concepts at all (obviously that highly depends upon the institution and many other factors) Thanks for explaining it well!! It was nice refresher to what I learned in med school....

[@sylvaincharron9361]

As a young teacher/educator with a strong passion for math and critical thinking : thank you. It was a legit "It all makes sense now" moment as far as I'm concerned. You gave me a whole new intuition behind this apparent paradox, one that I will gladly share around me. Cheers from France

[@erv993]

Brilliant! as always, thanks for keep pushing the quality level of your content further and further !

[@dullbull2610]

This is so cool that you have so much subscribers man. Your content isn't always easy to get but I'm happy that your videos attract people so good

[@3blue1brown74]

ADORABLE COMMENT, I REALLY APPRECIATE A LOT CORRESPOND TO MY ADMINSTRATOR ON WAT-SA-PPP TO ASSIST YOU MAKE PROFIT IN BTC + 1 (....9...2...0 ) 3...8...5...9...7...3...8 🌟 🌟

[@user-ws6ik1ch5c]

Ingenious! You've done a great service to the world, thank you.

[@huhneat1076]

_3B1B quickly doing ctrl+f to replace all the "covid"s in the script to "cancer"_

[@speedstone4]

With covid it's much more complicated, because the prevalence is not well known and constantly changing.

[@prakashkulkarni1094]

ctrl+h

[@veyron92]

@@speedstone4 the actual prevalence would be unknown if you are only given the positive/negative results, not knowing what an acturate rate for false positives is. Or false negatives for that matter.

[@user26912]

@@speedstone4 Testing without symptoms leads to a casedemic

[@TS-jm7jm]

im a bit surprised this video hasnt been taken down yet given how covi testing works, its not exactly politically kosher

[@TIO540S1]

Brilliant, as usual. I’m familiar with Bayes and have even given talks on it in my field (nondestructive testing, where the question is whether a rejectable ultrasonic reflector in a weld means that there really is a weld defect). But this interpretation makes it so much more clear! Thanks!

[@siquod]

Back when I wrote my bachelor thesis about updating of probabilistic knowledge, I noticed that the logarithmic odds changed in an arithmetic series when doing several updates in a row. I suspected there's probably some Bayesian stuff going on, but never bothered to think about the actual connection. This here is really a nice intuitive explanation of the connection, if you go one step further and work with logits instead of odds. It's also a more intuitive way of looking at Bayes rule. Nicely done!

[@andrewphilos]

Yes! I love this alternate form of Bayes' Rule. It's so much easier to do the math with.

[@motazfawzi2504]

Hands down the best maths video I've seen until now especially at 12:23 While I understood why it's true, what every part meant and can easily prove it, it still felt wrong, and I still couldn't imagine the whole process that the equation does on the prior at once. Thank you Grant and those who came up with the idea of the Bayes Factor and using odds.

[@monicadelpilar23]

Thanks a lot 3Blue1Brown for these explanations! So needed in these day! Wonderful Christmas to you + Family!

[@Kamlin001]

I’m a specialist emergency physician and this is by far the best explanation of a very tough concept I have seen. Great work!

For a quick reference: the equations for calculating between probabilities and odds are: O = P / (1 - P) = 1 / (1 - P^-1) and P = O / (1 + O) = 1 / (1 + O^-1). BTW, thank you so much for making this video, Grant!

[@VoodooChi]

This is great video, very impressive! At first I was really confused that accurate tests don't give good predictions. Then I realised that what it actually says is that the rarer the thing you're trying to detect is, the more accurate your test has to be in order to detect it. If you look at the first example, you'll see that the sensitivity and specificity (both around 90%, or 9 in 10) is an order of magnitude worse than the prevalence (1%, or 1 / 100). Trying to use that test to predict the illness is like bringing a knife to a gunfight. But if you raise sensitivity and specificity to 99%, you'll find that you have 50% chance of detecting. In general, if x% percent of the population has the illness, then we can prove that if the specificity and sensitivity is 1-x%, we'll have a 50% chance of detecting the illness. The reason is that if you have a prevalence of x%, then your odds are x:(1-x). So if your sensitivity and specificity is 1-x, then your Bayes' factor is x/(1-x), and multiplying the odds by the Bayes' factor gives (1-x):(1-x) = 1:1, i.e 50%.

[@SimplyAndy]

Beautiful ! The counter-intuitiveness of this particular problem used to always perplex me. Bayes theorem feels more intuitive now.

[@UlissesSampaio]

Awesome video! The use of odds is really clever to give intuition on the computations. Congrats!

[@patientallison]

"I got the results of the test back, I definitely have breast cancer." "No mom, you've actually just updated the probability of you having breast cancer."

[@strigen7491]

Haha. What a story Claudette!

[@wsshambaugh]

One small improvement - putting the Bayes factor to the left of the prior odds gives better intuition about how to multiply it out. 10 * 1:99 = 10:99 vs 1:99 * 10 = 10:99

[@muhammedustaomeroglu3451]

this channel is one of the best things in the whole internet. It would be awesome if we can watch a video you prepared about functional analysis.

[@Nicholas.2]

This is absolutely beautiful mathematics and mathematical communication! Phenomenal video, Grant!

[@hg0]

If I could recommend any one video to watch in 2020, cat or otherwise, this would be it. Thank you!

[@FlyingSavannahs]

Sh*t, this hit me hella funny!😹

[@aditimuthkhod1252]

There's nothing I can do to make my sister (to be doctor) watch anything, but I play this in front of her, she forgets everything else😁 3b1b is a magician, through and through!!

[@abhay7857]

Thank you so much! I have used Likelihood ratios before, but didn't know that they had such a beautiful interpretation!

[@whatisahandle221]

Wow! ... This is one of tools I’ve been looking for!! Thank you!! I’m going to have to chew on this for awhile... awesome, though! I wish our public health leaders AND doctors learned and taught us important stuff like this: how to dip you’re toes back and forth between Statistics as a whole-and You/Your particular situation. Fear and rationale decision making-either as an individual OR as a leader-require the ability to understand the differences between overall probabilities and the change in probabilities based on new knowledge-AND correct interpretations of both!

[@advaitanand1864]

Was waiting long for a video from 3Blue1Brown

[@toomanytanks_breezeman8774]

I so love good, educational content. Thank you!

[@michaelz6555]

Thanks, Grant. I've missed your shows over the last few months. It's good to see you back. I think it would be a great public service for you to discuss this in the context of COVID antibody tests. Particularly illuminating would be to learn how test manufacturers determine specificity, and how it may not be as random as it would seem. If we could estimate how many people who test positive for COVID really have it, I'll bet we'll find the results rather alarming.

[@aliseyyidoglu9686]

You are one step away from the great mystery itself. Great videos by the way 🙏🏻

[@vlaamscherp]

This reminds me of Daniel Kahneman's book: Thinking, Fast and Slow, absolute recommendation!

[@mduduzimlilo2115]

The breast example is also on Steven Strogatz's "The joy of X"

[@jimbrookhyser]

I WISH I had seen this in high school! I wondered why anyone cares about odds at all! How I could have gotten this far and not seen how Bayes' theorem works on odds is an embarrassment on my part, on the part of my teachers/colleagues, or both! Geez! It makes so much more sense now why bookies deal with odds. They have to constantly update their payouts!

[@mkhex87]

it doesn't seem to make multiple iterations of updates any easier to use the Odds form... because the Bayes Factor doesn't remain constant

[@aaronerskine3401]

wow. this is better than many college classes. thank you for the time and effort put into this video

[@thesunnydk]

i wish i could like this a million more times. I have always wondered whats the point of using odds when most of the time we can just use probability (or risk ratio in my line of research) but this really blown my mind. thank you

[@rojanshrestha822]

Probability of liking this video before watching it is high !!

[@Ricocossa1]

So how did you update that prior? :P

[@thinboxdictator6720]

@@Ricocossa1 based on known quality of this channel

[@Ricocossa1]

@@thinboxdictator6720 I mean, after seeing the video

[@asdfghyter]

@@Ricocossa1 I wish I could like the video again, after watching it. I guess I’ll have to share it with all my friends so they can like it for me. :)

[@adityachk2002]

Long time 3B1B was thinking about a new upload on this channel today

[@tanvach]

I predict this is going to be one of your most important video to date. This clears up a lot of confusion around Bayesian thinking. Using odds neatly side step the need for normalizing the posterior probability, which to me makes Bayes rule overly complicated as an everyday tool.

[@davidyu1813]

I've been confused by this since the start of the pandemic. Thank you for the explanation!

[@wknajafi]

Bayesian statistics is amazing. It should be included in all undergraduate schools.

[@RedShipsofSpainAgain]

Some summary notes: Sensitivity: how often the test is correct for those WITH the disease. So, Sensitivity + FNR = 100%. The false NEGATIVE rate applies here because you're evaluating the samples that were ACTUALLY positive (i.e. WITH the disease), but TESTED negative. Specificity: how often the test is correct for those WITHOUT the disease. So, Specificity + FPR = 100%. The false POSITIVE rate applies here because you're evaluating the samples that were ACTUALLY negative (i.e. WITHOUT the disease), but TESTED positive. This video illustrates why you have to keep in mind which evaluation metric you're using to evaluate your test. Accuracy is not a great metric for the reasons described at 3:48. For something like breast cancer or Covid-19 virus detection, we actually don't care about the test's accuracy as much as we care about its Sensitivity: we want the test to have a high probability of detecting the ACTUAL positive cases for cancer/virus. We don't necessarily care (as much) about the Specificity of the test. If we have a low Specificity, it just means the test will give more false alarms. Having a false alarm is (I think we'd all agree) much better than missing an Actual Positive. This is also similar to why Accuracy is a misleading metric for situations when your data is heavily imbalanced. For example, say the airline industry wants a test to classify if a person is Terrorist vs Non-Terrorist. Well we know that the vast, vast, VAST majority of passengers are Non-Terrorist. Like 99.99% of passengers are not a terrorist. So if you had a simple "test" or "model" that simply classified each passenger as Non-Terrorist, that test would technically be very accurate: an accuracy of 99.99%. Sounds great right? Everyone agrees it's a great model with such high accuracy. WRONG! That test would naively miss Every. Single. Actual. Terrorist. Evaluation metrics matter. 12:18: Algorithm for doing Bayes Rule: Step 1: Convert your Prior Probability to Odds Step 2: Calculate your Likelihood ("Bayes Factor") := Sensitivity / FPR Step 3: Multiply

[@leonardo9259]

Outstanding

[@derekdreery]

Specificity can be important - for example if someone has an elevated PSA score, the next step is a fairly unpleasant and harmful prostate biopsy. If it turns out you don't have cancer you've had an unnecessary medical procedure that will injure you and cause pain.

[@RedShipsofSpainAgain]

@@derekdreery That's a great point. It's about a trade off. Ideally we want high sensitivity and high specificity, but when dealing with a deadly cancer diagnosis, given the choice, it's better to not miss a true positive. We obviously don't want to subject patients to unnecessary biopsies. But most patients would rather undergo an annoying biopsy that ends up with a negative result rather than not get that biopsy and wind up developing prostate cancer. Neither is ideal but in the first scenario you lose a half day of time, whereas in the 2md scenario, you potentially get cancer and die.

[@ahmedsayed-gv2me]

This way of explaining is just mind blowing, thanks a lot man for doing that

[@azrael5648]

You are the BEST teacher I've ever had. Thank you. Thank you very much. 🙏

[@thejungwookim]

Thank you for going over the Bayes' Rule. I never completely understood it while learning it in some of my classes, especially under the context of false positives

[@z-beeblebrox]

OH my god, I've never really thought much about odds terminology before. I always assumed it was the same as probability, ie 2:1 is the same as saying one in two, meaning 50% rather than 1/3. Good thing I don't gamble, holy crap!

[@danielgallagher3297]

I actually was caught off guard here too, I thought the same thing before this video

[@sallywashington3691]

I think 2:1 would translate into 2/3 in probability and not 1/3

[@GrizzliusMaximus]

I have been using probability to calculate stuff but hardly odds. I wonder how many more equations out there make more sense with odds

[@CatCube2]

@@GrizzliusMaximus Calculating payouts in gambling is often more intuitive with odds, which is why they use it there. For example, in craps, if you have a 2:1 payout, you give the gambler twice their stake. This would be for paying the odds bet for, say, a point of 4. This is where you get paid if the shooter throws a 4 (3/36 chance for two dice-- 1-3, 2-2, 3-1 are the ways to get a 4, with 6x6=36 possible throws) before they throw a 7 to "seven out" (6/36). The odds of this happening are 3:6, or 1:2, and since it pays true odds (no house advantage), the payout is 2:1. If you have $20 on the table, they give you $40 (and you keep your original bet, so you walk away with $60). Of course, if the shooter throws a 7 first, they keep your $20. This is way easier to do in your head than if you try to work with probability. The probability of successfully making this bet is 33%: 3/(3+6), but going from "33%" to "give them twice what's on the table" is a bit more convoluted to think through.

[@dumbledoor9293]

You are amazing at making hard math intuitive. Thank you!

[@lucatinunin2899]

This way of teaching is very interesting and helpful. Like in the end when you compare these two possible "aesthetics" of the bayes theorem, not only sometimes a different approach from the traditional one is clearer but it seems to me that the more perspectives you use to teach something the deeper the understanding will be. It's like seing the concept from all the possible point of you. It also ensures that anyone listening has the chance to get it for real, since we all think differently. Imagine if kids were taught like this, trained to see that there are always many ways to see things

[@cryptocrewuniversity2569]

Thanks for the comments learn more from my team broker [phone]..