The Ups and Downs of Evidence Based Medicine

[@DanThePropMan]

If one is concerned about clinical practices lacking robust evidence, surely the answer is to GATHER MORE ROBUST EVIDENCE--not to throw out the very concept of evidence entirely.

[@jmarch_503]

DanThePropMan science

[@imaginareality]

I am a child care worker and I think that evidence-based parctice is super important, but also that there are limitations. Take ABA therapy for children with autism for example or psychopharmacotherapy for children with conduct disorder / disruptive behavioral disorders. Both are evidence-based (ABA more so than pharmacotherapy) but both should be discussed critically. Some outcomes cannot easily be measured, studies are often relatively short-term and conducted in a clinical setting and there are also ethical oncerns that need to be considered. We should always ask ourselves who is researching what and how? I will not implement ABA even if studies tell me that I should, because I think it's unethical and doesn't reflect the goals that I personally have when working with children. But maybe this is a specific problem for psychology and related fields?

[@orlendatube]

Its key to remember that every body is unique, and just because something worked for others, doesnt mean it will for that individual. I have many rare disorders and also tend to react strangely to things, and I have found that some dr's are so enamored with evidence based medicine that they arent willing to think "outside the box" with me. Which is sad because it denies me needed care....

[@Quagthistle]

I have the EXACT same problem! Their focus on "if you hear hoofbeats, look for horses" leaves "zebras" like us overlooked and often recommended for treatments that are harmful or dangerous to us. Since little evidence exists for rare conditions (it's hard to do a study on a disease that affects a handful of people in the US, for example), assumptions are often made that put people like us at serious risk of debilitating harm. And, to make matters worse, doctors often refuse to listen to what evidence we DO have (experience), and dismiss it outright with an "I'm the doctor and I know best" mentality. That said, I don't think we should dissmiss evidence-based medicine. Surely, there's room for compromise between the two extremes. Every rule, no matter how well-intentioned, has an exception. (For example, excersize is great for you... except when it causes the heart to go haywire and leaves the patient bedridden for days, especially when the underlying heart problem is untreatable.) Any system that cannot vet evidence and change based on valid evidence, or that lacks the flexibility to realize exceptions must always exist, will never be adequate, function properly, or be well-perceived.

[@avidrucker]

Thank you as always for taking care of us! I purchased just now your book for my family :) ~ Avi

[@aaroncobeng]

I love your work. Specially about the generic and branded medicines topic. Showed it to my students and made my lecture so much easier to explain!

[@ZodiacKillers]

Evidence-based medicine should include the evidence that shows that individualised care and beyond broad pathophysiology. Not to mention the importance of rapport, psychology and the end goal of medicine which is not just to simply extend life as much as possible. When I think of evidence-based medicine, I think of tailoring the best evidence to your patient.

[@AscendantCaleb]

Does HTC have a video on R.I.C.E. (rest, ice, compression, elevation) and it's apparent lack of scientific evidence? If not, that's something that seems like it'd be pretty interesting.

[@jeffbrownstain]

Your mention of AI intrigued me. We're all so aware of the dangers of it, yet we continue to voraciously approach that goal, and we have no plans on how to fix the potential problems that might arise. Humans are an interesting bunch.

[@MrBOGooooEATER]

Please add links in the description to all the things you said at the end

[@Victor-qy1uy]

3:45 slight but important thing to note is that having a large enough cohort will ensure that anything not clinically significant/generalisable will be rarer. Doing more studies on smaller cohorts will increase the chance of picking up error though

[@tjbrunson]

Hey Aaron! I love your videos. I make sure to keep up with the healthcare sphere because I am pre-med and am applying to med schools this June. Your videos are an integral part of increasing my healthcare knowledge. I am curious if you have ever considered going on the Joe Rogan podcast, or any podcast for that matter. I think it would be a great opportunity to share your channel, your new book, and your knowledge about how to understand scientific research. Again, thank you for your wealth of videos!

[@ghuegel]

I'd love to know Dr. Carroll's thoughts on science-based medicine. It seems to me that tis concept/approach mediates a lot of the problems associated with evidence-based medicine.

[@playc.holder6432]

We need to focus on gene therapy, neurology, and immunology. These three areas are most negatively affected by evidence-based medicine. And also the most likely areas of medicine to revolutionize individual treatment options.

[@whatareallofthis]

Any chance you could do an episode on the "hierarchy of research" you mentioned in this video? I have a science background so I'm pretty familiar, but I'm sure it would be helpful to a lot of people.

[@robertstuckey6575]

Whether in heath or other fields, including my own, (mental health) I often see EBP as an excuse to make radical shifts that are not always proven. Having discourse about research and effective practice is so challenging anytime humans get involved. We need more discussion and professional humility when attempting to care for others.

[@DCsk8rgoelz]

that last sentence a million times. tack on an unyielding inquisitive nature and we might get closer to a better system.

[@trentgraham465]

Great video! Are you planning to make one on your Upshot column on heart stents?

[@unvergebeneid]

So yeah, using a scientific approach seems like a reasonable idea when it comes to life and death decisions. Although to be fair, a lot of sciences have trouble with people not using effect sizes and only relying on statistical significance instead.

[@tenou213]

I agree with all points but the solution seems to be...more evidence based medicine in order to refine that path. Because evidence needs to be properly gained and findings transparently issued. Corporate funding distorting findings by 40% on average (Cochrane Collaboration) doesn't help.

[@Frexican54]

Don't a lot of these Medical studies also have problems being replicated?

[@davidabad469]

It's mysterious that the title uses the (incorrect) phrase "Evidence Based Medicine" while the (correct) phrase used everywhere in the video is "evidence-based medicine".

[@TreantmonksTemple]

I wasn't always following with this one. Some examples would be helpful, if you don't want to give real examples, some hypothetical ones would be fine.

[@vincetheworldly537]

What baffles me is why is there even an argument with research-based medicine. Whats the other side of the party? Homeopathy? Chiropractic medicine? Shamans?

[@bdallaway]

Amen brother

[@soyllamu]

Buen video, nuevo sub

[@me-hk4rl]

the "EVIDENCE" is often produced by people paid by special interests and skewed to ignore what they don't want to see and doesn't fit their desired outcome.

[@hana.gemini]

I really enjoy your videos but often feel like I'm being yelled at or judged..

[@malikathueler2529]

I agree he often looks super pissed off 😂

[@sneakerbabeful]

Please don't take it personally. He's judgy of systems that don't help people, not individual people.

[@antoonsoetenshealth]

Society is based on progress such as: Evidence-based constructions buildings - machinesThis exists because every part of the construction is known down to the detail, composition, size and objective. This allows a scientific certificate of the degree of quality to be delivered. Evidence-based medicine of the human body with 100% living cells that can grow, multiply and die, but nobody knows exactly how that happens. Consequently, delivering a scientific certificate with the grade of quality does not exist, because: - one does not know the composition of 1 living cell, let alone that medicine will determine how to repair the cells. - the totality of the number of cells is unknown. - one does not know the functions of the cells that make the body function optimally. - one does not know the totality of functions in the body.- the name, proteins, vitamins, ... and their composition are a vision, but nobody can determine the exact composition of the atoms and physical properties exactly. The same for neurons, genes ... are a name for something that is not known exactly. Conclusion: therefore there is no evidence-based medicine if the basic elements cannot be precisely described. The result: the human body is not a well-known medium for internal processing without causing damage. But the problem is much worse:- the scientists, doctors and politicians have laws approved whereby dead cells = chemicals come into contact with a medium of 100% living cells: the human body. The result: dead cells (chemicals) do not have a medicinal effect, but poison and a number of side effects have arisen. The law encourages the use of poison, this is not possible because a law cannot be used to harm people. The same reasoning applies to animals and nature that also consist of 100% living cells. The scientists are responsible for their absurd conclusions and actions.

[@captainhoratiobungleiii7147]

ME/CFS is a good example of this. "Evidence" said it would go away with exercise and happy thoughts. Turns out that evidence was based on some of the worst science ever done!

[@SaucerJess]

💚

[@jeffbrownstain]

Everything I know about healthcare comes from the past half a year of Healthcare Triage. So there's that.

[@martingoodman3300]

Another way to phrase some of what Dr. Carroll is trying to say here is that even when there is good clinical and scientific method used to study a particular issue, the results may not be relevant to what they are being cited to support, because many medical care and public health issues are affected by a huge number of factors, and looking at only one contributor to them myopically... no matter how well researched and accurate the information on the one factor is... can give a highly skewed and often even entirely false view of the reality of the complex bigger issue at hand. Either due to deliberate efforts to deceive, or honestly ignorant use of the information. Dr. Carroll is especially on target here... and in others of his videos... with the support of irrational fads in the mainstream media after one narrow area of research findings are announced. This is especially important in the case of risks vs benefits, where there may be a given risk identified, but that risk may be massively outweighed by the benefits. The picture becomes even more complex when one is looking at benefits that are subjective, not objectively quantifiable and measurable or applicable to all, and result from taste and life style preference. A love of hang-gliding or mountaineer, for example. Or: A love of the taste of red meat to the point where one can rationally decide "I like this SO much I am quite willing to give up 5 years of my life, if it means I'll lose that from eating as much of this as I do, in order to increase the quality of the life I have, as I see and perceive quality of my own life to be." Where the near always in his videos here brilliant, articulate, and exceptionally accurate and to the point Dr. Carroll may be either missing or somewhat neglecting an important issue relevant to his subject in the subject of this video is the basic truth that trying to provide medical care on a for profit basis, in the interests of a few astronomically wealthy individuals behind the pharmaceutical and insurance companies, will always result in massive and obscene perversions of the provision of care to any population. To be sure, any real world society has finite resources, and will have to make decisions how to best spend them on medical care. And be helped by science and evidence based findings in doing so. But first you have to eliminate the situation of putting huge fractions of that money spent on "public health" into the pockets of a small group (some of us would use the word "class") of powerful and super-wealthy individuals.

[@kimbyd24]

One of the things that can be extremely frustrating with regards to evidence-based medicine is when it is used when comparing evidence gathered in one population to create guidance in another. A very significant example here is that of transgender women and hormone therapy: the vast majority of the research regarding the benefits and harms of feminizing hormones (typically some combination of testosterone blockers, estradiol, and/or progesterone) has been done in post-menopausal cis women, which is an incredibly different population. The evidence for hormone therapy in transgender women is extremely poorly-studied. We do have good evidence that overall it is of tremendous benefit psychologically for transgender women. We do know that the largest health risk appears to be blood clotting. But there are many dimensions along which there just isn't any good evidence, such as: 1) What is the right hormone regimen to maximize breast growth? Transgender women tend to have less total breast growth on average than cis women, and tend to be rather unsatisfied with the amount of growth. Learning what regimen is most likely to be effective would be of enormous benefit. 2) As most of the studies on harms for hormone therapy have been in post-menopausal cis women, do some of the harms translate at all to the usually much younger and healthier trans women? Many claim that progesterone is a bad idea because it has not been proven to improve the health of post-menopausal cis women, and may increase their cancer risk. There's no good reason to believe that either statement is accurate for trans women. The benefits for breast growth and the risk of breast cancer are both poorly-studied in trans women (progesterone *may* lead to rounder breasts: most trans women have rather pointy breasts, but it may also suppress the action of estradiol in promoting primary breast growth). 3) Is the largest risk in trans women avoidable (deep venous thrombosis, i.e. blood clotting)? There is some supposition that it may be a result of swallowing estradiol pills. If the trans woman uses a patch, injection, or uses sublingual administration, is the risk of DVT still there? 4) What are the surgery risks associated with various hormone regimens? Should trans women stop taking hormones prior to surgery? How long before? Note that some trans women experience severe increases in psychological illnesses as a result of stopping hormones. I really do want more evidence-based medicine, but it seems like all too often it's applied poorly to underrepresented populations. Right now, trans women's best resources for understanding the medical implications of their care are usually other trans women, which is very inconsistent and harmful. I want more research, but in the mean time I wish more doctors would avoid applying evidence that does not apply.

[@docsamurai9199]

1?

[@BeCurieUs]

Heh, feels like this entire video was in response to a twitter beef Aaron had!

[@lightbox617]

20 years ago, my cholesterol levels were perfect. Now, the essentially same levels are "borderline high." should I not wonder if this is due to some Big Pharma company needing to sell more drugs? Has the data on "utcomes" changed that much? Please keep doing this and write more for the NYT.

[@fkangalov]

Timothy Dingman Science/Technology improves, more and better studies are made, we learn new things sometimes the hard way or by making a mistake and come to more relevant and accurate conclusions

[@Suph373]

Not only have guidelines changed in the last 20 years, but your body has too! I would consider those both, in addition to concerns about Big Pharma, when looking at your health.

[@ChristianMCI]

Cigarettes were also considered healthy back in the day. And butter was consumed in large quantities because it was thought to make our vessels 'slippery' so good for blood circulation. Look how that turned out to be.

[@StrongMed]

You are right when implying the cholesterol guidelines have changed (more more dramatically than hypertension guidelines). However, the expert consensus is now that treatment for high cholesterol should be based on overall cardiovascular risk, rather than just a person's absolute level of cholesterol. So a cholesterol of x in a non-smoker might not warrant treatment, whereas that same level in a smoker might. Since you are now 20 years older, that increased age is also a risk cardiovascular risk factor. Thus, even if the current guidelines had been in place 20 years ago, you could still have had a discrepancy between whether your cholesterol looked good then vs. now.

[@zomakaja]

3rd?

[@donedennison9237]

Taking a complicated topic of medicine and attempting to apply solutions to a general population in a capitalist environment just adds chaos. I love capitalism AND science, but feel mixing the two creates markets out of hokum. It's possible to do the science and then use capitalism to promote the proven outcomes, but I don't know anywhere where that happens. How can one discern if a prescribed treatment is truly evidence based best practices or merely tradition? How can one understand "increased risk" studies where something doubles, yet the initial risk was so low to make that increased risk remain insignificant? Flat out confusing as a patient and I bet as a medical pro too.

[@Aziraphale686]

Why would anyone be against evidenced-based anything? What else would you do, just guess? Even anecdotal evidence is still evidence ffs.

[@DinoRamzi]

Worthy subject but poor argument. I was going to point to Sacket’s work in the 70’s and 80’s and the first incarnation of EBM as “critical assessment.” In fact, the elements of scientific epistemology go back to Aristotle. That some of the western scientific tradition has succumbed to the “difficulty of understanding what their salary depends on” is likewise, not a new. Nothing in human nature is new. It can almost have been anticipated that monied interests would attempt to coopt or discredit modes of thought that hurt their growth plans. Much the same way, direct primary care is being coopted or, at least redefined, by elements that stand to lose if DPC gains much of a foothold. Sorry EBM is a useful framework for an external validation of evidence based on agreed-upon criteria. The easiest way to attack it, is undermine the agreement. Climate anyone?

[@Me-by8qi]

It's difficult to do good science.

[@TheStackeddeck77]

So wait evidence based medicine isn't based on math and science? More like anecdotal evidence? I'm confused anyway I live math and science and there isn't anything that we know today that proper application can't point you toward. So can't we have a systems of medical research based solely on the maths and sciences? Sure clinical trials are super important, but how do we get there, trying it on animals who have similarities to us like rats immune systems, but that is still way off it seems like math based trials would be a better start. However I had never heard of evidence based medicine so this is more just me asking why not.

[@cinnie63]

TheStackeddeck77 EBM is based on science, the problem is getting the evidence in the first place. Much of current medical practice is not evidence based. For example you might have heard it said that "breakfast is the most important meal of the day" but there is little actual scientific evidence to back that claim up. It is extremely hard to study diet in humans, you can't lock them in a cage and control their diets for long periods of time. More is scientifically known about the diets in animals because you can strictly control their diet for long periods of time. That is why there is a new diet fad every few months. We know only generalities about what diets work and why, there is still so much to be proven.

[@JorgeSanchez-je4bt]

I like to to that evidence does not exist, just interpretation. If one understands the concept of type I error one will realize that there is no such thing as absolute truth in science. It is only a reasonable guess towards something, knowing that if the same research was repeated 100 times it will be wrong less than 5% of the time. I read once that the inventor of the P value, which is standard statistics in journals these days, never intended it to be used to imply truth. But rather, depending on individual values each individual should decide on their own what the truth is. For one person maybe their critical alpha is .05 but to another it is .00000001. The trouble with guidelines is that it is misconstrued as truth. The public and the media live in this world of black and white, in good and bad. But as one KZbinr I follow says, truth has a habit of resisting simplicity. Have a good day.

[@Hephera]

I really dont think this video explained what youre even talking about when you say "evidence based" you were far to vague when describing the difference and didnt really use any examples of evidence and non-evidence based medicine. Im sitting ehre at the end of the video with basically no clue wtf you were walking about for 6 minutes

[@cinnie63]

Let me use a very non medical analogy to try and explain this. Say there are two tire companies that make very different tires but they both claim to have the best tire. To see which is the best tire for you you would have to use them in an identical way for a period of time. At the end of that time you would have a bunch of data. Some of that data would be objective (it can be measured) like how long each tire maintained a certain tread depth under your specific driving conditions. So if the only thing you were interested in was how long a tire lasted then you would have a definitive answer. But often there are multiple criteria to choosing any product ( or medical treatment) so your study would not help chose a tire that resisted puncture, performed well in rough or wet conditions and most importantly would not apply to all vehicles. So although your study is evidence based, it had a measured outcome, it would not apply to all people buying tires. But what if we expanded the tire study and enrolled your whole neighborhood? They all drive random vehicles, drove different mileage over vastly different roads and and yet we still only measured one outcome. Then that study would give the average wear on those tires across the whole neighborhood, but it would not give you the exact wear that could be expected for each individual. The first study had a narrow scope applied only to you and people like you. The second study had a wide scope and applied to a wider population.

[@tiborpejic2341]

Get a large enough cohort and you will achieve statistical significance??? That makes no sense.

[@unappropadope]

so the ups of evidence based medicine and the downs of poor communication/education

[@Overonator]

Medicine should not be evidence based. It should be science based. There is no reason to study acupuncture for blood pressure by doing double blind placebo controlled clinical trials. We don't throw out everything we already know about human body in order to perform double blind placebo controlled trials for something so implausible. We have to consider prior plausibility before considering doing clinical trials and that's what science based medicine does. Evidence based medicine would waste resources on doing clinical trials on implausible interventions. That is why science based is better than evidence based.

[@martyyu]

sciencebasedmedicine.org/

[@rugbyguy59]

So you’re rejecting acupuncture without testing it based on what you subjectively decide is plausibile. That’s not a science based approach. PS, I’m not an acupuncture believer.

[@jonwhite8815]

I was going to write a snarky comment about how science based is evidence based, but googled it to see if anybody has elaborated on the distinction between the two and found this link: www.painscience.com/articles/ebm-vs-sbm.php And I learned something, so thanks. One limitation I could see arising in that approach is the possibility that legitimately beneficial treatments could get dismissed as implausible and not get studied. Lots of things that initially seemed incredibly implausible turned out to be true.

[@Overonator]

Actually I'm not subjectively deciding anything. We aren't totally ignorant about medicine biology and physics. We can use what we already know to realize that acupuncture is an extremely unlikely intervention for blood pressure. By your logic maybe spinning in a circle will help with migraines. We won't know until we waste the resources to test it right?

[@Overonator]

No if we had more resources we could invest resources into implausible interventions. We don't have those resources so we have to have some degree of prior plausibility before we even start the clinical trial.