Not all scientific studies are created equal - David H. Schwartz

View full lesson: Not all scientific studies are created equal - David H. Schwartz
Every day, we are bombarded by attention grabbing headlines that promise miracle cures to all of our ailments -- often backed up by a "scientific study." But what are these studies, and how do we know if they are reliable? David H. Schwartz dissects two types of studies that scientists use, illuminating why you should always approach the claims with a critical eye.
Lesson by David H. Schwartz, animation by Augenblick Studios.

[@GZXC993]

Unfortunately, looking into studies is a lot easier said than done. I've tried reading actual studies to see how they came to their results, and most I came across are published in scientific journals that require you to pay money to see the contents. Most allow you to see the abstract, but not the actual contents of how the study was conducted. Additionally, even if the studies were publicly available, the studies are structured to support the conclusion in the abstract; such as only using selective data, as mentioned in the video. This makes it very difficult to discern the validity of the test because you can only see the test in the framework that the author intended.

[@mrs.hendrysyoutubechannel2355]

Who is funding the research? This is important too.

[@W1z3k]

Search for sci-hub

[@mauilawrenceangeles7802]

This is a more common problem on high schools who are conducting research studies since they are less likely to pay money and that they usually lack resources and knowledge (unless they search the internet that deep).

[@safir2241]

@@mrs.hendrysyoutubechannel2355 honestly i just look at that. a lot of ppl are saying meat doesnt contribute much to greenhouse gases, but those studies and their spokespersons have ties to the meat industry

[@namename1307]

Which is why some turns to privacy, such as scihub, for access to these scientific journals

[@CrzyLion]

its pretty hard for people with no university-level understanding of biochemistry to look up a study and form their own thoughts on the study. As a student in medicine, i have no problems (except for having to look up a term i dont get in english every now and again) with this, and i know the sites to check (but i can only check them with my university-provided-account). My friend who hardly did high school chemistry, but who is equally skeptical about studies, has no chance whatsoever to do this. He's more than bright enough, but if you look at how most of these things are written you'd think its a WW2 code language. Kind of sad considering most of the conclusions in studies shouldnt be hard to (re)write in human language. Am i the only one who thinks it's rather silly that in this day and age of information-abundance; its mostly the choices you made at 16-17 that determine whether or not you can understand whats going on in science? Especially when "whats going on in science" often translates to "am i being healthy or am i doing the exact opposite".

[@samshaw7087]

No it’s not. I bet you feel pretty smart though.

[@blauwbeer556]

I doubt it, you don't need to understand the contents, just that the experiment/study seems legit. That there is little to no bias in the way it is handled , everybody can see that after seeing this video.

[@DavidViviano]

Thank you for this video, I am tired of trying to explain this to people and they still do not get it

[@rtyzxc]

People simply don't understand the logic behind correlation and controlled experiment, so they continue using the wrong intuition that correlation is some kind of causation, regardless of how much it's being refuted. I completed an internet course in Udacity named Statistics, and I liked how they really hammered it down, asking in excercise problem whether the setup could be used to prove causal relationship, out of 90% of the time, the answer was no, even though many times you are tempted to answer yes.

[@blauwbeer556]

It isn't that hard with a good example.

[@Chadlite]

Being skeptical about most things will benefit you in the long run.

[@Chadlite]

Brandan09997 You could be, but my point would still stand. :)

[@ericcartmann]

Brandan09997 Vsauce: " how many legs does a goat have if you call its tail a leg? 4. Just because you think its tail is a leg, doesn't make it a leg" Abraham Lincoln

[@i_am_ergo]

***** Someone sold Abe some bad weed. Or Vsauce is bad at quoting.

[@XxOTheEpicCrewOxX]

***** The irony..

[@ericcartmann]

XxOTheEpicCrewOxX oh the misunderstanding.

[@alejandrahuertarivas4370]

As a neuropharmacologist and a professor, this is what i always teach my students at the school of medicine, but every person should also understand that. it can save lives

[@JtrainMedia]

As an epidemiologist, I enjoyed this video!

[@MikeLazarus]

Don't just read ... question everything you read

[@jayrush01]

i dont believe your comment

[@MikeLazarus]

jayrush01 It was a command :-)

[@ludakriss9094]

jayrush01 hahahahaha :D Dude.

[@ObeySilence]

+Mike Lazarus why?

[@MikeLazarus]

Obey Silence Because I said so, obviously ... :-)

[@Gyozomroka]

TED, why are you so good?

[@luamfernandez6031]

Or is it?

[@lannguyenkoler]

Because Ted is good already =))

[@JESUS-jk8lv]

it just have to be good

[@ankitadandekar8125]

A brilliant video. Many of us need to learn the importance of questioning each thing that comes our way instead of falling for something blindly. This is such an easy way of breaking down the belief that every heading or an advertisement saying ' Scientifically Proven' is the best thing available on the market. Great work!

[@despicableyou12]

The two phrases I remember are "correlation does not imply causation" and "take everything with a grain of salt". Since I have RA (I'm only 17), I am constantly bombarded with magazines and articles and random people off of the street telling me that they've found the cure because their mother's cousin's neighbor did this and now they're in remission. They'll also "tell me" that the reason I have arthritis is because I did this or because I didn't do enough of that. You have to take a step back and really do your research on new medications or diets etc. because why waste your time, energy, money, and even sometimes your health on fads that will become yesterday's news

[@Joel11111]

This is a good video. It really bugs me when people start a sentence with "studies show blah blah blah blah" as if the words "studies show" make what they say any more credible without more information on these "studies".

[@Silverizael]

It's especially annoying when people try to dismiss hundreds of studies of various types that came to the same conclusion because of their belief that confounding variables, supposed conflict of interest, and selection bias make them all suspect, but then those people cling to the one or two studies that share their pre-disposed bias, even though those studies have an extremely clear list of things wrong with them, far more clear issues than the hundreds of studies that said the opposite.

[@sparkloweb]

Conflicts of interest and selection bias make the entire study suspect. If the authors make the correlation->causation fallacy in the conclusion, then only that part of the conclusion should be considered suspect. If a Facebook meme or a news article says anything related to science, it should be considered bullshit until after reading the actual study. (Except if the news article is a verbatim copy of a press release from the paper's authors at a reputable institution. Then it's merely suspect.)

[@Silverizael]

Magnus Webb A lot of the holistic websites that people share on Facebook don't even link to the study they talk about or even give the name. It's always aggravating.

[@lavenderm]

RCT is Randomized Controlled Trial, not randomized clinical trial.

[@chumark54]

Yes, a randomized study is the gold standard--but you also need to replicate the study, ideally more than once. The 1st time might just be a coincidence. Unfortunately, in science, replications don't get enough attention; they're way less likely to get published, especially in social sciences.

[@TheSghetty]

Great video... I just want to point out that these tricks to identify clickbait titles are useful when you have a lot of time to spend for that particular study, because you have to find and read the original article, and probably you won’t understand much of it if you aren’t a doctor, so you need to find other sources or contact the authors of the article. All I want to say is... I understand that critical thinking is important, but can’t we have journalists to do this? Because I thought that was their job.

[@FirstLast-fr4hb]

A great piece of honest scientific realistic thinking.

[@changyang1230]

RCT is more commonly known as randomized _controlled_ trial rather than randomized clinical trial. Although the "RCT" name is sometimes expanded as "randomized clinical trial" or "randomized comparative trial", the methodologically sound practice, to avoid ambiguity in the scientific literature, is to retain "control" in the definition of "RCT" and thus reserve that name only for trials that contain controls. Not all randomized clinical trials are randomized controlled trials (and some of them could never be, in cases where controls would be impractical or unethical to institute).

[@AkiraSpectrum]

Great video. Very clear, concise and informative.

[@ChillinGames]

you should do a vid on the history of cigarettes, from when/why they were first made threw the crazy revenues they provided to their gradual decline and what kind of studies were used to discover their health risks!

[@YellowCrystalSun]

This is a serious problem we all need to address.

[@brod2man]

So they don't want to bring things into the market before they have been tested for possibly harmful side-effects. Well, I'm glad someones looking after us. Time for a cigarette break

[@williamsartorio5524]

One other problem is that sometimes scientists are paid(by government or other entity) to skew a study somehow that benefits the bribing party.

[@skelworthvods984]

William Sartorio I think he already mentioned that just saying

[@ElementalSkater001]

It's great that TED has addressed this issue. Now people can stop pontificating their arguments with ambiguous studies.

[@junesept234]

Hello. Can I ask for a review paper or a book that discusses how can we say that a research is reliable?

[@BlackEpyon]

THANKYOU! Now I can show my friends who shun science why their excuses of "not all scientists agree" or "science doesn't give all the answers" or "It worked for this person, it must work for me too, so I'm gonna buy myself a $4000 water filter" why their arguments are invalid.

[@Steele00]

I like how this TED video can sort of invalidate some other TED videos.

[@dpearce19]

Thank you SOOOO Much for creating this video!!! I wish everyone thought about this before making hasty conclusions!

[@RaphaelSloanYusukeUrimeshi]

Interesting and it will help me be better educated on studies as the come down that pike! This is why I subscribe to this channel!!!

[@darkenergy7291]

correlation is not causation. I get it now. thanks ted!

[@TheCheukhin]

This video is underrated......

[@harrysonofbob]

This is so good. Thank you.

[@watchingourwatchmen]

The guy at 1:46 really loves his purple drink!

[@vinceawesome9652]

This should be coupled with a stats course.

[@constitutionfrist7229]

Equally as important: who did the study, and why?

[@FastEddy1959]

No. Not equal. Not even close. The quality of the study & the strength of its results matter. You should NOT dismiss a study simply because you’re suspicious of its sources of funding. Studies are expensive, they get funding from wherever it might be available. It is simple-minded to dismiss results simply because the money came from “big oil”, “big pharma” or “giant, multinational corporations”.

[@SageAndOnions]

This is a very limited skepticism though. Philosophical skepticism extends much further than looking at science, and can question the very legitimacy of science in the first place, or of our senses etc.

[@NemesisAnother]

Downloadable?

[@jamessamsom6780]

Sensible, well put, non partisan. Like.

[@MegaFinny1]

Can you do a video on prejudices plz

[@soulsfood5691]

but thats the thing though, people are lazy to double check the claim of the said "scientefic study". 1. most of us plebeans dont know where to go, and 2. most companies who declared the said studies would not provide the research documents that was conducted to support thier decleration. its literally counter productive for a company to do that since it brings to question thier decleration as well as theyre afraid of competitors proving them wrong or much worse, use the said information correctly and gain more from it than them.

[@kaesarcggb]

How is it that there is no subtitles in this?! I would like to share this video with non-english speaking people. How can I help translate this?

[@marlie3867]

there is now

[@pur3pandor]

What makes a good study with high validity and high reliability?

[@neneklampir6664]

the study that already proven empirically.

[@NabilTouchie]

1:46 "People going about they usual behavior (black guy drinking grape soda)" I see what u did there! hahaha

[@DrVictorLopez]

You are Best Ted ed, thanks

[@castrocafe8049]

Ah science, Tis the true bliss of truth itself

[@Drudenfusz]

And now imagine fields of research like social science, how much worse the things there are, where it sometimes seems to me that every study I read is just crap and the authors just take the data they like to prove their points on gender studies or things like that...

[@erikslapins9227]

I study social sciences..

[@i_am_ergo]

***** Social sciences aren't actually sciences.

[@erikslapins9227]

Semyon Galtsev And why are you telling me this?

[@i_am_ergo]

***** Sorry, I thought you were defending your major.

[@erikslapins9227]

Semyon Galtsev I was trying to make Drudenfusz feel bad by appearing as somebody who has made a choice to study social sciences. It is easy to criticize ideas or numbers but not people. I agree on what is said at this video, but I don't think that what I study is crap. Even if it's not a field of science.

[@lydiatomlinson2712]

what a phenomenal video.

[@iwtd6]

Why does this sound like the Announcer from Portal 2

[@hamzahfarrasamulya1694]

I'm still a bit confused with the difference between 'correlation' and 'cause-effect' stated at the almost end of this video ._. Anyone can explain further? Thx

[@radhapatel217]

For example, if someone did a study that proved that cities with more churches have more violent crimes committed in them per year, this would be a correlation. However, one does not necessarily cause the other and the true cause for both may be a third factor like population size.

[@rsmith4379]

If you want a really lovely podcast that talks about this all the time, check out Maintenance Phase. The episodes are hilarious, well-researched, skeptical, and honest about the world we live in.

[@zarkoff45]

I'm waiting to hear more about the study that showed that blood from young mice reversed aging in old mice, rejuvenating their muscles and brains. Are there human trials on this yet?

[@NancyFreire-dv4pi]

Thank you for the information it's nice to keep an open mind With educational studies

[@AJWeeks]

Does the audio sound strange to anyone else? Like kind of wavy?

[@k1monfared]

well said. finally.

[@MedEighty]

And that's why I always ignore findings of epidemiological studies. How they get away with calling them scientific is beyond my comprehension.

[@DynoBug]

I don't know much about this myself, but surely you shouldn't just ignore them, rather they should be considered with wider research

[@MedEighty]

Jordan Edmondson In most cases, they're nothing but simple correlations of two unrelated or only loosely related variables, with one always being made to look like the cause of the other. Here are a couple of examples that come to mind immediately. People who drink fizzy drinks are more likely to be depressed and those who drink diet fizzy drinks are even more likely to be depressed. Now that has nothing to do with the contents of the drinks. Those who drink fizzy (sugary) drinks are more likely to be overweight and disappointed with their body shape, leading to depression. Those who drink the diet variety are more likely to have become so overweight and depressed that they're hoping that switching to the diet drink will magically make them thin. And, when that doesn't work, they get more depressed. The older the parent that conceives a child, the more likely the child is to suffer from an autism spectrum disorder. Why? The child doesn't have autism because of the parent's age. The parent probably already had an autism spectrum disorder (perhaps a mild one), which made it more difficult for them to connect with people, resulting in them meeting their partner at an older age. Then, when they had a child, the genes that caused autism in them were passed on to the child. Chances are that both parents carried genes that caused autism, which increased the chance of their child being autistic. So, yes. You should consider these so-called studies alongside wider research or, simply, more logical reasoning. Sadly, that's not how they're reported. Because correct reasoning and explanation would not make great attention-grabbing headlines.

[@christopherg2347]

So you do not believe that falling off Buildings is deadly? Because to my knowledge we never did a RCT by throwing people off buildings to check this. All we got is a few milennia of Epdimiological Studies. EM Studies are reliable with enough repitition/amount of data.

[@tokugagua2008]

CAN I DOWNLOAD & RE UPLOAD IT WITH SPANISH SUBLITLES?

[@KakashiDominates]

It's so sad that humankind as a whole is so constrained by social and societal factors that we are unable to find Truth efficiently

[@IZY2091]

Very cool and thought provoking. Subscribe

[@ABunchOfRandomAnons]

Do they alter their voice? Or does the microphone just suck?

[@MsRScienceClassK12]

BLINDED? The way you represent blinded visually without explaining it is a major flaw. As a science educator, I would hope that you could fix this error and repost. Many who are ELL may take this literally and have a negative view of such studies.

[@aarone1777]

Sagon says question everything!

[@harlemzed3124]

yeah if you read short resumed version of original study you might understand it wrong and make wrong conclusions

[@MADDMOODY516]

did ya'll do a study for this??

[@brentonwang6771]

Nice job ted!

[@smaklilu90]

good video!, it reminds me of my friend who avoids diet sweeteners like poison. :)

[@ericcartmann]

everyone should be avoiding diet sweeteners...and pop in general...There's not a single study in man kind that says otherwise.

[@smaklilu90]

If u cant avoid it make it moderate. too much of everything is bad! that's the golden rule :)

[@i_am_ergo]

***** The "artificial lab created chemicals added to other chemicals" part shows how panicked and unreasonable you are. Everything is chemicals and the Earth is one big lab. Use your brain instead of overhyped no-no rules.

[@smaklilu90]

***** what are your alternatives?? no GMO's no additives?? do you think we manage to feed the exploding population that way?? how long it would take to grow crops naturally?? how many labors it would take?? analyse all this questions!!

[@smaklilu90]

what do u mean by 'abundant water and food' ? that's very misleading. why do u think third world countries are starving ?? abundant water and food is everywhere . right!!

[@skicreature]

Randomized Clinical Trials do not show cause and effect! All studies no matter their type can do no better than a correlation. Albeit that correlation can have varying strength and a randomized clinical trial does show the case where quite a few variables were eliminated. Unfortunately, one can not eliminate all variables since you likely did not include someone from each and every culture and ethnicity on the globe. It's possible the drug may have the opposite effect in a culture with completely different diets and backgrounds. There are always unknowns and a randomized clinical trial does not get rid of all of them but it is the best we can get.

[@badoocee1967]

Thank you.

[@rajtechking]

I was bombarded with terms.

[@TheKartana]

0:25 scientifish studies

[@j.fabricioelias227]

Great video.

[@toomuchtruth]

That was brilliant.

[@owensspace]

This is why studies did at one time show that marijuana use effected motivation. It was honestly just a bad study... Studies since have shown no such thing.

[@int0x80]

Thank you

[@yokozombie]

Last scene is not statistically significant - instead of happy face he should have dropped book into litter bin to show more natural reaction for most actual study results.

[@R3KTANGL3]

i love you, ted

[@jacobmartinez618]

His voice arouses me

[@aedenthegreatyt]

So TL;DR, coorelation does not always mean causation!

[@shilohschwartz8671]

I am skeptical of being skeptical.

[@JoshuaChowabc]

What about sex? Isn't it silly to lump male and female together when testing drugs? Example: Ambien and other sleep meds.

[@sleepycowboy18]

ain't nobody got time for that jk. but, good vid :)

[@syabilaazri7834]

Huh.... so that what really happen in the halloween episode of "Community" ....

[@andreshuertas8642]

Before teaching people to be skeptical of research we have to first figure out how to make them skeptical of Facebook posts 🤦‍♂️

[@corduroy99]

well said, AMEN!

[@ivanthreeofnine4028]

. . . AGREES . . .

[@TheCheukhin]

Love Ted-Ed

[@karapapaxatzidimitrakopoulos]

Awesome!

[@avendanod]

3.101

[@realnoahsimpson]

why is he speaking so fast?

[@rrmanoukian5080]

Follow the money! It’s a good indicator of researchers bias...

[@Madayano]

👍

[@Snakeyes244]

This is Statistics

[@geoffwales8646]

Please don't upvote a commenter when he or she disables replies. Singling out any topic (eg climate change) because there are limits to the usefulness of studies is stupid. All studies have to be seen in the wider context. All studies need time to be properly examined, tested and compared to other studies.

[@helloworld2848]

TODD Talks

[@umpurgatorio]

nice but misses the money factor.

[@allevu]

Why arabic text?

[@teamgamerpro3998]

a

[@christaansadler]

anybody else notice the black guy downing the GRAPE SODA @1:51 after they said "observed going about their usual day" lol #Racist smh

[@4ci1ab]

why don't you make clones then test it on them so their are no flaws

[@leodaza2151]

They're working on these chips that allow you to put cells of any type containing your own DNA so you can test things on something that has the exact same DNA as you. They should be available to the public in a couple years or less.

slow man slow

[@politereminder6284]

2021 covid vaccine hesitancy intensifies😳

[@CadetGriffin]

3:10 Trump