How statistics can be misleading - Mark Liddell

Explore the statistical phenomenon known as Simpson’s paradox, and how it can lead to incorrect conclusions about data.
--
Statistics are persuasive. So much so that people, organizations, and whole countries base some of their most important decisions on organized data. But any set of statistics might have something lurking inside it that can turn the results completely upside down. Mark Liddell investigates Simpson’s paradox.
Lesson by Mark Liddell, animation by Tinmouse Animation Studio.
Support Our Non-Profit Mission
----------------------------------------------
Support us on Patreon: bit.ly/TEDEdPat...
Check out our merch: bit.ly/TEDEDShop
----------------------------------------------
Connect With Us
----------------------------------------------
Sign up for our newsletter: bit.ly/TEDEdNew...
Follow us on Facebook: bit.ly/TEDEdFac...
Find us on Twitter: bit.ly/TEDEdTwi...
Peep us on Instagram: bit.ly/TEDEdIns...
----------------------------------------------
Keep Learning
----------------------------------------------
View full lesson: ed.ted.com/less...
Dig deeper with additional resources: ed.ted.com/les...
----------------------------------------------

[@samsulh314]

That's what makes statistics both beautiful AND scary. It doesn't have to lie to trick you.

[@Drudenfusz]

Which is why I always look into the methodology and prefer to look at data directly and never trust a conclusion someone else says the statistic would provide.

[@AustinTexas6thStreet]

The problem is that you can NOT trust the data itself!!! You are just re-interpreting biased and faulty data!!

[@latoyacreary5588]

I am fast learning this necessary to do if you want to form a truly educated opinion.

[@rahulpandey2022]

Hiding a fact is a lie

[@cory.bryant]

@@rahulpandey2022 thus complicates the problem further. Relevancy is in the eye of the beholder.

[@davidbuschhorn6539]

The numbers will confess to anything if you torture them long enough.

[@jayje2536]

That applies to anything mate

[@brendahawkins8211]

Lol...Interesting....and a very good analyst with integrity can find where the numbers were tortured and why. =))

[@demonicdrn3037]

Just like people, and bunnies. Well, the bunnies died but it should’ve worked.

[@Johnof1000Suns]

The NUMBERS, MASON!!! THE NUMBERS!!!

[@CarlosJosephManlapaz]

A perfect way of looking at it.

[@gayflower900]

Accordion to a recent study, 90% of people fail to realize that the beginning of this sentence is an instrument

[@Alien42x]

.

[@beactivebehappy9894]

Ns ouch that hurts

[@renanmendes1299]

nice sentence

[@eclipsewrecker]

Papa Shpee got me

[@wiggle7766]

I was gonna say, that isn’t even a sentence because there’s no period.

[@soulextracter]

I remember that in middle school, our teacher would say that people in the past didn't live longer than 40. The statistics showed that the average lifespan was not more than 40, but that was because the child mortality rate was so high. People could still get 70 - 80 :P My teacher didn't know that!

[@user-ss1ky4tu8y]

It was average but compared to the past most people die at early age because of the diseases that were not curable at that time

[@TheFeldhamster]

@@user-ss1ky4tu8y the thing is that the distribution for life expectancy is U-shaped. Then and now. Ppl mostly either die when young or when old. Most other distributions we encounter are bell shaped (Gaussian, also called Normal). In a bell shaped distribution, the mean is very meaningful - most individuals will be around the mean. Eg height - most ppl will be of approximately average height, only few will be either very short or very tall. With U-shaped distributions, the mean isn't intuitively useful. There are very few ppl who die at the average life expectancy age, most ppl die when older or when very young. In former times, child mortality was a lot higher, so the average life expectancy was farther from the old age at which ppl would die who had managed to survive childhood. Nowadays, child mortality is quite low in developed nations and therefore, the average age is now closer to the end of life dying age of ppl. It's still U-shaped, though.

[@Pomagranite167]

Thank you for explaining this bc this never made sense to me

[@greyrock420]

@@TheFeldhamster That's why persons who do statistics corectly first do test if distribution of data is normal/gaussian. If not then they don't count mean. Since some so called mathematician did it, it actually proves that he was rather uneducated botcher ( like me when I use mean... But I'm not mathematician :) )

[@youcan_change_handle_3june_]

@@greyrock420 elaborate pls.🌝

[@anger_birb]

"A good liar tells mostly the truth. A masterful liar tells only the truth." I don't remember where I heard this but..

[@Pwnz0rServer2009]

ah, lying by omission

[@Fonzofication]

I love how TED-Ed never has ads. They're here to spread knowledge, no make profits.

[@beactivebehappy9894]

Yeah sure, but they still only make $ 3539 on patreon. A single guy- CGP Grey makes 15 times as much. Also, my point is the quantity:quality ratio of both channels I agree that TED-ed is an organization but still Grey also has writers, animators, editors, etc. too!

[@TheTonyspik]

that's because their profit is made from the 3k or so price to attend their conferences

[@faceless1434]

Idk if they've changed but I got an ad on this but I think they deserve the income, regardless of how they get it (well with legitimate means ofc)

[@tanvikhare9710]

So you're implying the people making these animations, doing the research etc. Don't deserve to be paid

[@TheNemesis442]

@@tanvikhare9710 shut up capitalist swine

[@tonyzan5268]

To many people in the comments are dismissing statistical analysis completely. That is a mistake. The moral here is that you need to be careful and properly recognize and incorporate relevant variables. Statistics when properly used can show trends and help policy makers implement solutions to negative trends. For example in the hospital example once you incorporated the information about the health of the patients before they were admitted to the hospital you could see,that perhaps hospital A has some, serious deficiencies.

[@YtubeUserr]

+Tony Zan Go away. Statistics show that more than 99% of the statistics have flaws and not to be trusted.

[@AustinTexas6thStreet]

Statistics can be minimally useful but they should Never be taken at face value or as being genuinely accurate. Statistics, at most, should be just one of many many tools for discerning Facts! The entire process of attaining statistics is highly flawed and shouldn't carry the weight it does!! In the end, Statistics are not much better than Anecdotal Evidence...It's something to be considered but that's it!!

[@YtubeUserr]

David Belcher "Statistics are much better than anecdotal evidence". ^ That is a flawed sweeping generalization.

[@tonyzan5268]

David Belcher disagree, obviously statistics can be biased, but they can also represent an accurate portrayal of reality. And are much more likely to show an accurate portrayal of reality than anecdotal evidence. To compare statistics to anecdotal stories is insane. Anecdotal stories take the perspective that one event is representative of a larger phenomenon. At the very least statistics use a much larger range of variables. That alone makes this takes more relevant than anecdotal evidence. To say that statistics are never accurate to say that polling is never accurate. And if that were true then someone like Nate silver must be a psychic because of how accurate his predictions of the 2008 and 2012 elections were. To say that statistics are not more relevant than anecdotal evidence. Is the dismiss the fact that statistics like the gini coefficient show that the u.s. is the most economically unequal developed nation in the world. The say that statistics aren't true used to say that the statistics that show, government provided Healthcare increases life expectancy is all by accident.

[@YtubeUserr]

***** Are you so sure?

[@cassandrad8970]

This guy's voice is so calming

[@mohdnasirshaikh3831]

true

[@user-ez5vq9fd2t]

+That Guy u w0t m8?

[@ldohlj1]

+That Guy LOOOOOL

[@WTKB82]

+Cassandra D I know right, it gets me wet as fuck

[@rawstarmusic]

+Cassandra D I think you find it trustworthy. That is calming. If he seems to have knowledge that's also calming. I find it more meant to reassure that this is how it is.

[@patfts2518]

another great statistic: In avarage, every human has one boob and one ball, ever thought about that?

[@matthewmaylin2268]

And 1.99 legs

[@WalkOnNick]

+Stingy Greindger Vsauce?

[@yoadhordan2809]

+BuFufilms lol exactly what I remembered it from

[@turun_ambartanen]

+BuFufilms could you please link me the vsauce video? thanks in advance :)

[@videogyar2]

+Stingy Greindger And the average penis length is 7cm:D

[@RichardHannay]

Man, I thought the "Simpson's Paradox" came from Homer saying "people can come up with statistics to prove anything. 14% of people know that." XD

[@empty_cognizance2234]

+Richard Hannay What a load of rich, creamery butter you just spouted.

[@novastar6112]

@@empty_cognizance2234 is that a complement or an insult

[@giasharie274]

Nova Star yes

[@Green24152]

@@eshelsh1905 thats the joke r/whoooosh

[@AloeEdits]

wasn't homer gay>+?

[@MasterThYoda]

I would like to congratulate the animators of this TED-Ed! The guest artists usually do a good job in the videos, but I presume this topic was really hard and the animation not only richly ilustrated it, but also was able to make me have fun while watching. Thank you very much. I hope to see more colaborations of this company with TED.

[@rafiq6521]

Theres a nice saying i wanna share: Statistics are like bikini, what they reveal is suggestive. But what's hidden is vital :)

[@colleen9493]

Rafiq me too

[@waynejshynzhinds4434]

Caveat: doesnt apply to nudist colonies. :0)

[@mondaysinmyfriday]

I like this! I’m stealing it! Thank you!!

[@Alien42x]

.

[@geo3106]

@@waynejshynzhinds4434 They are genuine statistsics!

[@ZoZo-wb1nr]

Ironically, my last job I use to host a lot of surveys for data & research & I’ve always learned that data/statistics are imperfect & definitely are flawed due to situational factors or “lurking” variables. This just took a deeper dive into things! Thanks TEDEd

[@Owen_loves_Butters]

That first example is huge. Better hospitals often have lower survival ratings because it’s better, so sicker people often to to that hospital instead.

[@myNAMEisKIRSTY]

I think it's important to understand that there is still a purpose for statistics. Just be selective with your sources, consider if what you are reading a biased source such as left or right wing media. What does the writer want to you to feel, angry? shocked? What is their objective? Will they be able to give you a balanced view? If you can't find an unbiased resource read opposing biased sources to get a better idea of reality. Always read critically.

[@deltavistastudio124]

Got news for you, EVERY STATISTICAL SOURCE IS BIASED! Someone had to pay for the study.

[@a.w.4708]

@@deltavistastudio124 some might be not, if every result is equally good for paying side, or accuracy is actually the most beneficial. For example statistics telling what kind of paying side's product people like the most. Also not every company is immoral probably and in some cases falsing statistics would be very immoral (for example about deadliness of some substance)

[@timbradshaw5481]

What is the purpose of statistics?

[@nuclearsimian3281]

I never stopped trusting something as fast as I stopped trusting statistics after I took a statistics class.

[@mdmegrimalkin1792]

Yes so very true, it's scary how often stats are used to determine where money is placed in government, policing, health, education- people's lives are determined by the faith in statistical gathering and interpretation as if it were incontrovertible truth. Terrifying.

[@DeAngeloYouKnow]

Ted-Ed don't you ever stop making these videos.:)

[@swifterfan42]

It's like gerrymandering of data!

[@ShawnRavenfire]

It's like Mark Twain said. "There are three kinds of lies. Lies, damn lies, and statistics."

[@hobobmo]

+Shawn Ravenfire That quote was first used by a British prime minister not Mark Twain.

[@ShawnRavenfire]

Hash Kazi I stand corrected.

[@AustinTexas6thStreet]

Yeah, but Twain won the rights to the quote in a Poker game!!

[@alexwang982]

He didn’t say that. Statistical error.

[@cliffordthies6715]

I very much appreciate the author's point that the issue of lurking variables isn't easily resolved. I would add: it's good to have the same phenomenon, even the same data sets, examined by statistics with multiple perspectives.

[@ChaoSpiritZ]

if you put the intro and the outro in reverse, you get piano sounds, i just checked it

[@woodfur00]

this fucked me up

[@mohammadjj]

I'm not convinced. It sounds more like a string instrument like a violin.

[@mohammadjj]

Or a trumpet or whatever it's called. All I know is that it's a continuous instrument

[@ChaoSpiritZ]

+Maddix I literally checked it, its a piano, with some more unknown instuments

[@nice3333333333]

+ChaoSpiritZ rotmg Could you perhaps upload verification?

[@animeyahallo3887]

Did you know that there's a 100% chance that you're currently reading my comment?

[@guesswho2590]

Did you know that your sentence is actually correct? "You" comes to be whoever is reading that sentence; so yeah, there is a 100% chance. I found that funny and interesting.

[@TheProGam3rHD]

Yes, and it's taken from the sample size of all peoples that will be reading your comment. I did my own expert calculations and well, well, I arrived at the same percentage! (Although in scientific notation: 1*10^2 %)

[@AloeEdits]

YAHALLO!!!

[@TheProGam3rHD]

@@AloeEdits What? Lmao

[@groznyentertainment]

Amazing educational piece. This is relevant in every aspect of our lives. Hearing politicians using manipulated statistics for their agenda is common

[@em-dp9ov]

Thank you so much, you probably won't see this since this video is 4 years old but this helped me a lot with one of my math classes.

[@SolusBatty]

I used this paradox in a lesson i've held to highschoolers as part of my last year exam. SO thank you for introducing me to it. It was a great finishing touch that got the kids really excited. And my lesson was about philosophy of science and the role of common sense.

[@nrgizmiraslanli1088]

I laugh everytime the grandma jumps up with an "eeh!"😂😭💀

[@davidmackintosh2010]

Good statistical analysis requires (1) expertise and (2) integrity. Statistics give power, so they will always attract (1) incompetent people and (2) immoral people.

[@TheProGam3rHD]

_A recent study shows that anyone will believe anything as long as the study in question is prefaced with "a recent study shows". This, of course, is all according to a recent study using recent statistics._ No, but in all seriousness, try googling or looking up "a recent study shows" or "according to a recent study". You'll be surprised by all the "recent studies" you'll find. Most of them claiming to be from or citing "reputable resources".

[@xFrOsTeDGsMeRx]

I’m glad this is mentioned in the video and I imagined this is what it would be about. This kinda just goes down into simple scientific method really. Just because one thing is found doesn’t mean that there can’t be more factors affecting the situation. I do take statistics into consideration for a lot of decisions of course but I always try to separate it into separate factors if possible and try to find stuff that might disprove certain personal ideas that come up either because of the statistics or because of emotional causes.

[@xFrOsTeDGsMeRx]

Sadly sometimes there just isn’t enough statistics to check for everything. So at that point you do the best with what you got

[@PriyankaSingh-tv4lw]

Simpson’s paradox is really scary and beautiful at the same time.

[@kizryuver]

Animation,sound effects are simple yet so good and interesting you guys Its engaging and edutaining so i have lesser chance of being distracted n is easily comprehend...keep it up ted ed..we need channels like these..thanks so much for the information.

[@stoppi89]

If you suspect a lurking Factor missed out in a statistic (on accident or intentionally), just look if things that should obviously be considered (the health of patients in their Hospital example is a good one) are missing or not mentioned and maybe look up other statistics of the same thing. Usually every statistic tells you what kind of data they took. So if you see a statistic that says "tested 1000 healthy people, around 50% of each gender", you might realize that that other statistic said nothing about the initial health of the subjects, but on turn said they where between age 15 and 50. This is a great way to get closer to the truth with statistics.

[@jemuelstanley9936]

Thanks a lot Ted ed for posting this video. It is really informative. I have always thought statistics can be relied. Today I learnt to look at different perspectives of statistics and the information it provides. Thanks a lot! Keep posting such videos!

[@jp4431]

To this day I'm still fascinated by Simpsons paradox

[@Yagyaansh]

Lurking variable background sound is what i have been missing my whole life

[@nacholibre611]

That dude’s laugh drove me insane.

[@Literaturenjoyerman]

“There is no one size fits all answer” the Ted Ed equivalent of “get gud”

[@friendlyyoutubecommenter4527]

Unless you have green eyes.

[@j777]

Pretty good explanation of one reason why correlation doesn't guarantee causation

[@d1want34]

Covid19: you've exposed me

[@doctorf7501]

Lack of awareness of Simpson’s Paradox is basically the bane of all doctor’s existence.

[@chipperz4497]

This could be pretty useful to know someday!

[@goober3097]

Chipperz Right now.

[@No-oj5ow]

Rn with covid it pretty useful.

[@FirdausIsmail1]

someone should make this into a videogame

[@bdasf2214]

I like how this video show how statistic can be misleading and then quantify all murders happening in florida to be of the same value, discarding context as if judgment case are broadly the same thing and are not unique in their combinations of circumstances and factors. Was it murder of children? First or second degree? Judgment rendered by judge or jury? The real world is not the world of theorics macroeconomics where every single product that does the same basic function has the same value.

[@Di3g0g0tzyn]

An easy example how statistics can being tricky: We have 2 men and 2 breads, and one of them eats the 2 breads, but statistically says every man eats 1 bread.

[@catguy7520]

A real lesson worth sharing

[@TeaCius1]

The whole world should have to watch this video on their first day of high school or its equivalent....it's common sense to some of us and painfully absent in others. I have no stats on these two groups :)

[@meijimarts111]

Another good example of statistics is that there’s 104 days of summer vacation but Phineas and Ferb has 189 episodes

[@ankurrai8677]

thanks Ted Ed there's so much learning in this video , it guided me

[@martonkormendy6150]

The laughter of the lurking variables is so hillarious :D :D

[@ezprogramming7887]

Great example, something that people should know to understand society better.

[@darpanvyas5160]

Tinmouse Animation Studio alone deserves a million ❤️s!

[@nojo9684]

I saw a stat where two different acts were compared . One was 99.99% safe. The other was reported as being "25× more deadly". Sounds scary, but that means the second one is 99.75% safe.

[@pustota7254]

Bruh

[@christinelemus7530]

@@pustota7254what

[@nickmagrick7702]

thanks TED, stuff like this changes the world. Simple as it is

[@insanecoolaid7299]

Yup, this is why i don't care about "Vague" statistics.

[@oranjizer]

EXCELLENT !! Ive been saying this for so long, nobody ever listened me.

[@annelin6376]

Very enlightening, thanks for the video!

[@zapador]

A nice and important video. Can also recommend a TED Talk by Peter Donnelly about statistics.

[@wwikn]

omg, this is so cool, it really caught the first time, thank you so much!

[@r-qube23]

This video pretty much summarises the whole book called "FACTFULNESS"

[@LaughingMan44]

Wow a TED presentation that isn't garbage.

[@Dartmorin]

this is ted-ed, not ted i think.

[@Ninterd2]

I have an idea! Ted-ed is a bit redundant, let's shorten it to Ted!

[@Drudenfusz]

TED-ed is the best one, and TEDx is just a bunch of ideological crap, normal TED is hit and miss.

[@BloodWolfBoy27]

that lurking variable crept me more out than it should have

[@christopherhalim2801]

Study shows that more than 80% viewers consider the lurking variable's animation to be scary

[@thalespro9995]

The 97 dislikes r people who make misleading graphs and now less people fall for them

[@danstrett6374]

this is a great video. definitely an example of simpson's paradox is the wage gap as evidence of sexism as it only takes into account the average earnings of men and woman and doesn't go into any variables that may affect that statistic such as the average hours men and woman work and the different fields men and woman go into.

[@FantasmaNaranja]

something that always annoyed me about statistics is that 50% doesnt mean one every two

[@josephjackson1956]

That smile. That darn smile.

[@ekananda9591]

As a statistician, I must agree with you, but there are more useful things in statistics. AI, machine learning, data mining is some example of it. Without it, there would be no search engines like google. Do not let politicians make you skeptic about statistics.

[@galacticgufus]

amazing. this is very tricky.

[@Andres_2004]

is as simple as to look at as much data as posible, things are often more complex than we think

[@some_random_dude13]

Basically the stats don't mean anything without context

[@paxtonb7202]

This channel is simply amazing

[@farmeryaeg]

Like when "campus rape statistic" data is determined by the surveyor, not the surveyed, and includes things such as feeling verbally threatened or groped as a rape?

[@vinnieg6161]

taking statistics class should be mandatory we're all getting fooled ='D

[@shortburrito4077]

This is how the world works: A green frog gets its legs cut off. A white frog does not. Therefore, scientists conclude green frogs cannot jump

[@kubraozyrt]

That's why you need to test their reliabilities. Cronbach's Alpha is one of the most common ones.

[@nerikacutie283]

1:23 when you get 100% in your test

[@griffinbrown2735]

The lurking variable is going to give me nightmares

[@kvlpnd]

70% of statistical data is misleading. Remaining 40% of statistical data is errorful. Wait, what did I say just now?

[@manuelmilla4042]

To compare two sistems, you must apply the same treatment (in this case, the same set of patients) to both sistems

[@johnnychang4233]

The Simpson's paradox pop ups everyday in a country named Venezuela, economically speaking.

[@ecaboots]

the conditional variable's character representation scares me

[@ServantOfSatania]

Video on one of TED channels talking about misleading statistics? I'm sure this aged well :)

[@jacobfredman9442]

So we interviewed people who drove at full speed at the highway both of them sustained no injuries therefore driving at full speed is not reckles driving and it is 100% safe

[@rufioh]

You should do a video on Jeremy Hunt and his 'statistics' about healthcare in the UK.

[@Rikortez]

Funny how the last example this video gives on it self can be misleading. You need to look into the context of the data(as you state so many times), it is possible that the victims that were black were due to gang violence and the victims that were white were due home invasions. They both are murder cases, but the context of them can greatly change the sentence, the only way is to look for the data(as was done in example one) and see it. In this case it seem that was omitted, giving it the(possible false) notion that the determining factor was racial bias instead of the type of murder and the way it was carry out. Yet another fact that was omitted was the age and sex of the victims. I just found somewhat ironic that in a video that's supposedly about addressing bias and manipulation of the persuasive nature of statistics(for possible ideological motives) might be committing the very same thing.

[@deltavistastudio124]

Whoa nelly! Rikortez, can't even suggest such a thing without triggering the snowflakes!!!

[@zyaicob]

"It is possible that" is a nice way of saying you don't know. Everything you're saying is technically correct of course, but it would be way more interesting and useful if you had some real data

[@timbradshaw5481]

That's his point, making the assumption that the race of the convicted affects how likely they are to die rather than the crime itself is a manipulation of statistics. You are saying, it's not wrong unless you prove it by looking at the data. But it's not right until looking at all the data, that is the point of the video. And like the guy said, that's exactly the point the video is making, yet the video is being misleading in the same way the video is trying to prevent viewers from being fooled.

[@balajishrinivas8404]

I don't get it, they say that blacks are sentenced to death comparatively higher than whites, and then proceed to say whites have a higher sentencing rate😅

[@heatherburton1629]

Kind of dark scenarios. I realize those are high stakes examples.

[@matthewcowell]

The animations didn't really offer much, but fantastic explanation.

[@AakashKumar-gl2fk]

I didn't understood that Florida death sentence thing.

[@sid98geek]

Great video. But it's not statistics to blame here because without a method to organize any of our data we would be in a greater disorder today. It is true that statistics have deliberately lied and data has been manipulated. But most of the time (am I saying this statistically?), it's the loopholes in statistical graphs that people (I said people, which includes politicians and companies) exploit to fulfill their agendas. I mean how can a SINGLE, error-filled, filtered, non-contextual graph prove a theory, claim or anything like that? Representation is key, especially when dealing with unobservant audiences.

[@ShisnoGaming]

This is why you need to question results that may seem favorable for the narrative pushed by main stream media. Who sponsored the study and what variables were used and what were left out?

[@nicholasdranfile9312]

THIS BLEW MY MIND. SO BASIC BUT SO VITAL.

[@abcd123906]

Simpson's paradox is kind of like the inverse of the logic behind gerrymandering if you think about it

[@karengomez3143]

I’m quite late, but this is pretty interesting, actual and current!

[@mastervader9796]

Misleading data; the backbone of gerrymandering.

[@2bitgirly007]

As Mark Twain once brilliantly put it, there are lies, damned lies, and then there are statistic.

[@slimyweasles4973]

Excellent explanation! :)

[@mangcut5203]

Great video. Um, but... if you have good health, why do you even go to a hospital?

[@goober3097]

Good health = just sick, bad health = almost dead.

[@missmorbid1439]

Well, you could have a broken wrist or something similar, or maybe you’re giving birth.

[@detectiveshaft9990]

This is one of my favourite ones yet. I'll be sure to have this on hand next time I get into a flame war!

[@LBoomsky]

plot twist: the statistics used in this video are misleading

[@chroniclesofpickles]

He meant like confounding variables. No studies are flawless, yet people still need statistics to get substantial evidence.