Measures of Spread: Crash Course Statistics #4

Today, we're looking at measures of spread, or dispersion, which we use to understand how well medians and means represent the data, and how reliable our conclusions are. They can help understand test scores, income inequality, spot stock bubbles, and plan gambling junkets. They're pretty useful, and now you're going to know how to calculate them!
Crash Course is on Patreon! You can support us directly by signing up at / crashcourse
Thanks to the following Patrons for their generous monthly contributions that help keep Crash Course free for everyone forever:
Mark Brouwer, Nickie Miskell Jr., Jessica Wode, Eric Prestemon, Kathrin Benoit, Tom Trval, Jason Saslow, Nathan Taylor, Divonne Holmes à Court, Brian Thomas Gossett, Khaled El Shalakany, Indika Siriwardena, Robert Kunz, SR Foxley, Sam Ferguson, Yasenia Cruz, Daniel Baulig, Eric Koslow, Caleb Weeks, Tim Curwick, Evren Türkmenoğlu, Alexander Tamas, Justin Zingsheim, D.A. Noe, Shawn Arnold, mark austin, Ruth Perez, Malcolm Callis, Ken Penttinen, Advait Shinde, Cody Carpenter, Annamaria Herrera, William McGraw, Bader AlGhamdi, Vaso, Melissa Briski, Joey Quek, Andrei Krishkevich, Rachel Bright, Alex S, Mayumi Maeda, Kathy & Tim Philip, Montather, Jirat, Eric Kitchen, Moritz Schmidt, Ian Dundore, Chris Peters, Sandra Aft, Steve Marshall
--
Want to find Crash Course elsewhere on the internet?
Facebook - / youtubecrashcourse
Twitter - / thecrashcourse
Tumblr - / thecrashcourse
Support Crash Course on Patreon: / crashcourse
CC Kids: / crashcoursekids

[@kennethfoster3913]

You explained in 30 seconds the intuition behind variance. In 4 years my college professors could only regurgitate the formula to me.

[@rstlesswarrior]

KENNETH FOSTER isn't it obvious?

[@Tntpker]

how the hell did you survive 4 years of college if you can't even understand something simple as the variance lmao

[@JuperLuperBruh]

im so offended that you said that variance was simple and obvoius im just a kid who wants to learn math

[@verdatum]

Kudos to whoever did the background for this course. It looks great; one of my favorites yet. Edit: Kudos to Dave Freeman. Yay for end credits!

[@Anaguma79]

A detailed explanation or example of how dividing the variance by n as opposed to n-1 biases the calculation would be nice. This never got explained in any of the statistics courses I've taken.

[@HeatherLKelly]

"This is beyond the scope of a high school level course." (This translates to: I'm a high school AP Stats teacher, and I have no idea either.)

[@aarontcao]

This is actually known as Bessel's correction, and it is related to the degrees of freedom in a sample. Imagine you have a sample of n independently sampled items. You can think of this as a vector with n entries. Our n-sized vector has each entry independent of each other entry, so there are n degrees of freedom in our sample. Now let's obtain n residuals by subtracting each of the entries from the sample mean. What happens? You have now initiated a constraint on this newly transformed vector! The constraint is that all of the entries now have to sum to zero, so there is one less degree of freedom! You can essentially think about Bessel's correction as a linear algebra result from placing a constraint on a finite sample by subtracting the sample mean from each entry (and thus requiring the residuals to sum to 0).

[@YashArya01]

Thank you! This was pretty helpful. But I've been away from math for way too long so I'm going to spell out some more details that took me some time to work out. So for dummies like me. :) Say you have a list of 5 elements: 2, 2, 5, 8, 8 Mean of data = sum of all elements/ total number of elements = 25/5 = 5 Subtracting the mean from each value in the list: -3, -3, 0, 3, 3 Sum of all elements here = 0 -> added one constraint. The reason is, by the definition of a mean, we have [sum of all values less than the mean] = [sum of all values greater than the mean]. So the residues cancel out.

[@devanbecker3542]

Don't think of it as dividing by the number of points, think of it as dividing by the number of pieces of information (aka degrees of freedom, but I hate that name). When you calculate the mean, you divide by n because there are that many pieces of information. When you calculate the variance, you lost a piece of information. Why is that? It's because you already calculated the mean! If I told you all of the numbers except for one, but I also gave you the mean, you would be able to figure out what that missing point is (this was actually a question I used on a test recently). So basically, you only need n-1 data points as well as the mean in order to have n pieces of information. The mean counts as that last piece of information. The standard deviation is the average deviation per piece of information. There's no good reason to divide by the number of points. The mean is the average of all of the information from each point. You want to find how much the information in your data set varies, on average (it's the average of the deviations from the mean). The amount of information left after calculating the mean is n-1. When they get to linear models, you will find a divisor of n-2 in the variance of the slope of the line. This is because the slope is based on the mean AND the variance (or, equivalently, the intercept and the residuals); you've made 2 points of information redundant. I hope that makes sense.

[@letstalkaboutmath2121]

It comes from a theoretical reasoning. Suppose that you have a random sample, that is a set of random variable X1,...,Xn that are mutually indipendent and identically distibuted. Suppose that they have each finite mean and variance, that is the same for any Xi cause they are all identically distributed. It happnes that (sum_{k=1}^n(xk-mean)^2)/n is itself another random variable, called the variance of the sample, of which you can calculate the mean and the variance. But the mean of this new random variable is not equal to the variance of the initial distribution, but is equal to (n-1)/n times the variance of the original distribution. So one multiplies for n/(n-1) the variance of the semple for obtain a more precise estimation of the variance of the original distribution. I hope that i've been clear, althaugh it is necessary some theoretical basis to really understand it.

[@iAmTheSquidThing]

Square wins makes perfect sense. You will often hear people claim that they have won fair and square.

[@horizon241]

Love the inverted bell curve scar on her forehead instead of a lightning bolt scar in the thought bubble sequence. Makes me think she was almost killed by some deranged math professor but was saved by the power of statistical analysis.

[@honquewastaken2298]

lol

[@nathanielng-carrion497]

As a muggle-born wizard, I feel quite offended by this video.

[@nicanornunez9787]

PC MudBlood.

[@dontaskaboutmeplease]

@Rowan Brown TELL US ABOUT HOGWARTS (or illvermorny or whatever)

[@flamedragon07]

I agree it's a bit offensive. But it just an example help us better understand the material.

[@bandanawaddledee4806]

Amazing explanations. Really cleared up my questions about my statistics class. One small suggestion though. Could you please include the formulas to calculate Variation, Standard Deviation, etc. It would mean a lot to me. Thank You!

[@verdatum]

Boy do I wish reporters would give std-dev more often. I feel like they don't because they don't want to confuse people who don't know stat. I'm curious how you guys are going to do this course with minimal math. My stat classes were practically nothing but math, and I felt sorry for the kids who hadn't already taken calc...

[@luisfdconti]

She's a blood purist! Muggle borns can fly as well as anyone!

[@the.invincible.9542]

Although I know one example of a Muggle-born, i.e. Hermione, who doesn't like flying.

[@rparl]

@@the.invincible.9542 And Harry was muggle raised, but not born.

[@the.invincible.9542]

@@rparl Harry had previous experiences of flying the broom ( a kid-version, although). But yeah. It doesn't matter even if you are a muggle.

[@danieltrigo2928]

shut up you dirty blood!

[@lorettap.925]

I was really offended, too😩

[@lukehebert6207]

Using this playlist as a primer before diving back into the world of statistics after changing careers. This is super helpful! I especially loved the visualization of variance with the squares :)

[@ganaraminukshuk0]

And never forget the standard deviation.

[@aarohibhavsar1520]

What about Mean Absolute Deviation

[@bradleyosbourne8561]

Unless you're the Mongols

[@OlleLindestad]

This video contains an important misconception! :O Namely, that the standard deviation is the average of the deviations from the mean. Actually, because half of all deviations are negative (below the mean) and the other half are positive (above the mean), *the average of all deviations is always zero.* This is in fact why we square the deviations when calculating the variance - squares are always positive, so this gets rid of the problem. Of course, you could simply average the sizes of the deviations, disregarding the plus or minus signs. The resulting value is called the absolute mean deviation, and it's similar to the standard deviation, but *not the same*. Also, for historical and practical reasons, the absolute mean deviation is almost never used.

[@tsunghan_yu]

what are the historical and practical reasons?

[@Lortagreb]

Sizes is the wrong term, absolute value of the deviations is correct english

[@deekshas3936]

When calculating the mean deviation about mean and median, we just take the modulus value of the deviations. Does that mean I'm calculating the absolute mean deviation?

[@JuperLuperBruh]

i mean i agree when you say that the sum of deviations are zero but you know... does it really matter to how the variances are calculated?

[@atlaswar2425]

I love that ending! I will no longer consider myself as part of the average!

[@JuperLuperBruh]

I think of myself as one of the outliers that cause the average to change

[@mosama22]

Guys just a quick note, if you're still confused about the variance and standard deviation Khan Academy has a very nice explanation for it.

[@deniseyang9723]

the best crash course video i've seen by far. thank you for putting standard deviation into perspective!

[@RobVicRJ]

It is sad that an EDUCATIONAL video has to have a "math warning"... Awesome video btw. They shouldnt shy away from math.

[@voxatl8817]

Its for people who aren't good at math so they can get ready or not take the vid off

[@Drkfed101]

she kept saying the 'mean' number but I kept hearing "MEME" number XD

[@ganaraminukshuk0]

Drkfed101 what if the mean is 420? Is the mean now a meme?

[@Drkfed101]

lol

[@simonclarke2276]

This is absolutely brilliant - I am a physician looking to do a PhD and my last formal maths course was at high school (shall we say 30+ years ago...). I REALLY needed to go back to basics on statistics and this course is perfect. I feel set up to attack the more specialised texts on medical stats as you have explained the principles behind the formulae... thankyou!! I have one minor point, it would have been useful to mention why you square the deviation of each data point when calculating the variance; it took my rusty brain quite a while to realise that all of the deviations above and below the mean would otherwise cancel themselves out!

[@uzomaohajekwe7149]

I forgot the name of the host but she's a good host. In fact, I never remember any of the hosts other than John and Hank Green, except for Stan, of course.

[@Eric-db8jp]

I’ve been wondering for a long time. Can someone explain why we want to average the square root instead of the absolute value of the difference? If you say variance is counter-intuitive because it’s squared and take its square root to use as standard deviation, it seems much more straight forward to average the absolute values of differences in the first place. By taking the square root, we are putting more weight on the greater differences. Even if that is our intention, square seems to be an arbitrary choice. Is there any good reason why we want to average the squared differences, take its square root and call it standard deviation to use as a measure of spread?

[@determinators]

this is following my stats class almost exactly lmao

[@ThePrimevalVoid]

Can someone explain why the biased variance is corrected by subtracting the sample size by one? I mean, I understand the motive behind it, but what's the mathematical reason for doing that?

[@aarontcao]

The Primeval Void See Jack Leonard's comment in this video for a full thread discussing this. I'll repost my reply here for you too "This is actually known as Bessel's correction, and it is related to the degrees of freedom in a sample. Imagine you have n samples. You can think of this as a vector with n entries. Our n-sized vector has each entry independent of each other entry, so there are n degrees of freedom in our sample. Now let's obtain n residuals by subtracting each of the samples from the sample mean. What happens? You have now initiated a constraint on this newly transformed vector! The constraint is that all of the entries now have to sum to zero, so there is one less degree of freedom! You can essentially think about Bessel's correction as a linear algebra result from placing a constraint on a finite sample by subtracting the sample mean from each entry (and thus requiring the residuals to sum to 0)."

[@Kiefer679]

In the information age and the rise of data analytics do you think it's a necessity to learn stats, or should it still be considered an optional math topic? Should it be nessesary in highschool, college, or both?

[@moonemonne2318]

Totally didn't expect a statistics lesson teach me some life lesson. Thanks for your kind words!

[@sandiagiego]

Telewizja polska TVP pozdrawia

[@sergioramos3437]

humblebrag on the subscribers

[@ruslanakhmedagaev1640]

Math is everywhere! :) Please, make the Crash Course Mathematics; it would be a pleasure to watch these series.

[@monasayed715]

لماذا هذه الحلقه ليست مترجمه للغه العربية؟

[@najrenchelf2751]

We started having Statistics as a subject starting last week. This series’s timing is amazing!

[@krtkosilva9288]

Okaaay. I gotta watch this one more time....

[@valentinhuiban2494]

I am trying to prepare for a resit exam for my statistics course which was so confusing. I want to say that this crash course is a blessing, and especially this episode which has done a marvelous job on explaining variance and deviation and more important what is the standard and why, more than my uni could. Thank you!

[@ReaperVXG]

Phenomenal videos and narration by Adrian, certain terminology is assumed and challenging to understand but it is afterall a "Crash Course"

[@ProfessorHerning]

Can we get some parentheses? (-3)^2=9, but -3^2=-9.

[@mathdreamerpi]

Agreed ! Graphic needs to fixed!

[@enlightedjedi]

I remember from school that there were measures akin to the standard deviation that were calculated from the 3rd power of the distance to the average and then cubic root applied to the sum. Also with the forth order. I think the name of one of them was skew :)!

[@aarontcao]

Yep! You're thinking of the skewness (which utilizes the third moment) and the kurtosis (which utilizes the fourth moment).

[@enlightedjedi]

Yes!

[@dinornis]

I've been trying to wrap my head around standard deviation for a while now and was always confused by it. This video lead me to that 'aha!' moment where I finally get it -- thank you!

[@dinornis]

I struggle a lot when I see formulas, and need to see each point in action to understand what's going on. This video did this really well. As someone else mentioned, it would maybe be nice to see the formulas alongside what step you're doing, to better allow [the viewer] to convert the logic to the math and vice versa.

[@sideb4242]

please add formula subtitles. i don't understand plain speech easily, it will translate it for me from human language.

[@teendailymotivation9817]

I agree she has to add that so that her viewers can get a better glimpse of what she is trying to explain.

[@Mukesh_Kuriya]

Pls make more on designnof experiment (DOE)

[@miekwavesoundlab]

What’s the meme number of crash course videos?

[@tedoymisojos]

I love how this illustrates what Im studying. Thank you :D

[@Lortagreb]

I love crash course its like cheating

[@oskgs1115]

Please upload crash course on stock market

[@KarimaKadaoui]

I always look forward to the creeper on the shelf

[@rkpetry]

...variance being square is like energy compared to momentum-but we don't subtract one atom to compute total energy, nor do we square-root, the energy, to calculate the standard deviation of momentum... ('hmmm') ...still, it doesn't seem like enough 'qualification' of the statistics to have only mean and variance/-deviation, like we need a sense of near:far slop; we'd also want a sense of talkup criterion how-close-to-an-elementary-statistics-function...

[@SigalTifferet]

Could you please approve my Hebrew subtitles? They are from last year. Tx

[@Joeviocoe]

Too bad a Wizarding School will never have a math class.

[@brandonhaidon6705]

They have numerology lol and astronomy.

[@LetsTalkOnePiece]

no they have arithmancy not numerology. and potions also require math skills.

[@nanamaplazmaboom]

Top-notch Arithmancy course, however, Madam Hooch never gained professorship

[@swimmer1121]

The idea about to comparing yourself to the average because it may not be a precise indicator is a really interesting point.

[@lukewest8552]

I like how they had a different picture for investors and gamblers

[@gohogoho23]

I think these courses may come in handy in some 10 years when my kids reach high school years... the question is - will they understand the Harry Potter references? :P

[@nehamujumdar1990]

Could you please explain the 'kurtosis' in a sample data? N It's relation to mean and median? I liked these videos. It's proving very helpful for me to understand the subject. Thank you!

[@galeop]

I'd recommend putting less emphasis in the prononciation of the words. Please try to have a more constant flow, without highlighting some syllables

[@teendailymotivation9817]

When she puts emphasis on words we pay more more mind/attention to those words. That's probably her "SPECIAL" way of teaching a topic to her viewers. But nice recommendation.

[@horsecrazy2266]

it's helpful for my adhd because it takes me out of my own world and brings me back

[@TabithaElkins]

Thanks for these videos! You are helping me study for my statistics exam!

[@abhishekkafle5214]

WOW, you might just have changed the way I think about Everything!

[@joshuaj.knowles3664]

This was beautifully helpful!

[@moeinhasani8718]

variance=(sum((X-mean)^2)/N) standard deviation=sqrt(variance)

[@RachelMiles]

"MADAM" Hooch. Love the reference though :D

[@flamedragon07]

I agree. That was a very silly one. It was fun to visualize that minus the muggle flyers.

[@naturalism609]

Overall a decent crash course but jumping from one example to another without fully explaining the conclusions we make with each is confusing for students. Stick with one use of sports scores: baseball, basketball, or QUidich (if I spelled that right) - jumping to all 3 confuses rather than clarifies for those who don't understand - otherwise, you end up preaching to the chior. Good example of standard deviation with murder but then it jumps away before explaining the example fully, another 5-10 seconds would have done it. As this is, if I show it to a class I would expect that it would clarify the concepts for about 40% but that 40% likely don't really need it. The other 60 might walk away more confused since they understood some and not the rest.

[@odinallfather4852]

6:33 Very 'mean' of you to exclude the muggles.

[@yuzhoutv]

I think the Harry Potter example is brilliant

[@binhan6432]

Grrrrr.. got into this to help me understand stock market but without background, it is seriously getting harder from this episode.

[@JuperLuperBruh]

are you here for pleasure or for study. if it's for pleasure this might be a bit hard

[@rajakrishnan7662]

hello, why do we have to use variance and standard deviation as measures for dispersion in the data as opposed to just use the sum of modulus value or absolute value of the difference between data point and mean.i.e. sum(|mean-Xi|).

[@rajakrishnan7662]

hello, why do we have to use variance and standard deviation as measures for dispersion in the data as opposed to just use the sum of modulus value or absolute value of the difference between data point and mean.i.e. sum(|mean-Xi|).

[@JaimeNyx15]

Aha! So this is what Hermione did in Arithmancy!

[@justinwakeling8669]

This women's american accent is really getting on my nerves. Shocking accent is the yank accent. Got drama written all over it

[@dishboy08]

Could anyone explain the mathematical reasoning behind squaring the deviation (which is also called the variance, I think) to get the squared deviation? Maybe more specifically, I don't understand what the squared deviation tells us about the spread of the data that the deviation doesn't? Many thanks in advance!

[@OlleLindestad]

I wish they'd mentioned this in the video! It used to confuse me for years. :( Basically, the variance is an attempt to answer the question "how far is the typical data point from the mean?" But the problem is that roughly half the data is above the mean (the deviation is positive), while the other half-ish is below the mean (the deviation is negative). So the average of this series of negative and positive deviations is always simply zero. But if we square all the numbers before averaging them, they'll all be positive (-2 squared and 2 squared are both 4). That's why we compute the variance as the average of *squared* means. (Or rather the sum of squared means divided by the number of data points minus one, but whatever, if the sample size is large enough that's basically the same thing.) Of course, as mentioned in the video, we then get a sort of uninterpretable number - so we have to square root that bastard again to get the standard deviation, which is *not* actually the average deviation from the mean... but it's close enough, and makes more sense to us intuitively than the variance.

[@dishboy08]

Awesome, thanks so much for the succinct and easy to follow explanation!

[@tsunghan_yu]

@@OlleLindestad why not take the absolute value of the deviation then if you just want to ensure the sum is not zero?

[@OlleLindestad]

​@@tsunghan_yu Yes, you can do that. The resulting statistic is called the mean deviation, and is very intuitive, but almost unused in statistical analysis. Unlike the mean deviation, variance can be manipulated in lots of useful ways, and split up into different sources, which is what ANOVA and related techniques are based on.

[@Ouvii]

The Astro's are looking a little too good in that graph

[@anthonyeaton9049]

It's almost like they could win a World Series... ...Oh wait.

[@Ouvii]

Anthony Eaton, Hahahaha, I had no clue; I stopped caring when I moved away from Houston 3 years before. Funny.

[@ananyapamde4514]

loved the last part! thanks

[@hablahabla6653]

can we get crash course mechanics too please?

[@popcorn_consumer]

Fair bit of Mechanics in Crash Course Physics if you're looking, it's pretty damn interesting.

[@hablahabla6653]

Peachy Keen Gloria Jean thanks! I'll check it out. Though some bits of mechanics follows different procedures than physics so I'm a little worried about that.

[@lilolebob]

Mmmm, blond hair and seems to hate muggles. She's really a Malfoy.

[@BusinessAnalystGuru]

great work for a person like me who has numerophobia

[@aliciadonadio2597]

Measures of Nutella I guess What's wrong with dispersion?

[@Biscuitsdefortune]

You don't want your peanut butter and jam ending everywhere in the kitchen

[@kareemjeiroudi1964]

Man, the joke at the end is really cool, and true as well 😄

[@teendailymotivation9817]

Agreed!

[@abhishekparihar7724]

thankyou :)

[@flamedragon07]

I love the Madm Hoouch thought bubble example. It was very funny to see that in the video.

[@brittanyscholl8156]

These videos are SO helpful! When does episode 5 publish? How many more are to come?

[@EduardoEscarez]

On average, they publish one episode per week, because sometimes they publish outtakes 😛

[@funkysagancat3295]

and usually around 40 episodes

[@3rl0y]

So a mean of 307 murders and a standard deviation of 353 murders. So around 15% of the states have -46 murders or less if it follows a normal distribution. Nice to see people getting revived so frequently!

[@eoghan.5003]

In school (in Scotland) we're told that to get standard deviation we divide by n-1 and THEN take the square root. This never made sense to me, and it's not how you explained it. Does anyone know what's going on?

[@skoockum]

I get that saying the SD can be thought of as the average deviation from the mean is just an analogy. But why is the SD calculated as the sqrt[(sum of the squares of the differences)/N] instead of being calculated as the (sum of the absolute values of the differences from the mean)/N?

[@iyerakshay]

Wouldn't the Mode be a better point for comparison in a social scenario - mode weight, mode salary, mode height, mode IQ points etc.? A mode is more likely to give a better indication of where the social "average" is.

[@amineaboutalib]

IQ points and Height are natural so the Mode is pretty close to the average , however salary isn't

[@abhilashkokkonda1713]

Great explanation. I feel a little more explanation of rationale behind going for variance and standard deviation is required. Anyways I just like the course. ^^

[@emperorplushies8242]

Yeah that’s most definitely not right

[@jakescheirer3022]

Nice job. Now to just obligate people to share their SD's when they produce a Mean fact...

[@rajakrishnan7662]

hello, why do we have to use variance and standard deviation as measures for dispersion in the data as opposed to just use the sum of modulus value or absolute value of the difference between data point and mean.i.e. sum(|mean-Xi|).

[@akkshayadwivedi]

in the basketball example, the interquartile range was closer to three in my opinion. What do you think about my opinion?

[@viannechang3961]

when there are an odd number of values, is the median used to find Q1 and Q3?

[@rajv7343]

the Median calculated at 6:45 is wrong. its 40 when the outliers are included not 30. please correct me if I am wrong.

[@nyborat401]

Thank for making feeling good with statistic course 🤭😬😜

[@Sordatos]

I love that Adriene Thought bubble version has thick eyebrows, resulting it be much more expressive than the others..

[@amitdesai9584]

I think the way the explanations are orchestrated is superb... The examples and choice of words are brilliant....makes me as a viewer understand things pretty easily.

[@Hartono25277]

I am amazed. At school we are tasked to memorise equations, but Adrienne didn't need to give equations for me to understand...

[@watermeloninsideawatermelon]

just started university and this is really saving my butt 😅🤩 thank you so much CC team!

[@StellarBlack001]

So what you are saying is... that we should always get rid of the muggle born!!! It all make sense now!

[@FrancoFluence]

deviation is -3 or -3 squared?

[@hamadihamadi9219]

Brilliant Video

[@TheIanSamuels]

I don’t get that how can it be 10 & 357

[@viktorianas]

Average is OVERRATED!

[@anthonygonzalez9148]

Was struggling with these concepts and got the hang of it in the first 4 minutes. Great explanation, thank you

[@robertofontiglia4148]

"You can think of the standard deviation as the average amount that a point deviates from the mean" -- uh. That is false... ?

[@lizageorge8923]

But...it's always been Madame Hooch...

[@crocopie]

Wrong, Adrienne. We often compare ourselves to the outliers. Hence we often feel depressed. And news often report the outliers in order to sell.