Why Dividing By N Underestimates the Variance

  Рет қаралды 128,601

StatQuest with Josh Starmer

StatQuest with Josh Starmer

Күн бұрын

This is the follow up video to:
Statistics Fundamentals: The Mean, Variance and Standard Deviation
• Calculating the Mean, ...
In it, we show exactly why, when we estimate the variance, dividing by 'n' underestimates the value we are interested in. It also describes why we square each term instead of taking the absolute value. The visuals used in this StatQuest make it easy to remember why we should divide by n-1, and this will save us from falling into a very common pitfall.
If you'd like to support StatQuest, please consider...
Support StatQuest by buying The StatQuest Illustrated Guide to Machine Learning!!!
PDF - statquest.gumr...
Paperback - www.amazon.com...
Kindle eBook - www.amazon.com...
Patreon: / statquest
...or...
KZbin Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshi...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer....
...or just donating to StatQuest!
www.paypal.me/...
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
Corrections:
3:23 I should have said "To understand why dividing by n underestimates the variation around the population mean".
3:40 The estimated mean was switched with the population mean.
#statquest #variance

Пікірлер: 628
@statquest
@statquest 4 жыл бұрын
Corrections: 3:23 I should have said "To understand why dividing by n underestimates the variation around the population mean". 3:40 The estimated mean was switched with the population mean. Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
@Viralvlogvideos
@Viralvlogvideos 4 жыл бұрын
BAM BUM hahah
@mayurihazarika6550
@mayurihazarika6550 Жыл бұрын
Please Give Video on degrees of freedom please🙇
@m3c4nyku43
@m3c4nyku43 Жыл бұрын
At around 8:35, you should've used asterisk '*' character instead of 'x' character for multiplication. I was a bit confused and thought you wrote 2*(x-v)*x-1 instead of 2*(x-v)*(-1). Great video by the way!
@statquest
@statquest Жыл бұрын
@@m3c4nyku43 noted
@paulpaschert6215
@paulpaschert6215 5 жыл бұрын
is there some sort of award we can give this guy? please?!
@statquest
@statquest 5 жыл бұрын
:)
@jacobmoore8734
@jacobmoore8734 5 жыл бұрын
I think we're encouraged to purchase a double dam t-shirt or sweatshirt, which is more of a financial incentive than an award but who doesn't like getting paid to be awesome? I'll probably pick one up this weekend
@statquest
@statquest 5 жыл бұрын
@@jacobmoore8734 Thanks! :)
@paulpaschert6215
@paulpaschert6215 5 жыл бұрын
@@jacobmoore8734 award + t-shirt = double bam! just ordered my own shirt. gonna wear it to my statistics test in 2 weeks
@karankartik1327
@karankartik1327 3 жыл бұрын
Really, your way is too unique. one of the best
@arun5351
@arun5351 4 жыл бұрын
Amazing Josh!! I can't imagine how much hard work goes into simplifying the complex statistics concepts and coming up with these amazing videos. And on top of that your ingenious ideas of adding humor and musical creativity, taking the content to another level. If there was an Oscar for tutoring you'd be the undisputed winner. BAMMM !!- simply the best educator on KZbin....
@statquest
@statquest 4 жыл бұрын
BAM! :)
@namanjain8939
@namanjain8939 5 жыл бұрын
I searched for this on a number of online resources, some mentioned "n" while others "n-1", leaving me confused. This is the best possible explanation to the problem you made it really easy for us to understand. Thanks a lot !!! Bammmm subscribed and shared with friends.
@statquest
@statquest 5 жыл бұрын
Awesome!!! Thank you very much for subscribing and sharing my videos with your friends. :)
@ramkotha4726
@ramkotha4726 4 жыл бұрын
Josh, This is a total hypnotism you did with BAMs, echos, and other sounds. You mastered the art of making us stick there. I been searching for statistics and machine learning videos where they have kind of a roadmap, and simple explanations for complex topics, and this is it. You saved my life for sure, my donation is on its way, I know anything is small for what goes into making these. Hats off to you, you are a LEGEND. We owe you...
@statquest
@statquest 4 жыл бұрын
Wow, thank you!
@ksrajavel
@ksrajavel 4 жыл бұрын
Came to this video for "Why Dividing By N Underestimates the Variance" but got to know why absolute values are not used in Variance calculation. Literally cried, Prof. Josh.. Kudos to you. You are supporting me to understand the topics in statistics. I will support you regularly after I get a job soon. And I'm sure your teachings are required for many of the upcoming students in the coming decades. In India we have a concept called "Guru Kulam", and I see you as my guru (Not the term commonly known in the western world, this is more about respect)
@statquest
@statquest 4 жыл бұрын
Thank you so much!!! It means a lot to me.
@nursahidassafaat6283
@nursahidassafaat6283 4 жыл бұрын
I've been 2 years asking how to plot variance, why sample variance (also sd) divided by n-1. And this is best explanation i ever had
@statquest
@statquest 4 жыл бұрын
Awesome! :)
@arbanafal
@arbanafal 5 жыл бұрын
I have nothing but admiration; this is the clearest explanation that I've seen so far that does not shy away from the underlying math, yet still keeping it understandable for those with minimal math background. I feel like a bit of a fool when I see the contrast between my own attempts to explain this correction factor and your explanation.
@statquest
@statquest 5 жыл бұрын
I'm glad you like the video so much. Thanks! :)
@wowZhenek
@wowZhenek 3 жыл бұрын
Yet another video from this channel that leaves me speechless. I've never really understood this concept until I've watched your video. Thank you very much, again.
@statquest
@statquest 3 жыл бұрын
Wow, thank you!
@dimiw5435
@dimiw5435 4 жыл бұрын
the best accessible explanation I can find in the whole internet for this mystery. then just as I was about to say "aha! you missed out something!" towards the end of the video, you seemed to have read my mind and "p.s. if you are wondering why n-1 and not 0.5 or 2 .... " you are so so spot-on!
@statquest
@statquest 4 жыл бұрын
Thank you very much! :)
@naysannaderi5135
@naysannaderi5135 4 жыл бұрын
@@statquest I agree - best explanation i have found and i'm sharing this video with all my students. THANK YOU! So.... any chance that next video is coming out soon? (or has come out already?)
@statquest
@statquest 4 жыл бұрын
@@naysannaderi5135 I hope the next video will come out soon. Possibly in the next 4 months or so. I hope!
@achannel9598
@achannel9598 2 жыл бұрын
came from calculating the mean, variance and SD video. Did not expect a proof for why variance = x-bar. This is a really good in depth video i've ever watched for statistics. Thank you very much.
@statquest
@statquest 2 жыл бұрын
bam!
@keej7146
@keej7146 Жыл бұрын
Thank you for this!! The first time I saw the formula for the sample variance I wondered why the n-1 was there, this is a great explanation.
@statquest
@statquest Жыл бұрын
Thanks!
@Calypso-rt5tf
@Calypso-rt5tf Жыл бұрын
hello, keej i hate u mate
@punktdotcom
@punktdotcom 4 жыл бұрын
I rather get a clear and understanding explanation with "BAMS" like i'm five, than a 50 pages long explanation with words like "trivial" and abbreviations (q.e.d) and just feel depressed and left clueless. And an other very important thing: Only if you *really* understood the topic, you can explain it with easy words. Very well done, Josh! Thank you very much!
@statquest
@statquest 4 жыл бұрын
Thank you very much!!!! :)
@chathurijayaweera1590
@chathurijayaweera1590 2 жыл бұрын
Thank you for this explanation. When I was learning stat in university, I did not understand well, why we divide by (n-1) instead of n to estimate sample variance. You explained it so clearly in a way that I will never forget what I learnt. Thank you Josh !!!
@statquest
@statquest 2 жыл бұрын
Hooray! :)
@mugssyy
@mugssyy 4 жыл бұрын
Michael Scott: Why don't you explain this to me like I'm five? Josh Starmer: Bammm!! and understood ... thank you : ) !
@statquest
@statquest 4 жыл бұрын
BAM! :)
@Deepak-uv8du
@Deepak-uv8du 3 жыл бұрын
@@statquest Can you provide the slides for all the statistics videos you used to explain the concepts
@statquest
@statquest 3 жыл бұрын
@@Deepak-uv8du I have PDF study guides for some of my videos here: statquest.org/studyguides/
@davidh1876
@davidh1876 5 жыл бұрын
Big thanks from Taiwan. I have been asking why not dividing by n since high school...but all I get from my teacher was only "a rule of thumb". Now I know the reason behind and thanks to statquest. BAM!!
@statquest
@statquest 5 жыл бұрын
Hooray! :)
@killua9369
@killua9369 2 жыл бұрын
I have always hated statistics but I just today found this channel and this guy explains everything elegantly! ❤😊
@statquest
@statquest 2 жыл бұрын
Wow, thank you!
@jsc3417
@jsc3417 4 жыл бұрын
Thank you, 10 years of confusion made clear by this 15 mins of video.
@statquest
@statquest 4 жыл бұрын
Hooray! I'm glad the video was helpful. :)
@libertarianPinoy
@libertarianPinoy 5 жыл бұрын
Kids today are so lucky they can review their stats online like this with great teachers.
@statquest
@statquest 5 жыл бұрын
:)
@Ana-wx8jm
@Ana-wx8jm 4 жыл бұрын
I click the like button before I watch it because I'm always sure I'll love it! Thanks so much for making this series. You'll never know how helpful it has been in my life
@statquest
@statquest 4 жыл бұрын
Hooray!!! Thank you very much! :)
@cristianleoni6852
@cristianleoni6852 4 жыл бұрын
Amazing explanation of why we use the square of the errors instead of the absolute value! I always asked myself that and all the teachers said it was just to give a bigger weight to the errors! We need the statquest on expected value!
@statquest
@statquest 4 жыл бұрын
Thanks! I'm working on the expected value, but it still might be a few months before it's ready.
@theblinkingbrownie4654
@theblinkingbrownie4654 8 ай бұрын
I think for even n there wouldn't even be a minimum point, rather a flat line between the 2 middle samples
@GbUnLimiteD
@GbUnLimiteD 5 жыл бұрын
Yes! I had already feared that the n-? question won't be explained. Glad to hear that you will explain this unsolved mystery in the next video!
@statquest
@statquest 5 жыл бұрын
Unfortunately, it will be a while before I get to it. I've got covariance and correlation coming up next, then a few machine learning videos, but then I'll loop back to expected values. It's a topic that I've wanted to work on for quite some time.
@2oqp577
@2oqp577 5 жыл бұрын
My uneducated guess about n-x is that the bigger the magnitude diff. between the population and your sample size, the larger x would be. Because as this magnitude get smaller and smaller, the need for x to have any significant value, disappears. My biggest question is why would x lead to this unitary value when your sample size is little. But we'll see what Josh explains about that.
@nickp7526
@nickp7526 4 жыл бұрын
Intuitively: the number you're dividing stands for the degrees of freedom you have. In other words: how many data points are allowed to vary freely. The reason that this is 1 less here is, as the video hinted at, because of the sample mean. If someone shared with you n-1 data points of their sample distribution of n points, and you know what the sample mean is, then you can easily calculate what the last data point is. I.e. that last data point doesn't have any freedom to vary, just because it was crucial in defining the sample mean. This doesn't matter if you know what the population mean is, precisely because the sample distribution didn't decide its value. Therefore all n values in a sample distribution with known population mean can be used to make an unbiased estimator, while only n-1 degrees of freedom can be used to have an unbiased estimator when all you known is the sample mean. Mathematically: en.m.wikipedia.org/wiki/Bias_of_an_estimator The first example (in the examples tab) shows why it should be n-1, and not n, or n-whatever.
@anishchhabra5313
@anishchhabra5313 2 жыл бұрын
This is epic, never got a better or clearer explanation for this particular problem. Hats off!🙌
@statquest
@statquest 2 жыл бұрын
Thanks a ton!
@ireneylhsiao
@ireneylhsiao 5 жыл бұрын
Bam!!! I've watched lots of your videos after I discovered the one explaining the standard error. You make me understand stats concepts more clearly. Please continue making these awesome videos (machine learning too)! 5 dollars donated!
@statquest
@statquest 5 жыл бұрын
Thank you very, very much. I really appreciate it. :)
@anujlahoty8022
@anujlahoty8022 5 жыл бұрын
Awesome and the best video with most simplified explaination.
@statquest
@statquest 5 жыл бұрын
Thank you! :)
@yildizkoca8878
@yildizkoca8878 5 ай бұрын
This video is such a gem! Thanks for explaining the root of this concept which is not easy to find even in statistics books.
@statquest
@statquest 5 ай бұрын
Glad it was helpful!
@ARM26878
@ARM26878 2 жыл бұрын
BAM! I have not seen this concept explained better anywhere else ever. Have you gotten around to making the follow-up video on 'expected values' ? Can't thank you enough for your channel
@statquest
@statquest 2 жыл бұрын
I've got the video on expected values kzbin.info/www/bejne/gX3WkGqYbLh-n5Y and kzbin.info/www/bejne/hYSzo2l9a7CUY7c , but there are still a few steps to go after that... :(
@PraveenKumar-yv5zn
@PraveenKumar-yv5zn 4 жыл бұрын
This is the best explanation that I've come across for this. And I really liked that you gave a proof for general set of observations. Thanks a lot.
@statquest
@statquest 4 жыл бұрын
Awesome, thank you!
@Michael-zn4oq
@Michael-zn4oq 4 жыл бұрын
Thank you so much for the clear and simple explanation. This is an example for when showing the proof is better than only trying to give an intuition.
@statquest
@statquest 4 жыл бұрын
Thanks
@ryanmckenna2047
@ryanmckenna2047 Жыл бұрын
This channel is just incredible, well done!
@statquest
@statquest Жыл бұрын
Thank you very much! :)
@haoqichen7610
@haoqichen7610 2 жыл бұрын
The last point about absolute value explains a lot! I was always wondering why squaring data is so much more common than taking absolute values!
@statquest
@statquest 2 жыл бұрын
bam! :)
@christopherchen4920
@christopherchen4920 3 жыл бұрын
The most impressive explanation I've ever seen.
@statquest
@statquest 3 жыл бұрын
Thanks!
@lelamakharadze727
@lelamakharadze727 5 жыл бұрын
"Future is nooow, BAM " - #LOL #respect #welldone #thanks
@statquest
@statquest 5 жыл бұрын
Thank you! :)
@rajarshibasak347
@rajarshibasak347 3 ай бұрын
Aah! Finally end. What a excellent work by you!! Statquest rocks ❤.. Thank you sir. You helped a lot in my carrier ❤.
@statquest
@statquest 3 ай бұрын
Thanks!
@tumul1474
@tumul1474 5 жыл бұрын
Statquest, JBstatistics and Khan Academy.....You guys are just amazing !!.....Thank you for all you have done for us
@statquest
@statquest 5 жыл бұрын
Thank you! :)
@izebit
@izebit 5 жыл бұрын
Thank you, I haven't known about these channels
@morenomartinovic4385
@morenomartinovic4385 4 жыл бұрын
I'm eagerly awaiting the expected values quest! Thank you so much for making these videos, I love watching them before sleep.
@statquest
@statquest 4 жыл бұрын
Awesome! It's on the to-do list, but it might not be done for awhile. :(
@morenomartinovic4385
@morenomartinovic4385 4 жыл бұрын
@@statquest That's cool, take your time to keep making awesome videos. I still have loads of your videos on my to-watch list!
@alexandermedina4950
@alexandermedina4950 3 жыл бұрын
I can only have love for these videos, thank you Josh and all the team if you have any.
@statquest
@statquest 3 жыл бұрын
Thank you! It's just me doing all this.
@edward8064
@edward8064 3 жыл бұрын
Mind = Blown. Thankyou from Indonesia.
@statquest
@statquest 3 жыл бұрын
Thanks!
@Igor-vb1hv
@Igor-vb1hv 4 жыл бұрын
Thanks for explanation! I understand that differences between the SAMPLE data and the sample mean are smaller than the differences between the SAMPLE data and the population mean. BUT! We are not interested in the difference between the SAMPLE data and the population mean, rather we are looking for the difference between the TRUE POPULATION data and the population mean (the population variance). And it's not clear why this value would be larger. I mean sample data is centered around sample mean the same way population data is centered around population mean. Comparing sample data with population mean feels to be misleading.
@statquest
@statquest 4 жыл бұрын
The best estimate we can do is the estimate of the variance around the sample mean, which is probably an underestimate, but not always. So this is the best we can do.
@OdysseusKingofIthaca-o4n
@OdysseusKingofIthaca-o4n 3 ай бұрын
Thank you St Josh for this illuminating explanation :)
@statquest
@statquest 3 ай бұрын
My pleasure!
@scuti7073
@scuti7073 2 жыл бұрын
Man, I always thought that statistics doesn’t make any sense at all and that people should just blindly chug into weird formulas without questioning, but this was absolutely mind opening. Not even khan academy could explain the proof!
@statquest
@statquest 2 жыл бұрын
Thanks!
@HamidNourashraf
@HamidNourashraf 2 жыл бұрын
I love the way you explain these topics, great work!
@statquest
@statquest 2 жыл бұрын
Thanks!
@tippyandfriend
@tippyandfriend 5 жыл бұрын
This is excellent, I am looking forward to the next one.
@Drugio24
@Drugio24 5 жыл бұрын
this is literally what I was trying to get a clear understanding on in the last few days? what are the chances? no seriously what are the chances?
@statquest
@statquest 5 жыл бұрын
That's awesome! :)
@nizarch22
@nizarch22 3 жыл бұрын
I don't even remember what I was confused about in particular, but I remember feeling very happy to see this video. Will revisit this in the following days. Psst, you're a gem ;)
@statquest
@statquest 3 жыл бұрын
Thank you very much! :)
@emmaning992
@emmaning992 4 жыл бұрын
I admire this explanation... Amazing. I really look forward to the expected values video!
@statquest
@statquest 4 жыл бұрын
Thank you. I started working on the expected value video, but it will still be awhile before I finish since I have many other projects to work on.
@marinasha2949
@marinasha2949 Жыл бұрын
Good job Josh!! Waiting for StatQuest on Expected Values! I am the one wondering why not dividing by 'n-0.5' or 'n-2'
@statquest
@statquest Жыл бұрын
Thanks!
@magtazeum4071
@magtazeum4071 2 жыл бұрын
8:22 `the way he said "Whaat" is so cute.. I'm in love
@statquest
@statquest 2 жыл бұрын
:)
@radosawszostak6104
@radosawszostak6104 Жыл бұрын
Great video! We clearly see that estimated variation is smaller than desired so we have to make it bigger. We can make it by dividing by n-1, but also by n-2 or n-1.5 or n-100. Why n-1?
@statquest
@statquest Жыл бұрын
One day I'll make that video, for now, see: online.stat.psu.edu/stat415/lesson/1/1.3
@mahdixareie2348
@mahdixareie2348 8 ай бұрын
Thanks Josh, Awesome video But one question, in the video you proved using sample mean to calculate the variance is always result to a value less then calculating using population mean. But why do we think if we divide by n-1 that would be enough to avoid underestimating the variance? why not n-2? or, how do we know if we plug n -1, it wouldn't result in overestimating the variance?
@mahdixareie2348
@mahdixareie2348 8 ай бұрын
Ahh, I just got to the PS part :))
@statquest
@statquest 8 ай бұрын
Small bam... :)
@DarkPrincess_M
@DarkPrincess_M 3 ай бұрын
​@@statquest And when the PS part would come? Cuz I would like to know the reason behind it
@timothymattnew
@timothymattnew 3 жыл бұрын
I really want to understand why we use n-1 instead of substituting any other number instead of 1. I'm guessing it has something to do with the way we approximate the mean and the variance. I think it's related to properties the normal distribution has and such. I think that to truly understand that analytically I'd have to integrate over all possible outcomes while taking into account all the probabilities and then calculating the average. It really excites me, but I don't know where I can find the information needed to understand the subject in more depth. Can you give me some advice on what textbooks I should read, please? I'd really really appreciate that!
@statquest
@statquest 3 жыл бұрын
See: online.stat.psu.edu/stat415/book/export/html/886
@timothymattnew
@timothymattnew 3 жыл бұрын
@@statquest thank you, I will definitely read that!
@brucewayne6744
@brucewayne6744 5 жыл бұрын
Great explanation!! I'm loving every second of your videos!!! Cheers!!
@statquest
@statquest 5 жыл бұрын
Thank you! :)
@lyrachang950
@lyrachang950 11 ай бұрын
im currently learning data analytics and trying to figure out ab testing and bam! here i am! thank you so much for making statistics fun and easy to understand! double bam!
@statquest
@statquest 11 ай бұрын
Happy to help!
@mukhtarbimurat5106
@mukhtarbimurat5106 Жыл бұрын
Greatest explanation so far!
@statquest
@statquest Жыл бұрын
Thank you! :)
@yufeizhan726
@yufeizhan726 3 жыл бұрын
I finally know why n-1 is used. Thank you so much!
@statquest
@statquest 3 жыл бұрын
Bam!
@ROTOBAfilms
@ROTOBAfilms Жыл бұрын
You are a very great teacher, i like your coaching style, keep going on!
@statquest
@statquest Жыл бұрын
Thank you! 😃
@thegamingannex5752
@thegamingannex5752 2 жыл бұрын
Your work is impeccable. BAM!
@statquest
@statquest 2 жыл бұрын
Thank you!
@mansoorbaig9232
@mansoorbaig9232 4 жыл бұрын
This is awesome explanation. Waiting for quest on 'Expected Values'....BAM!
@statquest
@statquest 4 жыл бұрын
Me too. Hopefully I can get to it soon.
@Ujjwalchhabra1
@Ujjwalchhabra1 4 жыл бұрын
You left in a cliff hanger of expected values :(( Love your videos tho, thanks for these!
@statquest
@statquest 4 жыл бұрын
I'm working on it, but everything I do takes longer than I would like. :)
@shubhamtalks9718
@shubhamtalks9718 4 жыл бұрын
Man, you are great. From where did you learn these concepts? Keep making videos and enlighten us. Thank you.
@statquest
@statquest 4 жыл бұрын
Thanks! :)
@shubhamtalks9718
@shubhamtalks9718 4 жыл бұрын
@@statquest When I try to learn these concepts they seem complicated to me. From where did you learn these concepts?
@statquest
@statquest 4 жыл бұрын
@@shubhamtalks9718 The concepts seem complicated because people that do not really understand them try to teach them. How did I learn them? Years of really hard work. I read everything I can about a subject, then I re-read it. Then I re-read it again. Then I make a program based on my ideas and see what happens. Then I re-read everything over again. And sooner or later I figure it out. But it takes a lot of time and a lot of work. Sometimes I worry I will not succeed, and sometimes I fail, but I keep trying anyway.
@shubhamtalks9718
@shubhamtalks9718 4 жыл бұрын
@@statquest Thanks😁
@nidhiarora4739
@nidhiarora4739 4 жыл бұрын
I have been SO stressed out about a project I'm working on, and 3:15 made me laugh so hard!!! I didn't even realize how stressed out I was until I caught myself laughing for the first time in weeks. Thank you Josh!!! **sob**
@statquest
@statquest 4 жыл бұрын
Hooray!!! Good luck with your project. I hope it goes well. :)
@patbentolilarhythmking
@patbentolilarhythmking Жыл бұрын
Clarity brings understanding
@statquest
@statquest Жыл бұрын
Bam! :)
@MirrorNeuron
@MirrorNeuron 11 ай бұрын
Hi Josh, where did you study about it, is it from Bessel's correction or Karl Pearson. I am interested to ready a bit about the history behind it. Can you please suggest a book or paper where the original discovery was made. Thanks in advance.
@statquest
@statquest 11 ай бұрын
The idea for this came from Bessel's correction.
@sinchanar9360
@sinchanar9360 9 ай бұрын
Thank you for this amazing video Prof. Josh. This really helped me understanding why we use n-1 instead of n. But I had a doubt, doesn't dividing by n-1 lead to overestimation?
@statquest
@statquest 9 ай бұрын
You can actually show that if you use n-1 that the estimator is unbiased, but it's a relatively complex proof.
@sinchanar9360
@sinchanar9360 9 ай бұрын
@@statquest Oh ok. Thank you the clarification : )
@fernandoaloisiohm
@fernandoaloisiohm 5 жыл бұрын
You gave an amazing explanation about why dividing by n underestimates the population variance. Your videos are awesome, i love them. I just don't get why n-1 to find the unbiased estimator. Is that because of degrees of freedom thing? Sorry about my english, and regards from Brazil.
@statquest
@statquest 5 жыл бұрын
Oi! It has to do with "expected values". That's a pretty involved topic, so I've saved it for another series of videos.
@ujjwal2912
@ujjwal2912 3 жыл бұрын
Although you make everything look so simple, your teaching pedagogy requires a lot of hardwork( to make the slides particulalry). I hope that every teacher puts in the same kind of hardwork and assume there students to be in 5th grade that way every class will be a pleasureable experience of life.
@statquest
@statquest 3 жыл бұрын
Wow, thank you!
@waddragon
@waddragon 3 жыл бұрын
it is because some teachers don't know how to teach. They learn from textbook's concept. Memorize them, then give those back to students. I am not being rude but it is the reality. In order to be able to explain well to new learners, teachers must be able to understand the concepts well. Teaching is a hard skill to master. Nowadays, lot of taught concepts are assumed true or left blank during teaching . That's why if those students become teachers, they won't be able to explain.
@chiragsomani101
@chiragsomani101 2 жыл бұрын
ASTOUNDING EFFECTS & EXPLAINATIONS! SUBSCRIBED TRIPLE BAM!!!
@statquest
@statquest 2 жыл бұрын
bam!
@coldbrewed8308
@coldbrewed8308 8 ай бұрын
Oh no... I'm falling deeper and deeper into this rabbit hole
@statquest
@statquest 8 ай бұрын
:)
@taotaotan5671
@taotaotan5671 3 жыл бұрын
I just read wiki and found that even divided by n-1, we still underestimate the standard deviation (although we don't underestimate the variance anymore). I feel that's somewhat mind-blowing, since calculating sample std is such an ordinary job for statisticians, and it is surprisingly BIASED (and I am sure the standard error formula is also biased)...
@statquest
@statquest 3 жыл бұрын
interesting
@taotaotan5671
@taotaotan5671 3 жыл бұрын
@@statquest Yeah. This is the wiki page. en.wikipedia.org/wiki/Unbiased_estimation_of_standard_deviation
@dver7349
@dver7349 Жыл бұрын
Super interesting! Thanks for your work!
@statquest
@statquest Жыл бұрын
Thanks!
@samarthpatil2599
@samarthpatil2599 3 жыл бұрын
Loved the video. But didn't understand something clearly. The variance is the least around the calculated mean. But that is only when the data x remains the same right? How can you compare it with the population variance which has a lot more data points and the summation is therefore different?
@statquest
@statquest 3 жыл бұрын
We are not comparing it to the population variance. We are simply comparing the variance of the data calculated around the sample mean compared to the variance of the data calculated around the population mean.
@SimónG.U
@SimónG.U 8 күн бұрын
Great video! I'd love to see the video on expected values to understand why precisely it's n-1 and not n-0.5 or n-2. Could someone PLEASE link the video? Thanks!
@statquest
@statquest 7 күн бұрын
I still have to do that. in the mean time, see: online.stat.psu.edu/stat415/lesson/1/1.3
@SimónG.U
@SimónG.U 7 күн бұрын
@@statquest Reply from the legend himself! Thanks!
@sephirothjc
@sephirothjc 2 жыл бұрын
This the best explanation ever
@statquest
@statquest 2 жыл бұрын
Thank you!
@inkevinsshoes4690
@inkevinsshoes4690 3 жыл бұрын
Great video! Which book did you get this explanation from?
@statquest
@statquest 3 жыл бұрын
Ummm....I just did the math.
@keysky_1622
@keysky_1622 5 жыл бұрын
wow that n-1 has something to do with E(X)? Im waiting for it!
@dan_mirnejhad
@dan_mirnejhad Жыл бұрын
if the population doesn't follow a normal distribution, will the sample variance (following the population formula and not the one with the correction) still be lower than the population variance? does the formula with the correction still hold up for samples from a population that does not follow a normal distribution? I'm 17, and pretty new to the world of statistics, this channel has completely changed my feelings about stats, I used to hate it and find it so boring but now I'm deeply interested and find it really cool, thank you, keep going!
@statquest
@statquest Жыл бұрын
Great question. To be honest, I'm not sure, but I believe this holds true for any underlying distribution. See: en.wikipedia.org/wiki/Bessel%27s_correction
@dan_mirnejhad
@dan_mirnejhad Жыл бұрын
@@statquest okay I see, thanks for the link. (sorry for the late reply, youtube didn’t send a notification for it) you would think that this bessel’s correction would be a variable relative to the size of the sample, because i can’t imagine n-1 helps the underestimation at all if the sample size is huge, would you say the small change makes a difference on a larger scale?
@statquest
@statquest Жыл бұрын
@@dan_mirnejhad The larger your dataset, the better your estimate will be, and the less it needs to be corrected.
@dan_mirnejhad
@dan_mirnejhad Жыл бұрын
@@statquest that makes so much sense thank you
@Lsazeh
@Lsazeh 2 жыл бұрын
Thanks so much for the explanation, super clear as always
@statquest
@statquest 2 жыл бұрын
Glad it was helpful!
@koreanbroadcastarchive306
@koreanbroadcastarchive306 3 жыл бұрын
Excellent. Thank you for a great explanation.
@statquest
@statquest 3 жыл бұрын
Glad you enjoyed it!
@ipmankus
@ipmankus 4 жыл бұрын
Very nice explanation, god bless you josh!
@statquest
@statquest 4 жыл бұрын
Thank you! :)
@Marius-vw9hp
@Marius-vw9hp 5 жыл бұрын
I wanted to know why we deduct exactly 1, but I guess that only takes 20 aditional minutes to explain. Hooraay! Thanks for the videos :)
@statquest
@statquest 5 жыл бұрын
It's true. We have to dive into expected values and that is a whole new topic.
@thanhtungnguyen7500
@thanhtungnguyen7500 4 жыл бұрын
I wonder population variance = sum((xi - mu)^2)/N, (i = 1,...,N). sum((xi - x_bar)^2)/n < sum((xi - mu)^2)/n (i = 1,...,n) doesn't mean it < sum((xi - mu)^2)/N, (i = 1,...,N) Could you explain on this
@statquest
@statquest 4 жыл бұрын
That's a good question and I don't know the answer.
@thanhtungnguyen7500
@thanhtungnguyen7500 4 жыл бұрын
thanks for your feedback, I will try to figure out anyway, really love your songs & explanations, it help me a lot
@88skewer
@88skewer 3 жыл бұрын
awesome video, do you have any recommended channel to learn derivative and calculus ?
@statquest
@statquest 3 жыл бұрын
I have a video on The Chain Rule Here: kzbin.info/www/bejne/rZ2Unqyup9mEfrM but I have heard that Khan Academy is good for learning derivatives.
@iAmTheSquidThing
@iAmTheSquidThing 4 жыл бұрын
This intuitively makes more sense to me now. If I take a sample, the sample mean may end up being larger or smaller than the population mean. But the sample variance can never be larger than the population variance, it might be equal to it, but most probably it will be smaller.
@statquest
@statquest 4 жыл бұрын
That's exactly right. :)
@wobwobvoid420
@wobwobvoid420 Жыл бұрын
I think you have to be a little bit careful with what you mean by "sample variance" and "population variance". As long as you're comparing an estimated population variance using the sample data and actual population mean vs an estimated population variance using the sample data and sample mean. But, comparing the estimated population variance using the sample data and sample mean vs the actual population variance (all data and actual population mean) doesn't have the guarantee that sample variance will be lower than population variance.
@sourabh513
@sourabh513 2 жыл бұрын
Great video! Can you share link of StatQuest on Expected Values that explains why divide by n-1 and not n-0.5 or not n-2? Thanks!
@statquest
@statquest 2 жыл бұрын
I hope to do that one day.
@thepahadiboi
@thepahadiboi 4 жыл бұрын
What a explanation. I don't have money, else I'd have contributed. The least I could do is share, which I already did. BAM !
@statquest
@statquest 4 жыл бұрын
BAM! :)
@ThalesBrunoM
@ThalesBrunoM 4 жыл бұрын
8:21 -> I will watch a thousand times and I will laugh out loud a thousand times 😂
@statquest
@statquest 4 жыл бұрын
Hooray! :)
@shashankupadhyay821
@shashankupadhyay821 4 жыл бұрын
I usually hit like after the first BAMMM. This is some super great stuff Josh.
@statquest
@statquest 4 жыл бұрын
Thank you very much! :)
@ginopeduto4264
@ginopeduto4264 4 жыл бұрын
THX!!! Looking forward for the STATQUEST on expected Values ;))))
@statquest
@statquest 4 жыл бұрын
Me too!
@a950721
@a950721 4 жыл бұрын
Sorry I am still getting confused. At 5:29, in the inequality, both left hand side and right hand side are using the same n, which is the number of samples. You argued that the right hand side is greater so that we need to make the left hand side larger by dividing n-1. However, the right hand side is not the actual population variance. The actual population variance should be using a much larger n to calculate. What we are doing here is to estimate the population variance but not the right hand side. Thinking to this point, all the linkage seems broken. How can I relate the right hand side to the population variance? It is true that the inequality holds. But it does not mean also the population variance is always greater than the left hand side. Thanks for your videos. They inspire me and teach me a lot.
@statquest
@statquest 4 жыл бұрын
Sometimes we know the population mean, but don't know the variance, so we sill have to estimate it. That is what is going on on the right side of the equation.
@truewarrior911
@truewarrior911 Жыл бұрын
I literally watch your videos for fun.
@statquest
@statquest Жыл бұрын
bam!
@MrBlissTube
@MrBlissTube 4 жыл бұрын
Great video! Where is the one about Expected Values? I cannot wait with such a cliffhanger! GoT finale can wait...
@statquest
@statquest 4 жыл бұрын
Very funny! Yes, I have my work to do. I hope to get to expected values before too long.
@MrBlissTube
@MrBlissTube 4 жыл бұрын
@@statquest Thanks a lot for responding! ... and sorry, as I noticed after reading more comments, that you had already answered this question many times. Quest on!
@groovearmada2008
@groovearmada2008 4 жыл бұрын
Has the announced video "expected values" been released? I couldn't find it... Thx
@statquest
@statquest 4 жыл бұрын
Not yet, but I'm working on it.
@michaelbruce4987
@michaelbruce4987 4 жыл бұрын
Just as I was typing the question Why not use N minus 0.5 or 2 and is there any time that we would use something other than 1, the ending of the video told me to stay tuned.
@statquest
@statquest 4 жыл бұрын
Yes! And I'm finally making progress towards the follow up video (it's very slow progress - so don't get too excited - but progress is progress!)
@eddy147Tennis
@eddy147Tennis 5 жыл бұрын
Question: why is it that the differences between the data and the sample mean is less than the differences between the data and the population mean? If I take the most extreme data points for the sample mean won’t I have a bigger variance for the sample data? P.S. I agree you need to get some award, videos are fun, educational and clear.
@statquest
@statquest 5 жыл бұрын
One of the properties of the sample mean (which was illustrated in the examples and in the calculus) is that it always in a location that minimizes the distances between it and the sampled data.
@shouryanand456
@shouryanand456 4 жыл бұрын
I wish you were my stats teacher!! Amazing job!!!
@statquest
@statquest 4 жыл бұрын
Thank you! :)
@shouryanand456
@shouryanand456 4 жыл бұрын
@@statquest really waiting for the expected value video to get explanation of n-1. When can we expect it?
@statquest
@statquest 4 жыл бұрын
@@shouryanand456 Unfortunately, it might be a while. I've got a full plate until after the summer.
@vkvkvkvk
@vkvkvkvk 5 жыл бұрын
baaammm! subscribed.
@statquest
@statquest 5 жыл бұрын
Awesome! :)
@nursahidassafaat6283
@nursahidassafaat6283 4 жыл бұрын
BAAM! me too
@NuclearSpinach
@NuclearSpinach 3 жыл бұрын
"The future is now" I'm dying
@statquest
@statquest 3 жыл бұрын
BAM! :)
@rajkumarguptafx3907
@rajkumarguptafx3907 4 ай бұрын
Your Voice is magical 🌹🌹🌹
@statquest
@statquest 4 ай бұрын
Thank you!
@alishashingade4456
@alishashingade4456 3 жыл бұрын
You are so awesome ! Thank you Josh :)
@statquest
@statquest 3 жыл бұрын
Thank you! :)
@hafidhrendyanto2690
@hafidhrendyanto2690 2 жыл бұрын
Amazing video! I think that you should teach another subject. Maybe MathQuest? That would be amazing!
@statquest
@statquest 2 жыл бұрын
Maybe one day!
Covariance, Clearly Explained!!!
22:23
StatQuest with Josh Starmer
Рет қаралды 560 М.
Calculating the Mean, Variance and Standard Deviation, Clearly Explained!!!
14:22
StatQuest with Josh Starmer
Рет қаралды 466 М.
Крутой фокус + секрет! #shorts
00:10
Роман Magic
Рет қаралды 41 МЛН
ТИПИЧНОЕ ПОВЕДЕНИЕ МАМЫ
00:21
SIDELNIKOVVV
Рет қаралды 1,8 МЛН
Dividing By n-1 Explained
14:18
PsychExamReview
Рет қаралды 6 М.
What are degrees of freedom?!? Seriously.
27:17
zedstatistics
Рет қаралды 198 М.
Expected Values, Main Ideas!!!
13:39
StatQuest with Josh Starmer
Рет қаралды 193 М.
ROC and AUC, Clearly Explained!
16:17
StatQuest with Josh Starmer
Рет қаралды 1,5 МЛН
The standard error, Clearly Explained!!!
11:44
StatQuest with Josh Starmer
Рет қаралды 222 М.
But what is the Central Limit Theorem?
31:15
3Blue1Brown
Рет қаралды 3,5 МЛН
Variance and Standard Deviation: Why divide by n-1?
13:47
zedstatistics
Рет қаралды 275 М.