Machine Learning Fundamentals: Bias and Variance

[@statquest]

Correction: 4:06 I say that the difference in fits between the training dataset and the testing dataset is called Variance. However, I should have said that the difference is a _consequence_ of variance. Technically, variance refers to the amount by which the predictions would change if we fit the model to a different training data set. Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

[@leif1075]

And at 4:55, why do you say straight line has low variance? That isn't necessarily true since those points on the graph could be anywhere else and if they are farther from the line, the sum of squares could easily be much greater. .

[@statquest]

@@leif1075 Given this dataset, the straight line has lower variance than the squiggly line. Given another dataset, things could be very different.

[@leif1075]

@@statquest Ok so you were only referring tp this dataset then? Sorry What I said is correct though in general right?

[@statquest]

@@leif1075 Regardless of the models and the data, you always have to test to see which one has the least variance.

[@leif1075]

@@statquest So what I said was correct then?

[@y.gromyk]

4 hours of the lectures with a lot of complicated math: got nothing 6 minutes with the singing guy: *DOUBLE BAM*

[@statquest]

Hooray! :)

[@vitaminprotein2789]

You can't get anywhere without the math

[@headyshotta5777]

@@silly2099 ima jus import da python library my G. math is for fools

[@ninobach7456]

Bam

[@0xh8h]

Better than lots of courses on Udemy. I really like your humor

[@statquest]

Thanks! :)

[@tahmidurrahmaneram]

@@statquest BAMMMM!!!!!!

[@statquest]

@@tahmidurrahmaneram :)

[@prashdash112]

@@statquest DOUBLE BAMMM

[@statquest]

@@prashdash112 Thanks! :)

[@reniellechavez3689]

This guy has united his two passions-Machine Learning and guitar.

[@statquest]

Yes! :)

[@LevanZoSoGharibashvili]

and mice :)

[@hannahdo980]

and singing & composing! Loved the intro in this video :)

[@funnybunnies3985]

and saying "Bam"

[@shaiguitar]

Josh! How about that transformers video? Eagerly awaiting your humorous and mad explanation skills. Perhaps how it relates to its predecessor models? Key Query Value bit would be great as well. Keep on rocking it.

[@Maha_s1999]

I went from BUMMED to DOUBLE BAM in six and a half minutes. God bless you!

[@statquest]

Hooray! :)

[@mjatx]

I did the same in just over three minutes with increased playback speed! BAM

[@mortysmith8980]

Notes for myself: Def. of Bias: The inability for a machine learning method to capture the true relationship is called Bias. Def. of Variance: The difference in fits between data sets is called Variance.

[@BrandonSLockey]

M-m-Morty huh? Learning some m-m-machine learning? Your grandpa rick would be p-p-proud of you **burp**, Morty.

[@sakata_gintoki007]

Thanks Morty for ur short note, which helps me to understand the definition more clearly. Good luck for ur adventure with ur crazy Grandpa

[@alephnull423]

Thank you Morty

[@ankur2893]

@@BrandonSLockey That doesn't sound like something Rick would say! He'd probably berate Morty for trying to learn this and then go on a soliloquy of how nothing actually will ever matter :D

[@Twinblade34]

These two definitions are completely counter-intuitive for me, have to re-define them for myself constantly. Because, bias sounds like the model is biased to the training data, but the bias in the definition is towards the model's assumptions (i.e linear model biased towards linearity). Variance sounds like the model's variation from the training set data (creating high variance), but the definition refers to the large variance of the error values (i.e residuals) when the model is fit to new data. Hope this helps if your intuition is similar to mine.

[@cherryandjaji5694]

The world of learning is still enjoyble cuz of people like you are still present

[@statquest]

Thank you! :)

[@sunwukong6268]

I am currently in a trainee program to learn machine learning...my teachers suggested this channel. This is awesome

[@statquest]

Welcome aboard!

[@maliknauman3566]

Sir I must say you are the Gem. This 6:35 Mins video has taught me what our Phd Dr. 3 Hrs with 50 slides cant.... Hat's off

[@statquest]

bam!

[@katiedunn7369]

I have paid for courses on edX and also have many free resources available to me through school- nothing has explained Bias and Variance as quickly and efficiently as you have in this video. Thank you, thank you, THANK YOU!

[@statquest]

Hooray! I'm glad my video was helpful.

[@jennydavies6973]

I have watched many of Josh's videos several times. Whenever I find myself trying to remember a concept, I know that a StatQuest video will sort me out in 10 minutes or less

[@statquest]

BAM!

[@BenStoneking]

My masters course in ML has been challenging. Getting washed over with lots of maths with greek (I've only taken calc I) and statistical jargon (never taken stats) when I am a simple computer science pleb has made class really hard. These videos are making light work of looking past the confusing figures and long-winded over-technical lectures! Thank you, Josh. Thanks, StatQuest!

[@statquest]

Hooray! I'm glad my videos are helpful! :)

[@mitchellsteindler]

How on earth did you get into a masters of ML without more background in relevant subjects?

[@BenStoneking]

@@mitchellsteindler I'm looking back at my previous reply and see that it sounds like I'm doing a masters program in ML. What I was trying to say is that I was taking an ML course in my masters program. My program is just computer science :) But I passed my class with an A with big thanks to these awesome videos!

[@mitchellsteindler]

@@BenStoneking ah okay

[@ghanilawal6798]

You are probably the best resource when it comes to understanding the fundamentals of Machine Learning... like it's not even close

[@statquest]

Thank you! :)

[@Sina_Z]

you just did it in a perfect way. I've read blogs, "best ML books", and other resources, but you just nailed this. thank you!

[@statquest]

Thank you!

[@VijaySharma-tl1ib]

After watching more than hundred of videos on machine learning, i find your way of explanation very easy to understand and digest. Plus, i am really amazed with the way you start your lectures and wait for 'BAM' to come.

[@statquest]

Wow, thanks!

[@genie52]

Wow this was so straight to the point with great visuals that I managed to figure out all in one go! Great stuff!

[@statquest]

Awesome!!! :)

[@aroonsubway2079]

Best explanation I have ever seen. I could remember "underfitting->bias" and "overfitting->variance", but I was always confused why bias/variance were used to describe these issues. Now I realize "bias" and "variance" are properties of models wrt future data: When a model overfit on the training data, its performance on a random test data is unpredictable, so it has a high variance. On the other hand, when a model underfit on the training data. its performance on a random test data would not change a lot (probably equally bad, so we say bias is large), so it has a low variance.

[@statquest]

bam!

[@aroonsubway2079]

@@statquest Hi, Thanks for posting this wonderful video. Can I ask a small question here about tradeoff btw bias and variance? Is it theorically impossible to have model with both lower bias and lower variance? Or if we spend effort on designing a model, it is possible to achieve both low bias and low variance?

[@statquest]

@@aroonsubway2079 Sure. For example, if your data is genuinely sinusoidal (like seasonal temperatures), then a sine function will give you less bias and variance than a straight line.

[@VictorAntonioLive]

LOL What a way to present dry material with a dry approach yet making it interesting and easy to follow :-) Great job!

[@youtubecommenter5122]

His dryness went full circle XD

[@MorganFABRE]

BAM ! Mindblowing how clearly explained these videos are, with even a sense of humour and some home made music. Really nice work, hats off.

[@statquest]

Wow, thanks!

[@amirmohammadjalili2676]

It couldn't have been made easier to understand these concepts.Great job, I hope your journey to making abstruse concepts easy to understands doesnt end here

[@statquest]

I hope not! :)

[@pranaysingh3950]

You are a life saver for so many poor kids. Even Humanity doesn't deserve you. Keep going. Thanks

[@statquest]

Thank you! :)

[@amribrahim7850]

You're gifted to turn unclear concepts to pretty clear ones. Baaam!

[@statquest]

Thank you! :)

[@shidulislam8168]

I have been working ML for the last 4 months but fortunately, I found this tutorial which helps me a lot to go deeper.

[@statquest]

Bam! :)

[@joebater7830]

Best, most intuitively understoood, explanation of this that I've ever seen!

[@statquest]

BAM! :)

[@starreachsocietybw]

Thank God I found this channel! I understand 2 hour lectures under 10 minutes - Thanks StatQuest!!

[@statquest]

Happy to help! :)

[@iho2437]

Josh , I don't know which i love more, your songs or your lessons on stats. You're amazing.

[@statquest]

bam! :)

[@davidngounou4472]

Your teaching is so clear and elaborate unlike others.

[@statquest]

Thank you!

[@nurwani556]

very clear, no extra unnecessary "noise". I really enjoyed this lesson.

[@statquest]

Awesome, thank you!

[@yusrashaikh1259]

MAN!!! i was reading about bias and variance trade off, but not a word got into my head...this video made it beyond clear!! thanks a ton!!

[@statquest]

Hooray! I'm glad the video was helpful. :)

[@akashdesarda5787]

This guy is awesome... this video actually explain bias and variance To Me finally. I have watch lots of other video but it was this video who taught me this concept

[@statquest]

Awesome!!! Thank you so much! :)

[@alokkumarsingh4641]

this channel is better than netflix for me , every recommendation is on point and cant help myself but watch it

[@statquest]

bam! :)

[@chrisg0901]

You're like the postal mailman of online videos. Neither snow nor rain nor heat nor gloom of night can stop StatQuest!

[@codeb3nder862]

i read it as post malone

[@shobhitsadwal756]

you have watched enough Seinfeld , haven't you?

[@aashishkarn]

Man I have been trying to learn this for 1 month finally I found this video no video on Internet beats this.

[@statquest]

bam! :)

[@aashishkarn]

@@statquest Double Bam ! AI in Nepal has your spark of knowledge.

[@statquest]

@@aashishkarn That is awesome! Go for it! :)

[@gardnmi]

Currently reading the Intro to Statistical Learning with Application in R and I can't tell you the number of times I've loaded up one of your videos to help me understand one of the concepts such as Bias and Variance because they do a poor job in explaining for a broader audience. Please keep it up!

[@statquest]

Hooray! One of my long term goals is to "translate" most of that book into StatQuest videos. This was the first, but I also just put out a vide on Ridge Regression and will soon put out a vide on Lasso Regression.

[@michaellamb8924]

Literally doing exactly the same thing

[@kenocontreras]

I was searching Bias and Variance for the same reason. Thankfully I found this channel!

[@erdenebilegb.379]

Came here for the exact same reason lol

[@hohohoupufuru]

You are a scholar and a gentleman. Thank you for explaining what my lecturer tried to explain in 2 hours in 6 minutes.

[@statquest]

You're very welcome!

[@dropfiremusic4752]

I have understood not only the Bias and Variance, but also even more ML terminology that has been quite difficult for me to understand until this point! Keep it up brother! Very good job :)

[@statquest]

Awesome!!! :)

[@chiragpalan9780]

Nothing can be better than this in 6.35 mins... It drives me crazy... stopped watching courses on ML of the bigger names.... will continue with #statequest. Its double BAM!!!. Love you Josh.

[@statquest]

Thank you very much! :)

[@BhanutejaAryasomayajula]

So much of quality content on Machine Learning!! I wish I knew about this channel a bit before. A must follow channel for ML & DS enthusiasts. Great job Josh :) Please continue the good work and serve the humanity!!

[@statquest]

Thank you very much! :)

[@sumandhamala4807]

This channel is so underrated, You are life saver sir. I tip you my hat.

[@statquest]

Thanks!

[@shaikansari6882]

Wow.. Go through many blogs.. Watched many videos and asked n no.of questions in quora and other platforms, but your single video (less than 7 minute video) explained well.. Really Thanks man.. Done a great job..

[@statquest]

Thank you!!! :)

[@hasibahmad3660]

I don't know why I subscribed to your channel a long ago and after a long time I have been searching for ML course and have found you. After watching the intro to ML, I have felt like wow I subscribed to a worthy channel

[@statquest]

Hooray!!! :)

[@srikarAilla]

3:09 psst. I can listen to this all day.

[@statquest]

:)

[@Fields_Forks]

Very concise and easily understandable video. In the past I have read this topic in books and seen other videos but never understood bias variance so clearly earlier.

[@statquest]

Thanks! I'm glad my video is helpful. :)

[@jenn6997]

Paid thousands of dollars on Udacity, but ALWAYS have to come to your channel for a clear explanation. Love the way you explained all these complicated concepts Josh :) (Btw, we met at IVADO's 100 Days Event haha:) )

[@statquest]

Hooray! I'm so glad my videos are helpful and IVADO's 100 Days Event was super cool. :)

[@jenn6997]

Your videos are AMAZING!! Thank you Josh for being such an inspiration :) Have a wonderful weekend! :)

[@Sbmbrk]

hands down the best explanation that i have ever seen. plus the humour is soo good

[@statquest]

Thanks!

[@joelvaz8115]

Thanks man, i do not know what the start was about, but your video really helped me. Thanks

[@katwoods8514]

I love that you explained why you square the differences! Most people don't bother explaining that and it always seemed strange to me.

[@statquest]

Thanks!

[@BeSharpInCSharp]

youtube should give option to add thousand likes. Your channel beats paid ML courses out there hands down.

[@statquest]

Thank you very much! :)

[@veeru06]

Good thing is there is no youtube ads coming for these lectures ❤️👌

[@mauropappaterra]

*Opens StatQuest Videos* -> Automatically clicks 'Like'

[@M0N0T1C0L0URS]

Thank goodness you exist... I've never ever understood why squaring the distances mattered until your foot note at 3:12

[@statquest]

Nice! :)

[@tymothylim6550]

Thank you, Josh, for this wonderful video on Bias and Variance in ML. It was a great visual-heavy explanation and the explanations were made very clear for these two concepts!

[@statquest]

Thank you very much! :)

[@enixling]

the best video so far on bias-variance tradeoff.

[@statquest]

Thanks!

[@pratiknabriya5506]

I loved your composition Miss Carolina. You have amazing voice Sir!

[@statquest]

Thank you very, very much!!!! :)

[@proggenius2024]

I will comment on every single video of yours. Just to show how much I love your teaching style.

[@statquest]

bam!

[@hari20001]

Brilliant and clear and concise explanation: the best i have seen!!! Congrats and many thanks.

[@statquest]

Thank you! :)

[@mehediazad1780]

this was so straight to the point, with some great visuals that I managed to figure out all in one go! BAM!!!!!

[@statquest]

Hooray! :)

[@hummus_boss]

Great job man! Seriously, you made my journey in data science easier 👍

[@statquest]

BAM! :)

[@felipecifuentes4981]

I'm amazed how many bams I've reached just in a couple of hours. Your videos have been enlightening, thank YOU very much!

[@statquest]

BAM! :)

[@tymothylim6550]

Thank you very much for this video! I am learning a lot from it and it helps me understand what people mean by Bias-Variance tradeoff!

[@statquest]

Thank you very much! :)

[@midhunanvn4130]

Although I know the subject already ... I watch these videos only for fun .... great humor! and great music!!! BAM!!!

[@statquest]

Hooray! :)

[@Max-sc8qj]

Thank you for your work Josh, I learn more from your six and a half minute videos than I do from six and a half of hours of textbooks and classwork

[@statquest]

Glad to help!

[@robertmurphy8084]

You should sell these videos as DVD sets. I bet a lot of educators would buy them.

[@dinara8571]

I got to the point where I first check statquest if I come across unfamiliar topics. Thank you so much for all of your hard work!!!

[@statquest]

Bam! :)

[@dhruvdatta1055]

I can't believe how mentally deficient my college professors had to be that they were unable to explain this simple concept

[@statquest]

Glad I could help!

[@engcuz]

The way you explain it , priceless . Thank you so much.

[@statquest]

Thank you!

[@yangwang6805]

Thank you so much for this video at this special moment! I hope you can keep safe during Florence hurricane! Good luck to you and the Carolinas!

[@statquest]

Thank you! We got a lot of flooding, but I stayed dry and now the sun is shining again. :)

[@LC-rw3lt]

WHAT IS STATQUEST? MY LIFE SAVER

[@statquest]

:)

[@johnhutton7313]

You’re on my list of guys I’ll buy a beer for if I ever see in a bar. You, Jeremy Howard, and the folks over at Deep Lizard.

[@statquest]

Wow! Thank you very much! :)

[@ramakrishnanchandran2801]

It was one such explanation where the concepts are crystal clear and crisp.. Thanks a lot

[@statquest]

Thank you! :)

[@jorgejgleandro]

Man, you're very didactic! For each statement, there is a 'because', so that your students never ends with a question mark in the head. Besides that, you don't mind to repeat the because's again and again in different ways, and that's what make things clearer. Why can't teachers, coaches, tutors realize that? Triple BAMMM!

[@statquest]

Thank you very much!! :)

[@osaabd390]

Triple Bam!!!! Thank you for the nice explanation. Seriously my genius professor, who is truly a genius, could not explain these two concepts this well. Genius does not mean good teacher

[@statquest]

Thanks!

[@sharanyar7092]

Tons of Thanks for You..your videos are really nice..pls do the video on regularization soon..

[@statquest]

I should have the first video on Regularization out in the next week or two. :)

[@sharanyar7092]

👍

[@infinisidoracle4487]

what an amazing start ...this is how every educational channel should start including some surprise element in their lectures. statquest (rhyming )

[@statquest]

bam!

[@soundbeans]

Just found this channel today. Also making my way through ISLR. They have a great video series to go along with the book, but still pretty technical. This channel is a god send. Thank you!

[@statquest]

You're welcome!

[@sajjadabouei6721]

man I love how you explianed it so easy to understand like butter 🔥🔥🔥🔥

[@statquest]

Thanks!

[@preranadas4037]

Hi Josh! You are the "God of ML and Stats". You really made me fall in love with these subjects. I had a query. According to you, if we cut the data into training and testing sets, what % should be assigned to test? I think it should vary with the amount of data, but is there a thumb rule?

[@statquest]

There are a handful of "rules of thumb". One simple one is if you do 10 fold cross validation, then you divide your data into 10 equally sized bins (see the StatQuest on cross validation: kzbin.info/www/bejne/nITcpa19rNx1jNk ). Another standard is to use 75% for training and 25% for testing. This is the default setting for Python's scikit-learn function train_test_split().

[@AdityaRajPVSS]

Are you a magician .....lots of respect to your knowledge and ability to explain

[@statquest]

Thank you! :)

[@justinberken]

Great video, very clear. Also, the graphics are intuitive. Thank you!

[@statquest]

Thanks! :)

[@unlink1649]

Your the biggest statistics nerd i have come across in a while. I love it

[@statquest]

Thanks! :)

[@atisafarkhah5923]

PERFECT AND CLEAR!

[@statquest]

Awesome!

[@philips2247540]

All your intro music give me a feeling tat the concepts are easy to understand....thanks you for building tat confidence.

[@statquest]

Hooray! :)

[@vaibhav_uk]

Who on the EARTH disliked this video? Probably other content creators...

[@statquest]

It's always a mystery why someone doesn't like StatQuest. Maybe they couldn't handle the BAM! :)

[@nursahidassafaat6283]

@@statquest could't agree more xD

[@martini3898]

thank you so much! i'm in a data science masters program and you explained this better than my professor and textbook.

[@statquest]

Wow, thanks!

[@kittipobkomjaturut8797]

StatQuest terminology : Bam with a high tone means this is the point you should understand. Little bam means something more important are coming. Double bams means at this point, you should be enlightened.

[@statquest]

That's perfect!!! You made me laugh out loud. :)

[@statquest]

@@samarthgoel1671 I think Tiny Bam means "boring but important."

[@marvelfan5444]

@@statquest TRIPLE BAM

[@jonathanuis]

I do love the way you explain and the way you keep people alert to upcoming information

[@statquest]

Thank you! :)

[@bhabeshroy3038]

Woah your original songs are beautiful too'

[@mauliknaik9520]

Have watched many of your videos and that have forced me to write a comment, Stat Quest is AWESOME!! and @Josh Starmer, I am you fan. The way you begin your videos and go about explaining some of the most difficult concepts in Statistics and Machine Learning is GREAT. Many books and tutorials mention making the complex simple, but rarely do so. This channel is not one of them, it truly makes things simple to understand. I have just one request (i think most of your followers would agree to this point), please write a book on Machine Learning and it's application of various algorithms (may be a series of books).

[@statquest]

Thanks so much! If I ever have time, I'll write a book, but right now I only have time to do the videos.

[@MontahaAlriQa]

awsome and very clear explanation!

[@bhagyashreedalvi7117]

thankyou so much for making machine learning so much fun! I never thought I could enjoy doing ML so much

[@statquest]

Hooray! :)

[@yulinliu850]

Such a GREAT video on bias-variance trade-off. Looking forward to your lectures on regularization and boosting~

[@sheikhrehman4867]

One of the best videos I have come so far

[@statquest]

Thank you!

[@Abdu572]

linear regression (aka least square) finally, now I can die in peace. you explain things in very nice way.

[@statquest]

Thanks!

[@thryce82]

i spent 2 hours reading some bs explanation in ISLR about this and got a much better explanation in < 7 minutes thru here. thanks for your work boss

[@statquest]

Glad it helped!

[@Yesuuh]

perfect video doesn't exist... wait nvm, found it!

[@statquest]

bam! :)

[@josephiles7515]

What an outstandingly simple and intuitive explanation, bravo!

[@statquest]

Thank you! :)

[@lucienzimmermann]

I could simply replace my tuition payments with payments for a KZbin Premium subscription. Much cheaper and easier to study :D

[@statquest]

bam! :)

[@zenicv]

I came here after taking a grad level course, but this simple explanation often stays longer in the mind :)

[@statquest]

Happy to help!

[@mauliknaik9520]

Also, can you tube customize the like button to BAM!! that would be Great.. ;)

[@statquest]

That would be awesome! :)

[@yasirmalik869]

Please live a long and healthy life. You're a gift to many

[@statquest]

Thanks!