The Main Ideas of Fitting a Line to Data (The Main Ideas of Least Squares and Linear Regression.)

Fitting a line to data is actually pretty straightforward.
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you'd like to support StatQuest, please consider...
Buying The StatQuest Illustrated Guide to Machine Learning!!!
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
KZbin Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
#statquest #regression

[@statquest]

Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

[@weixiangzhao561]

Everyone should buy this book if you want to learn machine learning. It is the greatest 20 bucks that I have ever spent in my entire life.

[@statquest]

@@weixiangzhao561 BAM! Thank you very much! :)

[@dipenpandit684]

This video isn't available in Nepal. Can you please make it available sir? I really love your content. 🙏

[@statquest]

@@dipenpandit684 I'm working on it. It's some strange thing with youtube and I've contacted them.

[@alicefriedman7607]

I love this book! I have taken Stats so many times but I still learned so much. Thank you for writing this book!

[@rmenchoachupicachu]

I have been enrolled in a graduate machine learning course for about a month now and you have just demystified so many details around Linear Regression. Please do more ML videos! They are so clear and helpful. If you can, please do one on Regularization and Decision Forests.

[@kenway346]

I actually took machine learning as the elective subject for my final year of engineering and I pretty much guess that this channel's going to teach me every thing!

[@statquest]

Hooray!!! :)

[@Squidward-v5u]

@@statquest your vids are so good I watch your videos for entertainment lol

[@PunmasterSTP]

How'd the machine learning class turn out?

[@majeedhussain3276]

You deserve more subscribers, the quality in your videos is so intuitive that even a high school student understands. Please do upload videos, keep going. I really think ur gonna get more subscribers in future.

[@adambis337]

You were right

[@wirotep.1210]

Can’t agree more

[@PunmasterSTP]

Definitely! I'm glad that as of March 2024, he's got over one million subs (1.13 to be exact)!

[@rishabh2892]

After weeks of research and frustration, I have finally understood the concept of least squares so well! You explained the concepts so simply and logically!! Thank you so much for this amazing video. Much appreciated.

[@statquest]

Thank you very much!!! :)

[@toandef3109]

You have no idea how much I appreciate the clarity and simplicity of this explanation, you deserve a medal

[@statquest]

Thank you!

[@homeboy6668]

I must admit sir, you are one of the best teachers I've ever had. Thank you for being so awesome!

[@statquest]

Wow, thank you!

[@maggiechen1141]

Thank you!! After somehow passing 2 PhD quantitative methods modules and not really understanding why we did any of it, your channel has finally cleared a lot of stuff up!

[@statquest]

Great to hear!

[@frankchen4229]

How the hell did you pass

[@maggiechen1141]

@@frankchen4229 do what you're told and don't ask why

[@frankchen4229]

@@maggiechen1141 huh. Maybe PhD isn't so bad.

[@kapricun]

@@maggiechen1141 best advice so far

[@blessinglotus]

I keep coming back for more! THANK YOU SO MUCH!!! This is so clearly explained than any other tutorial/video/in-class session I've ever listened to! You are the best!

[@statquest]

Wow! Thank you very much!

[@odins_claw]

Your presentation style is way ahead of anything else on this platform. You don't have the inefficient habit of deriving everything from first principles but allow a holistic intuition to develop. Absolute magic!

[@statquest]

Thank you! :)

[@Neoclassicalmaese]

I don't think I have ever clicked on the subscribe button that fast. Absolutely amazing

[@statquest]

Awesome! :)

[@aldikroos6290]

Me too

[@benparasa]

There are so many videos on this subject where they say what and how to use least squares method of find the line of best fit but you are the only one who explained the concept behind this method. Thank you.

[@statquest]

Thanks!

[@Jef-ur7zv]

the quality of these videos seem to have improved greatly over the years, but the simplicity was always there. Amazing!

[@statquest]

Some of the early videos are still the best.

[@daiannehofig3311]

Your way to explain these abstract concepts is simply AMAZING!!!! Thank you so much for these incredible videos!

[@statquest]

Thank you very much! :)

[@_aft]

It literally only took me 15 seconds to subscribe because of his unique 15 second intro

[@statquest]

That's awesome! :)

[@balajichandrasekaran6312]

@@statquest i think you should add fitting a curve to data in this playlist

[@munny0607]

This is absolutely fantastic! I am so glad that I found this channel on the KZbin while I am doing the data science self study. I am now understand the concept of OLS which stress me out in a week time before I found this video. Big thanks!

[@statquest]

I'm glad it was helpful! :)

[@guillermoalvarezbacame5909]

This is the best KZbin channel ever, thank you Josh for all your work you doing awesome!!

[@statquest]

Wow, thanks!

[@MrCentrax]

I always wanted to know why the formula squares the distance and then get the root instead of using the absolut value. You're the first one to explain this. Thank you!

[@statquest]

Thanks!

[@fertingoff]

So can you explain in your own words in short then? Thanks

[@uchuhikari9292]

I have watched and read so many articles but this video explains the use of sum of squared errors and why its important. Thank you!!!

[@statquest]

Hooray! :)

[@pravinsaraswatula5263]

StatQuest Team - Thank you so much for all your efforts. For the last few months, I felt like that mouse stuck in a wheel going round and round with concepts as I got deeper into ML. A definite recommendation to everyone and anyone irrespective of their ML proficiency.

[@statquest]

Awesome! Good luck with your ML studies! :)

[@avishekdey1943]

I never understood why we needed to plot a line. Now I do. It is amazing

[@datasciencewithjazz5854]

Josh your songs and teaching are excellent, you are doing something no one else has done in my life: inspiring me to become a Data scientist as well as a composer

[@statquest]

Wow! That is awesome! BAM! :)

[@avadhsavsani1148]

What a legend you are! No words to express my gratitude. You are a blessing to everyone wanting to learn these concepts! Wish you good health and loads of happiness. :)

[@statquest]

Thank you!

[@danielkhromov9724]

Oh my God this is just the best video I've ever seen about Linear Regression! Thank you very much! I subscribed just after the video, please do not stop!

[@statquest]

Awesome! Thank you!

[@reginango9009]

This video made me stop crying from stress of barely understanding anything in my class. Thank you

[@statquest]

Glad it helped!

[@bigvinweasel1050]

Right?! This channel is so underrated, we have to change that!

[@statquest]

@@bigvinweasel1050 Wow! Thank you!

[@bigvinweasel1050]

@@statquest I'm serious! If you ever need anything done on Python for content - I would be more than happy to write it out as clearly and as elegantly as I can so you can use it for content.

[@ramkotha4726]

Never excited more than after watching this video. Truly intuitive and amazing intro to linear regression.

[@statquest]

Thank you! :)

[@imranullah7355]

What a teaching style... 200% which I was searching about... Exellent work

[@statquest]

Thank you very much! :)

[@doge6154]

english is not my first language, but i can clearly understand your explanation. Thank you sir!

[@statquest]

Thank you!

[@moneyman2200]

I've read a few undergrad texts and none of them actually explain the origins of the idea behind least squares. At least not in a simplified visual form, you're usually just explained the problem and slapped with the simple linear model and then the generalized version....This gives some insight into the motivation behind this technique. Thank you for donating you're time to such a altruistic cause. You a real one!

[@statquest]

Thank you! :)

[@vincent3542]

bro im appreciate every single videos that you've made, i just want to say thanks a ton with not skip ads in your videos, love from indonesian

[@statquest]

I appreciate that!

[@asghaznavi]

Hello Josh, Thank you for sharing. Every time I sign in, I learn something new from you. Fantastic presentation . Stay blessed.

[@statquest]

Thank you! You too!

[@MONAD_0]

Hey josh, I absolutely love your explanations! It's given me a completely different perspective on how i think about machine learning and made the topics intuitive. I wonder if you can compile some notes for all of the content that can be reviewed in under a day and a mind-map that can be used to put all the pieces together. that would be really AWESOME!

[@statquest]

Something like this? app.learney.me/maps/StatQuest

[@mostinho7]

Done thanks We take the square to make all the errors positive (we want to find the total error of the points from the line) We want to find the optimal values for a and b in the equation of a line that minimize the sum of errors squared. We can express the sum of squares as a function of a and b and take the derivative to optimize it 5:45 We find the slope that minimizes the error by finding the minima of the multi variable function (variables are a,b) 7:30

[@ludelaire]

Josh, seriously, thanks for all these videos. My ML journey is smoother thanks to them.

[@statquest]

Glad you like them!

[@akeithkira]

OMG !! You really have the best in explaining this concept which most books just want to show how good their English are.

[@statquest]

Thanks! :)

[@halilibrahimcetin9448]

Awesome. I am screaming with happiness. Thanks Statquest. Intuition you conveyed to us priceless.

[@statquest]

Thanks!

[@nikhilsetty3451]

Great video.I love the way how intuitive your videos are! 👍🏻

[@statquest]

Thank you! :)

[@kawaiihellen2285]

Your videos are helping me write my thesis because I don't have a stats background and went into a science heavy masters. this is just the best!!!!

[@statquest]

Good luck! :)

[@aldikroos6290]

The best teaching video I have ever seen. What a great work!

[@statquest]

Thank you! :)

[@franciscoicarocs]

I just wanted to say that you were born to teach. The book? Are you kidding me? Perfection. I would advise to give the first chapter as free content so everybody can have a taste of your abilities. Favorite quote so far: "The Binomial Distribution makes me want to run away and hide. :) "

[@statquest]

Thank you very much! That's a good idea. I wonder how I can do that.

[@franciscoicarocs]

Favorite new quote just dropped: "the Normal distribution is awesome, and, to be honest, it sort of looks like you..." lol

[@statquest]

@@franciscoicarocs Haha! I'm glad you're enjoying it. I just started (in earnest) to write my next book on neural networks (from simple to start of the art)

[@divinity1170]

This is the real teaching. Respect

[@nayaksrigovind]

This is an amazing video on the intuition behind Fitting lines to data. Loved this video, it gave me a recap on some of the concepts I've learnt years ago and have forgotten.

[@statquest]

Hooray! :)

[@aakshay8440]

hello, you have done the wonderfull explanation. I loved it. and I am requesting you to do a video on assumptions made in linear regression. this will help us a lot.

[@gothams1195]

I'm new to data science, you just nailed it ....amazing explanation...after so many videos...finally understood what the heck Linear regression is ...Thank you so much...

[@statquest]

Hooray! :)

[@ian_owuor]

Yes this is exactly what I've been looking for. Great video. It has made my life a lot easier.

[@statquest]

Thanks!

[@abhi9029]

superb explanation, crystal clear with graphics is good way to comprehend. Thank you for this.

[@statquest]

Hooray!

[@kensarmoreto1863]

Thank you for the vid. It was so easy to understand the concept of Least Square using visualization. I will use this as my reference for my demo teaching in stat. Hoping fore more stat videos and data analysis trick & tips...

[@statquest]

Glad it was helpful!

[@gingerl6081]

I think that the Blue Lake in New Zealand has some competition for ~most clarity~. This channel is amazing!!

[@statquest]

BAM! :)

[@may4081]

Simply explained...just makes it beautiful to watch! Thanks!! :-)

[@statquest]

Thank you! :)

[@ian.ambrose]

You have the best teaching skill in the universe.

[@statquest]

Wow! Thanks!

[@iktiar]

your method of teaching is awesome, thank you!

[@statquest]

Thank you! :)

[@angelmotta]

Thanks for this content!!! I am very happy to understand these concepts watching this awesome explanation!

[@statquest]

Glad it was helpful!

[@zukofire6424]

Thanks very much for this. I watched it last year when I was looking to change careers. Re watching now that I'm enrolled in some real training. And Wow!

[@statquest]

BAM! Good luck with your course.

[@maheshgopalakrishnan5072]

Sir, your explanations are crystal clear. Thank you

[@statquest]

Thank you! :)

[@lucasbatista8169]

Great video! It is admirable the effort you put on teaching... Just a suggestion: It would be great if you could put the link of the complementary videos you describe in each video. That way, it is easier to keep on track.

[@statquest]

Thanks for the tip. I generally try to do that (add the links in the description), but sometimes I forget. If you have time, it would be great if you could post which videos need links to complementary videos in a comment and then I'll take care of the rest.

[@abhinav9561]

Hi Josh. Love your videos!! They give the best intuitive explanation. Can't thank you enough :). Please make a video on curve fitting for linear equations using normal equations vs using gradient descent. Thank You!

[@statquest]

My video on Gradient Descent compares that method to the analytical solution: kzbin.info/www/bejne/qXXZZZlqqJeGeJo

[@aakashgarg66]

This is the best tutorial channel ever!!

[@statquest]

Thank you very much! :)

[@mukossa4650]

Thank you, this is great. Easy to understand

[@statquest]

Thank you! :)

[@dariakrupnova6245]

Please do more of these, I think I will be able to pass my econometrics test thanks to you.

[@statquest]

Good luck!

[@lampkanocna6]

I just "discovered" by myself that lines generated by linear regression always passes (M(X), M(Y)) point (which, when thinked about it is quite intuitive) and thus, we can add constriction, that b=M(Y)-a*(M(X), which allows to solve the equation for just single varible (slope), instead of two (slope and intercept). Thats for sure basic, but im neverthless proud of myself :D Great channel BTW.

[@statquest]

bam!

[@Luxcium]

BAM 😅😅😅🎉 I reached out to the bottom of the _STACK_ Quest (finally) 🎉❤ Wow 😮 I am on this quest to find out where this will stop since I just learned that I must watch the video *Fitting a Line to Data* also know as *Linear Regression* before I can watch the *Gradient Descent Step-by-Step!!!* so that I can watch the video related to *Neural Networks part 2* that I must watch before I can watch the *The StatQuest Introduction To PyTorch...* before I can watch the *Introduction to coding neural networks with PyTorch and Lightning* 🌩️ (it’s something related to the cloud I understand) I am genuinely so happy to learn about that stuff with you Josh❤ I will go watch the other videos first and then I will back propagate to that first video... And now I feel like it took me 6 years to find out this wonderful video I can't wait to see it😊

[@statquest]

You finally made it!

[@kushbhomawat8722]

AMAZING VIDEO. MAKES STATS AND MATHS MORE LOGICAL AND REAL

[@statquest]

Thanks!

[@woodynani4394]

Concise and precise, well done Sir.

[@statquest]

Thank you! :)

[@motanoloto]

Thanks a lot making such a wonderful video, you have really simplified learning Linear regression...

[@statquest]

Hooray!!! I'm glad the video is helpful! :)

[@user-cb7it9cv9l]

Best and simplest explanation ever !!

[@statquest]

Glad you liked it!

[@supriyamanna715]

taken Andrew's course and bought books, but that intro and vibe is the rela into to ML--I learned that the hard way. Thanks for your songs.

[@statquest]

:)

[@supriyamanna715]

@@statquest hey man, I'd like to request you to kindly make a video on how to become an ML engineer from scratch! I am a self taught aspirant. please make the roadmap for people like us

[@statquest]

@@supriyamanna715 To be honest, you could just start at the top of this and work your way down, through the webinars: statquest.org/video-index/

[@gmayank32]

I like your videos. They are so easy to understand. Thank you so much. Can you please make a video on SGD with momentum, averaging and Adagrad?

[@johnrine9671]

Great Josh! Liked and subscribed, thank you very much!

[@Nakameguro97]

The slope explanation gives a good intuitive sense of how to find the best-fit slope and y-intercept for least squares, especially if you have a background in calculus. In contrast, the linear algebra solution of OLS is just that much more shocking/amazing; that the same result can be calculated algebraically for n-space >without< any geometric intuition, without any search in the solution space for slope and y-intercept. Your visual explanation is more intuitive and memorable. The linear algebra approach feels more magical, as I find it harder to remember the derivation.

[@statquest]

Noted

[@michaelvogt7787]

Josh. Ive seen this explanation for near 40 years, always re-watching for someone to explain the approach as well as the instructor who introduced it to me in 1980... your is the best so far. But, you got away w some simplification by starting w a 0- slope line, and the calculatiin of the line's value was sort of 'lost' when you lept to non-zero slope lines... Jussayin. Cheers DocV

[@statquest]

Glad it was helpful!

[@muksmart1]

awesome explanation, best ever explanation, made it look so easy.....

[@statquest]

Thanks!

[@RoRight]

Just the right amount of theory and math. You should consider teaching stat for students in health and biological studies.

[@TheSheekeyScienceShow]

I actually LOVE your video style!

[@statquest]

Thank you! :)

[@johnwilson3918]

This is a brilliant video. You're a few minutes from explaining how to derive the LR formula. I seem to recall from my high school days - that you work out two partial derivatives (dSum/da - treating 'b' as a constant and then dSum/db treating 'a' as a constant) and equate these to 0 (to find minimum). You should be left with a couple of equations for intercept and slope.

[@statquest]

That's exactly right (and I show how at least part of this works in my video on the Chain Rule kzbin.info/www/bejne/rZ2Unqyup9mEfrM ). However, what I don't like about the LR formula is that it only works in this specific situation. In contrast, Gradient Descent gives us pretty good parameter estimates and works in a million other situations. For more details on Gradient Descent, see: kzbin.info/www/bejne/qXXZZZlqqJeGeJo

[@christinagiannaros9817]

When a clip about stats starts of with singing that makes you laugh before you start it's a very good thing.

[@statquest]

bam!

[@joehutchison669]

maths is so crazy, the explanation was amazing - the people who figure this stuff out are geniuses

[@statquest]

bam! :)

[@tseckwr3783]

Thanks for the video. It was a great refresher.

[@statquest]

Thanks!

[@urjaswitayadav3188]

Thanks for this. Really helpful.

[@jasafraga]

I think another reason to take the square error is that it creates an actual geometric square of size of the error (area of the square, A= L*W). Add them all up together to get a 2D representation of the error. Where adding all the absolute error lines is only 1D. Sometimes shapes are easier to visualize. Amazing videos and pedagogy style. Props.

[@statquest]

Noted.

[@sivasrinivas3716]

Great video and many thanks! Lucid explanation!

[@statquest]

Thanks! :)

[@MrFredazo]

I've seen many of your videos, they are amazing good stuff! I just wanted to point out something in this one: you calculate "b" for the first horizontal line, and then you start rotating it to find the best slope. But you never explain WHERE IS THE ROTATION POINT! This is crucial!

[@MrFredazo]

I mean, to give the intuition that you can assume any intercept and the rotate using it as the rotation point, and it won't change the result.

[@statquest]

Noted.

[@mustafacakir__]

Your method of explanation is great. Please keep uploading tutorials. I would like to see tutorials about Deep Learning and Boosting (XGBoost, Catboost, etc.) algorithms which are popular lately. Thanks.

[@statquest]

I already have videos on deep learning here: kzbin.info/www/bejne/eaKyl5xqZrGZetk and XGBoost here: kzbin.info/www/bejne/haWnaaqMlqugbKc All of my videos are organized here: app.learney.me/maps/StatQuest

[@techproductowner]

Bro you are genius . it just sinks in the mind . .they way you are explaining , Kindly guide me with some youtube channel which explains other concepts related to math's like calculus etc. .in similar way . .or I request you to create them as well from prospective of DS

[@statquest]

Thanks! Lots of other people like 3Blue1Brown for math.

[@ayselceferzade8587]

clearly explained! lots of thanks

[@statquest]

Glad it was helpful!

[@naughtynoodle3271]

You have the best videos on the internet on Stat & data science.

[@statquest]

Thank you very much! :)

[@amribrahim7850]

You are an incredible teacher.

[@statquest]

Thank you! :)

[@jeanlanz2344]

Great explanation. Thank you. God bless you!

[@statquest]

Thank you!

[@MichaelKlimenko]

Really good explanation, thank you !

[@statquest]

Thanks!

[@tagoreji2143]

Awesome Information.Thank you Sir

[@statquest]

Thanks!

[@palakagrawal8856]

The best explanation to date!

[@statquest]

Thanks!

[@xiangnan-oz9hs]

Thanks very much. Like your sound, like your straightaway explanation, like your songs, and specially like your learning style.

[@alhello_game_of_everything]

Wowie kabawie wowie ZOWIE! Im still in highschool! And I understand most fundementals in the video!😊

[@statquest]

double bam!

[@liza10o]

Wow. I never looked at it this way!

[@ankuragarwala7558]

Great video on least square method.

[@statquest]

Thanks!

[@mostafa4321]

Thank you, Can you put a video with actual Data and interpret that data based on the model with possible decisions to make based on the outcome. I believe this will bring us to a closer understanding of linear regression relative to decisions making especially for those who have limited knowledge of statistics.

[@statquest]

I have a playlist of videos where I show how to do things in R. These are the "applied" videos where I go through a data set and show how it's handled and filtered and how the results are interpreted. Here is a link to the playlist: kzbin.info/aero/PLblh5JKOoLUJJpBNfk8_YadPwDTO2SCbx

[@apopheniac4231]

Excellent video. Thank you!

[@statquest]

Glad it was helpful!

[@parisaayazi8886]

Thank you so much for this incredible video!

[@statquest]

Glad you enjoyed it!

[@MiguelAngelMartinezFernandez]

Hi! First of all, congratulation for this awesome youtube channel. I was wondering if it would it be possible to re-upload this video in a higher quality? If I am not wrong, the max quality is 480p, which is not the best at these days. Thanks! Miguel

[@statquest]

I'm glad you like the video! That's a good idea to upload a higher quality version.

[@shrutivashishth541]

This was really helpful, thanks! :)

[@statquest]

Thanks!

[@dylanusyd]

Great work, thanks heaps!

[@statquest]

You're welcome! :)