The Main Ideas of Fitting a Line to Data (The Main Ideas of Least Squares and Linear Regression.)

[@statquest]

Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

[@weixiangzhao561]

Everyone should buy this book if you want to learn machine learning. It is the greatest 20 bucks that I have ever spent in my entire life.

[@statquest]

@@weixiangzhao561 BAM! Thank you very much! :)

[@dipenpandit684]

This video isn't available in Nepal. Can you please make it available sir? I really love your content. 🙏

[@statquest]

@@dipenpandit684 I'm working on it. It's some strange thing with youtube and I've contacted them.

[@alicefriedman7607]

I love this book! I have taken Stats so many times but I still learned so much. Thank you for writing this book!

[@rmenchoachupicachu]

I have been enrolled in a graduate machine learning course for about a month now and you have just demystified so many details around Linear Regression. Please do more ML videos! They are so clear and helpful. If you can, please do one on Regularization and Decision Forests.

[@majeedhussain3276]

You deserve more subscribers, the quality in your videos is so intuitive that even a high school student understands. Please do upload videos, keep going. I really think ur gonna get more subscribers in future.

[@adambis337]

You were right

[@wirotep.1210]

Can’t agree more

[@PunmasterSTP]

Definitely! I'm glad that as of March 2024, he's got over one million subs (1.13 to be exact)!

[@rishabh2892]

After weeks of research and frustration, I have finally understood the concept of least squares so well! You explained the concepts so simply and logically!! Thank you so much for this amazing video. Much appreciated.

[@statquest]

Thank you very much!!! :)

[@kenway346]

I actually took machine learning as the elective subject for my final year of engineering and I pretty much guess that this channel's going to teach me every thing!

[@statquest]

Hooray!!! :)

[@User-l3u6m]

@@statquest your vids are so good I watch your videos for entertainment lol

[@PunmasterSTP]

How'd the machine learning class turn out?

[@homeboy6668]

I must admit sir, you are one of the best teachers I've ever had. Thank you for being so awesome!

[@statquest]

Wow, thank you!

[@maggiechen1141]

Thank you!! After somehow passing 2 PhD quantitative methods modules and not really understanding why we did any of it, your channel has finally cleared a lot of stuff up!

[@statquest]

Great to hear!

[@frankchen4229]

How the hell did you pass

[@maggiechen1141]

@@frankchen4229 do what you're told and don't ask why

[@frankchen4229]

@@maggiechen1141 huh. Maybe PhD isn't so bad.

[@kapricun]

@@maggiechen1141 best advice so far

[@toandef3109]

You have no idea how much I appreciate the clarity and simplicity of this explanation, you deserve a medal

[@statquest]

Thank you!

[@odins_claw]

Your presentation style is way ahead of anything else on this platform. You don't have the inefficient habit of deriving everything from first principles but allow a holistic intuition to develop. Absolute magic!

[@statquest]

Thank you! :)

[@MrCentrax]

I always wanted to know why the formula squares the distance and then get the root instead of using the absolut value. You're the first one to explain this. Thank you!

[@statquest]

Thanks!

[@fertingoff]

So can you explain in your own words in short then? Thanks

[@mostinho7]

Done thanks We take the square to make all the errors positive (we want to find the total error of the points from the line) We want to find the optimal values for a and b in the equation of a line that minimize the sum of errors squared. We can express the sum of squares as a function of a and b and take the derivative to optimize it 5:45 We find the slope that minimizes the error by finding the minima of the multi variable function (variables are a,b) 7:30

[@benparasa]

There are so many videos on this subject where they say what and how to use least squares method of find the line of best fit but you are the only one who explained the concept behind this method. Thank you.

[@statquest]

Thanks!

[@Jef-ur7zv]

the quality of these videos seem to have improved greatly over the years, but the simplicity was always there. Amazing!

[@statquest]

Some of the early videos are still the best.

[@_aft]

It literally only took me 15 seconds to subscribe because of his unique 15 second intro

[@statquest]

That's awesome! :)

[@balajichandrasekaran6312]

@@statquest i think you should add fitting a curve to data in this playlist

[@doge6154]

english is not my first language, but i can clearly understand your explanation. Thank you sir!

[@statquest]

Thank you!

[@Neoclassicalmaese]

I don't think I have ever clicked on the subscribe button that fast. Absolutely amazing

[@statquest]

Awesome! :)

[@aldikroos6290]

Me too

[@avishekdey1943]

I never understood why we needed to plot a line. Now I do. It is amazing

[@blessinglotus]

I keep coming back for more! THANK YOU SO MUCH!!! This is so clearly explained than any other tutorial/video/in-class session I've ever listened to! You are the best!

[@statquest]

Wow! Thank you very much!

[@datasciencewithjazz5854]

Josh your songs and teaching are excellent, you are doing something no one else has done in my life: inspiring me to become a Data scientist as well as a composer

[@statquest]

Wow! That is awesome! BAM! :)

[@reginango9009]

This video made me stop crying from stress of barely understanding anything in my class. Thank you

[@statquest]

Glad it helped!

[@bigvinweasel1050]

Right?! This channel is so underrated, we have to change that!

[@statquest]

@@bigvinweasel1050 Wow! Thank you!

[@bigvinweasel1050]

@@statquest I'm serious! If you ever need anything done on Python for content - I would be more than happy to write it out as clearly and as elegantly as I can so you can use it for content.

[@moneyman2200]

I've read a few undergrad texts and none of them actually explain the origins of the idea behind least squares. At least not in a simplified visual form, you're usually just explained the problem and slapped with the simple linear model and then the generalized version....This gives some insight into the motivation behind this technique. Thank you for donating you're time to such a altruistic cause. You a real one!

[@statquest]

Thank you! :)

[@uchuhikari9292]

I have watched and read so many articles but this video explains the use of sum of squared errors and why its important. Thank you!!!

[@statquest]

Hooray! :)

[@pravinsaraswatula5263]

StatQuest Team - Thank you so much for all your efforts. For the last few months, I felt like that mouse stuck in a wheel going round and round with concepts as I got deeper into ML. A definite recommendation to everyone and anyone irrespective of their ML proficiency.

[@statquest]

Awesome! Good luck with your ML studies! :)

[@ramkotha4726]

Never excited more than after watching this video. Truly intuitive and amazing intro to linear regression.

[@statquest]

Thank you! :)

[@divinity1170]

This is the real teaching. Respect

[@guillermoalvarezbacame5909]

This is the best KZbin channel ever, thank you Josh for all your work you doing awesome!!

[@statquest]

Wow, thanks!

[@daiannehofig3311]

Your way to explain these abstract concepts is simply AMAZING!!!! Thank you so much for these incredible videos!

[@statquest]

Thank you very much! :)

[@munny0607]

This is absolutely fantastic! I am so glad that I found this channel on the KZbin while I am doing the data science self study. I am now understand the concept of OLS which stress me out in a week time before I found this video. Big thanks!

[@statquest]

I'm glad it was helpful! :)

[@akeithkira]

OMG !! You really have the best in explaining this concept which most books just want to show how good their English are.

[@statquest]

Thanks! :)

[@vincent3542]

bro im appreciate every single videos that you've made, i just want to say thanks a ton with not skip ads in your videos, love from indonesian

[@statquest]

I appreciate that!

[@ludelaire]

Josh, seriously, thanks for all these videos. My ML journey is smoother thanks to them.

[@statquest]

Glad you like them!

[@kawaiihellen2285]

Your videos are helping me write my thesis because I don't have a stats background and went into a science heavy masters. this is just the best!!!!

[@statquest]

Good luck! :)

[@avadhsavsani1148]

What a legend you are! No words to express my gratitude. You are a blessing to everyone wanting to learn these concepts! Wish you good health and loads of happiness. :)

[@statquest]

Thank you!

[@lampkanocna6]

I just "discovered" by myself that lines generated by linear regression always passes (M(X), M(Y)) point (which, when thinked about it is quite intuitive) and thus, we can add constriction, that b=M(Y)-a*(M(X), which allows to solve the equation for just single varible (slope), instead of two (slope and intercept). Thats for sure basic, but im neverthless proud of myself :D Great channel BTW.

[@statquest]

bam!

[@gothams1195]

I'm new to data science, you just nailed it ....amazing explanation...after so many videos...finally understood what the heck Linear regression is ...Thank you so much...

[@statquest]

Hooray! :)

[@franciscoicarocs]

I just wanted to say that you were born to teach. The book? Are you kidding me? Perfection. I would advise to give the first chapter as free content so everybody can have a taste of your abilities. Favorite quote so far: "The Binomial Distribution makes me want to run away and hide. :) "

[@statquest]

Thank you very much! That's a good idea. I wonder how I can do that.

[@franciscoicarocs]

Favorite new quote just dropped: "the Normal distribution is awesome, and, to be honest, it sort of looks like you..." lol

[@statquest]

@@franciscoicarocs Haha! I'm glad you're enjoying it. I just started (in earnest) to write my next book on neural networks (from simple to start of the art)

[@gingerl6081]

I think that the Blue Lake in New Zealand has some competition for ~most clarity~. This channel is amazing!!

[@statquest]

BAM! :)

[@MrFredazo]

I've seen many of your videos, they are amazing good stuff! I just wanted to point out something in this one: you calculate "b" for the first horizontal line, and then you start rotating it to find the best slope. But you never explain WHERE IS THE ROTATION POINT! This is crucial!

[@MrFredazo]

I mean, to give the intuition that you can assume any intercept and the rotate using it as the rotation point, and it won't change the result.

[@statquest]

Noted.

[@halilibrahimcetin9448]

Awesome. I am screaming with happiness. Thanks Statquest. Intuition you conveyed to us priceless.

[@statquest]

Thanks!

[@imranullah7355]

What a teaching style... 200% which I was searching about... Exellent work

[@statquest]

Thank you very much! :)

[@asghaznavi]

Hello Josh, Thank you for sharing. Every time I sign in, I learn something new from you. Fantastic presentation . Stay blessed.

[@statquest]

Thank you! You too!

[@zukofire6424]

Thanks very much for this. I watched it last year when I was looking to change careers. Re watching now that I'm enrolled in some real training. And Wow!

[@statquest]

BAM! Good luck with your course.

[@nayaksrigovind]

This is an amazing video on the intuition behind Fitting lines to data. Loved this video, it gave me a recap on some of the concepts I've learnt years ago and have forgotten.

[@statquest]

Hooray! :)

[@supriyamanna715]

taken Andrew's course and bought books, but that intro and vibe is the rela into to ML--I learned that the hard way. Thanks for your songs.

[@statquest]

:)

[@supriyamanna715]

@@statquest hey man, I'd like to request you to kindly make a video on how to become an ML engineer from scratch! I am a self taught aspirant. please make the roadmap for people like us

[@statquest]

@@supriyamanna715 To be honest, you could just start at the top of this and work your way down, through the webinars: statquest.org/video-index/

[@kushbhomawat8722]

AMAZING VIDEO. MAKES STATS AND MATHS MORE LOGICAL AND REAL

[@statquest]

Thanks!

[@aakashgarg66]

This is the best tutorial channel ever!!

[@statquest]

Thank you very much! :)

[@joehutchison669]

maths is so crazy, the explanation was amazing - the people who figure this stuff out are geniuses

[@statquest]

bam! :)

[@Nakameguro97]

The slope explanation gives a good intuitive sense of how to find the best-fit slope and y-intercept for least squares, especially if you have a background in calculus. In contrast, the linear algebra solution of OLS is just that much more shocking/amazing; that the same result can be calculated algebraically for n-space >without< any geometric intuition, without any search in the solution space for slope and y-intercept. Your visual explanation is more intuitive and memorable. The linear algebra approach feels more magical, as I find it harder to remember the derivation.

[@statquest]

Noted

[@ian.ambrose]

You have the best teaching skill in the universe.

[@statquest]

Wow! Thanks!

[@michaelvogt7787]

Josh. Ive seen this explanation for near 40 years, always re-watching for someone to explain the approach as well as the instructor who introduced it to me in 1980... your is the best so far. But, you got away w some simplification by starting w a 0- slope line, and the calculatiin of the line's value was sort of 'lost' when you lept to non-zero slope lines... Jussayin. Cheers DocV

[@statquest]

Glad it was helpful!

[@aldikroos6290]

The best teaching video I have ever seen. What a great work!

[@statquest]

Thank you! :)

[@Luxcium]

BAM 😅😅😅🎉 I reached out to the bottom of the _STACK_ Quest (finally) 🎉❤ Wow 😮 I am on this quest to find out where this will stop since I just learned that I must watch the video *Fitting a Line to Data* also know as *Linear Regression* before I can watch the *Gradient Descent Step-by-Step!!!* so that I can watch the video related to *Neural Networks part 2* that I must watch before I can watch the *The StatQuest Introduction To PyTorch...* before I can watch the *Introduction to coding neural networks with PyTorch and Lightning* 🌩️ (it’s something related to the cloud I understand) I am genuinely so happy to learn about that stuff with you Josh❤ I will go watch the other videos first and then I will back propagate to that first video... And now I feel like it took me 6 years to find out this wonderful video I can't wait to see it😊

[@statquest]

You finally made it!

[@jasafraga]

I think another reason to take the square error is that it creates an actual geometric square of size of the error (area of the square, A= L*W). Add them all up together to get a 2D representation of the error. Where adding all the absolute error lines is only 1D. Sometimes shapes are easier to visualize. Amazing videos and pedagogy style. Props.

[@statquest]

Noted.

[@user-cb7it9cv9l]

Best and simplest explanation ever !!

[@statquest]

Glad you liked it!

[@danielkhromov9724]

Oh my God this is just the best video I've ever seen about Linear Regression! Thank you very much! I subscribed just after the video, please do not stop!

[@statquest]

Awesome! Thank you!

[@aakshay8440]

hello, you have done the wonderfull explanation. I loved it. and I am requesting you to do a video on assumptions made in linear regression. this will help us a lot.

[@iktiar]

your method of teaching is awesome, thank you!

[@statquest]

Thank you! :)

[@ian_owuor]

Yes this is exactly what I've been looking for. Great video. It has made my life a lot easier.

[@statquest]

Thanks!

[@lucasbatista8169]

Great video! It is admirable the effort you put on teaching... Just a suggestion: It would be great if you could put the link of the complementary videos you describe in each video. That way, it is easier to keep on track.

[@statquest]

Thanks for the tip. I generally try to do that (add the links in the description), but sometimes I forget. If you have time, it would be great if you could post which videos need links to complementary videos in a comment and then I'll take care of the rest.

[@TheSheekeyScienceShow]

I actually LOVE your video style!

[@statquest]

Thank you! :)

[@abhi9029]

superb explanation, crystal clear with graphics is good way to comprehend. Thank you for this.

[@statquest]

Hooray!

[@AbhiJeevanantham]

10 videos in I have started dancing to the song at the beginning.

[@statquest]

bam! :)

[@MONAD_0]

Hey josh, I absolutely love your explanations! It's given me a completely different perspective on how i think about machine learning and made the topics intuitive. I wonder if you can compile some notes for all of the content that can be reviewed in under a day and a mind-map that can be used to put all the pieces together. that would be really AWESOME!

[@statquest]

Something like this? app.learney.me/maps/StatQuest

[@dariakrupnova6245]

Please do more of these, I think I will be able to pass my econometrics test thanks to you.

[@statquest]

Good luck!

[@maheshgopalakrishnan5072]

Sir, your explanations are crystal clear. Thank you

[@statquest]

Thank you! :)

[@may4081]

Simply explained...just makes it beautiful to watch! Thanks!! :-)

[@statquest]

Thank you! :)

[@kensarmoreto1863]

Thank you for the vid. It was so easy to understand the concept of Least Square using visualization. I will use this as my reference for my demo teaching in stat. Hoping fore more stat videos and data analysis trick & tips...

[@statquest]

Glad it was helpful!

[@Hevletica]

Sorry for commenting several times throughout the series, but I would like to point out you should probably move this video + "Linear Regression, Clearly Explained!!!" before ROC/AUC in your ML playlist since you suggest understanding these basics beforehand.

[@statquest]

Ok thanks!

[@johnwilson3918]

This is a brilliant video. You're a few minutes from explaining how to derive the LR formula. I seem to recall from my high school days - that you work out two partial derivatives (dSum/da - treating 'b' as a constant and then dSum/db treating 'a' as a constant) and equate these to 0 (to find minimum). You should be left with a couple of equations for intercept and slope.

[@statquest]

That's exactly right (and I show how at least part of this works in my video on the Chain Rule kzbin.info/www/bejne/rZ2Unqyup9mEfrM ). However, what I don't like about the LR formula is that it only works in this specific situation. In contrast, Gradient Descent gives us pretty good parameter estimates and works in a million other situations. For more details on Gradient Descent, see: kzbin.info/www/bejne/qXXZZZlqqJeGeJo

[@PunmasterSTP]

Linear regression? More like "Liking these videos and I'm in heaven!" 👍

[@statquest]

Double bam!

[@techproductowner]

Bro you are genius . it just sinks in the mind . .they way you are explaining , Kindly guide me with some youtube channel which explains other concepts related to math's like calculus etc. .in similar way . .or I request you to create them as well from prospective of DS

[@statquest]

Thanks! Lots of other people like 3Blue1Brown for math.

[@prasshanthvp1082]

Finding optimal slope and intercepts is done by the machine, however we would need to tell the machine "how to do it" in order to get the values of intercept and slope. There are various ways to do this. If you are a comfortable programmer, you could write a function which changes slope and intercept for varying values, and pick the values which brings the least sum of squares errors . If you know certain optimization techniques like Gradient Descent, Stochastic Gradient Descent, you could ask the machine to use these algorithm and solve for intercept and slopes, where it uses the concept of slope of function being least or the "derivatives". If you know linear algebra and matrices really well, you could solve for intercept and slopes using Normal Equation. Note: I have used 'slopes' instead of 'slope', this means I am using many input variables, which you will find out more in future StatQuest videos. (Proud promotion of StatQuest videos!) For smaller data, it's not very important to know these concepts, since using any of them will bring us to the same values of intercept and slopes. When the complexity increases a.k.a millions of data and thousands of input variables, each optimization techniques may not arrive at same results. Also, you will have practical challenges and constraints like available time, speed, processing power, etc. In such cases, it might be useful to know (i) what are few optimization techniques, and (ii) when to apply each of them. Hurray!! you have reached to the end of this comment. Smash that like button to the videos and subscribe StatQuest. - Biggest fan of Josh and StatQuest.

[@statquest]

bam

[@ankuragarwala7558]

Great video on least square method.

[@statquest]

Thanks!

[@RoRight]

Just the right amount of theory and math. You should consider teaching stat for students in health and biological studies.

[@tseckwr3783]

Thanks for the video. It was a great refresher.

[@statquest]

Thanks!

[@alhello_game_of_everything]

Wowie kabawie wowie ZOWIE! Im still in highschool! And I understand most fundementals in the video!😊

[@statquest]

double bam!

[@palakagrawal8856]

The best explanation to date!

[@statquest]

Thanks!

[@amribrahim7850]

You are an incredible teacher.

[@statquest]

Thank you! :)

[@angelmotta]

Thanks for this content!!! I am very happy to understand these concepts watching this awesome explanation!

[@statquest]

Glad it was helpful!

[@ayselceferzade8587]

clearly explained! lots of thanks

[@statquest]

Glad it was helpful!

[@MiguelAngelMartinezFernandez]

Hi! First of all, congratulation for this awesome youtube channel. I was wondering if it would it be possible to re-upload this video in a higher quality? If I am not wrong, the max quality is 480p, which is not the best at these days. Thanks! Miguel

[@statquest]

I'm glad you like the video! That's a good idea to upload a higher quality version.

[@woodynani4394]

Concise and precise, well done Sir.

[@statquest]

Thank you! :)

[@jeanlanz2344]

Great explanation. Thank you. God bless you!

[@statquest]

Thank you!

[@mukossa4650]

Thank you, this is great. Easy to understand

[@statquest]

Thank you! :)

[@MichaelKlimenko]

Really good explanation, thank you !

[@statquest]

Thanks!

[@christinagiannaros9817]

When a clip about stats starts of with singing that makes you laugh before you start it's a very good thing.

[@statquest]

bam!

[@meysamamini9473]

If software engineering world has uncle Bob, ML world has uncle Josh ! Hoorraay❤️!!

[@statquest]

:)

[@viethoalam9958]

each time i hear your song, i just laugh. you brighten my day!

[@statquest]

bam! :)

[@mustafacakir__]

Your method of explanation is great. Please keep uploading tutorials. I would like to see tutorials about Deep Learning and Boosting (XGBoost, Catboost, etc.) algorithms which are popular lately. Thanks.

[@statquest]

I already have videos on deep learning here: kzbin.info/www/bejne/eaKyl5xqZrGZetk and XGBoost here: kzbin.info/www/bejne/haWnaaqMlqugbKc All of my videos are organized here: app.learney.me/maps/StatQuest

[@nikhilsetty3451]

Great video.I love the way how intuitive your videos are! 👍🏻

[@statquest]

Thank you! :)

[@apopheniac4231]

Excellent video. Thank you!

[@statquest]

Glad it was helpful!

[@19.sairoopesh10]

wow i finally understood why we have to take the derivative

[@statquest]

Bam! :)

[@mohdaslam9092]

Best explanation ever!

[@statquest]

Wow, thanks!

[@shamsulazhar]

Amazing clarity.... and I don't just mean the singing.

[@statquest]

:)

[@muksmart1]

awesome explanation, best ever explanation, made it look so easy.....

[@statquest]

Thanks!

[@yourfriend988]

Now that's a good explanation.

[@statquest]

Thanks!

[@atanu_giri]

Subscribed under the influence of the comment section. Don't wanna loose it.

[@statquest]

:)

[@mostafa4321]

Thank you, Can you put a video with actual Data and interpret that data based on the model with possible decisions to make based on the outcome. I believe this will bring us to a closer understanding of linear regression relative to decisions making especially for those who have limited knowledge of statistics.

[@statquest]

I have a playlist of videos where I show how to do things in R. These are the "applied" videos where I go through a data set and show how it's handled and filtered and how the results are interpreted. Here is a link to the playlist: kzbin.info/aero/PLblh5JKOoLUJJpBNfk8_YadPwDTO2SCbx

[@ajayravishankar7797]

Pretty spot on. Happy I found this channel. I have a question. The distance between the line and a point, should'nt it be the perpendicular distance between the point and the line? I believe that is even more of an accurate model.

[@statquest]

For regression we use the vertical distance between the data and the line in order to preserve the relationship that the variable on the x-axis is supposed to be predicting the variable on the y-axis. By measuring the vertical distance (instead of the perpendicular distance), we can measure how good or bad that prediction is.

[@hardouthereforaph.d.2525]

God bless you and these videos. These are so helpful

[@statquest]

Thanks!