Regularization Part 2: Lasso (L1) Regression

Lasso Regression is super similar to Ridge Regression, but there is one big, huge difference between the two. In this video, I start by talking about all of the similarities, and then show you the cool thing that Lasso Regression can do that Ridge Regression can't.
NOTE: This StatQuest follows up on the the StatQuest on Ridge Regression:
• Regularization Part 1:...
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you'd like to support StatQuest, please consider...
Buying The StatQuest Illustrated Guide to Machine Learning!!!
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
KZbin Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
#statquest #regularization

[@statquest]

If you want to see why Lasso can set parameters to 0 and Ridge can not, check out: kzbin.info/www/bejne/jp6VdJKdiaafbsU Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

[@marisa4942]

I am eternally grateful to you and those videos!! Really saves me time in preparing for exams!!

[@statquest]

Happy to help!

[@citypunter1413]

One of the best explanation of Ridge and Lasso regression I have seen till date... Keep up the good work....Kudos !!!

[@statquest]

Thanks! :)

[@hughsignoriello]

Love how you keep these videos introductory and don't go into the heavy math right away to confuse; Love the series!

[@statquest]

Thank you!

[@Phobos11]

Good video, but didn't really explain how LASSO gets to make a variable zero. What's the difference between squaring a term and using the absolute value for that?

[@statquest]

Intuitively, the closer slope gets to zero, the square of that number becomes insignificant compared to the increase in the sum of the squared error. In other words, the smaller you slope, the square gets asymptotically close to 0 because it can't outweigh the increase in the sum of squared error. In contrast, the absolute value adds a fixed amount to the regularization penalty and can overcome the increase in the sum of squared error.

[@statquest]

@@theethatanuraksoontorn2517 Maybe this discussion on stack-exchange will clear things up for you: stats.stackexchange.com/questions/151954/sparsity-in-lasso-and-advantage-over-ridge-statistical-learning

[@programminginterviewprep1808]

@@statquest Thanks for reading the comments and responding!

[@statquest]

@@programminginterviewprep1808 I'm glad to help. :)

[@Phobos11]

@@statquest I didn't reply before, but the answer really helped me a lot, with basic machine learning and now artificial neural networks, thank you very much for the videos and the replies :D

[@patrickwu5837]

That "Bam???" cracks me up. Thanks for your work!

[@statquest]

:)

[@markparee99]

Every time I think your video subject is going to be daunting, I find you explanation dispel that thought pretty quickly. Nice job!

[@qiaomuzheng5800]

Hi, I can't thank you enough for explaining the core concepts in such short amount of time. Your videos help a lot! My appreciations are beyond words.

[@statquest]

Thank you!

[@JeanOfmArc]

(Possible) Fact: 78% of people who understand statistics and machine learning attribute their comprehension to StatQuest.

[@statquest]

bam! :)

[@gonzaloferreirovolpi1237]

Hi man, really LOVE your videos. Right now I'm studying Data Science and Machine Learning and more often than not your videos are the light at the end of the tunnel, sot thanks!

[@perrygogas]

Some video ideas to better explain the following topics: 1. Monte Carlo experiments 2. Bootstrapping 3. Kernel functions in ML 4. Why ML is black box

[@statquest]

OK. I'll add those to the to-do list. The more people that ask for them, the more I'll priority they will get.

[@perrygogas]

@@statquest That is great! keep up the great work!

[@gauravms6681]

@@statquest yes we need it please do plsssssssssssssssssssssssssssssssss plsssssssssssssssssssssssssssssssssssssssssssssss

[@InfinitesimallyInfinite]

Bootstrapping is explained well in Random Forest video.

[@miguelsaravia8086]

Do it for us... thanks good stuff

[@anuradhadas8795]

The difference between BAM??? and BAM!!! is hilarious!!

[@statquest]

:)

[@SaiSrikarDabbukottu]

​@@statquestCan you please explain how the irrelevant parameters "shrink"? How does Lasso go to zero when Ridge doesn't?

[@statquest]

@@SaiSrikarDabbukottu I show how it all works in this video: kzbin.info/www/bejne/jp6VdJKdiaafbsU

[@Jenna-iu2lx]

I am so happy to easily understand these methods after only a few minutes (after spending so many hours studying without really understanding what it was about). Thank you so much, your videos are increadibly helpful! 💯☺

[@statquest]

Great to hear!

[@Jan-oj2gn]

This channel is pure gold. This would have saved me hours of internet search... Keep up the good work!

[@statquest]

Thank you! :)

[@alexei.domorev]

Josh - as always your videos are brilliant in their simplicity! Please keep up your good work!

[@statquest]

Thanks, will do!

[@tymothylim6550]

Thank you, Josh, for this exciting and educational video! It was really insightful to learn both the superficial difference (i.e. how the coefficients of the predictors are penalized) and the significant difference in terms of application (i.e. some useless predictors may be excluded through Lasso regression)!

[@statquest]

Double BAM! :)

[@admw3436]

My teacher is 75 years old, explained us Lasso during one hour , without explaining it. But this is a war I can win :), thanks to your efforts.

[@statquest]

I love it!!! Glad my video is helpful! :) p.s. I got the joke too. Nice! ;)

[@ak-ot2wn]

Why is this scenario many times the reality? Also, I check StatQuest's vids very often to really understand the things. Thanks @StatQuest

[@luispulgar7515]

Bam! I appreciate the pace of the videos. Thanks for doing this.

[@statquest]

Thanks! :)

[@TeXtersWS]

Explained in a very simple yet very effective way! Thank you for your contribution Sir

[@statquest]

Hooray! I'm glad you like my video. :)

[@chrisg0901]

Don't think your Monty Python reference went unnoticed (Terrific and very helpful video, as always)

[@statquest]

Thanks so much!!! :)

[@ajha100]

Oh it absolutely did. And it was much loved!

[@ryanzhao3502]

Thx very much. Clear explanation for these similar models. Great video I will conserve forever

[@naomichin5347]

I am eternally grateful to you. You've helped immensely with my last assessment in uni to finish my bachelors

[@statquest]

Congratulations!!! I'm glad my videos were helpful! BAM! :)

[@atiqkhan7803]

This is brilliant. Thanks for making it publicly available

[@statquest]

You're welcome! :)

[@simsim2159]

Your videos make it so easy to understand. Thank you!

[@statquest]

Thank you! :)

[@takedananda]

Came here because I didn't understand it at all when my professor lectured about LASSO in my university course... I have a much better understanding now thank you so much!

[@statquest]

Awesome!! I'm glad the video was helpful. :)

[@kyoosikkim749]

The other day, I had homework to write about Lasso and I struggled.. wish I had seen this video a few days earlier.. Thank you as always!

[@AnaVitoriaRodriguesLima]

Thanks for posting, my new favourite youtube channel absolutely !!!!

[@statquest]

Wow, thanks!

[@sanyuktasuman4993]

Your intro songs reminds me of Pheobe from the TV show "Friends", and the songs are amazing for starting the videos on a good note, cheers!

[@statquest]

You should really check out the intro song for this StatQuest: kzbin.info/www/bejne/emHIl3t7f9iZftE

[@gregnelson8148]

You have a gift for teaching! Excellent videos!

[@statquest]

Thanks! :)

[@jasonyimc]

So easy to understand. And I like the double BAM!!!

[@statquest]

Thanks!

[@corneliusschramm5791]

Dude you are an absolute lifesaver! keep it up!!!

[@statquest]

Hooray! I'm glad I could help. :)

[@Endocrin-PatientCom]

Incredible great explanations of regularization methods, thanks a lot.

[@statquest]

Thanks! :)

[@DonnyDonowitz22]

The best explanation ever.

[@pelumiobasa3104]

this is awesome thank you so much for this u explained it so well . I will recommend this video to every one I know who is interested . I also watched your lasso video and it was just as good thank you

[@statquest]

Thank you very much! :)

[@ayush612]

Yeahhhh!!! I was the first to express Gratitude to Josh for this awesome video!! Thanks Josh for posting this and man! your channel is growing.. last time, 4 months ago it was 12k. You have the better stats ;)

[@statquest]

Hooray! Yes, the channel is growing and that is very exciting. It makes me want to work harder to make more videos as quickly as I can. :)

[@akashdesarda5787]

@@statquest please keep on going... You are our saviour

[@longkhuong8382]

Hooray!!!! excellent video as always Thank you!

[@statquest]

Hooray, indeed!!!! Glad you like this one! :)

[@hsinchen4403]

Thank you so much for the video ! I have watched several your videos and I prefer to watch your video first then see the real math formula. When I did that, the formula became so easier and understandable! For instance, I don't even know what does 'norm' is, but after watching your video then it would be very easy to understand!

[@statquest]

Awesome! I'm glad the videos are helpful. :)

[@whispers191]

Thank you once again Josh!

[@statquest]

bam!

[@yilinxie2457]

Thanks! I finally understand how they shrink parameters!

[@flyingdutchmanSimon]

Seriously the best videos ever!!

[@statquest]

Thanks!

[@arthurus8374]

so incredible, so well explained

[@statquest]

Thanks!

[@RussianSUPERHERO]

I came for the quality content, fell in love with the songs and bam.

[@statquest]

BAM! :)

[@sophie-ev1mr]

Thank you so much for these videos you are a literal godsend. You should do a video on weighted least squares!!

[@joshuamcguire4832]

a man of his word...very clearly explained!

[@statquest]

Thank you! :)

[@MrArunavadatta]

wonderfully explained

[@statquest]

Thank you! :)

[@ajha100]

I really appreciated the inclusion of swallow airspeed as a variable above and beyond the clear-cut explanation. Thanks Josh. ;-)

[@statquest]

:)

[@petrsomol]

Me too!

[@Azureandfabricmastery]

Hi Josh, Thanks for clear explanation on regularization techniques. very exciting. God bless for efforts.

[@statquest]

Glad you enjoyed it!

[@mrknarf4438]

Great video, clear explanation, loved the Swallows reference! Keep it up! :)

[@statquest]

Awesome, thank you!

[@johnholbrook1447]

Fantastic videos - very well explained!

[@statquest]

Thank you! :)

[@chadmadding]

Always amazing videos.

[@statquest]

Thank you!

[@rishabhkumar-qs3jb]

Amazing video, explanation is fantastic. I like the song along with the concept :)

[@statquest]

Bam! :)

[@emmanueluche3262]

Wow! so easy to understand this! Thanks very much!

[@statquest]

Thanks!

[@alecvan7143]

The beginning songs are always amazing hahaha!!

[@statquest]

Awesome! :)

[@add6911]

Excelent video Josh! Amazing way to explain Statistics Thank you so much! Regards from Querétaro, México

[@statquest]

Muchas gracias! :)

[@abdullahmoiz8151]

Brilliant explanation didnt need to check out any other video

[@statquest]

Thank you!

[@codewithsid2063]

Keep it up man. Awesome content.

[@statquest]

Thanks! :)

[@lihuang919]

Great video! I finally understand!

[@simrankalra4029]

Thankyou Sir ! Great Help.

[@lucianotarsia9985]

Great video! The topic is really well explained

[@statquest]

Thank you!

[@yuzaR-Data-Science]

Thanks a lot! Amazing explanation! Please, continue the great work and add more on statistics, probability in general and machine learning in particular. Sinse Data Science suppose to have a great future, I am certain that your channel also will prosper a great deal!

[@statquest]

Thank you! :)

[@kitkitmessi]

Airspeed of swallow lol. These videos are really helping me a ton, very simply explained and entertaining as well!

[@statquest]

Glad you like them!

[@raymilan2301]

Thanks a lot for the explanation !!!

[@statquest]

You are welcome!

[@TheBaam100]

Thank you so much for making these videos! Had to hold a presentation about LASSO in university.

[@statquest]

I hope the presentation went well! :)

[@TheBaam100]

@@statquest Thx. It did :)

[@hanadiam8910]

Million BAM for this channel 🎉🎉🎉

[@statquest]

Thank you!

[@faustopf-.]

Magnificent video

[@statquest]

Thanks!

[@gdivadnosdivad6185]

You are the best! I understand it now!

[@statquest]

Thanks!

[@zebralemon]

I enjoy the content and your jam so much! '~Stat Quest~~'

[@statquest]

Thanks!

[@user-ur2en1zq4f]

Great people know subtle differences which is not visible to common eyes love you sir

[@statquest]

Thanks!

[@apekshaagrawal6696]

Thanks for the Video. They make difficult concepts seem really easy..

[@statquest]

Thank you! :)

[@apekshaagrawal6696]

@@statquest Can u make a similar video for LSTM?

[@qorbanimaq]

Ah! A triple THANKSSSS!!!!. I finally got what they are really doing.

[@statquest]

Hooray! :)

[@2210duynn]

Very good video. You help me alot !!!!

[@statquest]

Thanks! :)

[@sandipansarkar9211]

Great explanation

[@statquest]

Thank you! :)

[@KrizTahimic]

Amazing! Thank you so much for this!

[@statquest]

Thanks!

[@praveerparmar8157]

Just love the way you say 'BAM?'.....a feeling of hope mixed with optimism, anxiety and doubt 😅

[@statquest]

:)

[@johnalvinm]

This was gold!

[@statquest]

Thank you!

[@davidmantilla1899]

Best youtube channel

[@statquest]

Thank you! :)

[@hosseineyvazi3122]

keep up ur work bro .

[@statquest]

Thanks, will do!

[@rezaroshanpour971]

Great....please continue to learn other models...thank you so much.

[@statquest]

Thanks!

[@RAJIBLOCHANDAS]

Nice explanation!

[@statquest]

Thanks!

[@gmshadowtraders]

This is dope fam!

[@statquest]

:)

[@yulinliu850]

Thanks Josh!

[@statquest]

You're welcome! :)

[@clementbourgade2487]

NOBODY IS GOING TO TALK ABOUT THE EUROPEAN / AFRICAN SWALLOW REFERENCE ????are you all dummies or something ? It made my day. Plus, video on top, congratulation. BAMM !

[@statquest]

bam!

[@hareshsuppiah9899]

Statquest is like Marshall Eriksen from HIMYM teaching us stats. BAM? Awesome work Josh.

[@statquest]

Thanks!

[@leandadearaujo5559]

awesome your explanation just simplifies everything request to make videos on rest of the algorithms as well thank you

[@statquest]

I'm working on them :)

[@mohitkushwaha8974]

awesome video

[@statquest]

Thanks!

[@arpitqw1]

why can't ridge reduce weight/parameter to 0 like lasso?

[@akronmom9037]

Thank you for uploading this video. Can you upload a video explaining the difference between Lasso and Group Lasso? Thanks again.

[@kd1415]

love the work, i remember reading books about linear regresion, when they spent like 5 pages for these 2 topics but i still have no clue what they really do =))

[@statquest]

Glad it was helpful!

[@kd1415]

Love the fact that you reply to every single comment here in YT haha

[@KansasFashion]

Love this song bro!

[@statquest]

Thanks!

[@nurainibrahim9251]

BAM!! clearly explained!!

[@statquest]

Hooray! I'm glad you like the video. :)

[@mahajanpower]

Hi Josh! I am a big fan of your videos and it is clearly the best way to learn machine learning. I would like to ask you if you will be uploading videos relating to deep learning and NLP as well. If so, that will be awesome. BAM!!!

[@statquest]

Right now I'm finishing up Support Vector Machines (one more video), then I'll do a series of videos on XGBoost and after that I'll do neural networks and deep learning.

[@mahajanpower]

StatQuest with Josh Starmer Thanks Josh for the updates. I’ll send you request at Linkedin.

[@alessandrofesti208]

You're great!! Thank You

[@statquest]

You're welcome!!! :)

[@gabrielowen7933]

I love you for this

[@statquest]

Thanks!

[@joaocasas4]

Me and my friend are studying. When the first BAM came, we fell for laught for about 5min. Then the DOUBLE BAM would cause a catrastofic laughter if we didn't stop it . I want you to be my professor please!

[@statquest]

BAM! :)

[@yoonchaena671]

Thank you so much~!!!!

[@statquest]

bam!

[@debatradas9268]

thank you so much sir

[@statquest]

Thanks!

[@pratiknabriya5506]

A StatQuest a day, keeps Stat fear away!

[@statquest]

I love it! :)

[@ainiaini4426]

Hahaha.. That moment you said BAM??? I laughed out loud 🤣🤣🤣

[@statquest]

:)

[@ninakumagai22]

My favourite youtuber!

[@statquest]

Thank you! :)

[@adwindtf]

love your videos.... extremely helpful and cristal clear explained.... but your songs..... let's say you have a very promising career as a statistician... no question

[@statquest]

;)