Machine Learning Tutorial Python - 17: L1 and L2 Regularization | Lasso, Ridge Regression

[@codebasics]

Check out our premium machine learning course with 2 Industry projects: codebasics.io/courses/machine-learning-for-data-science-beginners-to-advanced

[@bharathis9295]

Statquest theory+Codebasics Practical implementation=😍😍😍

[@codebasics]

ha ha .. nice :) Yes I also like statquest.

[@gokkulkumarvd9125]

Exactly!

[@ItsSantoshTiwari]

Same😂👌

[@abhinavkaul7187]

@@codebasics BAM!! :P Btw, the way you explained Yolo that was superb, bro!

[@sandydsa]

Yes! Minor comment, kindly please switch age and matches won. Got confused at first 😂

[@ManigandanThangaraj]

Nice explanation .. Adding to that L2 Ridge : Goal is to prevent multicollinearity and control magnitude of the coefficients where highly corelated features can be removed by shirking the coefficients towards to zero not exactly zero , stability and generalization. L1 Lasso : Goal is to prevent sparsity in the model by shirking the coefficients exactly to zero , importance in feature selection, preventing overfitting..

[@r0cketRacoon]

so, in what cases should we use L1 and L2?

[@AlonAvramson]

I have been following all 17 videos on ML you provided so far and found this is the best resource to learn from . Thank you!

[@DrizzyJ77]

Bro, you don't know how you've helped me in my computer vision journey. Thank you❤❤❤

[@Hari-xr7ob]

you should probably change the X and Y axes. Matches won is a function of Age. So, Age should be on X axis and Matches won on Y axis

[@hansamaldharmananda9605]

That will more familiar. :D

[@kj7767]

familiar where !@@hansamaldharmananda9605

[@parthasarothi2295]

you just said my words

[@gyanaranjanbal10]

Clean, crisp and crystal clear, I was struggling to understand this from a long time, your 20 mins video cleared it in one attempt, thanks a lot💌💌

[@bors1n]

thank you a lot, I'm from Russia and I'm student. I watch your video about ML and It helps me to understand better

[@codebasics]

Glad to hear that!

[@shashankdhananjaya9923]

Couldn't have explained it any simpler. Perfect tutorial.

[@codebasics]

Glad it helped!

[@ambujbaranwal9351]

00:04 L1 and L2 regularization help address overfitting in machine learning 02:12 Balancing between underfitting and overfitting is crucial for effective model training. 04:26 Regularization shrinks parameters for better prediction function 06:47 L2 regularization penalizes the overall error and leads to simpler equations. 09:14 Filtering and handling NA values in a dataset 12:02 Dropping NA values and converting categorical features into dummies for machine learning in Python. 14:28 Understanding the issues of overfitting in linear regression model 17:00 Regularization techniques like L1 and L2 improve model accuracy. 19:16 Encouraging viewers to like and share the video

[@RadioactiveChutney]

Note for myself: This is the guy... his videos can clear doubts with codes.

[@codebasics]

ha ha .. thank you 🙏

[@NafisAnsari-vr2xq]

For a different parameter of train test split ( train_X, test_X, train_y, test_y = train_test_split(X, y, test_size=0.3, random_state=2) the test score is 0.13%, using the regular train test split parameter the scores are similar to 67-68%. No point in trying the regularization on these scores. Yet I tried them on these with some changes and there wasn't any significant change. For anyone with the same scenario, Just try to remember that regularization can be used when the scores for training and testing data has a significant difference, could be underfitting or overfitting.

[@ajaykushwaha-je6mw]

Best tutorial on l1 and L2 Regularization.

[@tusharsethi2801]

One of the best videos out there for Regularization.

[@yash422vd]

As per the equation y = mX + c, you inter-changed the y & X axis, if I'm not wrong. Because you are trying to predict match won(yhat) which is your horizontal line and age(X) is on vertical line. Maybe using something unconventional mislead new-learners. As X is a horizontal line and y is vertical line, that's what we learned since school time. Assigning X & y to axis(as per your explanation) will be great help to learner. I hope you are not taking personally. My opologies if so!

[@lazzy5173]

Summary: - L1 regularization helps in feature selection. -L2 regularization helps in preventing overfitting.

[@atulupadhyay1542]

machine learning concepts and practicals made easy, Thank you so much Sir

[@codebasics]

I am happy this was helpful to you.

[@bruh-jr6wj]

I believe the most appropriate imputing method here is to group by the similar type of houses and then fill with the mean value of the group. For example, if the average is, say, 90 m^2, and the home is only a flat, the building area is incorrectly imputed.

[@nexthome1445]

Kindly make video on Feature selection for Regression and classification problem

[@bhavikjain1077]

A good video to understand the practical implementation of L1 and L2. Thank You

[@haintuvn]

Thank you for your interesting video. As far as I get from the video, L1, L2 regulations help to overcome the overfit problem from Linear regression! What is about other algorithms ( Support vector machine, logistic regression..) , how can we overcome the overfit problem?

[@nastaran1010]

best learning with very good explanation. Thanks

[@koustavbanerjee8195]

Please do videos about XGBoost, LGBoost !! You Videos Are Pure GOLD !!

[@mukeshkumar-kh2fh]

thank you for helping the DS community

[@javiermarchenahurtado7013]

Such a great video!! I was struggling to understand regularization and now it's crystal clear to me!

[@piyushlanjewar6274]

That's a really great explanation, Anyone can use this method in real use cases now. Keep it up.

[@amruth3]

Sir your all the vedios are really helpful...Now Iam giving you the feed back of the vedio Iam going to see.This is also an beautiful vedio and Hyperparamter tuning also an very best vedio......God Bless you..u..work hard in getting think to understand in easy manner..

[@nationhlohlomi9333]

I really love your content….. You change lives❤❤❤

[@DarkTobias7]

These are the videos we like!!!

[@codebasics]

Thanks DarkTobias. Good to see your comment.

[@NeekaNeeksz]

Clear introduction. Thanks

[@kaizen52071]

Nice video....good lesson......funny enough i see my house address in the dataset

[@king1_one]

good explanation sir and you need appreciation , i am here .

[@phil97n]

Awesome explanation, thanks.

[@Ultimate69664]

thank you ! this video save my exam :)

[@joehansie6014]

All your videos are totally great. Keep working on it

[@phuonglethithanh8498]

Thank you for this video. Very straightforward and comprehensive ❤

[@anvarshathik784]

achine learning concepts and practicals made easy, Thank you so much Sir

[@codebasics]

You are most welcome

[@ankitmaheshwari7310]

Good.model representation is good.hoping some deep knowledge in next video

[@nehareddy4619]

I really liked your way of explanation sir

[@vyduong276]

I can understand it now, thanks to you 🥳

[@ALLINONETV1]

Please continue ....

[@leonardomenar55]

Excellent Tutorial, Thanks.

[@mohammadrasheed9247]

Nice Explanation. Also Recommended to play on 2X

[@dylanloh5327]

Thank you vm for this video. This is straight-forward and simple to understand!

[@codebasics]

👍👍😊

[@vishvam1307]

Nice explanation

[@rohantalaviya136]

Really great video

[@PollyMwangi-cp3jn]

Thanks so much sir. Great content

[@davuthdy876]

Thank for your video for sharing to the world.

[@codebasics]

I am glad you liked it

[@priyankshekhar2454]

Very good videos by you on each topic..thanks !!

[@kouider76]

Just came across this video accidentally simply great thank you

[@joehansie6014]

Simple but powerful😎👍

[@marthanyarkoa9007]

Thanks so simple ❤😊

[@bryteakpakpavi637]

You are the best.

[@codebasics]

Glad it was helpful!

[@jongcheulkim7284]

Thank you. This is very helpful.

[@tanishsadhwani730]

Amazing sir thank you so much

[@ayenewyihune]

Cool video

[@analuciademoraislimalucial6039]

Thank you so much teacher

[@alielakroud1786]

Hi Sir, Thanks for all this tutorials in ML. I've tried to use this syntaxe above, but when i fit my model the score using trainning data is 0.68 whereas the reg.scores using Test data is just weird.score(X_test,Y_test) =--17761722756.9913 dummies=pd.get_dummies(df[['Suburb','Type','Method','SellerG','CouncilArea','Regionname']]) Merge=pd.concat([df,dummies],axis='columns') final=Merge.drop(['Suburb','Type','Method','SellerG','CouncilArea','Regionname'],axis='columns') final 2nd part of my question is when i use L1 and L2 Regularization the score seem correct 0.66 and 0.67 I would also mentionned that when i've used LabelEncoder i find a score test data 0.44 and Trainning data 0.42 Thanks in advance for your answers

[@ajgameboy6930]

Same here, I really don't know what went wrong...

[@ajgameboy6930]

Hey, quick update, I found out the problem in my scenario... I had filled NaN values of price with mean, which caused the problem... Now that I have dropped 'em, it's working fine... Hope you had also solved the problem (you must've, ur comment is from 2 years back XD)

[@swL1941]

Great video. However, It would have been better if you had provided the justification for assigning Zeros to few NaN values and giving mean to frew records. I know "its safest to assume" butt hen I believe in real world projects we cannot just assume things.

[@unifarzor7237]

Always excellent lessons, thank you

[@HA-bj5ck]

Appreciate the efforts, but there were issues with the foundational understanding. Additionally, the inclusion of dummy variables expanded the columns to 745 without acknowledgement or communication regarding its potential adverse effects to viewers was not expected.

[@noahrubin375]

Not all superheroes wear capes!

[@m.shiqofilla4246]

Very nice video sir but at first i hoped you show the plot of scatter plot of the data and how the curve of the L1/L2 regression...

[@aadityashukla8535]

good theory!

[@ravikumarrai7325]

Awesom video....really awesom..

[@codebasics]

Glad you liked it

[@denisvoronov6571]

Nice example. Thank you so much!

[@codebasics]

Glad you liked it!

[@nikolinastojanovska]

great video, thanks!

[@SahilAnsari-gl3xu]

Thank a lot Sir❤️ Very good teaching style (theory+practical)👍

[@sanooosai]

thank you great work

[@daretoschool4113]

Please make video for genetic algorithm

[@ayusharora2019]

Very well explained !!

[@codebasics]

Glad it was helpful!

[@JAVIERHERNANDEZ-wp6qj]

Maybe in the Cost formula, the indices for summation should be different (in general): for the MSE term the sum should be over the entire training dataset (in this case n), and the sum for the regularization term should run over the number of features or columns in the dataset

[@cvino0618]

there isnt a paid course on udemy better than the information I am gaining here

[@MrMadmaggot]

First when you apply lasso, you apply it apart from the first linear regression model you made right? Which means applying scikit Lasso is like making a linear regression but with regularization or it is applied to the linear regresion from the cell above?? So what if I use a knn or a forest?

[@anseljanson5171]

Thank you for this video why did you drop na value price column even though it had more than 7000 na values wont it affect the prediction??

[@mkt4941]

You cannot accurately make an assumption as to what the price is based on the available data, so you have to drop it.

[@anseljanson5171]

@@mkt4941 Thanks :)

[@victorbenedict8743]

Great tutorial sir.Its a privilege to be a fan of yours.Please sir could you please do a video on steps to carry out when doing data cleaning for big data.Thank you.

[@gouravsapra8668]

Hi...The equation, shouldn't it be : Theta0 + Theta1.x1 + Theta2.square (x1)+Theta3.cube (x1) rather than Theta0 + Theta1.x1 + Theta2.square (x2)+Theta3.cube (x3) because we have only one x feature ? 2) the Regularization expression (Lambda part), my understanding is that we should not take "i & n" , rather we should take "j & m" etc. The reason is that in first half of equation, we took "i & n" for number of rows whereas in second half, we need to take number of features, so different parameters should be used. Please correct me if my understanding is wrong.

[@EngineerNick]

Thankyou for this it was very useful :)

[@codebasics]

Glad it was helpful!

[@nomanshaikhali3355]

Kindly explain Boosting algos!!

[@thoeer913]

I don't know how can you explain such a simple topic in so complicated manner. Your explanation caused more confusion than the topic itself.

[@OceanAlves23]

👨‍🎓👏✔, from Brazil-Teresina-PI

[@codebasics]

Thanks Ocean. I wish you visit Brazil one day (especially Amazon rain forest :) )

[@arjunbali2079]

thanks sir

[@SGandhi]

Can you make a video of ensemble model of using decision tree,knn and svm code

[@adia9791]

I think one must not use those imputations(mean) before train test split as it leads to data leakage, correct me if I am wrong.

[@Piyush-yp2po]

Taking mean for prices would have been a better choice

[@nikhilsingh1296]

I really love learning from your Videos, they are pretty awesome. Just a concern, as in Line 11 we ran a missing value sum code where the Price Stated, 7610 and in the next line that is Line 12, we have dropped the 7610 rows, isn't it? Also, what was the other option if we would not have dropped the valued, can we not divide the data set and treat 50 percent of the missing values in Price and as a train dataset by imputing mean, and run the test on the missing price values. I am not sure, even if this is a valid question, but I am a bit curious. Also, what was the scope for PCA here?

[@slainiae]

I agree. The missing 'Price' values could have been estimated using one of the previously presented algorithms.

[@anjalipatel9028]

L1,L2 Regularization is valid for regression algorithm only?

[@rash_mi_be]

In L2 regularization, how can theta reduce when lambda increases, and increase when lambda decreases?

[@sudharsanb9391]

Sir pls put a video on xgboost, adaboost and gradient boosting

[@swaralipibose9731]

Yes please

[@DHAiRYA2801]

Yes!

[@codebasics]

sure. looks like there is lot of demand for these topics, I have added them in my todo list

[@duztv5370]

@@codebasics please sir, we will be expecting. Thanks

[@sunzarora]

Yes please!

[@bhoomi5398]

what is dual parameter and please explain what is primal formal & dual

[@TheOraware]

0.01*4=0.04 , here 0.01 is lambda and 4 is theta , 0.1*4=0.4 , here same 0.1 is lmada and 4 is theta. When i increase lambda then product of lambda increase hence cost function increase not theta value at @6:47

[@Piyush-yp2po]

Got 45% accuracy for normal reg, l1 and l2

[@_k_kd]

but you have dropped more then 7000 price with Na

[@PRIYASHARMA-cr8ff]

same I was also thinking the same

[@tjbwhitehea1]

Hey, great video thank you. Quick question - what's the best way to find the optimal alpha? Do you do a grid search?

[@codebasics]

Yes doing grid search would be a way

[@sunzarora]

Nice video, my question is what will u do so accuracy will jump on this dataset from 67 to 90+?

[@ajaysaroha2539]

Sir,I am fresher & want to make career in finance domain data analyst & I have no any experience in finance domain so how can I gain knowledge in finance domain so pls give some suggestion about it.

[@junaidlatif2881]

Amazing. But how to select best alpha value?

[@zappist751]

cross validation :)

[@gefett]

Thank's for class it's very clearly for me. But I had a problem to create a sending file my code from to Kaggle, help me please.

[@Ilya_4276]

thanks

[@codebasics]

Welcome

[@tigrayrimey6418]

Exchange the variables pls. Put matches won on Y-axis and age on X-axis.