This is showing that the quality and value of a video is not depending on how fancy the animations are, but how expert and pedagogue the speaker is. Really brilliant! I assume you spent a lot of time designing that course, so thank you for this!
@ritvikmath3 жыл бұрын
Wow, thanks!
@backstroke08102 жыл бұрын
Totally agree. I learn a lot from his short videos. Precise, concise, enough math, enough ludic examples. True professor mind.
@rez_daddy4 жыл бұрын
"Now that we understand the REASON we're doing this, let's get into the math." The world would be a better place if more abstract math concepts were approached this way, thank you.
@garbour4562 жыл бұрын
good point
@tzu-chunchen51399 ай бұрын
This is the best explanation of Ridge regression that I have ever heard! Fantastic! Hats off!
@GreenEyesVids3 жыл бұрын
Watched these 5 years ago to understand the concept and I passed an exam. Coming back to it now to refresh my memory, still very well explained!
@ritvikmath3 жыл бұрын
Nice! Happy to help!
@nadekang81985 жыл бұрын
This is awesome! Lots of machine learning books or online courses don't bother explaining the reason behind Ridge regression, you helped me a lot by pulling out the algebraic and linear algebra proofs to show the reason WHY IT IS THIS! Thanks!
@siddharthkshirsagar25454 жыл бұрын
I was searching for ridge regression on the whole internet and stumbled upon this is a video which is by far the best explanation you can find anywhere thanks.
@zgbjnnw93062 жыл бұрын
It's so inspiring to see how you get rid of the c^2! I learned Ridge but didn't know why! Thank you for making this video!
@aarshsachdeva57857 жыл бұрын
You should add in that all the variables (dependent and independent) need to be normalized prior to doing a ridge regression. This is because betas can vary in regular OLS depending on the scale of the predictors and a ridge regression would penalize those predictors that must take on a large beta due to the scale of the predictor itself. Once you normalize the variables, your A^t*A matrix being a correlation matrix of the predictors. The regression is called "ridge" regression because you add (lambda*I + A^t*A ) which is adding the lambda value to the diagonal of the correlation matrix, which is like a ridge. Great video overall though to start understanding this regression.
@taareshtaneja75235 жыл бұрын
This is, by far, the best explanation of Ridge Regression that I could find on KZbin. Thanks a lot!
@SarahPourmolamohammadi Жыл бұрын
You are the best of all.... you explained all the things,,, so nobody is gonna have problems understanding them.
@RobertWF427 ай бұрын
Excellent video! One more thing to add - if you're primarily interested in causal inference, like estimating the effect of daily exercise on blood pressure while controlling for other variables, then you want an unbiased estimate of the exercise coefficient and standard OLS is appropriate. If you're more interested in minimizing error on blood pressure predictions and aren't concerned with coefficients, then ridge regression is better. Also left out is how we choose the optimal value of lambda by using cross-validation on a selection of lambda values (don't think there's a closed form expression for solving for lambda, correct me if I'm wrong).
@surajshivakumar51243 жыл бұрын
This is literally the best video on ridge regression
@q0x8 жыл бұрын
I think its explained very fast, but still very clear, for my level of understanding its just perfect !
@BhuvaneshSrivastava4 жыл бұрын
Your data science videos are the best I have seen on KZbin till now. :) Waiting to see more
@ritvikmath4 жыл бұрын
I appreciate it!
@nikunjgattani9992 жыл бұрын
Thanks a lot.. I watched many videos and read blogs before this but none of them clarified at this depth
@charlesity4 жыл бұрын
Stunning! Absolute gold!
@wi8shad0w4 жыл бұрын
seriously!!!
@yxs84957 жыл бұрын
This really is gold, amazing!
@canernm3 жыл бұрын
Hi and thanks fr the video. Can you explain briefly why when the m_i and t_i variables are highly correlated , then the estimators β0 and β1 are going to have very big variance? Thanks a lot in advance!
@lanag8732 жыл бұрын
Hi same question here😶🌫
@alecvan71434 жыл бұрын
Amazing video, you really explained why we do things which is what really helps me!
@OmerBoehm2 жыл бұрын
Brilliant simplification of this topic. No need for fancy presentation to explain the essence of an idea!!
@xwcao19913 жыл бұрын
Thank you. I make the comment because I know I will never need to watch it again! Clearly explained..
@ritvikmath3 жыл бұрын
Glad it was helpful!
@TahaMVP6 жыл бұрын
best explanation of any topic i've ever watched , respect to you sir
@abhichels17 жыл бұрын
This is gold. Thank you so much!
@mohamedgaal5340 Жыл бұрын
I was looking for the math behind the algorithm. Thank you for explaining it.
@ritvikmath Жыл бұрын
No problem!
@cu76956 жыл бұрын
I subscribed just after watching this. Great foundation for ML basics
@ethanxia12888 жыл бұрын
Excellent explanation! Could you please do a similar video for Elastic-net?
@bettychiu73754 жыл бұрын
This really helps me! Definitely the best ridge and lasso regression explanation videos on KZbin. Thanks for sharing! :D
@mikeperez42223 жыл бұрын
Anyone else get anxiety when he wrote with the marker?? Just me? Felt like he was going to run out of space 😂 Thank you so much thoo, very helpful :)
@murraystaff5688 жыл бұрын
Brilliant! Just found your channel and can't wait to watch them all!!!
@e555t66 Жыл бұрын
I don't have money to pay him so leaving a comment instead for the algo. He is the best.
@akino.31926 жыл бұрын
You, Ritvik, are simply amazing. Thank you!
@Viewfrommassada4 жыл бұрын
I'm impressed by your explanation. Great job
@ritvikmath4 жыл бұрын
Thanks! That means a lot
@Lisa-bp3ec7 жыл бұрын
Thank you soooo much!!! You explain everything so clear!! and there is no way I couldn't understand!
@babakparvizi24256 жыл бұрын
Fantastic! It's like getting the Cliff's Notes for Machine Learning. These videos are a great supplement/refresher for concepts I need to knock the rust off of. I think he takes about 4 shots of espresso before each recording though :)
@aDifferentHandle6 жыл бұрын
The best ridge regression lecture ever.
@theoharischaritidis41736 жыл бұрын
This really helped a lot. A big thanks to you Ritvik!
@teegnas4 жыл бұрын
These explanations are by far the best ones I have seen so far on youtube ... would really love to watch more videos on the intuitions behind more complicated regression models
@soudipsanyal6 жыл бұрын
Superb. Thanks for such a concise video. It saved a lot of time for me. Also, subject was discussed in a fluent manner and it was clearly understandable.
@RAJIBLOCHANDAS2 жыл бұрын
Excellent approach to discuss Lasso and Ridge regression. It could have been better if you have discussed how Lasso yields sparse solutions! Anyway, nice discussion.
@nicolasmanelli7393 Жыл бұрын
I think it's the best video ever made
@Krishna-me8ly9 жыл бұрын
Very good explanation in an easy way!
@nickb68117 жыл бұрын
So so so very helpful! Thanks so much for this genuinely insightful explanation.
@wi8shad0w4 жыл бұрын
THIS IS ONE HELL OF A VIDEO !!!!
@mortezaabdipour55845 жыл бұрын
It's just awesome. Thanks for this amazing explanation. Settled in mind forever.
@jhhh06199 жыл бұрын
Your explanation is extremely good!
@nickwagner51736 жыл бұрын
We start out by adding a constraint that beta 1 squared + beta 2 squared must be less than c squared, where c is some number we choose. But then after choosing lamda, we minimize F and c ends up having no effect at all on our choice of the betas. I may be wrong but it doesn't seem like c has any effect on our choice of lambda either. I find it strange that we start out with the criteria that beta 1 squared + beta 2 squared must be less than c squared, but the choice of c is irrelevant. If someone can help me un-boggle my mind that would be great.
@RobertWF427 ай бұрын
Good question - I think it has to do with using the method of Lagrange multipliers to solve the constrained OLS optimization problem. The lambda gets multiplied by the expression in the parentheses at 11:17, which includes the c squared term. So whatever c squared value you choose, it's going to be changed anyways when you multiply by the lambda.
@yanlinwang57032 жыл бұрын
The explanation is so clear!! Thank you so much!!
@sanketchavan86 жыл бұрын
best explanation on ridge reg. so far
@prabhuthomas87705 жыл бұрын
SUPER !!! You have to become a professor and replace all those other ones !!
@SUBHRASANKHADEY5 жыл бұрын
Shouldn't the radius of the Circle be c instead of c^2 (at time around 7:00)?
@abhijeetsingh50498 жыл бұрын
Stunning!! Need more access to your coursework
@Sytch6 жыл бұрын
Finally, someone who talks quickly.
@adrianfischbach9496 Жыл бұрын
Huge thanks!
@zw74532 жыл бұрын
best explanation ever!
@prateekcaire41936 ай бұрын
It is unintuitive that we are constraining weights(betas) within value c^2, yet the regularization expression does not include the c but rather sum of squared weights. Certainly I am missing something here. Alternatively, why adding a sum of squared betas(or weights) to the cost function help optimize beta that stays within constraint so that betas don't become large and vary across datasets?
@vishnu2avv6 жыл бұрын
Awesome, Thanks a Million for great video! Searching you have done video on LASSO regression :-)
@intom16396 жыл бұрын
Brilliant! Could you make more videos about Cross validation, RIC, BIC, and model selection.
@sachinrathi78143 жыл бұрын
Can anyone explain the statement "The efficient property of any estimator says that the estimator is the minimum variance unbiased estimator", so what is minimum variance denotes here.
@kartikkamboj2954 жыл бұрын
Dude ! Hats off 🙏🏻
@HeduAI7 жыл бұрын
I would trade diamonds for this explanation (well, allegorically! :) ) Thank you!!
@shiva60166 жыл бұрын
simple and effective video, thank you!
@sagarsitap35404 жыл бұрын
Thanks! why lamba cannot be negative? What if to improve variance it is need to increase the slope and not decrease?
@tamoghnamaitra99017 жыл бұрын
Beautiful explanation
@faeritaaf7 жыл бұрын
Thank you! Your explaining is really good, Sir. Do you have time to make a video explaining the adaptive lasso too?
@LossAndWaste6 жыл бұрын
you are the man, keep doing what you're doing
@adityakothari1937 жыл бұрын
Excellent explanation .
@kamesh78186 жыл бұрын
Excellent explanation, thanks!
@meysamsojoudi39473 жыл бұрын
It is a brilliant video. Great
@JC-dl1qr7 жыл бұрын
great video, brief and clear.
@JoonHeeKim6 жыл бұрын
Great video. A (very minor) question: isn't it c instead of c^2 when you draw the radius vector of the circle for \beta restriction?
@Viewfrommassada4 жыл бұрын
think of it as an equation of a circle with center (0,0)
@youyangcao38378 жыл бұрын
great video, the explanation is really clear!
@SiDanil7 жыл бұрын
what the "level curve" means?
@Thaifunn18 жыл бұрын
excellent video! Keep up the great work!
@garbour4562 жыл бұрын
great video - thanks
@zhongshanhu73768 жыл бұрын
very good explanation in an easy way!!
@mnwepple8 жыл бұрын
Awesome video! Very intuitive and easy to understand. Are you going to make a video using the probit link?
@jakobforslin63012 жыл бұрын
You are awesome!
@Theateist6 жыл бұрын
Is the reason to not choose big LAMBDA because we maight get underfitting? If we choose big LAMBDA we get small W and then the output function (hypothesis) won’t reflect our data and we might see underfitting.
@sasanosia65585 жыл бұрын
Amazingly helpful. Thank you.
@zehuilin87834 жыл бұрын
Hey Ritvik, I have a question about this one, I don't really know why we are choosing the point that is far from the origin point. So which direction does the gradient descent and why? Please help me out here, thank you so much!
@ibrahimkarabayir89639 жыл бұрын
nice video , I have a question: lambda depends on c, isnt it?
@hunarahmad7 жыл бұрын
thanks for the nice explanation
@TURBOKNUL6668 жыл бұрын
great video! thank you very much.
@sergioperezmelo30905 жыл бұрын
Super clear
@tsrevo17 жыл бұрын
Sir, a question about 4:54: I understand that in tax/income example the VARIANCE of the beta0-beta1's is high, since there's an additional beta2 effecting things. However, the MEAN in the population should be the same, even with high variance, isn't it so? Thanks in advance!
@Hazit907 жыл бұрын
excellent video, thanks.
@lauraarmbrust16395 жыл бұрын
Thanks for this really helpful video! Could you explain why the independent variables in A should be standardized for Ridge and Lasso Regression?
@ronithsinha57026 жыл бұрын
Can someone explain why does Ridge regression leads to shrinkage of co-efficients but not entirely zero co-efficients, whereas Lasso causes some co-efficients to become zero entirely.
@jamiewilliams92716 жыл бұрын
Thank you so much!!!!
@kxdy8yg86 жыл бұрын
This is gold indeed!
@bnouadam4 жыл бұрын
Impressive
@ritvikmath4 жыл бұрын
Thanks !
@eDogBomb7 жыл бұрын
What is the intuition behind putting the constraint on the size of the Beta coefficient rather than the standard errors of the Beta coefficient?
@zhilingpan24867 жыл бұрын
Very clear. Thank you!
@vnpikachu46272 жыл бұрын
Amazing!!!
@myazdani29977 жыл бұрын
I love this video, really informative! Thanks a lot
@abeaumont106 жыл бұрын
Great videos thanks for making it
@1982sadaf8 жыл бұрын
How can both beta_1 and beta_2 be bounded by the same C? Are the independent variables normalized? (divided by their variance?) otherwise the scale of beta_1 and beta_2 can be drastically different.
@gueyenono7 жыл бұрын
In order to normalize data, we divide it by the standard deviation rather than the variance. And that's just the second step after subtracting the mean.
@LeCoolCroco6 жыл бұрын
also you can do apply MinMax Scaler... in sklearn for Ridge and Lasso there is "normalize" parameter. normalize : boolean, optional, default False
@AhmedAbdelrahmanAtbara6 жыл бұрын
Well, that is the whole point of regularization, isn't it? You don't want these coefficients to produce a polynomial with the exact fit of your data, i.e. over fitting; you want to have a rough fitting which can only happen when you reject any coefficients, and hence solutions, reside outside the bounded domain (the circle). The answer is you don't care how large are these values, no need to normalize them, you just reject any large that which are not subject to the constrain. If the variation is too high and not recommended to ignore then maybe the Ridge Regression is not the right regularization for your data!