Gradient Boosting Complete Maths Indepth Intuiton Explained| Machine Learning- Part2

[@vipindube5439]

IF you teach in this way people will become passionate about data science, thanks for your effort.

[@Arjit_IITH]

I am enrolled in an ML online course, but I was unable to understand Gradient Boosting there, but everything is cleared after watching this video. Thank you Krish Naik

[@someshjaiswal545]

Thanks for the explanation Krish. If someone wonders why Gamma_m in step 4 changed to alpha at 15:32. It is because alpha is a hyperparameter (something whose value you set), set it between 0-1 and it remains fixed through all iteration from m = {1...M}. In this case however you dont need step 3. If you dont want to set alpha by yourself, want to learn it from data itself, and that is adjusted automatically in each iteration for m={1..M}, use step 3, but from wikipedia link given in description. Modification at 15:32 is done to make things look simple I believe. Awesome explanation. Thanks Again.

[@alliewicklund6192]

You are brilliant, Krish! I've had trouble with the theory and intuition of data science before, but these videos make things so clear.

[@sandipansarkar9211]

watched it again today.Very important for interviews in product based companies

[@urmishachatterjee5127]

Thank you for making this video on gradient boosting. I am getting a better understanding of ML from your videos. Thanks a lot

[@rkaveti]

I am taking this class cs109a at Harvard and I tell you what- you beat the professor any day. So clear!

[@karunamayiholisticinc]

Thanks to Professor Leonard's Calculus classes on here that I could understand this. Great explanation. I don't think it would be as easy to get this so fast from Wikipedia. Thanks for taking out time to explain the concepts. Keep up the good work!

[@tengliyuan1988]

Thanks Krish, I cant tell how much I appreciate your sharing of the knowledge.

[@TheR4Z0R996]

Hey krish, I have a doubt, when we update the model shouldn't we multiply the base learner with gamma_m instead of the learning rate alpha? There is this little mismatch from your video and the wikipedia page. That being said, keep up the good work. You're such an amazing guy, 10x a lot.

[@krishnaik06]

Oh yes, well I missed that part..thank u for pointing it out...it helps everyone :)

[@gardeninglessons3949]

@@krishnaik06 sir can u point out the step and rectify it in the comments , thanku

[@mranaljadhav8259]

hey did you got the point means how to update the model? Here our gamma is y^ right? it's like 60+60(-10) ?

[@MrAbhiraj123]

@@mranaljadhav8259 no bro he missed that part kindly check the wiki page

[@celestialgamer360]

Thank you so much sir... I understood everything easily... i believe i can solve now complex problems...😊😊

[@SuperRia33]

Wikipedia scares me(with formulas) but Krish saves me ,Thank you for all your hardwork ,for simplifying complex things and keeping me motivated to learn!!

[@mainhashimh5017]

Krish man I'm so thankful for your work! Passionate and intelligent.

[@akhandshahi3337]

If we are euclidean distance then you are the standardization. You make our calculations very easy.

[@pramodkumargupta1824]

wow Krish, you made math equation so easy to understand that it really motives to me to look at equation in different angle. Great Job.

[@somalkant6452]

Hey Krish, Thanks a lot for the awesome explaination, just love watching your videos like m watching a TV series, continuously i can watch for 2-3 hours :) One request if time permits can we have video on LGBM and Catboost. There are no good explaination available.

[@ankitgupta1808]

awesome krish you have an amazing ability to describe complex things with ease

[@SK-ww5zf]

Krish -- Fantastic teaching! Thank you! You mention we first fit y to the independent variables, then fit the residual to the independent variables, and repeat that second step. When do we stop iterating? Will there be an iteration after which y-hat will start to deviate away from true y values, and how do we identify that?

[@utkarshsalaria3952]

Thanks a lot sIr for such clear explaination.!!!

[@anon44492]

amazing bro! i have been trying since months to get my head around this... thank you so much!

[@pushpitkumar99]

Sir you make things look so simple. Really learnt a lot from you.

[@amardeepsingh9001]

It's a good explanation. Just one thing: at last (at step 4), what you are referring to as alpha (learning rate), is a gamma(m) actually. It's an obtained coefficient for minimum loss at step 3. However, we can add alpha in a multiple of gamma there, to perform regularization. Just tried to understand from wiki ;)

[@abhinav02111987]

Thank you Krish for helping us understand many complex algorithms.

[@jewaliddinshaik8255]

Hi krish sir i am following all your videos ,easy explanations .....keep doing same ..thanks a lot sir

[@spicytuna08]

thanks. 11:30 - confusion between r and gamma.

[@ManishKumar-qs1fm]

U r doing well Sir Awesome 👍

[@satishbanka]

Very good explaination fo Gradient Boosting Complete Maths!

[@vijaymukkala27]

You are doing a great job .. This is bulletproof explanation of the whole algo .Actually you made me inspired in recording my own videos of my understanding which might help me in future ..

[@himanshubhusanrath212]

Very beautifully explained Krish

[@RajeevRanjan-u7z]

Basically, for getting minima of f(x) => we need to find x such that d(f(x))/dx = 0 or f'(x) = 0

[@priyabratamohanty3472]

Nice to see the gradinent boosting series

[@yashasvibhatt1951]

In the third sub-step, according to the formula doesn't that always makes things to 0. Since yi will be your original values, y_hat will be the residual, Fm-1(xi) will be the base estimator's value. These values makes it, 1/2(50 - (60 + (-10))) which is apparently equals to 0 and it is not for a single sample, it is for all the sample. Correct me if I am wrong.

[@subarnasubedi7938]

I exactly got the same problem if you minimize you will get y(hat)=y(bar)-60 which is 60-60

[@1pmcoffee]

Hello Krish, I have a doubt: at 14:00, you mentioned the previous value of the model as 60. But as calculated earlier in the video, the latest error was r11 that is -10. So, shouldn't we put -10 instead of 60?. A side note, I am enrolled in Applied AI course but couldn't understand this concept. You made it so much easier. Thank you so much.

[@radhakrishnapenugonda734]

If you have observed the equation clearly, Fm-1(x) is the value that obtained in the previous model. We are trying to find the gamma that minimizes the loss obtained in the present model.

[@kmnm9463]

Hi Krish, Excellent math discussion on Gradient Descent. I have one clarification and an observation. Clarification : at the start the loss function is defined as 1/2 summation y-y^. Want to know where the 1/2 came from.? Also in calculating the y(cap) in the first base model - it is also the direct average value of the initial dependent variables ( salary). This gives 60 ( the same as derivative route). Why to use derivative in the first step? Regards KM

[@saichaitanya9613]

Hi Krish, thanks for your explanation. So the column r11 is the residual we got when we subtracted y^ with actual target value,but in your explanation you said it is the output of decision tree trained with r11 as target. I am bit off here, may be I might have understood in a wrong way. Anyone can correct me :)

[@shashankbajpai5659]

The explanation is strikingly similar to StatQuest's explanation on gradient boosting.

[@sagarmunde3088]

Hi Krish. All your videos are really well explained but, can you please upload how to implement the algorithms using code also. so, it will be helpful for everyone

[@sandipansarkar9211]

Superb explanation Krish,Thanks

[@rajeshrajie1237]

Thanks Krish ...Good explanation ..Much Appreciated..

[@thepresistence5935]

nice explanation about this thankyou so much !

[@srinagabtechabs]

excellent teaching..thank u ..

[@Vignesh0206]

Awesome sir 👌✌️..also please do indepth videos for PCA too.. personally heard many people find it little difficult to understand. Please consider this as an humble request on behalf of all

[@rupeshsingh4012]

Hats off to you sir ji

[@shashirajak9997]

Hi krish. Just a request that whenever u make a video which is continuation of a video (part 2, part3 ) then plz put link of part 1 or last video related to it. This will really help . Thanks

[@sushanbastola947]

15:32 The moment when your teacher caught you dozing!

[@davidzhang4825]

Nice video. What's the connection in Step2 between (2) Fit a base learner and (3) Calculate the gamma using the argmin sum function ?

[@hiteshmalhotra183]

Thankyou sir for sharing your knowledge with us..

[@tanmoybhowmick8230]

Sir can you please show a full video on model deployment....

[@kalppanwala6439]

Wonderful !!! explained like an arrow ie. on point

[@ThePKTutorial]

Nice video please keep it up

[@chirodiplodhchoudhury7222]

Sir please makr the part 3 and part 4 of the Gradient Boosting series

[@dkm865]

Best lecture on the mathematics of gradient boosting regression. Thank you so much Krish Sir!

[@DharmendraKumar-DS]

Great explanation...but is it necessary to remember all these formulas from interview point of view?...or having understanding of concepts is enough?

[@fatmamansour8606]

excellent video

[@sohailhosseini2266]

Great work!!!

[@abhishekmaharia4837]

Thanks for the great explanation....my question is how do you select a loss function pertaining to a problem or is it like try different loss functions according to different ML models

[@MrKishor3]

hi krish, i've a doubt, you said d/dx(x^n ) is nx^n-1. so it will be d/dx(1/2(y-y^)^2)=2/2(y-y^)^2-1, but you are taking it to be 2/2(y-y^)*-1.please resolve my doubt.

[@ppsheth91]

Hey Krish, Can u please upload the remaining videos for Gradient boosting.. Thanks..

[@Vishal-rj6bn]

What i think is, learning rate is not the one in update model equation that is our multiplier gama(m). Learning rate is the one that we need to use while computing the multiplier. Since it is used to decide the rate at which we minimize the loss function.

[@mranaljadhav8259]

Can you explain how to update the model ? step 4) with that example

[@JalalUddin-xy7lf]

Excellent explansion

[@abhijeetjain8228]

thank you sir !

[@aminearbouch4764]

thank you my friend

[@talkswithRishabh]

thanks sir so much 😀

[@akashanande6725]

In step no. 4 it is gamma m instead of alpha as per Wikipedia

[@sauvikdas7755]

Hi Krish, excellent teaching. But just noticed that your expression 3 of gamma_m is different from the one you're referring to in Wikipedia (en.wikipedia.org/wiki/Gradient_boosting). According to the said reference, gamma_m is the multiplier or the "learning rate" for the additive decision tree, and for the loss function you're haven't written the entire updated function. Can you please clarify why have you written it differently?

[@srinathtripathy6664]

Thanks man . you have made my day 😊

[@khushboovyas5932]

Very informative.. thanks sir.. but i have one query here that how do we find optimal number of trees??

[@meetshah7989]

That you have to find using hyper parameter tuning

[@dheerendrasinghbhadauria9798]

In India , no research happens during masters or PhD degrees . Masters or PhD degree in india is not of much use . In such a case what should indian students do to become data scientist ??

[@jaysoni7812]

you said that there's part 2 will be come as gradient boosting classification, please make it bcz the classification of gradient boost is different compare to ada boost in ada boost it's easy but i found difficulty in gradient boost.

[@lijindurairaj2982]

thank you, was very helpful

[@shivadumnawar7741]

Thanks krish

[@subarnasubedi7938]

The last step of minimization is wrong because if you actually minimize you will get y (hat)=y(bar)-60 which is 60-60=0

[@roshankumargupta46]

3:25 Why 1/2 sir? Shouldn't it be 1/n?

[@harshavardhan3282]

Should be 1/n

[@tejasvigupta07]

It should be 1/(2n). Usually it's fine to have it as 1/n too but as you can see the loss function ,it is having power of 2 so when we will differentiate it the 2 will come forward and we can cancel it with 2 .In short we use 1/2n to make calculations simple.

[@anirudhagrawal5044]

hello krish , i have doubt regarding this video only as we use gradient descent technique and find the first order derivative of y^ we equate the equation with zero to find the local minima value for y^ but as we know gradient descent technique is a greedy technique we will never be able to reach best solution or global minima how can we implement gradient descent also and have global minima at the same time?

[@niladribiswas1211]

what is the use of gamma(m) in 3 rd step because later you changed the forth step to F(x)=Fm-1(x)+alpha*h(x),but in wiki it is gamma (multiplier) instead of alpha which makes quiet sense

[@suvarnadeore8810]

Thank you sir

[@hemantdas9546]

Great video

[@Abhishekpandey-dl7me]

wonderful explanation. please upload a video on xgboost

[@keerthi5006]

Awesome explanation. I want to know what is the better course to learn Python for data Science.

[@9604786070]

In step 4, h(x) is simply r_m i.e. residual calculated for that DT. Then why use different notation h(x)? And there should be summation over i in last term of eq.4, right?

[@avishgoswami2141]

Fantastic !!!

[@rafsunahmad4855]

Is knowing the math behind algorithm must or just knowing that how algorithms works is enough? please please please give a reply.

[@rashidquamar]

We need to step 2 for m --> 1 to M, what minimum M we should consider ?

[@SreeramPeela]

should the learning rate be fixed ahead or change over interations?

[@ganeshkharad]

that was a good explanation....

[@dheerendrasinghbhadauria9798]

Is Data Structure and Algorithms same for data science field and software developer field ?? Are OOPs & DSA of software developer field important for data science field as well ??

[@clivefernandes5435]

Well I would say maths is more important . becz most of the algorithms are already implemented in framework like sklearn , tensorflow , but if u have a good math foundation specially in statistics and linear algebra , probability that will take you a long way when reading research papers

[@nareshjadhav4962]

Very nice explained krish!...can we expect Xgboost after this or when?

[@krishnaik06]

Yes

[@16876]

thanks a lot

[@punithraj5478]

Sir videos on NLP??

[@helloworld7886]

It should be 1/n instead of 1/2 at time-stamp -3:40

[@ShashankMR]

will you start deep learning and neural network also

[@abdulbasith7665]

Where did the 1/2 came from? If considering the loss function as MSE then it should be 1/n sum((y-y_hat)**2)

[@pranabjena4438]

Could you please make a video on xgboost algorithm.

[@RoamingHeera]

shouldn't it be Gamma multiplied by h(x) in equation 3 (equation 3 on bottom right)?

[@subhadipchakraborty8997]

Could you please explain the same with a classification problem

[@willwoodward4150]

How is gamma_m calculated in step 3 used in subsequent steps?

[@anantvaid7606]

Sir, could you make a video to explain which boosting algo is suitable to appropriate scenario?

[@manishsharma2211]

Hello

[@anantvaid7606]

@@manishsharma2211 Hello bhai

[@parthsingh3473]

Hello I am first year btech student. How much maths is needed for ai. As I am average in mathematics should I choose Ai as my career option Please tell sir

[@amartyahatua]

Where are you using the gamma_m in the next step? Great tutorial.

[@satyamchatterjee1074]

exactly

[@satyamchatterjee1074]

gamma_m is used in place of learning rate

[@ArunKumar-sg6jf]

can u make video for light gbm maths intutiton please