Xgboost Classification Indepth Maths Intuition- Machine Learning Algorithms🔥🔥🔥🔥

[@krishnaik06]

We are near 250k. Please do subscribe my channel and share with all your friends. :)

[@_curiosity...8731]

Krish Naik please make video on decisions tree pruning with mathematical details

[@ArunKumar-sg6jf]

Lgbm is Missing

[@yashkhandelwal3877]

@@tamildramaclips8548 Depends on your college. Which college with these branches are you talking about?

[@yashkhandelwal3877]

@@tamildramaclips8548 You should definitely go with ECE. Since AI DS is a very new branch there is no surety how your college would groom the students with this branch. Also your college is not a national level college. So you shouldn't take any risk. That's all my suggestion.

[@hirdhaymodi]

sir could you make any video for a roadmap of machine learning engineer??

[@animeshsharma7332]

Man, this guy is now coming in my dreams. Who else have been binge watching his channel for months?

[@gauravpatil2926]

😂😂

[@thepresistence5935]

I am learnng from him for data science

[@geekyprogrammer4831]

Same here 😂😂😂 But this man should be given nobel prize for inspiring the present and future generations!

[@gandhalijoshi9242]

I have started following his machine learning series..And it's very nice.. I am also doing data science course simultaneously . His videos are helping a lot .

[@shaelanderchauhan1963]

HAHAHAHA ! You are being haunted by Ghost Naik

[@bhavikdudhrejiya852]

Great video. Understood in depth I have jotted down the processing steps from this video: 1. We have a Data 2. Constructing base leaner 3. Base learner takes probability 0.5 & computing residual 4. Constructing Decision as per below Computing Similarity Weights: ∑(Residual)^2 / ∑P(1-P) + lambda - Computing Similarity Weight of Root Node - Computing Similarity Weight of left side decision node & its leaf node - Computing Similarity Weight of right side decision node & its leaf node Computing Gain = Leaf1 Similarity W + Leaf2 Similarity W - Root Node Similarity W - Computing Gain of Root Node & left side of decision node and its leaf node - Computing Gain of Root Node & right side of decision node and its leaf node - Computing Gain of other combination of features of decision node and its leaf node - Selecting the Root Node, Decision node and leaf node have high information gain 5. Predicting the probability = Sigmoid(log(odd) of Prediction of Base Learner + learning rate(Prediction of Decision Tree)) 6. Predicting residual = Previous residual - Predicted Probability 7. Running the iteration from point 2 to 6 and at the end of the iteration, The residual will be the minimal. 8. Test Prediction on the model of iteration have minimal residual

[@manojsamal7248]

what if there are no. of classification in output (0,1,2,3) the average will be 1.5 but this is more than 1 i.e this cant be probality which 0.5 to base learner that time what we should do..? ]

[@pawanthakur-df2yk]

Thank you🙏

[@manojrangera]

@@manojsamal7248 yes bro..same question ...did you get the answer of this?..please let me know..

[@manojsamal7248]

@@manojrangera not yet bro

[@manojrangera]

@@manojsamal7248 I was thinking if there are 4 classes then probability will be 1/4 = .25 and if there are 5 then 1/5 =.20 because we are calculating probability ..I will confirm this but I think this is right..

[@johnnyfry2]

Great work Krish. Don't ever lose your passion for teaching, you're a natural. I appreciate how you simplify the details.

[@yashkhandelwal3877]

Hats off to you Krish for doing so much hardwork so that we can learn each and every concept of ML, DataScience!

[@nareshjadhav4962]

I was desparately waiting for this since last 7 months...now I will complete mashine learning playlist💥 Than you Krish..god bless you😀

[@joeljoseph26]

Guys, please watch for the mistake. There is a mistake made at 16:10 i.e. For credit >50 (G,B) = {-0.5,0.5} its not three, there is only two. The information gain for the right side is 0.67. However, you chose the right node. Btw, your teaching very simple and understandable. Keep doing more videos. Love your content.

[@moindalvs]

Thanks a lot, for eveyrthing you do. You did turn off the fan so that it doesn't interrupt the audio, you were sweating and breathing heavily with all this trouble and hardship you deserve more. I wish you success in life and a healthy and a prosperous life.

[@amitsahoo1989]

Hi krish, i have been watching ur videos for the last few months and it has helped me a lot in my interviews. A special thanks from my end. In this video, at 10:54 min 0.33 - 0.14 should be 0.19.

[@gshan994]

yes indeed bdw were u a fresher when u went for an interview?

[@sandipansarkar9211]

Very very important to crack in product based companies.Great explantion too.Thanks

[@mohitjoshi4209]

So much to learn from a single video, hats off to you sir

[@felixzhao9070]

This is pure gold! Thanks for the tutorial!

[@yashkhant5874]

Great Explanation sir... keep contributing to the community. We love your videos and most importantly you are serving your experience is the best thing.

[@sajidchoudhary1165]

i am most happiest person to see this videos thank you

[@marijatosic217]

This was amazing, I literally feel like I'm sitting in your class at a Uni.

[@mrzaidivlogs]

How do u stay so focused , strong and learn everything in a very efficient way?

[@yasharya1066]

Nation wants to know🙃

[@HistoryUnlocked-fi3er]

Willpower

[@dhruvenkalpeshkumarparvati4874]

Just what I was waiting for 🔥

[@abhishek_maity]

Great.... Clear explanation !! Thanks a lot 😄

[@narendradamodardasmodi3286]

Thanks, Krish for building the nation Towards AI Journey.

[@ajayrana4296]

chutiya nokri bhi to de

[@shashwattiwari4346]

"Day 1 or 1 Day your Choice" Thanks a lot Krish!

[@islamicinterestofficial]

what does this mean?

[@nitinahlawat2479]

Really Data science Bisham Pitama🙏 Respect you a lot👍

[@frozen1860]

Sir the way you teaching us is more better than any varsity classes. pls do a practical implementation on XGBoost. sir pls it will be very helpful for us...

[@MrPetarap]

Lovely explanation !

[@antonym9744]

Amazing !!!

[@ShahnawazKhan-xl6ij]

Great

[@vishnukv6537]

Sir you are too pleasant and amazing in teaching

[@mohamedgaal5340]

Thank You, Krish. Well explained!

[@raneshmitra8156]

Super explanation

[@Mazree152]

16:33 In my opinion there is a mistake in calculations. It should be computed for (>50K) but G & B are also included from

[@adityagupta8901]

I also noticed that, i guess maybe that is a mistake

[@amitupadhyay6511]

its tough to understand in first attempt ,but thanks for giving the outline so clearly, I will watch it untill I understand I implement it from scratch .

[@nukulkhadse5253]

Hey Krish, you should also have a video about Similarity Based Modelling (SBM) and Multivariate State Estimation Technique (MSET). They are actually widely used in the industries since 90s. There are many research papers to validate that. They also calculate similarity weight and residuals.

[@ishitachakraborty1362]

Please do a indepth maths intuition video on catboost

[@BatBallBites]

agree

[@thisismuchbetter2194]

I don't know why people don't talk about Catboost and LightGBM much..

[@stabgan]

Congratulations on your new job in E&Y. Checked you on LinkedIn. Very impressive profile.

[@muhammadsaqib2961]

Quite amazing and clear explanation

[@datakube3053]

thank you so much

[@sheikhshah2593]

Great sir🔥🔥

[@mihirjha1486]

Loved It. Thank You!

[@Kiddzzvideos]

hi, have one doubt, for p(1-p) + lambda in denominator to calculate similarity weight, if the residual is -0.5 it should be 0.5(1-(-0.5))= .75? or the negative sign does not matter?

[@navyamokmod1317]

In the denominator, we are not taking residuals for calculation, p = probability which is 0.5

[@nothing8919]

thank you alot sir, you are my best teacher

[@ppersia18]

1st view 1st like krish sir op

[@Amansingh-tr1cf]

the most awaited video

[@ayanmullick9202]

You are legend sir.

[@bayazjafarli3867]

Hi, thank you very much for this explanation! Great video! But I have one question. In 19:39 you first wrote 0 which is the probability of first row then you added learning rate*similarity weight. My question is instead of 0 shouldn't we write 0.5 which is the average probability of first (base model). 0.5+learning rate*similarity. Please correct me if I am wrong.

[@rutvikvatsa767]

base model comes after we put the first probability (0.5) through log(odds) at bottom right corner. Hence it is 0

[@modhua4497]

Good! Could you make a video explain the difference between XGB and Gradients Boosting? Thanks

[@govind1706]

Finally !!!!

[@jainitafulwadwa8181]

The similarity score is not the output value, there is a different formula for calculating the output based on residuals, you just have to remove the square in the numerator of the similarity score function.

[@arshaachu6351]

Is there any detailed videos about Adaboost regressor and gradient boosting classifier? Please help me

[@user-rw6iw8jg2t]

XGBoost fire nehi Wild Fire ML Algorithm 🔥only thing is it needs a bit careful hyperparameter tuning to prevent overfitting. Also Random Forest is a top notch algo.

[@saptarshisanyal4869]

Statquest Light !!!! Fantastic effort though.

[@IamGaneshSingh]

This video is "pretty much important!"

[@mohittahilramani9956]

Seriously thank u so much

[@RahulKumar-hb8cl]

Sir, How will the Prob value( 0.5 for the base tree ) be updated in each tree?

[@jamalnuman]

great

[@alokranjanthakur5746]

Sir can you refer some NLP projects using python. I mean with live implementation

[@ajiths1689]

what should be the new probability value we need to consider when we are considering the second decision tree?

[@brunojosebertora7935]

Krish, I have a question: when you compute the output value you are catching the similarity weighted. I think it is incorrect for classification, isn't it? To compute the output you shouldn't square the residuals. THANKS for the video!!

[@satwikram2479]

Finally❤

[@davidd2702]

Thank you for your fabulous video! I enjoy it and understand well! Could you tell me if the output from the xgb classifier gives 'confidence' in a specific output (allowing you to assign a class) ? or is this functionally equivalent to statistical probability of an event occuring?

[@vishaldas6346]

Hi Krish, I have a doubt, can you please confirm if XGBOOST is a part of ensemble technique or not as while importing from the library we are doing it separately not from sklearn library.

[@krishnaik06]

It is a seperate library

[@vishaldas6346]

@@krishnaik06 but is it an ensemble technique?

[@gshan994]

@@vishaldas6346 what is XGBoost and where does it fit in the world of ML? Gradient Boosting Machines fit into a category of ML called Ensemble Learning, which is a branch of ML methods that train and predict with many models at once to produce a single superior output.

[@saimanohar3363]

Grt teacher. Just a doubt, can't we take the credit as first node?

[@REHAN-ANSARI-]

XG-Boost is the secret of my energy

[@nandangupta727]

Thank you so much for such a step to step explanation. but I have a quick question what would we do if we have continuous variable than categorical. would we proceed as we do in decision tree for continuous features? or it's not recommended to use XGBoost in case of continuous features?

[@thepresistence5935]

i think we use all the models and will take the result by comparing those, I think It will be better for that.

[@subratakar4392]

for continous data, like salary , first it will sort that particular column in ascending, then for each consucutive value will create an avg.Now each avg will be taken as a spliting condition. The one where the gain is the highest will be considered for the split . Like suppose you have 5 salaries 10,20,30,40,50. first splt would be on salary

[@gardeninglessons3949]

sir please make a video on differences in all the boosting techniques , they are elaborate and couldn't find out the exact differences

[@accentureprep1092]

Hi @krish First of all kudos to you Great video Can you tell me how xgboost is different from Aprori alogrithm or does it cover every combination as in Aprori cover ( ie it's covers all the combination while creating tree as Aprori will cover for same problem statement) Thanks and love your work Keep rocking

[@deepsarkar2003]

Can anyone explain to me the video during 21:38 Mins ( 0-0.6)=-0.6 right not 0.4 right? or did I get it wrong Please Advise

[@sudiptodas6272]

I got the same question .

[@Jaydonj]

yeaaa me toooooooooooo....helpppwwwww meeee!! arghhh

[@pulakdas3216]

It started good but I got lost as the video ended. Can you please prepare something simpler and show that? as u did for adaboost and gradboost?

[@ashwinkrishnan4285]

Hi Krish, I have a doubt here. Here all the input features (salary, credit) are categorical. so we are making the decision tree easily based on the categories. Say suppose if we get the salary feature as continuous like 30k, 50k and not like 50k, how this split of decision tree will be done.

[@shubhambavishi5982]

Check out decision tree algorithm video in ml playlist. Inside it, he has mentioned how to handle numerical features..

[@vishaldas6346]

Hi Ashwin, for numerical features, you have to set a threshold for each value by taking the average of adjacent values for example for 30k - 40k you have to take (30+40)/2 i.e 35k and create a decision tree by setting value less than 35k i.e

[@sohinimitra7559]

Can you please do a video on feature selection approaches? Especially the use of Mutual Information. Thanks. Great videos!!

[@tarabalam9962]

Please upload a video on Light GBM.

[@VinodRS01]

Sir how does the model chooses which similarity weight should be multiplied with learning rate . Thank you sir u r doing great by helping us🙂

[@vishaldas6346]

its not the similarity weight which is multiplied, its the Output of the leaf node. Similiraity weight is used to calculate the Gain for splitting the nodes of the decision tree.

[@edwinokwaro9944]

is the formula for similarity score of the root node correct? since this is a classification problem?

[@amitshende5161]

It's lambda as hyper parameter, which u mentioned as alpha...

[@ManoharKumar-cw3ed]

Thank you sir! I have a question in this how we predict the probability value at the begging from 0-1

[@ArunKumar-sg6jf]

How u determine value of pr in base model

[@SRAVANAM_KEERTHANAM_SMARANAM]

Dear Krish, We have a course on machine learning. Around 40000 people subscribe to this course. But since they dont understand many of them will drop out in the middle. Why dont you start creating videos parallel to what is taught in the class and make a playlist for it. So that you can easily many views with one shot. Are u interested in this.

[@datakube3053]

250k coming soon

[@durjoybhattacharya250]

How do you decide on the Learning Rate parameter?

[@dulangikanchana8237]

can you do a video difference between statistical models and machine learning models

[@adityarajora7219]

How is Pr gonna change please explain!!!!

[@hemantsharma7986]

isnt gradient boosting and xgboost same with miner difference?

[@KOTESWARARAOMAKKENAPHD]

is any other value except 0 as a hyperparameter in XGboost algorithm

[@Acumentutorial]

Wht is the role of lambda in the similarity weight here.

[@ajayrana4296]

what is similarity weight why we use it what is its advantage what is the intution behind it

[@mainakray6452]

the max_depth in xgboost for each tree is 2? plz answer ,

[@titangamezone4379]

sir please make a video on gradient boosting for classification problem

[@seniorprog9144]

Sir . krish Do you have a code that deal with more than one target ( y1,y2,.. Y is 2 columns or 3 columns . (two target , three target )

[@SwethaSubramanian-l7v]

can someone please clear the log of odds part? similarity wt=1 means that's the output but to compute that we calculate the base model output with respect to 0.5 probability, why?

[@adireddy694]

How you have calculated the probability ?? How you have got 0.5 ??

[@RishikeshGangaDarshan]

When I training data first calculate residual and create dt but here we are not able to see how it classified the point and in this it say when new data point is come I am confused in this

[@KOTESWARARAOMAKKENAPHD]

what is the need of LOG(odd) function

[@stabgan]

23:00 that's lambda not alpha, please correct that

[@belxismarquez4447]

Please subtitle the videos in Spanish. There is a community that speaks Spanish and listens to your videos

[@swethanandyala]

Hi sir @Krish Naik. What will be the initial probability when there are multiple classes....if anyone knows the answer please share...

[@sachinjaisar5776]

Shouldn't your similarity weight be 1? Residuals must be squared first before adding up.

[@pratikbhansali4086]

U didn't upload gradient boosting classification videos i. e part 3 and part 4 of gradient boosting

[@mohana4179]

Please put lgbm mathematical explanation sir

[@biplabroy1406]

someone please explain 21:34 . 0-0.6 = -0.6.. then how it is converging to actual value.

[@snegas2849]

yes -0.6,may be in hurry he would have written wrong

[@biplabroy1406]

@@snegas2849 in that case the residual became -0.5 to -0.6... But it should converge to 0. Isn't it?

[@snegas2849]

@@biplabroy1406 actually when we compute the sigmoid the answer is 0.5,not 0.6 so the residual again will be -0.5, he had written 0.6 by mistake I think and in one go it will not become 0...it constructs many dt and finally it becomes zero

[@ramnareshraghuwanshi4737]

Dude!! 3.29 residual = actual - probability? how come?

[@subhodipgiri2924]

how can we subtract probability of a value from that value. if suppose i take approvals in terms of Y and N then also their probability remains same at 0.5. but we cannot subtract 0.5 from Y or N. I did not get your concept of subtracting the probability from value.

[@naveenvinayak1088]

Krish How do u stay so focused

[@dheerendrasinghbhadauria9798]

How he is taking probability = 0.5 in the whole process. What is the calculation of that probability??