Part 5-Hindi- Decision Tree Regression Indepth Intuition With Variance Reduction|Krish Naik

Decision tree builds regression or classification models in the form of a tree structure. It breaks down a dataset into smaller and smaller subsets while at the same time an associated decision tree is incrementally developed. The final result is a tree with decision nodes and leaf nodes.
github url : github.com/krishnaik06/Machin...
Subscribe @Krish Naik Channel For Data Science Videos In English.
channel link: bit.ly/3aeve4r
ML playlist in hindi: bit.ly/3NaEjJX
Stats Playlist In Hindi:bit.ly/3tw6k7d
Python Playlist In Hindi:bit.ly/3azScTI
Now I will be uplaoding videos of Data Science In Hindi.
#KrishNaik #decisiontreeregressor #decisiontreepython #prepruning
Connect with me here:
Twitter: / krishnaik06
Facebook: / krishnaik06
instagram: / krishnaik06

[@sayyedaghazal7477]

just completed your full playlist and trust me this was so so amazing. I really learnt many things here

[@kmishy]

7:55 correction : we don't call it mean square error. Here you are taking an error from mean so it is called variance. Thank you sir for giving a great explanation, I feel debt

[@Heisenberg69696]

actually without the 1/n part its mse so he was talking about that

[@listeniitians877]

rarely one helps and work hard like u for us

[@niranjanware2646]

thankyou krish bhai... very useful

[@varshabhambhani4819]

thank you for so informative videos, you're god sent🙏a tiny request, would be helpful if you could quickly walk through actual code and implication of these algorithms after explaining the model as this will help us relate more

[@shubhamkhatri7905]

Hindi looking forward for more GURUJI 🙏

[@ganeshgunjal4220]

very nice explanation sir. course ka video nahi smj aaya. yaha pe sub idea clear ho gaya. thank u so much sir.

[@avbendre]

these contents are gem

[@mrityunjayupadhyay7332]

Amazing explanation sir

[@SanjanaGupta-jt1so]

thank you so much for this easiest explanation of every topic .pls make video ANN,CNN too

[@MishraJiLive]

I am littile bit confuse that how can I know that what machine learning algorithms should i prefer for classification either KNeibhour, Logistic or Decision Tree and for same scenarios of Regression Type...please help

[@pratiknaikwade95]

Nice video sir

[@nitishkushwaha3821]

sir I think you have done a mistake ! you should use the sample instead of whole mean other wise you would always get 0 as variance reduction.

[@Sandeep-uj4kk]

Thanks for such Indepth session - could you please upload the PDF of this session in Git....would be helpful to go though .

[@sujatapatil9152]

Can you please add theory into the repo for this algorithm...

[@akshaypatil8155]

Decission tree regressor practical video not uploaded on hindi channel????? am i right? i cannot see it

[@aftab6417]

It is loaded . U can see it

[@deepeshjha9502]

sir the variance reduction for the second split will be 0.004. you have done it wrong in the video.

[@mehulgoyal5-yeariddcivilen832]

Hello Krish sir As you said the dt regressor will output the avg value then in case if input is less than 2.5 then avg will be taken ifall the input values's less than 2.5 then the output for all the values is the avg (constant) ,the error in model is so big .

[@jasanimihir4994]

As always you rocked it. I have a query, in the case of first root node of the tree, it would have all the data points?

[@harshal.rathore]

Yes, that is correct. The first root node of the decision tree regression will represents the entire dataset at the beginning of the training process.

[@Arceus948]

2:26 which software is this u r using??

[@poojansolanki9850]

I think it's Microsoft OneNote

[@Sandeep-uj4kk]

There was a mistake in calculation both the cases the variance reduction is coming as zero only as 60.8-[32.80 + 27.996] = 60.8-60.796 ( which is 60.8 if rounded off ) = 0 , hence which node to select for further distribution?

[@dr.yashveeryadav1711]

What about NLP playlist?

[@kindaeasy9797]

what about selection of feature , i mean to say why did you specifically start with years of experience

[@kavankomer3048]

CAN ANYONE PLEASE ANSWER? For the second independent variable gap, since its a categorical variable and output is continuous value, how to split it? This is not explained anywhere!!!

[@keshavaher2437]

@Krish. Which feature can we select as a root note according to this method? Or what basis we can choose that feature is a root feature? Please tell, if any video for this please share link.

[@harshal.rathore]

search for feature splitting or feature selection in decision tree regressor. In a nutshell The algorithm uses a criterion function to determine the best feature and the best split point for each node.

[@anirbankarak1936]

@Krish Naik Hindi Could you please upload video on NLP and Deep Learning as well

[@VyomKumaraes]

sir if we not convrting 140/3 =46.66666 ,and remain 140/3 then both the root became equal variance , i.e.zero

[@shubhamkhatri7905]

what if we chose the categorical variable as a root node to split ?

[@user-nz5nr9cm9l]

Suppose there are 3 features, 2 continuous and 1 categorical: age, height, gender and suppose there are 10 rows of data Step 1. The RandomForestRegression model will first pick the "age" feature and sort the dataset in ascending order based on the "age" column. Then the process that Krish Naik has shown here will be carried out by the model to see the datapoint of which row should be picked to maximize the variance reduction. Suppose out of 10 rows of data, picking the 4th datapoint and putting it inside node is giving highest Variance Reduction. Then that 4th data point will be put inside the Parent Node. Then the gini impurity of that Parent Node will be calculated. Suppose the Gini Impurity is .62 Step 2. The model will next pick the height column and sort the dataset based on the ascending order of the height column. Then again the same process which was carried out for the "age" column above will be carried out for the "height" column. Suppose the data point of the 6th row is giving highest Variance Reduction. Therefore we will keep 6th datapoint in the node and find out the Gini Impurity. Suppose the Gini Impurity of the Parent Node in this case is coming out to be .46. Step 3. The model will next pick the gender column. Here we don't need to carry out the process that is shown in this video, since this is a categorical variable. We can simply put this Categorical variable inside parent node and see what the gini impurity is coming out to be. Suppose in this case the gini impurity comes out to be .23 Since out of all the 3 features that we tried keeping in the parent node above, the Gini Impurity for the feature Gender came out to be the lowest, the Gender feature (which is categorical) will be entered inside the Parent Node. Same process will be carried out for the subsequent nodes using the remaining features. Note: Had the Gini impurity of, say, "height" feature been the lowest, we would have kept the Height feature inside the parent node instead of the Gender feature

[@soumyadeepsaha7853]

How can a single number have non zero variance? I am little bit confused.

[@subhajitdey4483]

Sir if you kindly help me to clear my doubt. 1. Sir why are taking the first independent variable (Exp) to our tree ? if there any 2nd input column then which col I will take ? 2. And next is that why we are taking 2 and 2.5 first ? after that how it will continue ? Sir I have seen this full video, and this are the doubts where I've stucked. please help. If anyone has the solution, please help me out. Thank you...🙂🙂

[@shaneebkottakkal3491]

DO you English Video of this Topic

[@HaiderAli-dm4iy]

The variance reduction in second split has come out to be 0.304 which is incorrect, as you have approximated 140/3 to 46.66, if you take 140/3 as it is the variance reduction will come out to be 0. I think, there is a problem in calculating variance at child nodes, you are using the mean of root nodes to calculate the variance at child nodes. Anyone having any idea about this, please reply

[@Garrick645]

Yeah I have same doubt...In stat quest videos, for sum of squared residuals( variance * N) method, they have used the mean of each child node to calculate the total SSR. for a parent node. They did the same for all data points in the feature 1 column , then choose the one split having least SSR. Now they did same with Feature 2,3,4 etc... and compared the least SSR of each column with each other and choose the feature having lowest SSR among all features for the split. Hope I make sense to you. Also I have a doubt, that how will we compare categorical value splits to numerical ones like in the Example Krish Sir has used

[@jackfrost8969]

this is incomplete. You didn't show how the second independent variable will be used.

[@Garrick645]

@jackfrost8969 I have same doubt, if you find the answer anywhere Goku Black, please share

[@veera.akasapu]

8:00 MISS INFORMATION on saying variance is calculated by using MSE. Variance and MSE are completely different things although their formulae look similar. 12:38 Wrong way of calculating Variance: You have calculated variance of a sample(child nodes in this case) using the mean of whole population. Variance is an average squared difference from the mean. If we have only one observation(as in child 1), we cannot do this. The mean is the observation itself and the difference is zero. We need at least two observations. More over as you have taken one numerical and one categorical variables, how would you choose which feature to select as the ROOT node?

[@Garrick645]

if you have answer to the second question on how to compare numerical and categorical features for feature split then please help me

[@zaafirc369]

Awesome video Krish! I had one doubt and if im wrong someone please correct me With regards to the calculation of the variance of the child nodes, unless mistaken, the mean of the child node should be taken and not the mean of the root note. Hence for calculation of variance of child nodes: mean(child node 1) = 40/1 = 40 var(child node 1) = 1/1 * (40 - 40)^2 = 0 mean(child node 2) = 42+52+60+56/4 = 52.50 var(child node 2) = 1/4 * ((42 - 52.5)^2 +(52 - 52.5)^2 +(60 - 52.5)^2 +(56 - 52.5)^2 ) = 44.75

[@adhirajsingh483]

same doubt with me too i dont think mean of the whole data should be taken with regards to child node , the mean of the data at child nodes should be taken .

[@vikramsingh1784]

Yes. Correct 👍👍

[@ChandanRakholia]

Yes, You are correct, He has done error 12:33

[@blindprogrammer]

You didn't explain how to select the feature on which the split will happen. You only told how the split happen on an already selected feature.

[@krishnaikhindi]

Already explained in decision tree classifier

[@blindprogrammer]

@@krishnaikhindi yes Krish, I watched that. But that is in the case of categorical features where we calculate information gain to find the feature to split on, but how to select a feature when suppose all features have continuous values. Please answer

[@krishnaikhindi]

@@blindprogrammer search for decision tree split for continuous variable u will get the video

[@blindprogrammer]

One more thing, can you please explain how the entropy formula is derived? Everywhere, I have seen the formula is directly used except the StatQuest video, but I couldn't understand much from that video. Sigma - p_i(x)*log(p_i(x) this one.

[@amolharsh7375]

@@blindprogrammer kzbin.info/www/bejne/j6XIk3qMrZJ5rtk watch the above video by StatQuest. It will help you in understanding entropy.