Let’s Write a Decision Tree Classifier from Scratch - Machine Learning Recipes #8

[@dabmab2624]

Why can't all professors explain things like this? My professor: "Here is the idea for decision tree, now code it"

[@exoticme4760]

agreed!

[@pauls60r]

I realized years after graduation, many professors either have received no training in teaching or have little interest in teaching, undergrads in particular. I can't say I've learned more on KZbin than I did in college but I have a whole lot of "OOOOOH, that's what my professor was talking about!" moments when watching videos like this. This stuff would've altered my life 20 years ago.

[@carol8099]

Same! I really wish they could dig more into the coding part, but they either don't cover it or don't teach coding well.

[@avijitmandal9124]

hey can someone give the link for doing pruning

[@Skyfox94]

Whilst I definitely agree, I have to say that, in order to understand algorithms like this one, you'll have to just work through them. No matter how many interesting and well thought out videos you watch, it'll always be most effective if you afterwards try and build it yourself. The fact that you're watching this in your free time shows that you are interested in the topic. That's also worth a lot. Sometimes you'll only be able to appreciate what professors taught you, after you get out of college/uni and realize how useful it would have been.

[@nbamj88]

In nearly 10 min, he explained the topic extremely well Amazing job.

[@learnsharegrowwithgh2181]

right

[@bluefiendish]

Because he knows how to write an explanation tree

[@FacadeMan]

Thanks a lot, Josh. To a very basic beginner, every sentence you say is a gem. It took me half hour to get the full meaning of the first 4 mins of the video, as I was taking notes and repeating it to myself to grasp everything that was being said. The reason I wanted to showcase my slow pace is to say how important and understandable I felt in regard to every sentence. And, it wasn't boring at all. Great job, and please, keep em coming.

[@Leon-pn6rb]

I'm curious, how did your career pan out? Still in ml?

[@learnsharegrowwithgh2181]

you are right he is

[@donking6996]

I am crying tears of joy! How can you articulate such complex topics so clearly!

[@riadhsaid3548]

Even it took me more than 30 minutes to complete & understand the video. I can not tell you how this explanation is amazing ! This is how we calculate the impurity ! PS: G(k) = Σ P(i) * (1 - P(i)) i = (Apple, Grape,Lemon) 2/5 * (1- 2/5) + 2/5 * (1- 2/5) + 1/5 *(1-1/5)= 0.4 * (0.6) + 0.4 * (0.6) + 0.2 * (0.8)= 0.24 + 0.24 + 0.16 = 0.64

[@ksenyaisavnina]

or 1 - (2/5)^2 - (2/5)^2 - (1/5)^2

[@vardhanshah8843]

Thank you very much for this explanation I went to the comment section to ask this question but you answer it very nicely.

[@cbrtdgh4210]

This is the best single resource on decision trees that I've found, and it's a topic that isn't covered enough considering that random forests are a very powerful and easy tool to implement. If only they released more tutorials!

[@georgevjose]

Finally after a year. Pls continue this course.

[@sundayagu5755]

As a beginner, this work has given me hope to pursue a career in ML. I have red and understood the concepts of Decision Tree. But the code becomes a mountain which has been levelled. Jose, thank you my brother and may God continue to increase you 🙏.

[@hbunyamin]

I have already known the concept; however, when I have to translate the concept into code ... I find it quite difficut and this video explains that smoothly. Thank you so much for the explanation!

[@techteens694]

The same case here man

[@learnsharegrowwithgh2181]

humm he is great teacher

[@gautambakliwal826]

You have saved weeks amount of work. So short yet so deep. Guys first try to understand the code then watch the video.

[@hyperealisticglass]

This single 9-minute video does a way better job than what my ML teacher did for 3 hours.

[@marklybeer9038]

I know, right? I had the same experience with an instructor. . . it was a horrible memory. Thanks for the video!

[@shreyanshvalentino]

a year later, finally!

[@WilloftheWinds]

Welcome back Josh, thought we would never get another awesome tutorial, thanks for your good work.

[@AyushGupta-kp9xf]

So much value in just 10 mins, this is Gold

[@JulitaOtusek]

I think you might confusing Information Gain and Gini Index. Information gain is reduce of entropy, not reduce of gini impurity. I almost did a mistake in my Engineering paper because of this video. But I luckily noticed different definition of information gain in a different source. Maybe it's just thing of naming but it can mislead people who are new in this subject :/

[@liuqinzhe508]

Yes. Information gain and Gini index are not really related to each other when we generate a decision tree. They are two different approaches. But overall still a wonderful video.

[@leonelp9593]

thanks for clarify this!

[@BestPromptHub]

You have no idea how your videos helped me out on my journey on Machine Learning. thanks a lot Josh you are awesome. 回复

[@leoyuanluo]

best video about decision tree thus far

[@teosurch]

Incredibly clear explanation. Thank you!

[@TomHarrisonJr]

One of the clearest and most accessible presentations I have seen. Well done! (and thanks!)

[@mindset873]

I've never seen any other channels like this. So deep and perfect.

[@aryamanful]

I don't generally comment on videos but this video has so much clarity something had to be said

[@Po-YuJuan-g9k]

Sooo dooope !!!! Helpful 🔥🔥🔥

[@rohitgavirni3400]

The script is tightly edited. Much appreciated.

[@falmanna]

Please keeps this series going. It's awesome!

[@alirezagh1456]

One of the best course i ever seed

[@ryanp9441]

so INSTRUCTIVE. thank you so much for your clear & precise explanation

[@tymothylim6550]

Thank you very much for this video! I learnt a lot on how to understand Gini Coefficient and how it is used to pick the best questions to split the data!

[@BlueyMcPhluey]

loving this series, glad it's back

[@dunstantough5134]

This video has saved my life 😆

[@anupam1]

Thanks, was really looking for this series...nice to see you back

[@BreakPhreak]

Started to watch the series 2 days ago, you are explaining SO well. Many thanks! More videos on additional types of problems we can solve with Machine Learning would be very helpful. Few ideas: traveling salesman problem, generating photos while emulating analog artefacts or simple ranking of new dishes I would like to try based on my restaurants' order history. Even answering with the relevant links/terminology would be fantastic. Also, would be great to know what problems are still hard to solve or should not be solved via Machine Learning :)

[@Sanchellios]

OH MYYYYYYYYY!!!! You're back! I'm SOSOOOOOOSOSOSOSOSOSOOO happy!

[@gautamgadipudi8213]

Thank you Josh! This is my first encounter with machine learning and you made it very interesting.

[@andrewbeatty5912]

Brilliant explanation !

[@dcarter666]

Ty

[@senyotsedze3388]

you are awesome, man! but why is it that, the second question on if the color is yellow? you separated only apple when two grapes are red. Or is it because they are already taken care of at the first false split of the node?

[@dinasamir2778]

It is great course. I hope you continue and make videos to all machine learning algorithms. Thanks Alot.

[@mingzhu8093]

Question about calculating impurity. If we do probability, we first draw data which give us probability of 0.2 then we draw label which give us another 0.2. Shouldn't the impurity be 1 - 0.2*0.2=0.96?

[@msctube45]

Thank you Josh for preparing and explaining this presentation aa well as the software to help the understanding of the topics. Great job!

[@avijitmandal9124]

do you have link for doing pruning

[@rodrik1]

best video on decision trees! super clear explanation

[@huuhieupham9059]

Thanks for your sharing. You made it easy to understand for everybody

[@बिहारीभायजी]

Amazing tutorial but confused too.. 6:22 Here it is not clear , information gain for what? 7:45 Here we are finding IG corresponding to each question at each node

[@jaydevparmar9876]

great to see you back

[@sajidbinmahamud2414]

Long time! i've been waiting for so long

[@johnstephen399]

This was awesome. Please continue this series.

[@lenaara4569]

You explained it so well. I have been struggling to get it since 2 days. great job !!

[@alehandr0s]

In the most simple and comprehensive way. Great job!

[@browneealex288]

At 8:41 He says Now the previous call returns and this node become decision node. What does that mean? How is this possible to return to the root node(false branch(upper line ))after executing the final return of the function. Please give your thoughts it will help me a lot.

[@fathimadji8570]

Excuse me, I am still not clear about how the value of 0.64 comes out, can you explain a little more?

[@csorex2376]

Can you cover Random Forest and SVM too

[@AbdulRahman-jl2hv]

thank you for such a simple yet comprehensive explanation.

[@IvanSedov-i7f]

I like your video, man. Its real simple and cool.

[@moeinhasani8718]

very useful.this the best tutorial out on web

[@stefanop.6097]

Please continue your good work! We love you!

[@slr3123]

I understood it as "when Gini Impurity of parent node is zero, Information Gain with child nodes is also zero. So we don't have to ask more question to classify." Is it right?

[@muslimbekabduganiev7483]

You are creating a question with only one value, what if I want to have a question like "Is it GREEN OR YELLOW?". So, basically, I will have to test all combinations of values of size 2 to find the best info_gain for a particular attribute. Furthermore, we could test all possible sizes of a question. Would that give a better result or is it better to use only one value of the attribute to build the question?

[@muslimbekabduganiev7483]

On top of that, why do we use binary partitioning? Can't we use the same attribute to ask a new question on the false rows, but excluding attribute values used in the true rows?

[@omarsherif88]

Awesome tutorial, many thanks!

[@guitarheroprince123]

Gosh I remember when this series first started, I knew nothing about AI or machine learning and now I'm like full on neural nets and TensorFlow. Gotta admit since I don't have formal education on ml, I don't classical models as much I understand neural nets.

[@ritikvimal4915]

well explained in such a short time

[@allthingsmmm]

Could you do an example in which the output triggers a method that changes it's self based on success or failure? An easier example, iterations increase or decrease based on probability; Or left, right up, down memorizing a maze pattern?

[@Conk-bepis]

Please cover ID 3 algorithm, explanation for CART was great!

[@aryamanful]

I have a follow up question. How did we come up with the questions. As in..how did we know we would like to ask if the diameter is > 3, why not ask if diameter > 2?

[@jakobmethfessel6226]

I thought CART determined splits solely on gini index and that ID3 uses the average impurity to produce information gain.

[@saimmehmood6936]

Would be glad to see English subtitles added to this episode as well.

[@hamza-325]

His english is very clear for me

[@gorudonu]

Was waiting for the next episode! Thank you!

[@learnsharegrowwithgh2181]

yes

[@j0kersama669]

6:22 Impurity = 0.62? How? What is the formular?

[@adampaxton5214]

Great video and such clear code to accompany it! I learned a lot :)

[@Xiaoniana]

Thank Thank's it was very informative. It took me hours to understand what is meant. Keep going

[@congliulyc]

best and most helpful tutorial ever seen! Thanks!

[@debanjandhar6395]

Awesome video, helped me lot.... Was struggling to understand these exact stuffs.....Looking forward to the continuing courses.

[@andreachristelle5359]

Clear with good English and Python explanations. So nice to find both together! Thank you!

[@houjunliu5978]

Yaaaay! Your back!

[@ricardohincapie1537]

Such a good video! I have very clear now

[@erikslatterv]

You’re back!!!

[@njagimwaniki4321]

How come at 6:20 he calls it average but doesn't divide it by 2? Also the same thing in a stack overflow question it seems to be called entropy after. Is this correct?

[@mcab2222]

perfect video on the implementation and the topic

[@Dedsman]

Why Impurity is calculated one way on 5:33 and on the code it's calculated differently? (1-(times the # of possible labels) vs 1-(# of possible labels)**2)?

[@yizhang8106]

same question..

[@ThePujjwal]

The wiki explains this one line derivation en.wikipedia.org/wiki/Decision_tree_learning#Gini_impurity

[@aseperate]

The Gino impurity function in the code does not output the same responses listed in the video. It’s quite confusing.

[@doy2001]

Impeccable explanation!

[@mrvzhao]

At first glance this almost looks like Huffman coding. Thanks for the great vid BTW!

[@guccilover2009]

amazing video!!! Thank you so much for the great lecture and showing the python code to make us understand the algorithm better!

[@shadowfox87]

This is the best tutorial on the net but this uses CART. I was really hoping to use C5.0 but unfortunately the package is only available in R. I used rpy2 to call the C50 function in Python. It would be great if there'd be a tutorial on that.

[@mohammadbayat1635]

Why Impurity is 0.62 after partitioning on "Is color green" on the left subtree?

[@panlis6243]

I don't get one thing here. How do we determine the number for the question. Like I understand that we try out different features to see which gives us the most info but how do we choose the number and condition for it?

[@adamtalent3559]

Thanks for your lovely lecture.how to catagorize more than 2 prediction classes at the same time ?

[@سميرشيخ-ب1س]

After such a long time!

[@sergior.m.5694]

Best explanation ever, thank you sir

[@uditarpit]

It is easy to find best split if data is categorical. How do split happens in a time optimized way if variable is continuous unlike color or just 2 values of diameter? Should I just run through min to max values? Can median be used here? Please suggest!!

[@MrAlekoukos]

Thanks Google Gods. Please accept my data.The tutorial was brilliant!

[@kwarnkham3836]

Love the music!

[@bhuvanagrawal1323]

Could you make a similar video on fuzzy decision tree classifiers or share a good source for studying and implementing them?

[@Yaxoi]

Great series!

[@supriyakarmakar1111]

I get lots of idea , thanks sir.But my question to you that if the data set is too large then what will i do ?

[@christospantazopoulos8049]

Excellent explanation keep it up!

[@sarrakharbach]

That was suuuuper amazing!! Thanks for the video!

[@HarpreetKaur-qq8rx]

Why is the impurity at the decision node "color=green" equal to 0.62

[@KamEt-69]

How comes that in the calculation of the GINI Impurity we remove from the impurity the square of the probability of each label?

[@arminhejazian5306]

Amazing and Tnx for sharing the code

[@MW2ONLINEGAMER100]

Thank you so much, beautifully written code too.

[@tooniatoonia2830]

I built a tree from scratch but I am stuck making a useful plot like is obtainable in sklearn. Any help?

[@nadhirafasyaghasani5568]

Impurity 4:45 Information Gain 5:56