How to implement Decision Trees from scratch with Python

In the fourth lesson of the Machine Learning from Scratch course, we will learn how to implement Decision Trees. This one is a bit longer due to all the details we need to implement, but we will go through it all in less than 40 minutes.
You can find the code here: github.com/AssemblyAI-Example...
Previous lesson: • How to implement Logis...
Next lesson: • How to implement Rando...
Welcome to the Machine Learning from Scratch course by AssemblyAI.
Thanks to libraries like Scikit-learn we can use most ML algorithms with a couple of lines of code. But knowing how these algorithms work inside is very important. Implementing them hands-on is a great way to achieve this.
And mostly, they are easier than you’d think to implement.
In this course, we will learn how to implement these 10 algorithms.
We will quickly go through how the algorithms work and then implement them in Python using the help of NumPy.
▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬▬
🖥️ Website: www.assemblyai.com/?...
🐦 Twitter: / assemblyai
🦾 Discord: / discord
▶️ Subscribe: kzbin.info?...
🔥 We're hiring! Check our open roles: www.assemblyai.com/careers
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#MachineLearning #DeepLearning

[@KitkatTrading2024]

OMG, so glad I found your channel. Your way of education is super useful. Intuition, hands-on coding , what else can I expect?

[@TheWorldwideapple]

by far the best set of videos on ML algorithms from scratch and I have seen so many githubs and youtube videos on the same subject. this content is 10/10

[@samuellefischer9596]

Unbelievable how helpful this was! More than I learned in lecture

[@sherimshin9319]

I love you...💖💖 your tutorials saved my machine learning assignments....😭💖

[@Karankaran-mx3lb]

One of the best videos i watched on the subject. Great Job and Thank you for creating it. 👍🙏

[@yusmanisleidissotolongo4433]

So nice from you sharing. Thanks so much.

[@thangha9100]

thanks, i have been searching for this

[@AssemblyAI]

You're very welcome :)

[@yusmanisleidissotolongo4433]

You are just GREAT. Thanks so very much.

[@Amir7Souri]

very nice and smooth presentation

[@sunnybhatti9567]

Thanks for a clear presentation, it helped me a lot!

[@AssemblyAI]

Great to hear!

[@patbentolilarhythmking]

Very nice presentation.

[@Wizhka]

Excelente! Me tomará un par de días digerir todo lo expuesto en 37 min. pero no dudo que valdrá la pena. Gracias.

[@AssemblyAI]

🙏

[@KleinSerg]

Many thanks

[@MrThought2012]

I think there is a small error in the entropy calculation at 23:37. The formula shown uses the logarithm to the base 2, so I think np.log2() would be correct as np.log() represents the natural logarithm. Nevertheless, the Videos as well as the whole series is great! Thank you for the content!

[@anatoliyzavdoveev4252]

Thank you 🙏

[@jurnalphobia7089]

you are so cool... im still learning from the bottom

[@runggp]

wonderful tutorial! I found I enjoy watching ppl coding :)

[@AssemblyAI]

Great!

[@vidumini23]

Thank you..

[@XuyenNguyen-jb9sj]

The example in the beginning shows a dataset with mixed data types (categorical and numerical), how ever it looks like the code you provided only handle numerical data points, right?

[@chineduezeofor2481]

Great video!

[@AssemblyAI]

Thanks!

[@sinethembamakoma5485]

Super!!! 10/10

[@morzen5894]

what type of decision tree are you making ?

[@kunalkhurana7822]

you are awesome! it's just that you use basically a lot. maybe 245 times in a 30 minutes video!

[@MiroKrotky]

im learning a lot of python while copying your code

[@AssemblyAI]

That's great!

[@mahatmaalimibrahim6631]

fantastic

[@AssemblyAI]

Thank you!

[@user-lp6qb2og3t]

You give an incorrect explanation of p(X) at 4:46 - n is not the total number of nodes, but rather total number of data points in the node

[@LouisDuran]

How might this code change if I were to use the Gini Index instead of Entropy to decide on splits? Does that make sense?

[@sumanthk2781]

Same code can you do the chi square for decision tree

[@slyceryk5639]

Bruh, seeing people translate human language to programming language that easy is amazing. I want that experience too

[@anupamvashishtha1970]

Maam explain more while writing code as we are beginners

[@ajithdevadiga9939]

Great implementation, May I know why GINI index was not considered as It can give better result compared to entropy and information gain. the code complexity might have reduced. kindly share your thoughts 🙂

[@BleachWizz]

what is X and y?

[@pranavmadhyastha190]

How do i print the decision tree in the output

[@hansmcconnell1418]

thank you sm for this video!! may i ask if this is id3 based?

[@mikeoxmaul8678]

yep

[@mattayres9748]

Great!

[@AssemblyAI]

Thanks!

[@rareschiuzbaian]

Running this multiple times sometimes results in: File "", line 97, in _most_common_label value = counter.most_common(1)[0][0] IndexError: list index out of range I haven't figured out the reason yet, but I will update this when I find it.

[@icam8784]

Same for me

[@noringo46]

modify your check the stopping area. I solve with depth>self.max_depth and n_samples

[@gokul.sankar29]

Make sure the entropy is a negative sum. That was what fixed it for me

[@samuelsykora9246]

I think you need to check gain in stopping area, when gain is 0 you don't want to make split right ? If someone search for solution. This helped me.

[@bendev6807]

Super

[@marcelohuerta2727]

greeat video, is this id3 algorithm?

[@chirantanbiswas9330]

At around 4:50, I do not understand how p(x) is defined. Can some one help me out on it?

[@bhavyajain4973]

p(x) is basically probability of getting a sample with category/label x if we select a sample randomly. So its calculated as (num of all samples with label x / num of all samples)

[@wesinec]

Amazing, but only works for numeric attributes, right?

[@roonywalsh8183]

I guess classification dec trees work for non numeric attributes... regression ones for numeric

[@justinlimjg3052]

What about one for GINI

[@zinebtighremt2305]

Did you find any thing please ?

[@saahokrish7219]

predict some model not seen data

[@MohammadBahiraei]

💪💪💪👍

[@hadiashehwar9505]

You are using self.value, threshold and many other......what is self used for

[@omerisik1032]

hello ı from turkey.can you share your decision tree code with me in mail?

[@AssemblyAI]

You can find it here: github.com/AssemblyAI-Examples/Machine-Learning-From-Scratch

[@calidata]

This looks like reciting the code previously learned from somewhere, like rapping in a foreign language. I would first implement the functions, explain them and only then combine them in the class. You can't just build a class and functions, without even explaining or understanding what they do.

[@philtoa334]

[@AssemblyAI]

🙌

[@Hsnfci83]

Super

[@AssemblyAI]

Thanks!