4 жыл бұрын
4 жыл бұрын
4 жыл бұрын
4 жыл бұрын
4 жыл бұрын
4 жыл бұрын
5 жыл бұрын
Пікірлер
@nrted3877 3 ай бұрын
you look like brad pitt, awseome video
@ashutoshsinghai713 3 ай бұрын
Really like your all the videos, I always look for videos where things are explained with examples, as well as implementation from scratch. And All your videos fulfils my expectations. Thank you
@SebastianMantey 3 ай бұрын
Thanks! Nice to hear!
@sethwilliams501 3 ай бұрын
thank you friend
@lucutes2936 3 ай бұрын
pessimistic vs optimistic pruning?
@affanahmedkhan7362 3 ай бұрын
Phenomenal job ❤❤❤❤❤
@affanahmedkhan7362 3 ай бұрын
جزاکم اللہ خیرا بھائی
@kabeersekhri 4 ай бұрын
I am getting error 'dict' is not callable when calling getbestsplit. How do i fix that
@izb1275 8 ай бұрын
Thanks for the tutorial finally I understand pre-pruning and post-pruning!
@FloraInn 9 ай бұрын
Thank you so much for this excellent series! This helps me a lot.
@abdallahmohamed8629 10 ай бұрын
thank you you are the best teacher i have ever seen as i take long time to understand it but i got it with you clearly so don't stop please ❤
@FaktariaID 11 ай бұрын
Why we use sigmoid? Isnt on the multi class problem we use softmax? Please answer please i really confusseeee
@SebastianMantey 11 ай бұрын
Yes, that is correct, softmax would be better. However, my goal for this series was to cover just the very basics that one would need to make a neural net work. Therefore, I would consider softmax to be beyond the scope of basics since it’s a technique to improve the performance of the neural net (and not an element that you absolutely need to “just” make the neural net work).
@sachinkaushik8580 Жыл бұрын
Wonderful video, thank you so much. I tried to do decision tree from scratch from myself and faced several issues, then I watched some KZbin videos and those were too complicated. I gave up on coding decision tree by myself. but you made it super easy, the way you explained. I am extremely surprised why this video has only 146 likes, it deserves millions. Thanks a ton for your efforts!
@SebastianMantey 11 ай бұрын
I’m glad that the videos were helpful to you. And thanks for the kind words!
@tushargoel3521 Жыл бұрын
can you also provide us this sample data for decision tree
@abdulalinawrozie8070 Жыл бұрын
looks like there is no bias, how to do back-propagation of bias values ?
@tobiasdamaske1468 Жыл бұрын
Can only agree with what has been said so far. Such good content and easy to grasp explanations.
@SebastianMantey Жыл бұрын
Thanks! I appreciate it!
@dangpham1547 Жыл бұрын
Thank you so much, your contents deserve much more attention!
@luckytraderchow Жыл бұрын
exactly what i need! thank you
@SebastianMantey Жыл бұрын
Clarification on Information Gain vs Overall Entropy (17:45) The formula for Information Gain is basically like this: Information Gain = Entropy before split - weighted Entropy after split Or in other words: Information Gain = Entropy before split - Overall Entropy So, to determine the Entropy before the split, we need to calculate the following: Entropy before split = 42/130 * (-log2*42/130) + 42/130 * (-log2*42/130) + 46/130 * (-log2*46/130) = 1.584 So, the Information Gain for split 1 is: Information Gain = 1.584 - 0.646 = 0.938 And the Information Gain for split 2 is: Information Gain = 1.584 - 0.804 = 0.780 So, split 1 results in a higher Information Gain and we would chose it over split 2. Therefore, we get the same result compared to the video, where we just used Overall Entropy. The reason I decided to just use Overall Entropy and not Information Gain is because they are essentially the same. With Overall Entropy you focus on the fact that the entropy decreases from 1.584 to 0.646 after the split. And with Information Gain you focus on the fact that the entropy of 1.584 decreases by 0.646 to get an Information Gain of 0.938 after the split. In my opinion, using Overall Entropy is simply more intuitive. Additionally, it requires one less calculational step.
@relaxthegamer Жыл бұрын
:/ hey bro it seems like information gain and overall gain causes different split condition. If I use information gain the tree matches with sklearns Decision Tree algorithm (entropy).
@SebastianMantey Жыл бұрын
I don’t think that the reason you are getting different split conditions is because of the difference between information gain and overall entropy since they are essentially the same. For clarification, please have a look at my pinned comment of this video: kzbin.info/www/bejne/hZOvgpamqrF0oqssi=fWJvJ7xElXx6KJ4a&t=1048
@eladiomendez8226 Жыл бұрын
Great explanation!
@marcelohuerta2727 Жыл бұрын
is this id3 algorithm?
@SebastianMantey Жыл бұрын
Please have a look at the comment from mustajabhussain9167 for the answer.
@ryantwemlow1798 Жыл бұрын
This is so helpful, thank you for making this!
@SebastianMantey Жыл бұрын
I’m glad it was helpful!
@tobe7602 Жыл бұрын
It's seem so simple when you do it, but for me need hours to understand your work so your work is truly impressive
@SebastianMantey Жыл бұрын
Thanks! But I also spent hours before recording the video. So, you are not alone. 😉
@MDRizwanulAmin Жыл бұрын
hi can anyone tell, where has that variable "overall_entropy=999" has come from at 20:35 ???? plz
@SebastianMantey Жыл бұрын
As I said in the video, it is just an arbitrary high number. And we need it because of the if-statement that follows later within the for-loops (“if current_overall_entropy <= overall_entropy:”). We need an initial value for the variable “overall_entropy”. Otherwise, we would get an error in the first iteration of the for-loops because “overall_entropy” gets assigned/updated within the if-statement. So, in the first iteration it would not exist yet. That being said, maybe a better way of writing the code would have been to use “np.inf” from NumPy. So, that there is not this magic number. Hope that helps!
@HansPeter-lx6qk Жыл бұрын
This videos are not famous enough. So good.
@SebastianMantey Жыл бұрын
Thanks!
@IvarHilferink Жыл бұрын
switch to dark mode powerpoints.
@hauntinglycelestial Жыл бұрын
thanks a lot for those videos!! these seriously just saved my grades in data science!!
@SebastianMantey Жыл бұрын
I'm glad that they were helpful!
@hopelesssuprem1867 Жыл бұрын
could you pls explain what type of pruning is it i.e. is it cost complexity pruning like in CART or something another and why did you decide to use this method?
@SebastianMantey Жыл бұрын
I am assuming you are referring to post-pruning: As I mention at 14:44, the process is called “Reduced Error Pruning”. And I used it simply because that’s the process that was described in the book I was using, namely “Fundamentals of Machine Learning for predictive data analytics”.
@hopelesssuprem1867 Жыл бұрын
@@SebastianMantey oo, thanks. Now I've understood everything.
@hopelesssuprem1867 Жыл бұрын
Overall_metric might be used as np.inf against True/False with 1'st iteration. In my ipinion it looks clearer on an intuitive level.
@SebastianMantey Жыл бұрын
Just to clarify, do you want to set “best_overall_metric = np.inf” at the beginning of the function and then check in the if-statement if “current_overall_metric <= best_overall_metric”? I think that should work.
@hopelesssuprem1867 Жыл бұрын
@@SebastianMantey yes, you understood correctly. I've also checked and it works. Now I'm trying to cope with regression tree. I used california housing dataset and I got an accuracy in 3% but sklearn model shows near 45%. I'll be checking where's a mistake😂.
@hopelesssuprem1867 Жыл бұрын
May I ask a question: why did u split data taken the average between two unique values? Is it a rule or may I split data just have been taking every unique value? I thought that in this case we shouldn't care about type of value (categorical and so on).
@SebastianMantey Жыл бұрын
No, it’s not a rule. I just did it because I based the code on my “Decision Tree Algorithm explained” video (kzbin.info/www/bejne/hZOvgpamqrF0oqs). By the way, in part 9 (kzbin.info/www/bejne/jGnbi2icfs6HjNk) I update this function so that the potential splits are actually the unique values themselves. Hope that helps!
@hopelesssuprem1867 Жыл бұрын
@@SebastianMantey thanks for answer. Your explanation is the best I've seen and it helped me to create my own version of decision tree.
@hopelesssuprem1867 Жыл бұрын
Thank u for a great tutorial. Is it a CART version?
@SebastianMantey Жыл бұрын
I didn’t specifically look at the ID3 or CART algorithms, for example, and then tried to implement them in code. What I did was (as far as I can remember): I had a high-level understanding of how decision trees basically work. Then, I tried to explain the algorithm in simple terms and in my own words in these two videos: kzbin.info/www/bejne/jZ2qppywmbuUn7s kzbin.info/www/bejne/hZOvgpamqrF0oqs And then, finally, from the diagram that you can see at 0:19 in part 1 of this series (kzbin.info/www/bejne/r2ennqN9lLWnhJI), I created the code from scratch by myself just using this particular diagram and no other references. Hope that answers your question.
@hopelesssuprem1867 Жыл бұрын
@@SebastianMantey thanks a lot
@HanJiang-hs5di 2 жыл бұрын
Hi Sebastian, thanks for creating such useful videos. I'm also wondering what I should do if I would like to know the number of observations and entropy value of each leaf node. Thank you!
@SebastianMantey Жыл бұрын
I’m glad that you found them helpful! That’s difficult to say because it would require a lot of little changes within the code. For example, you would probably need to change how the decision tree is structured, e.g. use a dictionary where the keys are “question”, “yes_answer”, “no_answer”, “entropy” and “n_observations”. And then, you would need to make the according adjustments in the code to provide the respective values for those keys. It's not a very specific answer, but maybe it helps a little bit.
@rajvaghela4378 2 жыл бұрын
Hats off to you sir! Thank you for such a deep and detailed video for post pruning a decision tree!!
@SebastianMantey 2 жыл бұрын
Glad to hear that it was helpful!
@doggydoggy578 2 жыл бұрын
Is this classification or regression ?
@vglez8088 2 жыл бұрын
Hi Sebastian Great job! Listen, I tried your calculate_r_squared function () but there seems to be an error Basicly, python does not like your statement grid_search = pd.DataFrame() It's only when I set the grid_search name to df that the error vanishes, however then I have to set a loop for printing - in this case I write if max_depth = 6 then print(df) and I get a list of 17 items. [see below] This is not very elegant but works after all. Can you please explain explain why I get this error? Many thanks ViG df = pd.DataFrame(grid_search) df.sort_values("r_squared_val", ascending=False).head() dfs = df.groupby('max_depth') dfs_updated = [ ] for _, df in dfs: df['id'] = df.groupby(['max_depth','min_samples', 'r_squared_train', 'r_squared_val']).ngroup()+1 dfs_updated.append(df) df_new = pd.concat(dfs_updated) df_new if max_depth == 6: print(f"") print(df_new)
@SebastianMantey 2 жыл бұрын
It is hard to say without actually seeing the code. But I find it strange that the code would work just because you change the name of the variable. That doesn’t make sense to me since Python doesn’t care what the name of a variable is. Sorry!
@saisaratchandra7693 2 жыл бұрын
hidden gem!
@maheshmadhavan9486 2 жыл бұрын
Decision Tree is working fine with small dataset. Its not finishing a binary classification for data with 3 Lakhs rows. How can we reduce the iterations in determine_ best_split() ??
@SebastianMantey 2 жыл бұрын
One thing that you could do, is to reduce the number of potential splits that are determined by the “get_potential_splits” function. For example, if the number of “unique_values” for a particular feature is greater than, let’s say, a thousand, then you could write the code in such a way that you just randomly pick a certain percentage (e.g. 20%) of all those splits. This way, you (probably) won’t include the best split, but at least you will include a split that is very close to the best split. And this percentage could then become another hyper-parameter for the decision tree algorithm, where you then have to trade-off speed versus accuracy. Hope that helps!
@Girly.tomboy123 2 жыл бұрын
How to open a jupyter notebook from conda environment
@SebastianMantey 2 жыл бұрын
stackoverflow.com/questions/58068818/how-to-use-jupyter-notebooks-in-a-conda-environment
@mathanimation7563 2 жыл бұрын
why you stop making videos ,your videos is outstanding
@daljeetsinghranawat6359 2 жыл бұрын
i have watched whole series on Basics of Deep Learning Part 1-15: Coding a Neural Network from Scratch in Python......got hooked to it and completed on binge.
@SebastianMantey 2 жыл бұрын
Thanks, for the kind feedback! I appreciate that. I hope it was helpful.
@ahmadmponda3294 2 жыл бұрын
This is indeed unique style from the entire youtube i've ever watched.Excellent OO methods.
@reemnasser9105 2 жыл бұрын
I'm lucky to find your videos..Thanks for this effort :)
@redforestx7371 2 жыл бұрын
SUPERB EXPLANATION! THANK YOU!
@Guiltyass 2 жыл бұрын
What a great tutorial. But how can I visualize the random forest by plot?
@SebastianMantey 2 жыл бұрын
To be frank: I don’t know, sorry! Someone also asked it before under my Decision Tree from Scratch series and then I looked a little bit into it. But it seemed too complicated to be worth the effort.
@Bryan-eg7si 2 жыл бұрын
Awesome video . I have a school assignment and your videos are a literal life saver. Am trying to use your code for a multiple split classification problem that im working on now
@SebastianMantey 2 жыл бұрын
Thanks! I'm glad my videos were helpful.
@dunjaorevic2382 2 жыл бұрын
Really nice one! :D
@AayushRampal 2 жыл бұрын
great video, i always wondered how decision trees are taking splits. like ifunique values are 2,3,4. does it split lik. e < and > 2.5,3.5 or does it use < & > 3,4 like that. are you doing it in same way like in sklearn ?
@SebastianMantey 2 жыл бұрын
Thanks! You can do it either way. Here, I am using the middle between two unique values as a splitting point. Later on in the series (kzbin.info/www/bejne/jGnbi2icfs6HjNk), however, I actually use the unique values as splitting points. With regards to sklearn, I actually don’t know how they exactly do it.
@thedanger49 2 жыл бұрын
Hey Sebstian,you’re an amazing teacher thank you for sharing this informations with us , please can you help me ? I have a question ,how can we handle categorical columns that have many unique features ? I have a columns named « country « that have 174 unique value I want to use it to solve a classification problem using a decision tree please ,if a use on hot encoder I get too many columns and my tree became very big I can’t even see it how can I solve this problem please
@SebastianMantey 2 жыл бұрын
Thanks! As I explained in part 8 (kzbin.info/www/bejne/a5bSd5SgrJ2Hjc0), the downside of this implementation is that the algorithm is pretty slow when there are features with many different unique values. However, as you can see in the video, there is a feature with 80 different values (“LIMIT_BAL”) and the algorithm is still fast. It only gets slow for the feature “BILL_AMT1” which has over 18,000 different values. Therefore, I would assume that, in your case, the algorithm is slow because you didn’t specify a “max_depth”, for example a max depth of 5 (since you also said that the tree is very big). So, I would suggest testing out different values for the “max_depth” parameter. This way, the algorithm is faster, and you will also get a better performing tree since a tree with many layers tends to overfit the training data anyway. Hope that helps!
@supriyakumarmishra1437 2 жыл бұрын
Sir, could you please make a visual representation of the tree, it seems good when it is interactive.
@SebastianMantey 2 жыл бұрын
Please have a look at my response to the comment from "Nicholas Neo" in part 8 (kzbin.info/www/bejne/a5bSd5SgrJ2Hjc0&lc=Ugy00Z86PRw77vtZgXZ4AaABAg) since he had a similar question.