6.6 Improvements & dealing with overfitting (L06: Decision Trees)

Sebastian's books: sebastianrasch...
This video covers some issues with decision trees (like overfitting) and discusses some improvements such as the gain ratio, pre-pruning, and post-pruning.
-------
This video is part of my Introduction of Machine Learning course.
Next video: • 6.7 Code Example Imple...
The complete playlist: • Intro to Machine Learn...
A handy overview page with links to the materials: sebastianrasch...
-------
If you want to be notified about future videos, please consider subscribing to my channel: / sebastianraschka

[@talsveta]

When you mentioned the pre-pruning technique of minimum number of data points per node, it made me think how this algorithm kind of serves as an approximation for the K-NN classifier (even though it isn't optimized for this specific task of course). But still, you learn a tree that finds you the at least K most-related data points and use majority vote for classification.

[@mehdimaboudi2703]

Is there any solution for the staircase problem of DT (which works on real datasets)?

[@SebastianRaschka]

No, I don't think there is a solution for this because it is just a side-effect of how a decision tree is split into nodes. This is related with the fact that there is no single ML algorithm that is best in all scenarios. In practice, it is probably rare to encounter a problem where you have the optimal decision boundary as an exact diagonal in some hyperspace so I wouldn't worry about it too much I guess.