4 Significant Limitations of SHAP

SHAP is the most powerful Python package for understanding and debugging your machine learning models. Yet, it still has its limitations. Understanding these is critical to avoid incorrect conclusions when using the package. We explore the 4 most significant limitations of SHAP: issues with the package, feature dependencies, causal inference and human error.
*NOTE*: You will now get the XAI course for free if you sign up (not the SHAP course)
SHAP course: adataodyssey.c...
XAI course: adataodyssey.c...
Newsletter signup: mailchi.mp/409...
Read the companion article (no-paywall link): towardsdatasci...
Medium: / conorosullyds
Twitter: / conorosullyds
Mastodon: sigmoid.social...
Website: adataodyssey.com/

[@adataodyssey]

*NOTE*: You will now get the XAI course for free if you sign up (not the SHAP course) SHAP course: adataodyssey.com/courses/shap-with-python/ XAI course: adataodyssey.com/courses/xai-with-python/ Newsletter signup: mailchi.mp/40909011987b/signup

[@saremish]

I really enjoyed such a deep discussion about the clear distinction between correlation and causation!

[@adataodyssey]

Thanks Sarem! A very important concept when it comes to XAI. I am definitely guilty of jumping to causality conclusions without enough evidence.

[@Hoxle-87]

Great video series. Don’t stop making them. Maybe take another app/tool/methodology and break it into parts like you did with SHAP. Very digestible.

[@adataodyssey]

Thanks! Planning some more videos soon

[@zahrabounik3390]

WOW! Such an amazing explanation on SHAP! I really enjoyed. Thank you.

[@adataodyssey]

No problem Zahra! I'm glad you found it useful

[@shrishchandrapandey801]

Amazing work, Conor! Keep them coming. These 6 mins have helped clarify so many topics!

[@adataodyssey]

Great to hear! I’m glad I could help.

[@yijunfu6808]

best youtuber explaining SHAP I have found!

[@adataodyssey]

Thank you! I am here to help :)

[@jenilsaliya3769]

good explanation on topic , thank you sir

[@adataodyssey]

Thank you Jenil!

[@yashnarang3014]

Great video!!!

[@adataodyssey]

Thanks Yash!

[@cesarepiscopo2549]

AMAZING WORK!

[@adataodyssey]

I really appreciate that!

[@AZ-ph7gg]

Great explanation!

[@adataodyssey]

Thank you :)

[@azizjedidi1180]

Great video man. Thank you very much.

[@adataodyssey]

I’m glad you enjoyed it Aziz!

[@NA-ug5eq]

Amazing video. Thank you so much. I have one question please: When explaining kernelShap, what do you mean by permuting values, please? What does mean grey circles in the graph at time 2.28, please? Does permuting refer to changing features order ( this is not clear in the graph in video at 2.28) or it refers to replacing some feature values with random values? Thank in advance for your response

[@adataodyssey]

Take a look at the theory videos in thius playlist. They should help :) kzbin.info/www/bejne/g4KZl3l6rM-omdE&pp=gAQBiAQB

[@sasaglamocak2846]

Great video. You mentioned that KernelSHAP suffers from extrapolation if features are correlated, like other permutation based methods. What about TreeSHAP with e.g., XGBoost?

[@adataodyssey]

Hi Sasa, this is a great question. To be honest, I don't completely understand the TreeSHAP algorithm. Looking into some other literature, it seems like TreeSHAP is not effected by correlations in the same way as KernelSHAP. "KernelSHAP ignores feature dependence. ... TreeSHAP solves this problem by explicitly modeling the conditional expected prediction." Then they go on to say "While TreeSHAP solves the problem of extrapolating to unlikely data points, it does so by changing the value function and therefore slightly changes the game. TreeSHAP changes the value function by relying on the conditional expected prediction. With the change in the value function, features that have no influence on the prediction can get a TreeSHAP value different from zero." You can read more here: christophm.github.io/interpretable-ml-book/shap.html

[@sasaglamocak2846]

@@adataodyssey great, thanks for the answer

[@escolhifazeremcasa]

Is there any way to deal with limitation 2: Feature Dependencies ?

[@adataodyssey]

Often, even if you have highly correlated features, SHAP will still work. It is just important to keep in mind that it may have problems if you do have highly correlatated features. In this case, you just need to confirm the results from shap using a method that is robust to them like ALEs or simple data exploration methods.

[@tschess7]

I am confused. You said that Machine Leaning only cares about correlations not association but should it be said "only cares about correlations not causation"?

[@adataodyssey]

Yes, "causation" is correct. Thanks for pointing out the mistake

[@mdabubakarchowdhurysunny2846]

can show some code about LIME

[@adataodyssey]

Keep an eye for the next video on Monday ;)