Пікірлер
@shikhinmehrotra 2 күн бұрын
Is there a more systematic or algorithmic way to understand the cross interaction between features in ICE plots, instead of manually adding hues?
@adataodyssey 2 күн бұрын
Yes! Friedman's h-stat. It is a metric that quantifies interaction strength by comparing the PDP to the PDP under the assumption there is no interaction. I have a video on the topic: kzbin.info/www/bejne/mIuldqWAZ6ZobJo
@shikhinmehrotra 17 сағат бұрын
@@adataodyssey awesome. Thanks! Will check this out.
@mdabubakarchowdhurysunny2846 9 күн бұрын
thank you my dear can you show me another video which completed relate yearand week bilstm and it depends with many column like temperature , turbidty , ph and so on
@adataodyssey 2 күн бұрын
I will see what I can do. However, I will take a break from youtube for a while :)
@Ericfrodrigues 12 күн бұрын
Sou outra pessoa depois de conhecer Shap Values!! ----> I'm a different person after getting to know Shap Values!! Thanks a lot!
@adataodyssey 11 күн бұрын
I agree Eric! It is an interesting topic :)
@mdabubakarchowdhurysunny2846 14 күн бұрын
can show some code about LIME
@adataodyssey 14 күн бұрын
Keep an eye for the next video on Monday ;)
@jujeep3204 14 күн бұрын
I have a question, I am following on your code but I can not use this command " from artemis.interactions_methods.model_agnostic import FriedmanHStatisticMethod" although I installed the artemis library.
@adataodyssey 14 күн бұрын
Sorry about that! But it is hard to debug this based on the comment. What version are you using?
@Bokbind 10 күн бұрын
Likely solution: Try installing pyartemis instead of artemis. Someone published another package named artemis on PyPi (The Python Package Index). The package used in the video is pyartemis.
@arvinflores5316 9 күн бұрын
@@adataodyssey he installed the wrong artemis library. I made that mistake too.
@muhammadawais581 15 күн бұрын
hats off to you for such a nice explanation.
@adataodyssey 15 күн бұрын
Thanks Muhammad!
@possakornkittipipatthanapo1737 15 күн бұрын
Hi Shapley value is very amazing in various interpretation and model understanding. However, I didn't see application related to the multi model like visual language model for example CLIP. Could you please provide any explanation or reference to further research?
@adataodyssey 14 күн бұрын
Hi I'm not too familiar with this area. I think SHAP is not the best for LLMs or generative models as you are not making predictions.
@tschess7 16 күн бұрын
I am confused. You said that Machine Leaning only cares about correlations not association but should it be said "only cares about correlations not causation"?
@adataodyssey 14 күн бұрын
Yes, "causation" is correct. Thanks for pointing out the mistake
@danielsanchez-gomez566 16 күн бұрын
Excellent video. I have a concern: I'm not quite sure about the interpretation of negative values in softmax. Isn't softmax supposed to return values between 0 and 1?
@adataodyssey 14 күн бұрын
I see how the wording is confusing! They are kind of like the softmax version of logodds. You need to apply softmax to those values to get probabilities. This article might help: medium.com/towards-data-science/shap-for-binary-and-multiclass-target-variables-ff2f43de0cf4?sk=f23afbb01aa2f552d5df8c7ac6efbde0
@adataodyssey 17 күн бұрын
🚀 Free Course 🚀 Signup here: mailchi.mp/40909011987b/signup XAI course: adataodyssey.com/courses/xai-with-python/ SHAP course: adataodyssey.com/courses/shap-with-python/
@NA-ug5eq 20 күн бұрын
Amazing video. Thank you so much. I have one question please: When explaining kernelShap, what do you mean by permuting values, please? What does mean grey circles in the graph at time 2.28, please? Does permuting refer to changing features order ( this is not clear in the graph in video at 2.28) or it refers to replacing some feature values with random values? Thank in advance for your response
@adataodyssey 14 күн бұрын
Take a look at the theory videos in thius playlist. They should help :) kzbin.info/www/bejne/g4KZl3l6rM-omdE&pp=gAQBiAQB
@Felipe90820 22 күн бұрын
Great content once again! Very helpful =)
@adataodyssey 22 күн бұрын
Thanks Felipe :) glad it could help!
@Gustavo-nn7zc 22 күн бұрын
Hi @adataodyssey , great video, thanks! Is there a way to use SHAP for ARIMA/SARIMA?
@adataodyssey 22 күн бұрын
Hi Gustavo, it's been a while since I've done time series analysis. If I remember correctly, those models are "interstitially interpretable." This means you can look directly at the parameters in the model to understand how it works and don't need model-agnostic methods like SHAP. That being said, you can still apply SHAP to linear models (see the article below). So it may be useful for ARIMA but I haven't seen it applied before. medium.com/towards-data-science/8-plots-for-explaining-linear-regression-to-a-layman-489b753da696?sk=ae508ca38771f36045312a27b81ffa75
@adataodyssey 24 күн бұрын
🚀 Free Course 🚀 Signup here: mailchi.mp/40909011987b/signup XAI course: adataodyssey.com/courses/xai-with-python/ SHAP course: adataodyssey.com/courses/shap-with-python/
@wexwexexort 23 күн бұрын
Is SHAP course not available free, only the XAI?
@adataodyssey 23 күн бұрын
@@wexwexexort Hi Eredin, yes only the XAI course. You need to signup for the newsletter and then you will get a coupon sent to you :)
@wexwexexort 23 күн бұрын
Thanks for the reply and the great content
@adataodyssey 22 күн бұрын
No problem! If it helps, I have quite a comprehensive playlist of SHAP videos on YT: kzbin.info/aero/PLqDyyww9y-1SJgMw92x90qPYpHgahDLIK
@fupopanda 28 күн бұрын
Jumping between what you are explaining and yourself is distracting
@adataodyssey 28 күн бұрын
Thanks for the feedback!
@wojpaw5362 29 күн бұрын
Liked, and subscribed! Amazing content keep it up ! Can you suggest 2 or 3 data sets I could test this on?
@adataodyssey 28 күн бұрын
Thanks, Woj! You can try these. They have some interesting interactions. archive.ics.uci.edu/dataset/1/abalone www.kaggle.com/datasets/conorsully1/pdp-and-ice-plots
@makefly3305 Ай бұрын
Hi I have a question at 5:45, wanna know based on which pattern of the plot you said the "km_driven" is less equally distributed and skewed to the left? 😄
@adataodyssey Ай бұрын
I'm looking at the bars on the x-axis. This is known as a "rug plot". 10% of the dataset falls before the first bar, 20% before the second bar and so on... You can see that the bars are shifted towards the left. This means that most of the dataset has a lower km_driven value. I hope that makes sense?
@adataodyssey Ай бұрын
🚀 Free Course 🚀 Signup here: mailchi.mp/40909011987b/signup XAI course: adataodyssey.com/courses/xai-with-python/ SHAP course: adataodyssey.com/courses/shap-with-python/
@soniaspisak645 Ай бұрын
Hi, I'm struggling with explaining GRU and LSTM models with SHAP. Encouraged by your videos, I am considering buying the course, but does it cover working with 3D data? Is even possible to implement SHAP and obtain reliable plots (without flattening the data) for time-series models?
@adataodyssey Ай бұрын
Hi Sonia, unfortunately, the course focuses on tabular data and models like XGBoost, Random Forest and CatBoost. There is one lesson on SHAP for image data but it doesn't sound like that will help you much. If you are working with PyTorch, these articles might help you get started with applying SHAP: towardsdatascience.com/image-classification-with-pytorch-and-shap-can-you-trust-an-automated-car-4d8d12714eea?sk=b04dcbb8a09f049f605d2110b5c8d851 towardsdatascience.com/using-shap-to-debug-a-pytorch-image-regression-model-4b562ddef30d?sk=7eb3016839186f1ba2a6f1f105f8ff64
@santizdr Ай бұрын
Best channel to dig deep into XAI. It would be great a video about the state of art of XAI applied on LLMs.
@adataodyssey Ай бұрын
Thanks Santi! I will consider this however my interests are more in computer vision at the moment
@Brickkzz Ай бұрын
Great recommendation from the youtube algorithm! Loving the content- keep it up!
@adataodyssey Ай бұрын
Thanks Theo! Will do
@sirireddy3102 Ай бұрын
I am getting error near model.fit my data has text and numeric So can you help me resolving it
@adataodyssey 14 күн бұрын
You will probably need to SHAP text explainer
@adataodyssey Ай бұрын
🚀 Free Course 🚀 Signup here: mailchi.mp/40909011987b/signup XAI course: adataodyssey.com/courses/xai-with-python/ SHAP course: adataodyssey.com/courses/shap-with-python/
@adataodyssey Ай бұрын
🚀 Free Course 🚀 Signup here: mailchi.mp/40909011987b/signup XAI course: adataodyssey.com/courses/xai-with-python/ SHAP course: adataodyssey.com/courses/shap-with-python/
@mahsadehghan-ws1kn Ай бұрын
Thank you so much for this awesome video. When I use this code in the #Train model section, I encounter this error. What is the solution?[17:50:59] C:\buildkite-agent\builds\buildkite-windows-cpu-autoscaling-group-i-0b3782d1791676daf-1\xgboost\xgboost-ci-windows\src\data\array_interface.h:492: Unicode-7 is not supported.
@adataodyssey Ай бұрын
There could be many things going wrong. You can try creating a Python environment and downloading the XGBoost package and only the other ones necessary to train the model.
@abdelbaki8625 Ай бұрын
what is the article reference for this information i need it for my studies emergency, please
@adataodyssey Ай бұрын
kzbin.info?event=video_description&redir_token=QUFFLUhqbktFYXFNVHVzc3NsTWpaYkc4Y3l3alZ0N3dmZ3xBQ3Jtc0trX2c3WmlOUVQwYW1USmJsaDh4YnpLV191dk5tOEdnOUtnVF9vZm5BbG8yTmRTaU56RXZNSE12Nkh2MjRITUZSLUZINUNPWmM3WFRlbnVGZWlscDFLZnFOZy1Xb0JiYm1RMnlQbVU2MEJ4R0hoUmJxMA&q=https%3A%2F%2Ftowardsdatascience.com%2Ffrom-shapley-to-shap-understanding-the-math-e7155414213b%3Fsk%3D329a1f042a0167162487f7bb3f0ffd46&v=UJeu29wq7d0
@TheCsePower Ай бұрын
Would be nice if the pdp had some kind of confidence interval that varied with the feature value.
@adataodyssey Ай бұрын
That's a good idea! You might be able to use the std of the prediction around each point. It would be related to the ICE plot where a point would have a larger std if not all the individual lines follow the same trend.
@abdelbaki8625 Ай бұрын
I don't understand
@v-ba Ай бұрын
Great explanation, thank you very much
@adataodyssey Ай бұрын
Thanks!
@shubhanshisinghms7745 Ай бұрын
Can you make a video on how recruitment decision is made?
@adataodyssey Ай бұрын
Do you mean how automated decisions are made or decisions for data scientists in general?
@youmustbenewhereguy Ай бұрын
how do you know which parameter of image manipulation that will be robust for any data will be faced in the future?
@adataodyssey Ай бұрын
This is a difficult question to answer as it will depend on your problem. In general, you will need a robust dataset that includes images taken under all conditions for which the model is expected to operate. Then you can evaluate the models trained using different feature engineering methods on this dataset.
@adataodyssey Ай бұрын
🚀 Free Course 🚀 Signup here: mailchi.mp/40909011987b/signup XAI course: adataodyssey.com/courses/xai-with-python/ SHAP course: adataodyssey.com/courses/shap-with-python/
@felicebugge 2 ай бұрын
Really useful , thank you
@adataodyssey 2 ай бұрын
No problem Felice!
@slimeminem7402 2 ай бұрын
Personally, the distinction is not necessary.
@adataodyssey 2 ай бұрын
I agree :) But I did think it was important when I first got into XAI.
@graziaedumabagraham5346 2 ай бұрын
Great content.
@adataodyssey 2 ай бұрын
Thank you, Grazia!
@graziaedumabagraham5346 2 ай бұрын
Great content
@adataodyssey 2 ай бұрын
🚀 Free Course 🚀 Signup here: mailchi.mp/40909011987b/signup XAI course: adataodyssey.com/courses/xai-with-python/ SHAP course: adataodyssey.com/courses/shap-with-python/
@seanjohn6956 2 ай бұрын
just stick to the explanations no need for the jarring adlibs
@adataodyssey 2 ай бұрын
That's boring...
@yelancho 2 ай бұрын
Appreciate a lot Prof Odyssey！Shaply values is now a more clear concept in my mind!
@adataodyssey 2 ай бұрын
Thanks Ye! I'm glad you found it useful :)
@adataodyssey 2 ай бұрын
🚀 Free Course 🚀 Signup here: mailchi.mp/40909011987b/signup XAI course: adataodyssey.com/courses/xai-with-python/ SHAP course: adataodyssey.com/courses/shap-with-python/
@youtubeuser4878 2 ай бұрын
Hello. Thanks for the tutorial. Regarding your XAI and SHAP courses, is there an order to how we should take the courses. Should we take the XAI before SHAP or vice versa. Thanks
@adataodyssey 2 ай бұрын
No problem! It is better to take XAI first then SHAP. XAI covers more of the basics in the field and other useful model agnostic methods. But the SHAP course still gives some basics so it is not necessary to do the entire XAI course (or even any of it) if all you care about it learning SHAP :)
@youtubeuser4878 2 ай бұрын
@@adataodysseyAwesome. Thank you.
@innocentjoseph9084 2 ай бұрын
Excellent explanation, just what I needed. Thank you.
@adataodyssey 2 ай бұрын
I’m glad you found it useful, Innocent :)
@shazajmal9695 2 ай бұрын
Thanks Bruh! Great Content! Would be happy if you upload a video comparing Shap with LIME and Integrated Gradients. Its a hot topic rn in data science interviews.
@adataodyssey 2 ай бұрын
Thanks for the suggestion! Would this be w.r.t. computer vision models and deep learning?
@franciscoletelier5598 2 ай бұрын
Great! always clear
@adataodyssey 2 ай бұрын
That’s my goal!
@adataodyssey 2 ай бұрын
🚀 Free Course 🚀 Signup here: mailchi.mp/40909011987b/signup XAI course: adataodyssey.com/courses/xai-with-python/ SHAP course: adataodyssey.com/courses/shap-with-python/
@dantedt3931 2 ай бұрын
This is awesome!
@adataodyssey 2 ай бұрын
Thanks!
@kalebomb5018 2 ай бұрын
Thanks for the content on XAI and particularly SHAP, it's given me a good overview before I jump into the details. I have a sci-fi book recommendation for you: Hyperion and The Fall of Hyperion by Dan Simmons =) The first book is told from the perspective of 7 characters as they visit/revisit the planet of Hyperion that they've had dealings with in the past. Hyperion is a fringe planet in the Hegemony of Man, not connected via Farcaster, and thus a visit incurs significant time dilation. On the planet are artefacts from another intelligent force: the Time Tombs, a location with whacky time reversal effects, a 3 meter tall metallic creature covered in spikes known as the Shrike (which also has time manipulation abilities), and more. Identified as the only significant anomaly in the AI faction's predictions, everything seems to be converging on Hyperion as the Time Tombs open... Genuinely incredible read
@adataodyssey 2 ай бұрын
Thanks! I actually just finished a book so this is good timing :)
@ShivSingh-zv1xw 2 ай бұрын
I have recently joined a course on eXplainable Artificial Intelligence (XAI) of yours and I am interested in applying the concepts of interpretability to image data while ensuring that the model's accuracy is preserved. please do create some videos on that topic.
@adataodyssey 2 ай бұрын
You're in luck! The next course I want to create will be XAI for computer vision. So expect to see some content soon.
@ShivSingh-zv1xw 2 ай бұрын
I have recently joined a course on eXplainable Artificial Intelligence (XAI) of yours and I am interested in applying the concepts of interpretability to image data while ensuring that the model's accuracy is preserved. please do create some videos on that topic. Thank you!
@karthikeyapervela3230 2 ай бұрын
Thanks, I was recently reading a post in LinkedIn how to eliminate highly correlated features with hierarchical clustering, but that was not clear but this is much better explained.
@adataodyssey 2 ай бұрын
Thanks Karthikeya! I'm glad you found it useful. I have another video coming out tomorrow about explaining linear models.