The mathematics behind Shapley Values

Shapley values are a fair way to divide the value of a game amongst its players. We explain the mathematics behind the Shapley value formula. To understand why it is fair, we also discuss the Shapley value axioms that the formula is derived from. The formula may seem scary but you will find it has an intuitive explanation.
*NOTE*: You will now get the XAI course for free if you sign up (not the SHAP course)
SHAP course: adataodyssey.c...
XAI course: adataodyssey.c...
Newsletter signup: mailchi.mp/409...
Read the companion article (no-paywall link): towardsdatasci...
Medium: / conorosullyds
Twitter: / conorosullyds
Mastodon: sigmoid.social...
Website: adataodyssey.com/

[@adataodyssey]

*NOTE*: You will now get the XAI course for free if you sign up (not the SHAP course) SHAP course: adataodyssey.com/courses/shap-with-python/ XAI course: adataodyssey.com/courses/xai-with-python/ Newsletter signup: mailchi.mp/40909011987b/signup

[@abstractqqq]

Good job sir. Very few in the ML (industry) will go this deep. Seeing more like-minded people always feel great.

[@adataodyssey]

Thank you! I always feel a bit uncomfortable if I don't have some sort of an understanding of the theory :)

[@santizdr]

Best channel to dig deep into XAI. It would be great a video about the state of art of XAI applied on LLMs.

[@adataodyssey]

Thanks Santi! I will consider this however my interests are more in computer vision at the moment

[@HenryCagnini]

That's a great video to explain the concept of Shapley values. Many thanks!

[@adataodyssey]

Thanks Henry! I’m glad you found it useful :)

[@helenli615]

Amazing job explaining this! I'm using it for macroeconomic analysis and thank you so much!

[@adataodyssey]

Glad I could help :)

[@umarkhan-hu7yt]

Dear Odyssey you are doing great. Keep continue and hit hard on all XAI models for a layman.

[@adataodyssey]

Thank you Umar! Will do :)

[@mugiwxrx6282]

thank you sir, that seems more clear in my mind !

[@adataodyssey]

I’m glad it helped!

[@v-ba]

Great explanation, thank you very much

[@adataodyssey]

Thanks!

[@muhammadawais581]

hats off to you for such a nice explanation.

[@adataodyssey]

Thanks Muhammad!

[@shadmohammed618]

Great explanation, thanks very much. 🙂

[@adataodyssey]

No problem Shad! I’m glad you found it useful

[@yelancho]

Appreciate a lot Prof Odyssey！Shaply values is now a more clear concept in my mind!

[@adataodyssey]

Thanks Ye! I'm glad you found it useful :)

[@miguelgarciaortegon]

Great explanation, thank you!

[@adataodyssey]

I'm glad you found it useful Miguel :)

[@Ericfrodrigues]

Sou outra pessoa depois de conhecer Shap Values!! ----> I'm a different person after getting to know Shap Values!! Thanks a lot!

[@adataodyssey]

I agree Eric! It is an interesting topic :)

[@silver_soul98]

Bro that was a nice explanation. thanks so much.

[@adataodyssey]

No problem :) I’m glad it was useful

[@abdelbaki8625]

I don't understand

[@smithanair787]

Great video! Can you make a video on how exactly the Kernel SHAP and TreeSHAP works?

[@adataodyssey]

Thank you Smitha! I will consider that. But, to be honest, it will take me some time to fully understand the algorithms first.

[@smithanair787]

@@adataodyssey Thank you!

[@adataodyssey]

@@smithanair787 by the way, the course goes into a bit more detail on the difference between kernelSHAP and treeSHAP. Otherwise you might find this article helpful: towardsdatascience.com/kernelshap-vs-treeshap-e00f3b3a27db

[@yaelefrat7677]

Amazing Video! Thank you very much! I do have one question regarding the coalition values of 3 players game: if the value of the prize (first, second, or third) can only be obtain when there is a coalition of 3, then what is the meaning of a coalition of 2 players: C_12, C_13, C_23 ?

[@adataodyssey]

These are the coalition values (i.e. prize money) if only the 2 players were in a team. To get these, I said "we need to go back in time a few more times". This is so that each alternative team could play the game and win prize money. Obviously, this is not possible. In reality, we must estimate these values. So, the actual prize money of $10000 must be split across the 3 players. However, to find a fair way to divide the money we need to know how much players would have won in alternative teams of 2 or less members.

[@civilengineeringonlinecour7143]

Awesome lecture

[@adataodyssey]

@@civilengineeringonlinecour7143 thanks glad it could help!

[@adataodyssey]

@@civilengineeringonlinecour7143 I’m glad you found it useful!

[@dennisestenson7820]

I really don't like that this subject is presented and studied as "games" when the underlying math is so incredibly enlightening and important.

[@adataodyssey]

This is a term that comes from "game theory". Rest assured that the "games" it deals with are very serious! Perhaps the example I've chosen is a bit silly but I was hoping that it would help the target audience relate to the concepts :)

[@elenagolovach384]

Thanks very much

[@adataodyssey]

No problem, Elena!

[@Empobaer]

Great video, formula was well explained! Though, I do have a question. What is the Intuition behind P(C1-C0) = 2/6 at 7:11 ? Because intuitively I would have thought there is only one way how player 1 can start its new coalition and thus I would have thought the weight should be 1/6. I see that if we look into the formula on how to calculate the weights, we obtain (p-|S|-1)!=2 and thus (1*2)/6=1/3, but I still miss the intuition behind why we need the (p-|Sl-1)!. Where am I thinking wrong?

[@adataodyssey]

Great question! Keep in mind that to receive the full value of the game all 3 players need to participate. So, after P1 joins, there are 2 ways for the full coalition to form -- P2 joins then P3 or P3 joins then P2. In other words, there will be 2 scenarios where P1 makes a marginal contribution to a team of no players. Does that make sense?

[@adataodyssey]

If not, then this article may help. It calculates the values in a slightly different way which may be more intuitive to you. www.analyticsvidhya.com/blog/2019/11/shapley-value-machine-learning-interpretability-game-theory/

[@Empobaer]

​@@adataodyssey Thank you for your fast reply, indeed the blog post was very useful for an intuitive understanding! Anyways your series on SHAP and Shapley values was a very helpful introduction. Now I only need to fully understand Kernel SHAP, which will probably take a bit longer :)!

[@lakshman587]

The time machine thing really got me hahaha!! I was wondering how can individual values be calculated!! Thanks for clear explanation!!

[@adataodyssey]

No problem Lakshman! Are there any other related concepts you're interested in learning about?

[@lakshman587]

@@adataodyssey I would like to learn about ChatGPT like how transformers work.

[@adataodyssey]

@@lakshman587 This is a bit out of my comfort zone tbh. My content is more aimed towards computer vision and explainable AI. I was considering doing a tutorial on how you can use the GPT API though!

[@lakshman587]

@@adataodyssey Ok No problem, can we have a video about how diverse counterfacuals work under the hood, we currently are using DiCE package from interpretml repo. I would like to know how these counterfacuals are getting generated!

[@adataodyssey]

@@lakshman587 will look into that!

[@rajivh4760]

Great video!!! Why do we consider the players joining in sequence. Why not just use the unique ways in which the teams can be formed?

[@adataodyssey]

You need to calculate the average marginal contribution of a player. The marginal contribution will change depending on which team members are in the team before the player joins. So you need to consider all possible sequences in which the players can join to form the team.

[@QindeelZahra]

Thanks for the video really useful, i am facing issue in understanding how you calculated the value for Player (P2) in your companion article. A little explanation on the weights would help there as i am struggling to get the value $3750. Thanks

[@adataodyssey]

Embrase the struggle! Will you will learn more through trying to solve it :)

[@hasnainayub2369]

Great explanation! I have a question though. Why don't the shap values for each feature (from the waterfall plot) add up to the predicted output at that particular observation?

[@adataodyssey]

They do if you also add the average prediction across all the instances in the dataset: f(x) = E[f(x)] + sum(shap values) You can see the average prediction, i.e. E[f(x)], on the bottom of the waterfall plot :)

[@hasnainayub2369]

Got it ! Thanks mate @@adataodyssey

[@QuantizedFields]

This is a very good explanation. However, I found it a bit confusing when you were referring to "Player-1" as "You". Because it is not clear who I am from the animation, am I "Player-1" or "Player-2" ? It would be better if you simply refer to the animation/picture and say "Player-1" or "Player-2" instead of "You". Thanks for your great work!

[@adataodyssey]

Thanks for the input Daniya! When it comes to technical content, it is difficult to strike a balance between making it interesting and easy to understand. My goal was to get the audience engaged but I see how this can be confusing.

[@adityababel3998]

Hello, can you make a video explaining the calculation of TreeSHAP??

[@adataodyssey]

Hi Aditya, this is already on my list of videos to do! I want to make a video about both Kernel SHAP and tree SHAP that go more in-depth into the algorithms.

[@hasnainayub2369]

Could you please explain P(C1-C0) = 1/3 ? (at 7:12). The rest is very well explained.

[@adataodyssey]

Remember, to get the prize money all players must eventually join the coalition. So, there are 2 ways that P1 can contribute to a coalition with no players (i.e. C0): - P1 joins then P2 then P3 - P1 joins the P3 then P2 So they make the marginal contribution C1 - C0 in 2/6 = 1/3 ways the coalition of 3 players can form.

[@hasnainayub2369]

@@adataodyssey thank you so much ! How did i not see that :/

[@adataodyssey]

@@hasnainayub2369 don't stress! It took me forever to understand

[@cheeseybox]

Ur a legend

[@adataodyssey]

Coming from you cheese vision, I take that as a great compliment!

[@TheOzpad]

Lekker vid

[@abdelbaki8625]

what is the article reference for this information i need it for my studies emergency, please

[@adataodyssey]

kzbin.info?event=video_description&redir_token=QUFFLUhqbktFYXFNVHVzc3NsTWpaYkc4Y3l3alZ0N3dmZ3xBQ3Jtc0trX2c3WmlOUVQwYW1USmJsaDh4YnpLV191dk5tOEdnOUtnVF9vZm5BbG8yTmRTaU56RXZNSE12Nkh2MjRITUZSLUZINUNPWmM3WFRlbnVGZWlscDFLZnFOZy1Xb0JiYm1RMnlQbVU2MEJ4R0hoUmJxMA&q=https%3A%2F%2Ftowardsdatascience.com%2Ffrom-shapley-to-shap-understanding-the-math-e7155414213b%3Fsk%3D329a1f042a0167162487f7bb3f0ffd46&v=UJeu29wq7d0

[@mathieucordier9248]

Does Shap Value is adapted for imbalanced data set ? Because one assumption is that we consider equality of chance for players combination (at 6:00).

[@adataodyssey]

That's a good question! I haven't really thought about that. In ML we don't assume equal chance for all feature values. We use the empirical distributions of the features. So you don't have to worry about that assumption for unbalanced features. For unbalanced targets, I'd say you should be fine as long as the model is still making accurate predictions. E.g. if it is always predicting one class then the SHAP values won't be meaningful.

[@seanjohn6956]

just stick to the explanations no need for the jarring adlibs

[@adataodyssey]

That's boring...