The order indifference of Shapely values only makes sense from a perspective where there is perfect knowledge of what other players will do, but if you don’t have that, a party that spent a huge amount of money on a project that was almost certainly going to be wasteful and ended up being saved when by sheer happenstance another party appeared to save the project was not making good spending decisions. Similarly, many agents won’t be optimising for Shapely value, say a government which spends money on infrastructure not caring about whether it’ll be used or not just to win political points, so they don’t properly deserve a share of the gains when someone else intervenes with notifications to make the project actually effective.
I feel that this article presents Shapley value as just plain superior, when instead a combination of both Shapley value and counterfactual value will likely be a better metric. Beyond this, what you really want to use is something more like FDT where you take into account the fact that the decisions of some agents are subjunctively linked to you and that the decisions of some other agents aren’t. Even though my current theory is that very, very few agents are actually subjunctively linked to you, I suspect that thinking about problems in this fashion is likely to work reasonably well in practise (I would need to dedicate a solid couple of hours in order to be able to write out my reasons for believing this more concretely)
Hey Chris!
It was nice seeing you at the EA Hotel, and I’m glad we could talk about this. I’m writing down some of my notes from our conversations. Is there anything I’ve forgotten, or which you’d like to add?
a. What are you using Shapley values / counterfactual values for?
You might want to use different tools depending on what your goal is; three different goals migh be: Coordination / Analysis / Reward / Award.
For example, you might want a function which is easier to understand when announcing an award. If you’re rewarding a behavior, you might want to make sure you’re incentivizing the right thing.
b. The problem of choosing who to count is more complicated than I originally thought, and you should in fact exclude some agents from your calculations.
The example of: “If a bus driver falls off a cliff and Superman rescues them and brings them safely to their destination, earlier, the bus driver gets half the credit” is silly, but made the thing really crisp for me.
Hearing that, we then thought that:
Yes, the driver gets half the credit under Shapley values, but the same value as Superman under counterfactual value.
(also, if the driver distracts Superman from saving a different bus, then the driver gets 0 or negative value in both cases)
(if the driver was intelligent enough to know that Superman wasn’t doing anything important, he might actually get half the credit, but only of getting there earlier. In this scenario, had there been no Superman, the driver wouldn’t have fallen off the cliff.).
(if the driver was a paperclip maximizer who didn’t know that Superman was going to be around, then Superman should take all the credit).
So the answer would seem to be something like:
-Counting only over people who are broadly similar to you?
Who are optimizing over the same thing, or whose decisions can be changed because of yours? It seems like this is more of a case of causal, rather than subjunctive dependence.
c. Shapley values and uncertainty
How do SVs deal with uncertainty? Can you do expected value over SVs? [Yes, you can]. For example, if you have a 1% chance of a SV of 100, you can say that the E[SV] = 1. Even thought the SV formalism is more complicated than the counterfactual, it still works elegantly / is well-defined, etc.
Fair point re: uncertainty. The situation seems pretty symmetric, though: if a politician builds roads just to get votes, and an NGO steps in and does something valuable with that, the politician’s counterfactual impact is still the same as the NGO’s, so both the Shapley value and counterfactuals have that problem (?). Maybe one can exclude agents acording to how close their goals are to yours, e.g., totally exclude a paperclip maximizer from both counterfactual and Shapley value calculations, and apply order indifference to allies only (?). This is something I haven’t though about; thanks for pointing it out.
Fair point re: epistemic status. Changed my epistemic status.
“The situation seems pretty symmetric, though: if a politician builds roads just to get votes, and an NGO steps in and does something valuable with that, the politician’s counterfactual impact is still the same as the NGO’s”—true, but the NGO’s counterfactual impact is reduced when I feel it’s fairer for the NGO to be able to claim the full amount (though of course you’d never know the government’s true motivations in real life)
The order indifference of Shapely values only makes sense from a perspective where there is perfect knowledge of what other players will do, but if you don’t have that, a party that spent a huge amount of money on a project that was almost certainly going to be wasteful and ended up being saved when by sheer happenstance another party appeared to save the project was not making good spending decisions. Similarly, many agents won’t be optimising for Shapely value, say a government which spends money on infrastructure not caring about whether it’ll be used or not just to win political points, so they don’t properly deserve a share of the gains when someone else intervenes with notifications to make the project actually effective.
I feel that this article presents Shapley value as just plain superior, when instead a combination of both Shapley value and counterfactual value will likely be a better metric. Beyond this, what you really want to use is something more like FDT where you take into account the fact that the decisions of some agents are subjunctively linked to you and that the decisions of some other agents aren’t. Even though my current theory is that very, very few agents are actually subjunctively linked to you, I suspect that thinking about problems in this fashion is likely to work reasonably well in practise (I would need to dedicate a solid couple of hours in order to be able to write out my reasons for believing this more concretely)
Hey Chris! It was nice seeing you at the EA Hotel, and I’m glad we could talk about this. I’m writing down some of my notes from our conversations. Is there anything I’ve forgotten, or which you’d like to add?
a. What are you using Shapley values / counterfactual values for?
You might want to use different tools depending on what your goal is; three different goals migh be: Coordination / Analysis / Reward / Award.
For example, you might want a function which is easier to understand when announcing an award. If you’re rewarding a behavior, you might want to make sure you’re incentivizing the right thing.
b. The problem of choosing who to count is more complicated than I originally thought, and you should in fact exclude some agents from your calculations.
The example of: “If a bus driver falls off a cliff and Superman rescues them and brings them safely to their destination, earlier, the bus driver gets half the credit” is silly, but made the thing really crisp for me.
Hearing that, we then thought that:
Yes, the driver gets half the credit under Shapley values, but the same value as Superman under counterfactual value.
(also, if the driver distracts Superman from saving a different bus, then the driver gets 0 or negative value in both cases)
(if the driver was intelligent enough to know that Superman wasn’t doing anything important, he might actually get half the credit, but only of getting there earlier. In this scenario, had there been no Superman, the driver wouldn’t have fallen off the cliff.).
(if the driver was a paperclip maximizer who didn’t know that Superman was going to be around, then Superman should take all the credit).
So the answer would seem to be something like: -Counting only over people who are broadly similar to you?
Who are optimizing over the same thing, or whose decisions can be changed because of yours? It seems like this is more of a case of causal, rather than subjunctive dependence.
c. Shapley values and uncertainty
How do SVs deal with uncertainty? Can you do expected value over SVs? [Yes, you can]. For example, if you have a 1% chance of a SV of 100, you can say that the E[SV] = 1. Even thought the SV formalism is more complicated than the counterfactual, it still works elegantly / is well-defined, etc.
Fair point re: uncertainty. The situation seems pretty symmetric, though: if a politician builds roads just to get votes, and an NGO steps in and does something valuable with that, the politician’s counterfactual impact is still the same as the NGO’s, so both the Shapley value and counterfactuals have that problem (?). Maybe one can exclude agents acording to how close their goals are to yours, e.g., totally exclude a paperclip maximizer from both counterfactual and Shapley value calculations, and apply order indifference to allies only (?). This is something I haven’t though about; thanks for pointing it out.
Fair point re: epistemic status. Changed my epistemic status.
“The situation seems pretty symmetric, though: if a politician builds roads just to get votes, and an NGO steps in and does something valuable with that, the politician’s counterfactual impact is still the same as the NGO’s”—true, but the NGO’s counterfactual impact is reduced when I feel it’s fairer for the NGO to be able to claim the full amount (though of course you’d never know the government’s true motivations in real life)