Hey Chris!
It was nice seeing you at the EA Hotel, and I’m glad we could talk about this. I’m writing down some of my notes from our conversations. Is there anything I’ve forgotten, or which you’d like to add?
a. What are you using Shapley values / counterfactual values for?
You might want to use different tools depending on what your goal is; three different goals migh be: Coordination / Analysis / Reward / Award.
For example, you might want a function which is easier to understand when announcing an award. If you’re rewarding a behavior, you might want to make sure you’re incentivizing the right thing.
b. The problem of choosing who to count is more complicated than I originally thought, and you should in fact exclude some agents from your calculations.
The example of: “If a bus driver falls off a cliff and Superman rescues them and brings them safely to their destination, earlier, the bus driver gets half the credit” is silly, but made the thing really crisp for me.
Hearing that, we then thought that:
Yes, the driver gets half the credit under Shapley values, but the same value as Superman under counterfactual value.
(also, if the driver distracts Superman from saving a different bus, then the driver gets 0 or negative value in both cases)
(if the driver was intelligent enough to know that Superman wasn’t doing anything important, he might actually get half the credit, but only of getting there earlier. In this scenario, had there been no Superman, the driver wouldn’t have fallen off the cliff.).
(if the driver was a paperclip maximizer who didn’t know that Superman was going to be around, then Superman should take all the credit).
So the answer would seem to be something like:
-Counting only over people who are broadly similar to you?
Who are optimizing over the same thing, or whose decisions can be changed because of yours? It seems like this is more of a case of causal, rather than subjunctive dependence.
c. Shapley values and uncertainty
How do SVs deal with uncertainty? Can you do expected value over SVs? [Yes, you can]. For example, if you have a 1% chance of a SV of 100, you can say that the E[SV] = 1. Even thought the SV formalism is more complicated than the counterfactual, it still works elegantly / is well-defined, etc.
Hey Chris! It was nice seeing you at the EA Hotel, and I’m glad we could talk about this. I’m writing down some of my notes from our conversations. Is there anything I’ve forgotten, or which you’d like to add?
a. What are you using Shapley values / counterfactual values for?
You might want to use different tools depending on what your goal is; three different goals migh be: Coordination / Analysis / Reward / Award.
For example, you might want a function which is easier to understand when announcing an award. If you’re rewarding a behavior, you might want to make sure you’re incentivizing the right thing.
b. The problem of choosing who to count is more complicated than I originally thought, and you should in fact exclude some agents from your calculations.
The example of: “If a bus driver falls off a cliff and Superman rescues them and brings them safely to their destination, earlier, the bus driver gets half the credit” is silly, but made the thing really crisp for me.
Hearing that, we then thought that:
Yes, the driver gets half the credit under Shapley values, but the same value as Superman under counterfactual value.
(also, if the driver distracts Superman from saving a different bus, then the driver gets 0 or negative value in both cases)
(if the driver was intelligent enough to know that Superman wasn’t doing anything important, he might actually get half the credit, but only of getting there earlier. In this scenario, had there been no Superman, the driver wouldn’t have fallen off the cliff.).
(if the driver was a paperclip maximizer who didn’t know that Superman was going to be around, then Superman should take all the credit).
So the answer would seem to be something like: -Counting only over people who are broadly similar to you?
Who are optimizing over the same thing, or whose decisions can be changed because of yours? It seems like this is more of a case of causal, rather than subjunctive dependence.
c. Shapley values and uncertainty
How do SVs deal with uncertainty? Can you do expected value over SVs? [Yes, you can]. For example, if you have a 1% chance of a SV of 100, you can say that the E[SV] = 1. Even thought the SV formalism is more complicated than the counterfactual, it still works elegantly / is well-defined, etc.