To arrive at the 12.5% value, you were assuming that you knew with certainty that the other two teams will try to create the vaccine without you (and that they each have a 50% chance of succeeding). And I still think that under that assumption, 12.5% is the correct figure.
If I understand your reasoning correctly for why you think this is incoherent, it’s because:
If the 3 teams independently arrive at the 12.5% figure, and each use that to decide whether to proceed, then you might end up in a situation where none of them fund it, despite it being clearly worth it overall.
But in making this argument, you’ve changed the problem. The other 2 teams are now no longer funding the vaccine with certainty, they are also making decisions based on counterfactual cost-benefit. So 12.5% is no longer the right number.
To work out what the new right number is, you have to decide how likely you think it is that the other 2 teams will try to make a vaccine, and that might be tricky. Whatever arguments you think of, you might have to factor in whether the other 2 teams will be thinking similarly. But if you really do all have the same goals, and there’s only 3 of you, there’s a fairly easy solution here, which is to just talk to each other! As a group you can collectively figure out what set of actions distributed among the 3 of you will maximize the global good, and then just do those. Shapley values don’t have to come into it.
It gets more complicated if there’s too many actors involved to all get together and figure things out like this, or if you don’t all have exactly the same goals, and maybe there is scope for concepts like Shapley values be useful in those cases. And you might well be right that EA is now often in situations like these.
Maybe we don’t disagree much in that case. I just wanted to push back a bit against the way you presented Shapley values here (e.g. as the “indisputably correct way to think about counterfactual value in scenarios with cooperation”). Shapley values are not always the right way to approach these problems. For example, the two thought experiments at the beginning of Parfit’s paper I linked above are specific cases where Shapley values would leave you predictably worse off (and all decision theories will have some cases where they leave you predictably worse off).
Let’s make the problem as simple as possible; you have a simple intervention with 3 groups pursuing it. Each has an independent 50% chance of success, per superforecasters with an excellent track record, and talking about it and coordinating doesn’t help, because each one is a different approach that can’t benefit from coordination.
And I agree with you that there are cases where it’s the wrong tool—but as you said, I think “EA is now often in situations like these,” and we’re not getting the answer right!
Edit: Vasco Grilo has pointed out a mistake in the final paragraph of this comment (see thread below), as I had misunderstood how to apply Shapley values, although I think the conclusion is not affected.
If the value of success is X, and the cost of each group pursuing the intervention is Y, then ideally we would want to pick N (the number of groups that will pursue the intervention) from the possible values 0,1,2 or 3, so as to maximize:
(1-(1/2)^N) X—N Y
i.e., to maximize expected value.
If all 3 groups have the same goals, they’ll all agree what N is. If N is not 0 or 3, then the best thing for them to do is to get together and decide which of them will pursue the intervention, and which of them won’t, in order to get the optimum N. They can base their decision of how to allocate the groups on secondary factors (or by chance if everything else really is equal). If they all have the same goals then there’s no game theory here. They’ll all be happy with this, and they’ll all be maximizing their own individual counterfactual expected value by taking part in this coordination.
This is what I mean by coordination. The fact that their individual approaches are different is irrelevant to them benefiting from this form of coordination.
‘Maximize Shapley value’ will perform worse than this strategy. For example, suppose X is 8, Y is 2. The optimum value of N for expected value is then 2 (2 groups pursue intervention, 1 doesn’t). But using Shapley values, I think you find that whatever N is, the Shapley value of your contribution is always >2. So whatever every other group is doing, each group should decide to take part, and we then end up at N=3, which is sub-optimal.
Since the marginal contribution only depends on the size of the coalition, not on its specific members, the Shapley value is (note it tends to -c as n increases, as expected):
In David’s example, N = 3, and p = 0.5, so S = (7/24) V—c. For your values of V = 8, and c = 2, S = 1⁄3. This is not higher than 2.
From the formula for the Shapley value, maximising it is equivalent to maximising:
f(N)=1−(1−p)NN.
I have concluded in this Sheet the above is a strictly decreasing function of N (which tends to 0), so Shapley value is maximised for the smallest possible number of players. This makes intuitive sense, as there is less credit to be shared when N is smaller.
The smallest possible number of players is 1, in which case the Shapley value equals the counterfactual value. In reality, N is higher than 1 in expectation, because it can only be 1 or higher. So, since the Shapley value decreases with N, assuming a single player game will tend to overestimate the contribution of that single player. I think David was arguing for this.
In any case, I do not think it makes sense to frame the problem as deciding what is the best value for N. This is supposed to be the number of (“live”) agents in the problem we are trying to solve, not something we can select.
Thank you for this correction, I think you’re right! I had misunderstood how to apply Shapley values here, and I appreciate you taking the time to work through this in detail.
If I understand correctly now, the right way to apply Shapley values to this problem (with X=8, Y=2) is not to work with N (the number of players who end up contributing, which is unknown), but instead to work with N’, the number of ‘live’ players who could contribute (known with certainty here, not something you can select), and then:
N’=3, the number of ‘live’ players who are deciding whether to contribute.
With N’=3, the Shapley value of the coordination is 1⁄3 for each player (expected value of 1 split between 3 people), which is positive.
A positive Shapley value means that all players decide to contribute (if basing their decisions off Shapley values as advocated in this post), and you then end up with N=3.
Have I understood the Shapley value approach correctly? If so, I think my final conclusion still stands (even if for the wrong reasons) that a Shapley value analysis will lead to sub-optimal N (number of players deciding to participate). Since the optimal N here is 2 (or 1, which has same value).
As for whether the framing of the problem makes sense, with N as something we can select, the point I was making was that in a lot of real-world situations, N might well be something we can select. If a group of people have the same goals, they can coordinate to choose N, and then you’re not really in a game-theory situation at all. (This wasn’t a central point to my original comment but was the point I was defending in the comment you’re responding to)
Even if you don’t all have exactly the same goals, or if there’s a lot of actors, it seems like you’ll often be able to benefit by communicating and coordinating, and then you’ll be able to improve over the approach of everyone deciding independently according to a Shapley value estimate: e.g. Givewell recommending a funding allocation split between their top charities.
A positive Shapley value means that all players decide to contribute (if basing their decisions off Shapley values as advocated in this post), and you then end up with N=3
Since I was calculating the Shapley value relative to doing nothing, it being positive only means taking the action is better than doing nothing. In reality, there will be other options available, so I think agents will want to maximise their Shapley cost-effectiveness. For the previous situation, it would be:
SCE(N)=1−(1−p)NNVc.
For the previous values, this would be 7⁄6. Apparently not very high, considering donating 1 $ to GWWC leads to 6 $ of counterfactual effective donations as a lower bound (see here). However, the Shapley cost-effectiveness of GWWC would be lower than their counterfactual cost-effectiveness… In general, since there are barely any impact assessments using Shapley values, it is a little hard to tell whether a given value is good or bad.
In a single person game, or one where we’re fully aligned and cooperating, we get to choose N. We should get to the point where we’re actively cooperating, but it’s not always that easy. And in a game-theoretic situation, where we’re only in control of one party, we need a different approach than either saying we can choose where to invest last, when we can’t, and I agree that it’s more complex than Shapley values.
Shapley values are not always the right way to approach these problems. For example, the two thought experiments at the beginning of Parfit’s paper I linked above are specific cases where Shapley values would leave you predictably worse off
You are most likely aware of this, but I just wanted to clarify Shapley value is equal to counterfactual value when there is only 1 (“live”) agent. So Shapley value would not leave us predictably worse off if we get the number of agents right.
Point taken, although I think this is analogous to saying: Counterfactual analysis will not leave us predictably worse off if we get the probabilities of others deciding to contribute right.
To arrive at the 12.5% value, you were assuming that you knew with certainty that the other two teams will try to create the vaccine without you (and that they each have a 50% chance of succeeding). And I still think that under that assumption, 12.5% is the correct figure.
If I understand your reasoning correctly for why you think this is incoherent, it’s because:
If the 3 teams independently arrive at the 12.5% figure, and each use that to decide whether to proceed, then you might end up in a situation where none of them fund it, despite it being clearly worth it overall.
But in making this argument, you’ve changed the problem. The other 2 teams are now no longer funding the vaccine with certainty, they are also making decisions based on counterfactual cost-benefit. So 12.5% is no longer the right number.
To work out what the new right number is, you have to decide how likely you think it is that the other 2 teams will try to make a vaccine, and that might be tricky. Whatever arguments you think of, you might have to factor in whether the other 2 teams will be thinking similarly. But if you really do all have the same goals, and there’s only 3 of you, there’s a fairly easy solution here, which is to just talk to each other! As a group you can collectively figure out what set of actions distributed among the 3 of you will maximize the global good, and then just do those. Shapley values don’t have to come into it.
It gets more complicated if there’s too many actors involved to all get together and figure things out like this, or if you don’t all have exactly the same goals, and maybe there is scope for concepts like Shapley values be useful in those cases. And you might well be right that EA is now often in situations like these.
Maybe we don’t disagree much in that case. I just wanted to push back a bit against the way you presented Shapley values here (e.g. as the “indisputably correct way to think about counterfactual value in scenarios with cooperation”). Shapley values are not always the right way to approach these problems. For example, the two thought experiments at the beginning of Parfit’s paper I linked above are specific cases where Shapley values would leave you predictably worse off (and all decision theories will have some cases where they leave you predictably worse off).
Let’s make the problem as simple as possible; you have a simple intervention with 3 groups pursuing it. Each has an independent 50% chance of success, per superforecasters with an excellent track record, and talking about it and coordinating doesn’t help, because each one is a different approach that can’t benefit from coordination.
And I agree with you that there are cases where it’s the wrong tool—but as you said, I think “EA is now often in situations like these,” and we’re not getting the answer right!
Edit: Vasco Grilo has pointed out a mistake in the final paragraph of this comment (see thread below), as I had misunderstood how to apply Shapley values, although I think the conclusion is not affected.
If the value of success is X, and the cost of each group pursuing the intervention is Y, then ideally we would want to pick N (the number of groups that will pursue the intervention) from the possible values 0,1,2 or 3, so as to maximize:
(1-(1/2)^N) X—N Y
i.e., to maximize expected value.
If all 3 groups have the same goals, they’ll all agree what N is. If N is not 0 or 3, then the best thing for them to do is to get together and decide which of them will pursue the intervention, and which of them won’t, in order to get the optimum N. They can base their decision of how to allocate the groups on secondary factors (or by chance if everything else really is equal). If they all have the same goals then there’s no game theory here. They’ll all be happy with this, and they’ll all be maximizing their own individual counterfactual expected value by taking part in this coordination.
This is what I mean by coordination. The fact that their individual approaches are different is irrelevant to them benefiting from this form of coordination.
‘Maximize Shapley value’ will perform worse than this strategy. For example, suppose X is 8, Y is 2. The optimum value of N for expected value is then 2 (2 groups pursue intervention, 1 doesn’t). But using Shapley values, I think you find that whatever N is, the Shapley value of your contribution is always >2. So whatever every other group is doing, each group should decide to take part, and we then end up at N=3, which is sub-optimal.
Hi Toby,
I do not think this is correct. If we consider a game with N players where each has to pay c to have a probability p of achieving a value of V:
The actual contribution of a coalition with size n is:
va(n)=(1−(1−p)n)V−nc.
The marginal contribution to a coalition with size n is (note it tends to -c as n increases, as expected):
vm(n)=va(n+1)−va(n)=(1−(1−p)(n+1))V−(n+1)c−(1−(1−p)n)V+nc==p(1−p)nV−c.
Since the marginal contribution only depends on the size of the coalition, not on its specific members, the Shapley value is (note it tends to -c as n increases, as expected):
S(N)=1N∑N−1n=0vm(n)=1N(pV1−(1−p)Np−Nc)=1−(1−p)NNV−c.
In David’s example, N = 3, and p = 0.5, so S = (7/24) V—c. For your values of V = 8, and c = 2, S = 1⁄3. This is not higher than 2.
From the formula for the Shapley value, maximising it is equivalent to maximising:
f(N)=1−(1−p)NN.
I have concluded in this Sheet the above is a strictly decreasing function of N (which tends to 0), so Shapley value is maximised for the smallest possible number of players. This makes intuitive sense, as there is less credit to be shared when N is smaller.
The smallest possible number of players is 1, in which case the Shapley value equals the counterfactual value. In reality, N is higher than 1 in expectation, because it can only be 1 or higher. So, since the Shapley value decreases with N, assuming a single player game will tend to overestimate the contribution of that single player. I think David was arguing for this.
In any case, I do not think it makes sense to frame the problem as deciding what is the best value for N. This is supposed to be the number of (“live”) agents in the problem we are trying to solve, not something we can select.
Thank you for this correction, I think you’re right! I had misunderstood how to apply Shapley values here, and I appreciate you taking the time to work through this in detail.
If I understand correctly now, the right way to apply Shapley values to this problem (with X=8, Y=2) is not to work with N (the number of players who end up contributing, which is unknown), but instead to work with N’, the number of ‘live’ players who could contribute (known with certainty here, not something you can select), and then:
N’=3, the number of ‘live’ players who are deciding whether to contribute.
With N’=3, the Shapley value of the coordination is 1⁄3 for each player (expected value of 1 split between 3 people), which is positive.
A positive Shapley value means that all players decide to contribute (if basing their decisions off Shapley values as advocated in this post), and you then end up with N=3.
Have I understood the Shapley value approach correctly? If so, I think my final conclusion still stands (even if for the wrong reasons) that a Shapley value analysis will lead to sub-optimal N (number of players deciding to participate). Since the optimal N here is 2 (or 1, which has same value).
As for whether the framing of the problem makes sense, with N as something we can select, the point I was making was that in a lot of real-world situations, N might well be something we can select. If a group of people have the same goals, they can coordinate to choose N, and then you’re not really in a game-theory situation at all. (This wasn’t a central point to my original comment but was the point I was defending in the comment you’re responding to)
Even if you don’t all have exactly the same goals, or if there’s a lot of actors, it seems like you’ll often be able to benefit by communicating and coordinating, and then you’ll be able to improve over the approach of everyone deciding independently according to a Shapley value estimate: e.g. Givewell recommending a funding allocation split between their top charities.
Since I was calculating the Shapley value relative to doing nothing, it being positive only means taking the action is better than doing nothing. In reality, there will be other options available, so I think agents will want to maximise their Shapley cost-effectiveness. For the previous situation, it would be:
SCE(N)=1−(1−p)NNVc.
For the previous values, this would be 7⁄6. Apparently not very high, considering donating 1 $ to GWWC leads to 6 $ of counterfactual effective donations as a lower bound (see here). However, the Shapley cost-effectiveness of GWWC would be lower than their counterfactual cost-effectiveness… In general, since there are barely any impact assessments using Shapley values, it is a little hard to tell whether a given value is good or bad.
In a single person game, or one where we’re fully aligned and cooperating, we get to choose N. We should get to the point where we’re actively cooperating, but it’s not always that easy. And in a game-theoretic situation, where we’re only in control of one party, we need a different approach than either saying we can choose where to invest last, when we can’t, and I agree that it’s more complex than Shapley values.
Hi Toby,
Regarding:
You are most likely aware of this, but I just wanted to clarify Shapley value is equal to counterfactual value when there is only 1 (“live”) agent. So Shapley value would not leave us predictably worse off if we get the number of agents right.
Point taken, although I think this is analogous to saying: Counterfactual analysis will not leave us predictably worse off if we get the probabilities of others deciding to contribute right.
Agreed. Great point, Toby!