The counterfactual impact of agents acting in concert
Summary: Consider some project worked on by multiple organisations A, B, C and D. The benefit of the project is x. Each of the organisations is a necessary condition of the benefit x. The counterfactual impact of A is x; the counterfactual impact of B is x; etc. Despite this, the counterfactual impact of A, B, C, and D acting together is not 4*x, rather it is x. This seems paradoxical but isn’t. This is relevant in numerous ways to EAs.
Cross-posted from my blog.
Much of the time when organisations are working to produce some positive outcome, no single organisations would have produced the benefit acting on their own. Usually, organisations work in concert to achieve valuable outcomes and in many cases the outcome would not have been produced if some of the organisations taken individually had not acted as they did. This is a particularly pervasive feature of policy advocacy and EA advocacy. This gives rise to apparent paradoxes in the assessment of counterfactual impact.
For example, suppose Victoria hears about EA through GWWC and wouldn’t have heard about it otherwise. She makes the pledge and gives $1m to ACE charities, which she wouldn’t have found otherwise (and otherwise would have donated to a non-effective animal charity let’s suppose). Who counterfactually produced the $1m donation benefit: Victoria, GWWC or ACE? Each of them is a necessary condition for the benefit: if Victoria hadn’t acted, then the $1m wouldn’t have been donated; if GWWC hadn’t existed, then the $1m wouldn’t have been donated; and if ACE hadn’t existed then the $1m wouldn’t have been donated effectively. Therefore, Victoria’s counterfactual impact is $1m to effective charities, GWWC’s counterfactual impact is $1m to effective charities, and ACE’s impact is $1m to effective charities.
Apparent paradox: doesn’t this entail that the aggregate counterfactual impact of Victoria, GWWC and ACE is $3m to effective charities? No. When we are assessing the counterfactual impact of Victoria, GWWC and ACE acting together, we now ask a different question to the one we asked above viz. “if Victoria, GWWC and ACE had not acted, what benefit would there have been?”. This is a different question and so gets a different answer: $1m. The difference is that when we are assessing the counterfactual impact of Victoria acting, the counterfactual worlds we compare are
Actual World: Victoria, GWWC, and ACE all act.
Counterfactual world (Victoria): Victoria doesn’t act, GWWC and ACE act as they would have done if Victoria had not acted.
The benefits in the Actual World are +$1bn compared to the Counterfactual World (Victoria). We take the same approach for each actor, changing what needs to be changed. In contrast, when assessing the collective counterfactual impact of the actors, we ask:
Actual World: Victoria, GWWC, and ACE all act.
Counterfactual world (Victoria+GWWC+ACE): Victoria, GWWC and ACE don’t act.
When multiple actors are each a necessary condition for some outcome, it is inappropriate to sum the counterfactual impact of actors taken individually to produce an estimate of the collective counterfactual impact of the actors.
[Edit: adding this to further clarify the point, as maybe the point wasn’t clear enough.]
To assess Vic’s counterfactual impact, we compare to the world in which Vic doesn’t act, and GWWC and ACE act as they would have done had Vic not acted.
To assess GWWC’s counterfactual impact we compare to the world in which GWWC doesn’t exist, and ACE and Vic act as they would have done had GWWC not existed.
To assess ACE’s counterfactual impact, we compare to the world in which ACE doesn’t exist, and GWWC and Vic act as they would have done had ACE not existed.
This is the correct way to assess these agents’ counterfactual impact. The principle that we can aggregate them is wrong and so the absurd conclusion about their collective impact that follows is not a reductio of that method of measuring counterfactual impact. The absurd conclusion is a reductio of the aggregation principle. We cannot aggregate the counterfactual impact taken individually because then we would be comparing the actual world to the world in which “Vic doesn’t act, and GWWC and ACE act as they would have done had Vic not acted, AND GWWC doesn’t exist, and ACE and Vic act as they would have done had GWWC not existed, AND ACE doesn’t eixst GWWC and Vic act as they would have done had ACE not existed”. This condition does not describe the relevant counterfactual world—if Vic had not acted, then GWWC and ACE would still have existed, but according to this proposition GWWC and ACE do not exist. The problem is the aggregation principle, not the method of estimating counterfactual impact.
Applications
Voting
This has obvious applications when assessing the social benefits of voting. Suppose that there is an election and Politician A would produce $10bn in benefit compared to Politician B. The election is decided by one vote. (For simplicity suppose that B wins if there is a tie.) Emma cast a decisive vote for option A and therefore her counterfactual impact is $10bn. It is correct to say that the counterfactual impact of each other A voter is also $10bn.
The alternative approach (which I argue is wrong) is to say that each of the n A voters is counterfactually responsible for 1/n of the $10bn benefit. Suppose there are 10m A voters. Then each A voter’s counterfactual social impact is 1/10m*$10bn = $1000. But on this approach the common EA view that it is rational for individuals to vote as long as the probability of being decisive is not too small, is wrong. Suppose the ex ante chance of being decisive is 1/1m. Then the expected value of Emma voting is a mere 1/1m*$1000 = $0.001. On the correct approach, the expected value of Emma voting is 1/10m*$10bn = $1000. If voting takes 5 minutes, this is obviously a worthwhile investment for the benevolent voter, as per common EA wisdom.
Assessing the impact of EA organisations
When evaluating their own impact, EA organisations will often face this issue. Community orgs like CEA, REG, and Founders Pledge will recruit new donors who donate to charities recommended by GiveWell, ACE and others. How do we calculate the counterfactual impact here? If I am right, in the manner above. This does mean we should be careful when making claims about the collective impact of EA as a whole movement. It would be a mistake to aggregate the counterfactual impact of all the EA orgs taken one by one.
Assessing the impact of policy organisations
For most policy advocacy campaigns, numerous actors are involved and in some cases numerous organisations will be a necessary condition for some particular policy change. Notwithstanding difficulties in finding out which actors actually were necessary conditions, their counterfactual impact should be calculated as per the above methodology.
A note of caution: leveraging and funging[1]
The approach I have outlined needs to be applied with care. Most importantly, we need to be careful not to confuse the following two counterfactual comparisons:
Comparison 1 (correct)
Actual World: A, B, and C all act.
Counterfactual world (A): A doesn’t act, B and C act as they would have done if A had not acted.
Comparison 2 (incorrect)
Actual World: A, B, and C all act.
Counterfactual world (A): A doesn’t act, B and C act as they did in the actual world.
Confusing these two comparisons can lead one to neglect leveraging and funging effects. Organisations can leverage funds from other actors into a particular project. Suppose that AMF will spend $1m on a net distribution. As a result of AMF’s commitment, the Gates Foundation contributes $400,000. If AMF had not acted, Gates would have spent the $400,000 on something else. Therefore, the counterfactual impact of AMF’s work is:
AMF’s own $1m on bednets plus Gates’ $400,000 on bednets minus the benefits of what Gates would otherwise have spent their $400,000 on.
If Gates would otherwise have spent the money on something worse than bednets, then the leveraging is beneficial; if they would otherwise have spent it on something better than bednets, the leveraging reduces the benefit produced by AMF.
Confusing the two comparisons can also lead us to neglect funging effects. Suppose again that AMF commits $1m to a net distribution. But if AMF had put nothing in, DFID would instead have committed $500,000 to the net distribution. In this case, AMF funges with DFID. AMF’s counterfactual impact is therefore:
AMF’s own $1m on bednets minus the $500,000 that DFID would have put in plus the benefits of what DFID in fact spent their $500,000 on.
The effect of funging is the mirror image of the effect of leveraging. If DFID in fact spent their $500,000 on something worse than bednets, then the funging reduces AMF’s benefit; if DFID spent the $500,000 on something better than bednets, then the funging increases AMF’s benefits.
END
Thanks to James Snowden and Marinella Capriati for discussion of some of the ideas developed here.
There’s a game theory concept for this called Shapley value which allocates credit in a way that does sum to 1, but takes into account a bunch of people each independently being required for all of the impact to happen.
In this example, Shapley value would give everyone x/4 of the credit, adding together to x as we’d naturally expect.
This doesn’t really seem like an argument to me… it seems like you start from the premise that voting must be rational, and that something like Shapley value would make it irrational, and thus that can’t be the case. But this seems to me to be assuming the conclusion?
I guess this is a case where the expected value of the action and the Shapley value are not the same, because in one case you’re analyzing an entire system and in the other case you’re analyzing the individual action. But just as it may seem weird that Shapley value assigns such a small value to each vote, the expected value approach is basically saying that the first few votes have a value of literally 0, which also seems nonsensical to me.
It needs to be explained why there is a paradox. I have not yet seen an explanation of why there might be thought to be one. EAs are concerned with having counterfactual impact. If you were a necessary condition of some benefit B occurring, then you have had counterfactual impact.
Re voting I’m appealing to how almost everyone in the academic literature assesses the expected value of voting, which is not by dividing the total value by each voter. I’m also appealing to a common EA idea which is discussed by Parfit and mentioned in Will’s book, which is that voting is sometimes rational for altruistic voters. On your approach, it would pretty much always be irrational to vote even if the social benefits were extremely large: every social benefit would always be divided by the number of decisive voters, and so would be divided by many millions in any large election
I don’t understand why the expected value approach says that the first few votes have a value of 0. Also, the ordering in which votes are cast is completely irrelevant to judging a voter’s counterfactual imapct because all votes are indistinguishable wrt causing the outcome: it doesn’t matter if I voted first and Emma voted last, we would still be decisive voters.
It’s not a paradox. The problem is just that, if everyone thought this way, we would get suboptimal outcomes—so maybe we should figure out how to avoid that.
Suppose there are three possible outcomes: P has cost $2000 and gives 15 utility to the world Q has cost $1000 and gives 10 utility to the world R has cost $1000 and gives 10 utility to the world
Suppose Alice and Bob each have $1000 to donate. Consider two scenarios:
Scenario 1: Both Alice and Bob give $1000 to P. The world gets 15 more utility. Both Alice and Bob are counterfactually responsible for giving 15 utility to the world.
Scenario 2: Alice gives $1000 to Q and Bob gives $1000 to R. The world gets 20 more utility. Both Alice and Bob are counterfactually responsible for giving 10 utility to the world.
From the world’s perspective, scenario 2 is better. However, from Alice and Bob’s individual perspective (if they are maximizing their own counterfactual impact), scenario 1 is better. This seems wrong, we’d want to somehow coordinate so that we achieve scenario 2 instead of scenario 1.
Are you neglecting to count the negative impact from causing other people to do the suboptimal thing? If I use my funds to set up an exploding matching grant that will divert the funds of other donors from better things too a less effective charity, that is a negative part of my impact.
Yes, that’s right. I agree that a perfect calculation of your counterfactual impact would do the right thing in this scenario, and probably all scenarios. This is an empirical claim that the actual impact calculations that meta-orgs do are of the form that I wrote in my previous comment.
For example, consider the impact calculations that GWWC and other meta orgs have. If those impact calculations (with their current methodologies) showed a ratio of 1.1:1, that seems nominally worthwhile (you still have the multiplicative impact), but I would expect that it would be better to give directly to charities to avoid effects like the ones Joey talked about in his post.
A true full counterfactual impact calculation would consider the world in which GWWC just sends the money straight to charities and convinces other meta orgs to do the same, at which point they see that more money gets donated to charities in total, and so they all close operations and send money straight to charities. I’m arguing that this doesn’t happen in practice. (I think Joey and Peter are arguing the same thing.)
Firstly, people who believe in the correct account of counterfactual impact would have incentives to coordinate in the case you outline. Alice would maximise her counterfactual impact (defined as I define it) by coordinating with Bob on project R. The counterfactual impact of her coordinating with Bob would be +5 utility compared to scenario 1. There is no puzzle here.
Secondly, dividing counterfactual impact by contribution does not solve all these coordination problems. If everyone thought as per the Shapely value, then no rational altruists would ever vote, even when the true theory dictates that the expected value of doing so was very high.
Also consider the $1bn benefits case outlined above. Suppose that the situation is as described above but my action costs $2 and I take one billionth of the credit for the success of the project. In that case, the Shapely-adjusted benefits of my action would be $1 and the costs $2, so my action would not be worthwhile. I would therefore leave $1bn of value on the table.
For the first point, see my response to Carl above. I think you’re right in theory, but in practice it’s still a problem.
For the second point, I agree with Flodorner that you would either use the Shapley value, or you would use the probability of changing the outcome, not both. I don’t know much about Shapley values, but I suspect I would agree with you that they are suboptimal in many cases. I don’t think there is a good theoretical solution besides “consider every possible outcome and choose the best one” which we obviously can’t do as humans. Shapley values are one tractable way of attacking the problem without having to think about all possible worlds, but I’m not surprised that there are cases where they fail. I’m advocating for “think about this scenario”, not “use Shapley values”.
I think the $1bn benefits case is a good example of a pathological case where Shapley values fail horribly (assuming they do what you say they do, again, I don’t know much about them).
My overall position is something like “In the real world when we can’t consider all possibilities, one common failure mode in impact calculations is the failure to consider the scenario in which all the participants who contributed to this outcome instead do other altruistic things with their money”.
At this point, i think that to analyze the $1bn case correctly, you’d have to substract everyone’s opportunity cost in the calculation of the shapley value (if you want to use it here). This way, the example should yield what we expect.
I might do a more general writeup about shapley values, their advantages, disadvantages and when it makes sense to use them, if i find the time to read a bit more about the topic first.
^ This is what I wanted to say, but even better than how I was going to say it.
Here’s another way of putting it. Imagine that you are deciding whether to support a project P with some action A. Lots of other agents have been involved in P and have contributed more than you in the past. You can make some small contribution (action A) which tips the balance making P a success, producing $1bn benefits, otherwise P fails. Should I say “by acting I personally will ensure that $1bn are produced”? Or should I say that “by acting I personally will ensure that $1bn divided by other’s contributions are produced?” The difference between my acting and not acting is $1bn, so the former.
Simply, the term ‘counterfactual impact’ refers to the difference between the world in which I act and the world in which I don’t.
Where are you actually disagreeing with Joey and the conclusions he is drawing?
Joey is arguing that the --EA Movement—might accidentally overcount its impact by adding each individual actor’s counterfactual impact together. You point out a scenario in which various individual actor’s actions are necessary for the counterfactual impact to happen so it is legitimate for each actor to claim the full counterfactual impact. This seems tangential to Joey’s point, which is fundamentally about the practical implications of this problem. The question of who is responsible for the counterfactual impact and who should get credit are being asked because as the EA Movement we have to decide how to allocate our resources to the different actors. We also need to be cautious not to overcount impact as a movement in our outside communications and to not get the wrong impression ourselves.
I don’t see Joey’s article cited anywhere. Can someone help pointing to that article?
Found it.
https://ea.greaterwrong.com/posts/fnBnEiwged7y5vQFf/triple-counting-impact-in-ea?hide-nav-bars=true
If that is what he is arguing I agree, but I don’t think he is arguing that. He writes
“This person would become quadruple counted in EA, with each organization using their donations as impact to justify their running.”
Each organisation would in fact be right to count the impact in the way described.
To try to narrow down the disagreement: Would you donate to GWWC instead of AMF if their impact calculation (using their current methodology) showed that $1.10 went to AMF for every $1 given to GWWC? I wouldn’t.
In Joey’s example, I can donate $500 to GWWC instead of AMF. If I donate to AMF, AMF gets $500 compared to the world in which i don’t donate. If I donate to GWWC, then AMF gets $1000 compared to the world in which I don’t donate. Clearly, I should donate to GWWC if I care about counterfactual impact. If GWWC donates the $500 directly to AMF, then value has been lost.
The coordination problem is a separate question to how individual organisations should count their own counterfactual impact.
Forget about the organization’s own counterfactual impact for a moment.
Do you agree that, from the world’s perspective, it would be better in Joey’s scenario if GWWC, Charity Science, and TLYCS were to all donate their money directly to AMF?
yes
A good way of seeing this is to think about a single actor taking three actions. Suppose that you come across a child drowning in a pond. You pull the child out, call emergency services, and perform CPR until an ambulance arrives. While it may be the case that each of your actions saved the child’s life (in the sense that the child would have died if any one of the actions had not been taken), it is certainly not the case that your three actions collectively saved three lives. And if that’s true of three actions taken by a single person, it should also be true of three actions taken by three separate people.
Great point!
“The alternative approach (which I argue is wrong) is to say that each of the n A voters is counterfactually responsible for 1/n of the $10bn benefit. Suppose there are 10m A voters. Then each A voter’s counterfactual social impact is 1/10m$10bn = $1000. But on this approach the common EA view that it is rational for individuals to vote as long as the probability of being decisive is not too small, is wrong. Suppose the ex ante chance of being decisive is 1/1m. Then the expected value of Emma voting is a mere 1/1m$1000 = $0.001. On the correct approach, the expected value of Emma voting is 1/10m*$10bn = $1000. If voting takes 5 minutes, this is obviously a worthwhile investment for the benevolent voter, as per common EA wisdom.”
I am not sure, whether anyone is arguing for discounting twice. The alternative approach using the shapley value would divide the potential impact amongst the contributors, but not additionally account for the probability. Therefore, in this example both approaches seem to assign the same counterfactual impact.
More generally, it seems like most disagreements in this thread could be resolved by a more charitable interpretation of the other side (from both sides, as the validity of your argument against rohinmshah’s counterexample seems to show)
Right now, a comment from someone more proficient with the shapley value arguing against
“Also consider the $1bn benefits case outlined above. Suppose that the situation is as described above but my action costs $2 and I take one billionth of the credit for the success of the project. In that case, the Shapely-adjusted benefits of my action would be $1 and the costs $2, so my action would not be worthwhile. I would therefore leave $1bn of value on the table.”
might be helpful for a better understanding.
So, on the Shapely value approach, we would ignore the probability of different states of affairs when deciding what to do? This seems wrong. Also, the only time the discount would be the same is when the probability of counterfactual impact happens to equal the discount applied according to the shapely value. This would only happen coincidentally: e.g. the chance of being decisive could be 1⁄1,000 and the shapely discount 1/1m.
I don’t understand the difference between the following in “Notes on leveraging and funging”. Is that a typo? They look the same to me.
“B and C act as they would have done if A had not acted.”
“B and C act as they did in the actual world”