On the practical point, one help is that I think cases like these are fairly uncommon:
The previous example used donations because it’s easy and clear cut to make the case that this is the wrong move without getting into more difficult issues, but it generalizes to talent as well. For example, recently, Fortify Health was founded. Clearly the founders deserve 100% impact- without them, the project certainly would not have happened. But wait a second: both of them think that without Charity Science’s support, the project would definitely not have happened. So, technically, Charity Science could also take 100% credit. (Since from our perspective, if we did not help Fortify Health it would not have happened, so it is a 100% counterfactually caused by Charity Science project). But wait a second, what about the donors who funded the project early on (because of Charity Science’s recommendation)? Surely they deserve some credit for impact as well! What about the fact that without the EA movement, it would have been much less likely for Charity Science and Fortify Health to connect? With multiple organizations and individuals, you can very easily attribute a lot more impact than actually happens.
In our impact evaluations, and in my experiences talking to others in the community, we would never give 100% of the impact to each group. For instance, if Charity Science didn’t exist, the founders of Fortify might well have ended up doing a similar idea anyway—it’s not as if Charity Science is the only group promoting evidence-based global health charities, and if Charity Science didn’t exist, another group like them probably would have sprung up eventually. What’s more, even if the founders didn’t do Fortify, they would probably have done something else high-impact instead. So, the impact of Charity Science should probably be much less than 100% of Fortify. And the same is true for the other groups involved.
At 80,000 Hours, we rarely claim more than 30% of the impact of an event or plan change, and we most often model our impact as a speed-up (e.g. we assume the career changer would have eventually made the same shift, but we made it come 0.5-4 years earlier). We also sometimes factor in costs incurred by other groups. All this makes it hard for credit to add up to more than 100% in practice.
good points. This can also go the other way though—an org could leverage money from otherwise very ineffective orgs. Especially with policy changes, it can sometimes be the case that a good org comes up with a campaign that steers the entire advocacy ecosystem to a more effective path. A good example of this is campaigns for ordinary air pollution regulations on coal plants, which were started in the 1990s by the Clean Air Task Force among others and now have hundreds of millions in funding from Bloomberg. If these campaigns weren’t started, environmental NGOs in the US and Europe would plausibly be working on something much worse.
I don’t think the notion of ‘credit’ is a useful one. At FP, when we were looking at orgs working on policy change, we initially asked them how much credit they should take for a particular policy change. They ended up saying things like “40%”. I don’t really understand what this means. It turned out to be best to ask them when the campaign and policy change would have happened had they not acted (obviously a very difficult question). It’s best to couch things in terms of counterfactual impact throughout and not to convert into ‘credit’.
Similarly with voting, if an election is decided by one vote and there are one million voters for the winning party, I think it is inevitably misleading to ask how much of the credit each voter should get. One naturally answers that they get one millionth of the credit, but this is wrong as a proposition about their counterfactual impact, which is what we really care about.
Indeed, focusing on credit can lead you to attribute impact in cases of redundant causation when an org actually has zero counterfactual impact. Imagine 100 orgs are working for a big policy change, and only 50 of them were necessary to the outcome (though this could be any combination of them and they were all equally important). In this case, funding one of the orgs had zero counterfactual impact because the change would have happened without them. But on the ‘credit approach’, you’d end up attributing one hundredth of the impact to each of the orgs
I agree—I was talking a bit too loosely. When I said “assign credit of 30% of X” I meant “assign counterfactual impact of 30% of X”. My point was just that even if you do add up all the counterfactual impacts (ignoring that this is a conceptual mistake like you point out), they rarely sum to more than 100%, so it’s still not a big issue.
I’m not sure I follow the first paragraph about leveraging other groups.
You argued that counterfactual impact may be smaller than it appears. But it may also be larger than it first appears due to leveraging other orgs away from ineffective activities. e.g. an NGO successfully advocates for a policy change P1 - the benefits of P1 is their counterfactual impact. But as a result of the proven success of this type of project, 100 other NGOs start working on similar projects where before they worked on ineffective projects. This latter effect should also be counted as the first org’s counterfactual impact. This could be understood as leveraging additional money into an effective space.
I don’t agree. His logic entails that money/effort you leverage shouldn’t be counted as your own counterfactual impact. If FHI convinces e.g. the UK government that biorisk is worth spending money on, then on Joey’s approach, FHI would be wrong to count this additional money as it’s own impact.
On the practical point, one help is that I think cases like these are fairly uncommon:
In our impact evaluations, and in my experiences talking to others in the community, we would never give 100% of the impact to each group. For instance, if Charity Science didn’t exist, the founders of Fortify might well have ended up doing a similar idea anyway—it’s not as if Charity Science is the only group promoting evidence-based global health charities, and if Charity Science didn’t exist, another group like them probably would have sprung up eventually. What’s more, even if the founders didn’t do Fortify, they would probably have done something else high-impact instead. So, the impact of Charity Science should probably be much less than 100% of Fortify. And the same is true for the other groups involved.
At 80,000 Hours, we rarely claim more than 30% of the impact of an event or plan change, and we most often model our impact as a speed-up (e.g. we assume the career changer would have eventually made the same shift, but we made it come 0.5-4 years earlier). We also sometimes factor in costs incurred by other groups. All this makes it hard for credit to add up to more than 100% in practice.
good points. This can also go the other way though—an org could leverage money from otherwise very ineffective orgs. Especially with policy changes, it can sometimes be the case that a good org comes up with a campaign that steers the entire advocacy ecosystem to a more effective path. A good example of this is campaigns for ordinary air pollution regulations on coal plants, which were started in the 1990s by the Clean Air Task Force among others and now have hundreds of millions in funding from Bloomberg. If these campaigns weren’t started, environmental NGOs in the US and Europe would plausibly be working on something much worse.
I don’t think the notion of ‘credit’ is a useful one. At FP, when we were looking at orgs working on policy change, we initially asked them how much credit they should take for a particular policy change. They ended up saying things like “40%”. I don’t really understand what this means. It turned out to be best to ask them when the campaign and policy change would have happened had they not acted (obviously a very difficult question). It’s best to couch things in terms of counterfactual impact throughout and not to convert into ‘credit’.
Similarly with voting, if an election is decided by one vote and there are one million voters for the winning party, I think it is inevitably misleading to ask how much of the credit each voter should get. One naturally answers that they get one millionth of the credit, but this is wrong as a proposition about their counterfactual impact, which is what we really care about.
Indeed, focusing on credit can lead you to attribute impact in cases of redundant causation when an org actually has zero counterfactual impact. Imagine 100 orgs are working for a big policy change, and only 50 of them were necessary to the outcome (though this could be any combination of them and they were all equally important). In this case, funding one of the orgs had zero counterfactual impact because the change would have happened without them. But on the ‘credit approach’, you’d end up attributing one hundredth of the impact to each of the orgs
I agree—I was talking a bit too loosely. When I said “assign credit of 30% of X” I meant “assign counterfactual impact of 30% of X”. My point was just that even if you do add up all the counterfactual impacts (ignoring that this is a conceptual mistake like you point out), they rarely sum to more than 100%, so it’s still not a big issue.
I’m not sure I follow the first paragraph about leveraging other groups.
You argued that counterfactual impact may be smaller than it appears. But it may also be larger than it first appears due to leveraging other orgs away from ineffective activities. e.g. an NGO successfully advocates for a policy change P1 - the benefits of P1 is their counterfactual impact. But as a result of the proven success of this type of project, 100 other NGOs start working on similar projects where before they worked on ineffective projects. This latter effect should also be counted as the first org’s counterfactual impact. This could be understood as leveraging additional money into an effective space.
Makes sense. I don’t think Joey would object if orgs were counting this though.
I don’t agree. His logic entails that money/effort you leverage shouldn’t be counted as your own counterfactual impact. If FHI convinces e.g. the UK government that biorisk is worth spending money on, then on Joey’s approach, FHI would be wrong to count this additional money as it’s own impact.