Similar to not costing others work, you can end up in situations where the same impact is counted multiple times across all the charities involved, giving an inflated picture of the total impact.
Eg. If Effective Altruism (EA) London runs an event and this leads to an individual signing the Giving What We Can (GWWC) pledge and donating more the charity, both EA London and GWWC and the individual may take 100% of the credit in their impact measurement.
Just a quick note that ‘double counting’ can be fine, since the counterfactual impact of different groups acting in concert doesn’t necessarily sum to 100%.
Also note that you can also undercount for similar reasons. For instance, if you have impact X, but another org would have had done X otherwise, you might count your impact as zero. But that ignores that by doing X, you free up the other org to do something else high impact.
I think I’d prefer to frame this issue as something more like “how you should assign credit as a donor in order to have the best incentives for the community isn’t the same as how you’d calculate the counterfactual impact of different groups in a cost-effectiveness estimate”.
When you notice that counterfactual values can sum up to more than 100%, I think that the right answer is to stop optimizing for counterfactual values.
It’s less clear cut, but I think that optimizing for Shapley value instead is a better answer—though not perfect.
I think of Shapley values as just one way of assigning credit in a way to optimise incentives, but from what I’ve seen, it’s not obvious it’s the best one. (In general, I haven’t seen any principled way of assigning credit that always seems best.)
DOUBLE COUNTING
Similar to not costing others work, you can end up in situations where the same impact is counted multiple times across all the charities involved, giving an inflated picture of the total impact.
Eg. If Effective Altruism (EA) London runs an event and this leads to an individual signing the Giving What We Can (GWWC) pledge and donating more the charity, both EA London and GWWC and the individual may take 100% of the credit in their impact measurement.
Just a quick note that ‘double counting’ can be fine, since the counterfactual impact of different groups acting in concert doesn’t necessarily sum to 100%.
See more discussion here: https://forum.effectivealtruism.org/posts/fnBnEiwged7y5vQFf/triple-counting-impact-in-ea
Also note that you can also undercount for similar reasons. For instance, if you have impact X, but another org would have had done X otherwise, you might count your impact as zero. But that ignores that by doing X, you free up the other org to do something else high impact.
I think I’d prefer to frame this issue as something more like “how you should assign credit as a donor in order to have the best incentives for the community isn’t the same as how you’d calculate the counterfactual impact of different groups in a cost-effectiveness estimate”.
I’d also point to Shapley values.
When you notice that counterfactual values can sum up to more than 100%, I think that the right answer is to stop optimizing for counterfactual values.
It’s less clear cut, but I think that optimizing for Shapley value instead is a better answer—though not perfect.
I think of Shapley values as just one way of assigning credit in a way to optimise incentives, but from what I’ve seen, it’s not obvious it’s the best one. (In general, I haven’t seen any principled way of assigning credit that always seems best.)
Good point, thanks! :)