Just a quick note that ‘double counting’ can be fine, since the counterfactual impact of different groups acting in concert doesn’t necessarily sum to 100%.
Also note that you can also undercount for similar reasons. For instance, if you have impact X, but another org would have had done X otherwise, you might count your impact as zero. But that ignores that by doing X, you free up the other org to do something else high impact.
I think I’d prefer to frame this issue as something more like “how you should assign credit as a donor in order to have the best incentives for the community isn’t the same as how you’d calculate the counterfactual impact of different groups in a cost-effectiveness estimate”.
When you notice that counterfactual values can sum up to more than 100%, I think that the right answer is to stop optimizing for counterfactual values.
It’s less clear cut, but I think that optimizing for Shapley value instead is a better answer—though not perfect.
I think of Shapley values as just one way of assigning credit in a way to optimise incentives, but from what I’ve seen, it’s not obvious it’s the best one. (In general, I haven’t seen any principled way of assigning credit that always seems best.)
Just a quick note that ‘double counting’ can be fine, since the counterfactual impact of different groups acting in concert doesn’t necessarily sum to 100%.
See more discussion here: https://forum.effectivealtruism.org/posts/fnBnEiwged7y5vQFf/triple-counting-impact-in-ea
Also note that you can also undercount for similar reasons. For instance, if you have impact X, but another org would have had done X otherwise, you might count your impact as zero. But that ignores that by doing X, you free up the other org to do something else high impact.
I think I’d prefer to frame this issue as something more like “how you should assign credit as a donor in order to have the best incentives for the community isn’t the same as how you’d calculate the counterfactual impact of different groups in a cost-effectiveness estimate”.
I’d also point to Shapley values.
When you notice that counterfactual values can sum up to more than 100%, I think that the right answer is to stop optimizing for counterfactual values.
It’s less clear cut, but I think that optimizing for Shapley value instead is a better answer—though not perfect.
I think of Shapley values as just one way of assigning credit in a way to optimise incentives, but from what I’ve seen, it’s not obvious it’s the best one. (In general, I haven’t seen any principled way of assigning credit that always seems best.)