Thanks for this interesting post. As I argued in the post that you cite and as George Bridgwater notes below, I don’t think you have identified a problem in the idea of counterfactual impact here, but have instead shown that you sometimes cannot aggregate counterfactual impact across agents. As you say, CounterfactualImpact(Agent) = Value(World with agent) - Value(World without agent).
Suppose Karen and Andrew have a one night stand which leads to Karen having a baby George (and Karen and Andrew otherwise have no effect on anything). In this case, Andrew’s counterfactual impact is:
Value (world with one night stand) - Value (world without one night stand)
The same is true for Karen. Thus, the counterfactual impact of each of them taken individually is an additional baby George. This doesn’t mean that the counterfactual impact of Andrew and Karen combined is two additional baby Georges. In fact, the counterfactual impact of Karen and Andrew combined is also given by:
Value (world with one night stand) - Value (world without one night stand)
Thus, the counterfactual impact of Karen and Andrew combined is an additional baby George. There is nothing in the definition of counterfactual impact which implies it can be always be aggregated across agents.
This is the difference between “if me and Karen hadn’t existed, neither would George” and “If I hadn’t existed, neither would George, and if Karen hadn’t existed neither would George, therefore if me and Karen hadn’t existed, neither would two Georges.” This last statement is confused, because the babies referred to in the antecedent are the same.
I discuss other examples in the comments to Joey’s post.
**
The counterfactual understanding of impact is how almost all voting theorists analyse the expected value of voting. EAs tends to think that voting is sometimes altruistically rational because of the small chance of being the one pivotal voter and making a large counterfactual difference. On the Shapely value approach, the large counterfactual difference would be divided by the number of winning voters. Firstly, to my knowledge almost no-one in voting theory assesses the impact of voting in this way. Secondly, this would I think imply that voting is never rational since in any large election the prospective pay-off of voting would be divided by the potential set of winning voters and so would be >100,000x smaller than on the counterfactual approach
I don’t exactly claim to have identified a problem with the counterfactual function, in itself. The counterfactual is perfectly well defined, and I like it, and it has done nothing wrong. I understand this. It is clear to me that it can’t be added just like that. The function, per se, is fine.
What I’m claiming is that, because it can’t be aggregated, it is not the right function to think about in terms of assigning impact to people in the context of groups. I am arguing about the area of applicability of the function, not about the function. I am claiming that, if you are optimizing for counterfactual impact in terms of groups, pitfalls may arise.
It’s like, when you first see for the same time: −1 = sqrt(-1)*sqrt(-1) = sqrt((-1)*(-1)) = sqrt(1) = 1, therefore −1 = 1, and you can’t see the mistake. It’s not that the sqrt function is wrong, it’s that you’re using it outside it’s limited fiefdom, so something breaks. I hope the example proved amusing.
I’m not only making statements about the counterfactual function, I’m also making statements about the concept which people have in your head which is called “impact”, and how that concept doesn’t map to counterfactual impact some of the time, and about how, if you had to map that concept to a mathematical function, the Shapley value is a better candidate.
Thanks for this interesting post. As I argued in the post that you cite and as George Bridgwater notes below, I don’t think you have identified a problem in the idea of counterfactual impact here, but have instead shown that you sometimes cannot aggregate counterfactual impact across agents. As you say, CounterfactualImpact(Agent) = Value(World with agent) - Value(World without agent).
Suppose Karen and Andrew have a one night stand which leads to Karen having a baby George (and Karen and Andrew otherwise have no effect on anything). In this case, Andrew’s counterfactual impact is:
Value (world with one night stand) - Value (world without one night stand)
The same is true for Karen. Thus, the counterfactual impact of each of them taken individually is an additional baby George. This doesn’t mean that the counterfactual impact of Andrew and Karen combined is two additional baby Georges. In fact, the counterfactual impact of Karen and Andrew combined is also given by:
Value (world with one night stand) - Value (world without one night stand)
Thus, the counterfactual impact of Karen and Andrew combined is an additional baby George. There is nothing in the definition of counterfactual impact which implies it can be always be aggregated across agents.
This is the difference between “if me and Karen hadn’t existed, neither would George” and “If I hadn’t existed, neither would George, and if Karen hadn’t existed neither would George, therefore if me and Karen hadn’t existed, neither would two Georges.” This last statement is confused, because the babies referred to in the antecedent are the same.
I discuss other examples in the comments to Joey’s post.
**
The counterfactual understanding of impact is how almost all voting theorists analyse the expected value of voting. EAs tends to think that voting is sometimes altruistically rational because of the small chance of being the one pivotal voter and making a large counterfactual difference. On the Shapely value approach, the large counterfactual difference would be divided by the number of winning voters. Firstly, to my knowledge almost no-one in voting theory assesses the impact of voting in this way. Secondly, this would I think imply that voting is never rational since in any large election the prospective pay-off of voting would be divided by the potential set of winning voters and so would be >100,000x smaller than on the counterfactual approach
I don’t exactly claim to have identified a problem with the counterfactual function, in itself. The counterfactual is perfectly well defined, and I like it, and it has done nothing wrong. I understand this. It is clear to me that it can’t be added just like that. The function, per se, is fine.
What I’m claiming is that, because it can’t be aggregated, it is not the right function to think about in terms of assigning impact to people in the context of groups. I am arguing about the area of applicability of the function, not about the function. I am claiming that, if you are optimizing for counterfactual impact in terms of groups, pitfalls may arise.
It’s like, when you first see for the same time: −1 = sqrt(-1)*sqrt(-1) = sqrt((-1)*(-1)) = sqrt(1) = 1, therefore −1 = 1, and you can’t see the mistake. It’s not that the sqrt function is wrong, it’s that you’re using it outside it’s limited fiefdom, so something breaks. I hope the example proved amusing.
I’m not only making statements about the counterfactual function, I’m also making statements about the concept which people have in your head which is called “impact”, and how that concept doesn’t map to counterfactual impact some of the time, and about how, if you had to map that concept to a mathematical function, the Shapley value is a better candidate.