a) What is the relevant units that you attribute impact to?
b) Why is it relevant to measure impact of past performance in the first place?
To clarify these questions, consider the following example. Suppose that country C has 349 members of parliament (MPs), elected in a UK-style first-past-the-post system. Now compare two scenarios:
1) One party, A, gets 175 MPs, whereas the other, B, gets 174 MPs
2) A gets 176 MPs and the other 173 MPs.
Now suppose that a win for A is worth 1 trillion dollars. Then in scenario 1), each A MP can claim to have caused 1 trillion dollars for C, in the sense that if they hadn’t won their race against their specific B opponent, C would have lost 1 trillion dollars. In scenario 2), however, none of them can claim to have had any impact at all, in that sense, because even if they had lost their race, A would have won.
Now firstly, note that this analysis is heavily dependent on the unit of impact attribution being individual MPs. Suppose that all but one consituencies had two MPs rather than one. In that case, the appropriate unit of impact attribution rather becomes these pairs of MPs, in which case each pair of A MPs actually did cause a 1 dollars gain (since if it weren’t for them, A would have lost 174-175).
Now I think that it can be that it’s not always obvious what the appropriate unit of impact attribution is (even though it might have been in the scenarios involving MPs). Suppose, e.g., that an EA org, according to a certain analysis, has caused 100 000 dollars to be moved to cost-effective charities, but that this is all due to a certain individual. Why are we then to attribute this impact to the EA org, rather than the individual? (This question obviously becomes all the more important if the individual subsequently has left the organisation.) Or conversely, why are we to attribute it to the EA org, rather than to the EA movement as a whole? (It might be that some organisations grow more thanks to the general momentum of the EA movement than thanks to any effort of their own.) Why is the EA org the appropriate level of analysis? (Not saying it isn’t, but that it is something that needs to be explicitly argued for.)
Let us turn to question b), and settle, for the sake of the argument, that individual MP is the appropriate unit of impact attribution. This means that in 1), each individual MP has had an enormous impact, whereas in 2), they had no impact whatsoever. Seemingly, we have very strong reasons to donate to each individual MP is 1), but very weak reasons in 2). Can this be right, given how similar the cases are?
No, it can’t, the reason being that 2) give you approximately as strong reasons as 1) to believe that the next election will be a close race as well. For, to answer question a), the ultimate point of this whole impact exercice is not to evaluate past performance, but to learn how to maximise the expected impact of future donations. In a sense, the MPs were “morally lucky” in 1), and we shouldn’t take luck into account when we’re thinking of where to donate (since donations should be future-facing, and there is by definition no reason to believe that we will continue to be lucky).
I think that at least part of the answer regarding the first question lies in the answer to this second question. To the extent that we want to assess past impact at all, we want to choose a level of impact attribution analysis that allows us to assess the expected impact of future donations accurately.
I think we have to consider two things here:
a) What is the relevant units that you attribute impact to?
b) Why is it relevant to measure impact of past performance in the first place?
To clarify these questions, consider the following example. Suppose that country C has 349 members of parliament (MPs), elected in a UK-style first-past-the-post system. Now compare two scenarios:
1) One party, A, gets 175 MPs, whereas the other, B, gets 174 MPs 2) A gets 176 MPs and the other 173 MPs.
Now suppose that a win for A is worth 1 trillion dollars. Then in scenario 1), each A MP can claim to have caused 1 trillion dollars for C, in the sense that if they hadn’t won their race against their specific B opponent, C would have lost 1 trillion dollars. In scenario 2), however, none of them can claim to have had any impact at all, in that sense, because even if they had lost their race, A would have won.
Now firstly, note that this analysis is heavily dependent on the unit of impact attribution being individual MPs. Suppose that all but one consituencies had two MPs rather than one. In that case, the appropriate unit of impact attribution rather becomes these pairs of MPs, in which case each pair of A MPs actually did cause a 1 dollars gain (since if it weren’t for them, A would have lost 174-175).
Now I think that it can be that it’s not always obvious what the appropriate unit of impact attribution is (even though it might have been in the scenarios involving MPs). Suppose, e.g., that an EA org, according to a certain analysis, has caused 100 000 dollars to be moved to cost-effective charities, but that this is all due to a certain individual. Why are we then to attribute this impact to the EA org, rather than the individual? (This question obviously becomes all the more important if the individual subsequently has left the organisation.) Or conversely, why are we to attribute it to the EA org, rather than to the EA movement as a whole? (It might be that some organisations grow more thanks to the general momentum of the EA movement than thanks to any effort of their own.) Why is the EA org the appropriate level of analysis? (Not saying it isn’t, but that it is something that needs to be explicitly argued for.)
Let us turn to question b), and settle, for the sake of the argument, that individual MP is the appropriate unit of impact attribution. This means that in 1), each individual MP has had an enormous impact, whereas in 2), they had no impact whatsoever. Seemingly, we have very strong reasons to donate to each individual MP is 1), but very weak reasons in 2). Can this be right, given how similar the cases are?
No, it can’t, the reason being that 2) give you approximately as strong reasons as 1) to believe that the next election will be a close race as well. For, to answer question a), the ultimate point of this whole impact exercice is not to evaluate past performance, but to learn how to maximise the expected impact of future donations. In a sense, the MPs were “morally lucky” in 1), and we shouldn’t take luck into account when we’re thinking of where to donate (since donations should be future-facing, and there is by definition no reason to believe that we will continue to be lucky).
I think that at least part of the answer regarding the first question lies in the answer to this second question. To the extent that we want to assess past impact at all, we want to choose a level of impact attribution analysis that allows us to assess the expected impact of future donations accurately.