The way to solve this is to look at actions in terms of both their positive and negative consequences. You want a way to reach some kind of summary conclusion about the set of consequences of a particular action (so that you can say, for example, “Overall, the action was altruistic”), but you also want a distinction between the summary and its components, so that individual consequences don’t go ignored.
EA folks think that they can find a useful probability for most any outcome, a probability that reflects something different than how well the outcome matches the consequences of a system of causes. They can’t, but they go on assigning epistemic confidence to their important decisions and assertions.
An EA person can say “I 70% believe taking action A will have consequence X, and 30% believe that it will have consequence Y”, and seem reasonable to their peers.
Suppose consequence X was “the most good” and consequence Y was “the most bad”. From this they calculate an “Expected Value”, multiplying their %’s of beliefs by the utility (the good) of their possible consequences.
So .70 times “the most good” (a positive value) plus .30 times “the most bad” (a negative value) , in the example, would give an expected value of “kinda good”.
The actual consequence, though, is either “the most good” or “the most bad”, not “kinda good”. When EA folks talk about doing “the most good” they are using estimates like expected value, and the calculation results are probably closer to “kinda good” or maybe “pretty good”, rather than “the most good”, which might have been one of the outcomes, but not what they were going for, ironically.
I think that’s a partial answer to your question. Your idea that incremental improvements can be more reliable than a big jump is a good one, I agree that it’s less risky and more wise, in some cases.
The way to solve this is to look at actions in terms of both their positive and negative consequences. You want a way to reach some kind of summary conclusion about the set of consequences of a particular action (so that you can say, for example, “Overall, the action was altruistic”), but you also want a distinction between the summary and its components, so that individual consequences don’t go ignored.
EA folks think that they can find a useful probability for most any outcome, a probability that reflects something different than how well the outcome matches the consequences of a system of causes. They can’t, but they go on assigning epistemic confidence to their important decisions and assertions.
An EA person can say “I 70% believe taking action A will have consequence X, and 30% believe that it will have consequence Y”, and seem reasonable to their peers.
Suppose consequence X was “the most good” and consequence Y was “the most bad”. From this they calculate an “Expected Value”, multiplying their %’s of beliefs by the utility (the good) of their possible consequences.
So .70 times “the most good” (a positive value) plus .30 times “the most bad” (a negative value) , in the example, would give an expected value of “kinda good”.
The actual consequence, though, is either “the most good” or “the most bad”, not “kinda good”. When EA folks talk about doing “the most good” they are using estimates like expected value, and the calculation results are probably closer to “kinda good” or maybe “pretty good”, rather than “the most good”, which might have been one of the outcomes, but not what they were going for, ironically.
I think that’s a partial answer to your question. Your idea that incremental improvements can be more reliable than a big jump is a good one, I agree that it’s less risky and more wise, in some cases.