I was just thinking about this again and I don’t believe it works.
Suppose we want to maximize expected value over multiple value systems. Let’s say there’s a 10% chance that we should only care about the current generation, and a 90% chance that generational status isn’t morally relevant (obviously this is a simplification but I believe the result generalizes). Then the expected utility of AMF is
You could say it’s wrong to maximize expected utility across multiple value systems, but I don’t see how you can make reasonable decisions at all if you’re not trying to maximize expected utility. If you’re trying to “diversify” across multiple value systems then you’re doing something that’s explicitly bad according to a linear consequentialist value system, and you’d need some justification for why diversifying across value systems is better than maximizing expected value over value systems.
I am not familiar with the moral uncertainty literature, but in my mind it would make sense to define the utility scale of each welfare theory such that the difference in utility between the best and worst possible state is always the same. For example, always assigning 1 to the best possible state, and −1 to the worst possible state. In this case, the weights of each welfare theory would represent their respective strength/plausibility, and therefore not be arbitrary?
Okay, can you tell me if I’m understanding this correctly?
Say my ethical probability distribution is 10% prior existence utilitarianism and 90% total utilitarianism. Then the prior existence segment (call it P) gets $1 and the total existence segment (call it T) gets $9. P wants me to donate everything to AMF and T wants me to donate everything to MIRI, so I should donate $1 to AMF and $9 to MIRI. So that means people are justified in donating some portion of their budget to AMF, but not all unless they believe AMF also is the best charity for helping future generations.*
I was just thinking about this again and I don’t believe it works.
Suppose we want to maximize expected value over multiple value systems. Let’s say there’s a 10% chance that we should only care about the current generation, and a 90% chance that generational status isn’t morally relevant (obviously this is a simplification but I believe the result generalizes). Then the expected utility of AMF is
Far future effects still dominate.
You could say it’s wrong to maximize expected utility across multiple value systems, but I don’t see how you can make reasonable decisions at all if you’re not trying to maximize expected utility. If you’re trying to “diversify” across multiple value systems then you’re doing something that’s explicitly bad according to a linear consequentialist value system, and you’d need some justification for why diversifying across value systems is better than maximizing expected value over value systems.
The scaling factors there are arbitrary. I can throw in theories that claim things are infinitely important.
This view is closer to ‘say that views you care about got resources in proportion to your attachment to/credence in them, then engage in moral trade from that point.’
Hi Carl,
I am not familiar with the moral uncertainty literature, but in my mind it would make sense to define the utility scale of each welfare theory such that the difference in utility between the best and worst possible state is always the same. For example, always assigning 1 to the best possible state, and −1 to the worst possible state. In this case, the weights of each welfare theory would represent their respective strength/plausibility, and therefore not be arbitrary?
Okay, can you tell me if I’m understanding this correctly?
Say my ethical probability distribution is 10% prior existence utilitarianism and 90% total utilitarianism. Then the prior existence segment (call it P) gets $1 and the total existence segment (call it T) gets $9. P wants me to donate everything to AMF and T wants me to donate everything to MIRI, so I should donate $1 to AMF and $9 to MIRI. So that means people are justified in donating some portion of their budget to AMF, but not all unless they believe AMF also is the best charity for helping future generations.*