the results here don’t depend on actual infinities (infinite universe, infinitely long lives, infinite value)
This seems pretty important to me. You can handwave away standard infinite ethics by positing that everything is finite with 100% certainty, but you can’t handwave away the implications of a finite-everywere distribution with infinite EV.
(Just an offhand thought, I wonder if there’s a way to fix infinite-EV distributions by positing that utility is bounded, but that you don’t know what the bound is? My subjective belief is something like, utility is bounded, I don’t know the bound, and the expected value of the upper bound is infinity. If the upper bound is guaranteed finite but with an infinite EV, does that still cause problems?)
I think someone could hand-wave away heavy tailed distributions, too, but rather than assigning some outcomes 0 probability or refusing to rank them, they’re assuming some prospects of valid outcomes aren’t valid or never occur, even though they’re perfectly valid measure-theoretically. Or, they might actually just assign 0 probability to outcomes outside those with a bounded range of utility. In the latter case, you could represent them with both a bounded utility function and an unbounded utility function, agreeing on the bounded utility set of outcomes.
You could have moral/normative uncertainty across multiple bounded utility functions. Just make sure you don’t weigh them together via maximizing expected choiceworthiness in such a way that the weighted sum of utility functions is unbounded, because the weighted sum is a utility function. If the weighted sum is unbounded, then the same arguments in the post will apply to it. You could normalize all the utility functions first. Or, use a completely different approach to normative uncertainty, e.g. a moral parliament. That being said, the other approaches to normative uncertainty also violate Independence and can be money pumped, AFAIK.
This seems pretty important to me. You can handwave away standard infinite ethics by positing that everything is finite with 100% certainty, but you can’t handwave away the implications of a finite-everywere distribution with infinite EV.
(Just an offhand thought, I wonder if there’s a way to fix infinite-EV distributions by positing that utility is bounded, but that you don’t know what the bound is? My subjective belief is something like, utility is bounded, I don’t know the bound, and the expected value of the upper bound is infinity. If the upper bound is guaranteed finite but with an infinite EV, does that still cause problems?)
I think someone could hand-wave away heavy tailed distributions, too, but rather than assigning some outcomes 0 probability or refusing to rank them, they’re assuming some prospects of valid outcomes aren’t valid or never occur, even though they’re perfectly valid measure-theoretically. Or, they might actually just assign 0 probability to outcomes outside those with a bounded range of utility. In the latter case, you could represent them with both a bounded utility function and an unbounded utility function, agreeing on the bounded utility set of outcomes.
You could have moral/normative uncertainty across multiple bounded utility functions. Just make sure you don’t weigh them together via maximizing expected choiceworthiness in such a way that the weighted sum of utility functions is unbounded, because the weighted sum is a utility function. If the weighted sum is unbounded, then the same arguments in the post will apply to it. You could normalize all the utility functions first. Or, use a completely different approach to normative uncertainty, e.g. a moral parliament. That being said, the other approaches to normative uncertainty also violate Independence and can be money pumped, AFAIK.
Fairly related to this is section 6 in Beckstead and Thomas, 2022. https://onlinelibrary.wiley.com/doi/full/10.1111/nous.12462