Nitpick: If you believe infinite expected utility is a thing then every action has infinite expected utility. I assume you both just mean “extremely large expected utility.”
I don’t believe it is infinitely valuable, but freedomandutility did mention it literally, so I’m applying a simplicity prior and thinking he did mean that literally.
Freedomandutility stated one example: making AGI safe has infinite expected utility under a techno-utopian view, which may not be a hypothetical.
Nitpick: If you believe infinite expected utility is a thing then every action has infinite expected utility. I assume you both just mean “extremely large expected utility.”
I don’t believe it is infinitely valuable, but freedomandutility did mention it literally, so I’m applying a simplicity prior and thinking he did mean that literally.
As I replied in that thread, raising the stakes just makes it all the more important to be actually prudent!