Really great post! As a person who subscribes to hedonistic utilitarianism (modulo a decent amount of moral uncertainty), this is the most compelling criticism I’ve come across.
I do want to assuage a few of your worries, though. Firstly, as Richard brought up, I respect normative uncertainty enough to at least think for a long time before turning the universe into any sort of hedonium. Furthermore, I see myself as being on a joint mission with all other longtermists to bring about a glorious future, and I wouldn’t defect against them by bringing about a future they would reject, even if I was completely certain in my morality.
Also, my current best-guess vision of the maximum-hedonium future doesn’t look like what you’ve described. I agree that it will probably not look like a bunch of simulated happy people living in a way anything like people on Earth. But hedonistic utilitarianism (as I interpret it) doesn’t say that “pleasure”, in the way the world is commonly used, is what matters, but rather that mental states are what matter. The highest utility states are probably not base pleasures, but rich experiences. I expect that the hedonistic utility a mind can experience scales superlinearly in the computational resources used by that mind. As such, the utopia I imagine is not a bunch of isolated minds stuck repeating some base pleasure, but interconnected minds growing and becoming capable of immensely rich experiences. Nick Bostrom’s Letter from Utopia is the best description of this vision that I’m aware of.
Possibly this still sounds terrible to you, and that’s entirely fair. Hedonistic utilitarianism does in fact entail many weird conclusions in the limit.
Really great post! As a person who subscribes to hedonistic utilitarianism (modulo a decent amount of moral uncertainty), this is the most compelling criticism I’ve come across.
I do want to assuage a few of your worries, though. Firstly, as Richard brought up, I respect normative uncertainty enough to at least think for a long time before turning the universe into any sort of hedonium. Furthermore, I see myself as being on a joint mission with all other longtermists to bring about a glorious future, and I wouldn’t defect against them by bringing about a future they would reject, even if I was completely certain in my morality.
Also, my current best-guess vision of the maximum-hedonium future doesn’t look like what you’ve described. I agree that it will probably not look like a bunch of simulated happy people living in a way anything like people on Earth. But hedonistic utilitarianism (as I interpret it) doesn’t say that “pleasure”, in the way the world is commonly used, is what matters, but rather that mental states are what matter. The highest utility states are probably not base pleasures, but rich experiences. I expect that the hedonistic utility a mind can experience scales superlinearly in the computational resources used by that mind. As such, the utopia I imagine is not a bunch of isolated minds stuck repeating some base pleasure, but interconnected minds growing and becoming capable of immensely rich experiences. Nick Bostrom’s Letter from Utopia is the best description of this vision that I’m aware of.
Possibly this still sounds terrible to you, and that’s entirely fair. Hedonistic utilitarianism does in fact entail many weird conclusions in the limit.