In this (purely hypothetical, functionally impossible) scenario, I would choose option B—not because of the mild, transient suffering in scenario A, but the possibility of the emergence of serious suffering in the future (which doesn’t exist on B).
Happiness is also extremely subjective, and therefore can’t be meaningfully quantified, while the things that cause suffering tend to be remarkably consistent across times, places, and even species. So basing a moral system on happiness (rather than suffering-reduction) seems to make no sense to me.
In this (purely hypothetical, functionally impossible) scenario, I would choose option B—not because of the mild, transient suffering in scenario A, but the possibility of the emergence of serious suffering in the future (which doesn’t exist on B).
Scenario A assumed “your perfect utopia forever”, so there would be no chance for serious suffering to emerge.
Then that would make Scenario A much more attractive to me (not necessarily from a moral perspective), and I apologize for misunderstanding your hypothetical. To be honest, with the caveat of forever, I’m not sure which scenario I’d prefer. A is certainly much more interesting to me, but my moral calculus pushes me to conclude B is more rational.
I also get that it’s an analogy to get me thinking about the deeper issues here, and I understand. My perspective is just that, while I certainly find the philosophy behind this interesting, the issue of AI permanently limiting human potential isn’t hypothetical anymore.
It’s likely to happen in a very short period of time (relative to the total lifespan of the species), unless indefinite delay of AI development really is socio-politically possible (and from what evidence I’ve seen recently, it doesn’t seem to be). [epistemic certainly — relatively low — 60%]
In this (purely hypothetical, functionally impossible) scenario, I would choose option B—not because of the mild, transient suffering in scenario A, but the possibility of the emergence of serious suffering in the future (which doesn’t exist on B).
Happiness is also extremely subjective, and therefore can’t be meaningfully quantified, while the things that cause suffering tend to be remarkably consistent across times, places, and even species. So basing a moral system on happiness (rather than suffering-reduction) seems to make no sense to me.
Scenario A assumed “your perfect utopia forever”, so there would be no chance for serious suffering to emerge.
Then that would make Scenario A much more attractive to me (not necessarily from a moral perspective), and I apologize for misunderstanding your hypothetical. To be honest, with the caveat of forever, I’m not sure which scenario I’d prefer. A is certainly much more interesting to me, but my moral calculus pushes me to conclude B is more rational.
I also get that it’s an analogy to get me thinking about the deeper issues here, and I understand. My perspective is just that, while I certainly find the philosophy behind this interesting, the issue of AI permanently limiting human potential isn’t hypothetical anymore.
It’s likely to happen in a very short period of time (relative to the total lifespan of the species), unless indefinite delay of AI development really is socio-politically possible (and from what evidence I’ve seen recently, it doesn’t seem to be). [epistemic certainly — relatively low — 60%]