Executive summary: The author argues that the EA community often takes classical utilitarianism for granted without sufficiently addressing its theoretical challenges, which could lead to potentially catastrophic outcomes if flawed utilitarian assumptions influence future AI systems.
Key points:
Many EA discussions assume classical utilitarianism without critically examining its foundations and issues like the repugnant conclusion.
Derek Parfit’s work on population ethics and “Perfectionism” offers important critiques of utilitarianism that deserve more attention.
Uncritically adhering to utilitarianism could lead to disastrous outcomes, like an AI optimizing for “happy ants” instead of human flourishing.
The author questions whether EA is funding research to address utilitarianism’s challenges or just accepting it as the default ethical framework.
A thought experiment (“Countless Ants Trolley”) illustrates how some values may be incommensurable, challenging utilitarian aggregation.
The author calls for more rigorous examination of metaethical issues before proceeding with longtermist efforts that could have cosmic consequences.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
Executive summary: The author argues that the EA community often takes classical utilitarianism for granted without sufficiently addressing its theoretical challenges, which could lead to potentially catastrophic outcomes if flawed utilitarian assumptions influence future AI systems.
Key points:
Many EA discussions assume classical utilitarianism without critically examining its foundations and issues like the repugnant conclusion.
Derek Parfit’s work on population ethics and “Perfectionism” offers important critiques of utilitarianism that deserve more attention.
Uncritically adhering to utilitarianism could lead to disastrous outcomes, like an AI optimizing for “happy ants” instead of human flourishing.
The author questions whether EA is funding research to address utilitarianism’s challenges or just accepting it as the default ethical framework.
A thought experiment (“Countless Ants Trolley”) illustrates how some values may be incommensurable, challenging utilitarian aggregation.
The author calls for more rigorous examination of metaethical issues before proceeding with longtermist efforts that could have cosmic consequences.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.