Executive summary: The author argues that alignment approaches closer to encoding generalized concern for beings’ preferences are more likely to benefit non-humans, but believes current research agendas have <5% chance of solving alignment, so these distinctions likely have limited practical impact.
Key points:
The author frames alignment methods on a spectrum from optimizing for users’ immediate preferences to embedding respect for all beings’ preferences, with the latter more favorable to non-human welfare.
The author estimates that “all of today’s research agendas combined have less than a 5% chance of solving alignment,” limiting the real-world importance of prioritizing non-human-friendly approaches.
Iterative alignment methods like RLHF are likely “bad for non-humans” because training pressures will remove unsolicited concern for animal welfare to satisfy user preferences.
Alignment theory and multi-agent cooperation are judged “good for non-humans” because they may encode “concern-for-all-welfare” or include non-humans in cooperative frameworks, though both are difficult to advance.
Model psychology interventions (e.g., constitutions including non-human welfare) are “somewhat good” and tractable, but the author doubts they will influence “an ASI’s true preferences.”
Several categories (e.g., interpretability, scalable oversight, honesty, data-level safety) are labeled unclear due to uncertainty about how they would affect whether AI systems ultimately consider non-human interests.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
Executive summary: The author argues that alignment approaches closer to encoding generalized concern for beings’ preferences are more likely to benefit non-humans, but believes current research agendas have <5% chance of solving alignment, so these distinctions likely have limited practical impact.
Key points:
The author frames alignment methods on a spectrum from optimizing for users’ immediate preferences to embedding respect for all beings’ preferences, with the latter more favorable to non-human welfare.
The author estimates that “all of today’s research agendas combined have less than a 5% chance of solving alignment,” limiting the real-world importance of prioritizing non-human-friendly approaches.
Iterative alignment methods like RLHF are likely “bad for non-humans” because training pressures will remove unsolicited concern for animal welfare to satisfy user preferences.
Alignment theory and multi-agent cooperation are judged “good for non-humans” because they may encode “concern-for-all-welfare” or include non-humans in cooperative frameworks, though both are difficult to advance.
Model psychology interventions (e.g., constitutions including non-human welfare) are “somewhat good” and tractable, but the author doubts they will influence “an ASI’s true preferences.”
Several categories (e.g., interpretability, scalable oversight, honesty, data-level safety) are labeled unclear due to uncertainty about how they would affect whether AI systems ultimately consider non-human interests.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.