Executive summary: The author, who previously expected aligned ASI to be good for all sentient beings through coherent extrapolated volition, now expresses uncertainty about whether current alignment approaches would achieve this, though estimates a 70% probability that aligned ASI would be good for animals.
Key points:
The author previously believed coherent extrapolated volition would lead aligned ASI to recognize and address animal suffering, but current alignment research has abandoned this approach.
Current alignment work using constitutions and RLHF locks in values like “virtues” rather than achieving coherent extrapolation, and it remains unclear how virtue ethics could be formalized into a coherent decision theory for ASI.
Claude’s Constitution treats animal welfare as one value among many to weigh, leaving unclear whether an ASI following such a constitution would take action on issues like factory farming.
The author identifies a positive correlation between alignment techniques that actually work and those good for animals, suggesting barbell outcomes: either good for all sentient beings or bad for all.
The field prioritizes alignment techniques unlikely to work well long-term, and if these “streetlight effect” techniques somehow succeed, they would likely benefit humans but not animals.
The author estimates that aligned ASI has a 70% probability of being good for animals, derived from a 30% probability of “deep” solutions (80% animal-friendly) and a 15% probability of popular techniques (50% animal-friendly).
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
The author identifies a positive correlation between alignment techniques that actually work and those good for animals
Not what I said. I said there’s a correlation between alignment techniques that are more likely to work. There are no known alignment techniques that “actually work”.
Executive summary: The author, who previously expected aligned ASI to be good for all sentient beings through coherent extrapolated volition, now expresses uncertainty about whether current alignment approaches would achieve this, though estimates a 70% probability that aligned ASI would be good for animals.
Key points:
The author previously believed coherent extrapolated volition would lead aligned ASI to recognize and address animal suffering, but current alignment research has abandoned this approach.
Current alignment work using constitutions and RLHF locks in values like “virtues” rather than achieving coherent extrapolation, and it remains unclear how virtue ethics could be formalized into a coherent decision theory for ASI.
Claude’s Constitution treats animal welfare as one value among many to weigh, leaving unclear whether an ASI following such a constitution would take action on issues like factory farming.
The author identifies a positive correlation between alignment techniques that actually work and those good for animals, suggesting barbell outcomes: either good for all sentient beings or bad for all.
The field prioritizes alignment techniques unlikely to work well long-term, and if these “streetlight effect” techniques somehow succeed, they would likely benefit humans but not animals.
The author estimates that aligned ASI has a 70% probability of being good for animals, derived from a 30% probability of “deep” solutions (80% animal-friendly) and a 15% probability of popular techniques (50% animal-friendly).
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.
Not what I said. I said there’s a correlation between alignment techniques that are more likely to work. There are no known alignment techniques that “actually work”.