Executive summary: Our fundamental moral beliefs about good and bad may arise from motivated reasoning rather than evidence, with implications for how we view moral judgments and the potential for AI systems to have good or bad experiences.
Key points:
Basic moral judgments like “pain is bad” seem to stem from desires rather than evidence-based reasoning.
This theory elegantly explains the universal belief in pain’s badness as motivated by our desire to avoid pain.
If moral beliefs arise from motivated reasoning, it raises questions about their truth status and validity.
Language models may be capable of good/bad experiences if they engage in motivated reasoning about preferences.
Consistent judgments may be necessary for beliefs about goodness/badness, creating uncertainty about whether current AI systems truly have such experiences.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
Executive summary: Our fundamental moral beliefs about good and bad may arise from motivated reasoning rather than evidence, with implications for how we view moral judgments and the potential for AI systems to have good or bad experiences.
Key points:
Basic moral judgments like “pain is bad” seem to stem from desires rather than evidence-based reasoning.
This theory elegantly explains the universal belief in pain’s badness as motivated by our desire to avoid pain.
If moral beliefs arise from motivated reasoning, it raises questions about their truth status and validity.
Language models may be capable of good/bad experiences if they engage in motivated reasoning about preferences.
Consistent judgments may be necessary for beliefs about goodness/badness, creating uncertainty about whether current AI systems truly have such experiences.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.