Executive summary: The author shares their understanding of cooperative AI as an emerging field focused on making things go well in a world with multiple powerful AI systems and diverse human values, distinct from but related to AI alignment.
Key points:
Cooperative AI lacks a single agreed-upon definition, but broadly aims to promote cooperation and social welfare among multiple AI systems and humans with diverse values.
Cooperative intelligence is an agent’s ability to achieve goals in socially beneficial ways across varied environments and interactions, and is relevant for addressing social dilemmas between AI systems.
The capabilities involved in cooperative intelligence (understanding, communication, commitment, norms) are dual-use and require caution.
Cooperative AI overlaps with AI safety on multipolar scenarios and catastrophic risks, and with beneficial AI on using AI to foster large-scale human cooperation.
Key uncertainties include the boundaries and overlaps between cooperative AI, AI ethics, and AI safety.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
Executive summary: The author shares their understanding of cooperative AI as an emerging field focused on making things go well in a world with multiple powerful AI systems and diverse human values, distinct from but related to AI alignment.
Key points:
Cooperative AI lacks a single agreed-upon definition, but broadly aims to promote cooperation and social welfare among multiple AI systems and humans with diverse values.
Cooperative intelligence is an agent’s ability to achieve goals in socially beneficial ways across varied environments and interactions, and is relevant for addressing social dilemmas between AI systems.
The capabilities involved in cooperative intelligence (understanding, communication, commitment, norms) are dual-use and require caution.
Cooperative AI overlaps with AI safety on multipolar scenarios and catastrophic risks, and with beneficial AI on using AI to foster large-scale human cooperation.
Key uncertainties include the boundaries and overlaps between cooperative AI, AI ethics, and AI safety.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.