Cooperative AI

TagLast edit: 12 May 2025 19:23 UTC by Dane Valerie

Cooperative AI is a subfield of artificial intelligence that aims to improve the ability of AI systems to engender cooperation between humans, machines, and institutions. It can therefore also be seen as part of the AI safety landscape, aimed at preventing AI risks and existential risks resulting from cooperation failure.

Further Reading

Baumann, Tobias (2022) Cooperative artificial intelligence, arXiv:2202.09859.

Dafoe, Allan et al. (2020) Open problems in cooperative AI, arXiv:2012.08630.

Dafoe, Allan et al. (2021) Cooperative AI: machines must learn to find common ground, Nature, vol. 593, pp. 33–36.

Related Entries

AI safety | Center on Long-Term Risk | Cooperative AI Foundation

Cooperative AI: Three things that confused me as a beginner (and my current understanding)

C Tilli16 Apr 2024 7:06 UTC

59 points

12 comments6 min readEA link

Brian Tomasik on cooperation and peace

Vasco Grilo🔸20 May 2024 17:01 UTC

34 points

1 comment4 min readEA link

(reducing-suffering.org)

Boomerang—protocol to dissolve some commitment races

Filip Sondej30 May 2023 16:24 UTC

20 points

0 comments8 min readEA link

(www.lesswrong.com)

Project ideas: Backup plans & Cooperative AI

Lukas Finnveden4 Jan 2024 7:26 UTC

25 points

2 comments13 min readEA link

(www.forethought.org)

[Question] Game theory work on AI alignment with diverse AI systems, human individuals, & human groups?

Geoffrey Miller2 Mar 2023 16:50 UTC

22 points

2 comments1 min readEA link

AGI Battle Royale: Why “slow takeover” scenarios devolve into a chaotic multi-AGI fight to the death

titotal22 Sep 2022 15:00 UTC

55 points

12 comments15 min readEA link

Cosmic AI safety

Magnus Vinding6 Dec 2024 22:32 UTC

24 points

5 comments6 min readEA link

Being honest with AIs

Lukas Finnveden21 Aug 2025 3:57 UTC

48 points

1 comment17 min readEA link

(blog.redwoodresearch.org)

Launching Foresight Institute’s AI Grant for Underexplored Approaches to AI Safety – Apply for Funding!

elte17 Aug 2023 7:27 UTC

48 points

0 comments2 min readEA link

Stackelberg Games and Cooperative Commitment: My Thoughts and Reflections on a 2-Month Research Project

Ben Bucknall13 Dec 2021 10:49 UTC

18 points

1 comment9 min readEA link

New Report: Multi-Agent Risks from Advanced AI

Lewis Hammond23 Feb 2025 0:32 UTC

40 points

3 comments2 min readEA link

(www.cooperativeai.com)

Applications Open for the Cooperative AI Summer School 2025!

C Tilli13 Jan 2025 12:31 UTC

25 points

0 comments1 min readEA link

Lewis Hammond: Introduction to Cooperative AI

Group Organizer29 Apr 2022 9:52 UTC

3 points

0 comments1 min readEA link

Open Problems in AI X-Risk [PAIS #5]

TW12310 Jun 2022 2:22 UTC

44 points

1 comment36 min readEA link

Apply to the Cooperative AI PhD Fellowship by October 14th!

Lewis Hammond5 Oct 2024 12:41 UTC

35 points

0 comments1 min readEA link

2024 S-risk Intro Fellowship

Center on Long-Term Risk12 Oct 2023 19:14 UTC

89 points

2 comments1 min readEA link

Why Moral Conflict Resolution Still Breaks Our Best Safety Tools

JBug18 Nov 2025 7:49 UTC

6 points

0 comments2 min readEA link

Apply to the Cooperative AI Summer School!

reddington3 Apr 2024 12:13 UTC

26 points

0 comments1 min readEA link

⿻ Symbiogenesis vs. Convergent Consequentialism

plex21 Oct 2025 10:40 UTC

17 points

1 comment20 min readEA link

[Job] Managing Director at the Cooperative AI Foundation ($5000 Referral Bonus)

Lewis Hammond3 Jul 2023 16:02 UTC

31 points

0 comments1 min readEA link

Refer the Cooperative AI Foundation’s New COO, Receive $5000

Lewis Hammond16 Jun 2022 13:27 UTC

42 points

0 comments3 min readEA link

Center on Long-Term Risk: Annual review and fundraiser 2023

Center on Long-Term Risk13 Dec 2023 16:42 UTC

79 points

3 comments4 min readEA link

CLR Summer Research Fellowship 2024

Center on Long-Term Risk15 Feb 2024 18:26 UTC

89 points

2 comments8 min readEA link

Announcing the Cooperative AI Research Fellowship

leo_hyams11 Sep 2025 13:48 UTC

10 points

0 comments8 min readEA link

GovAI Annual Report 2021

GovAI5 Jan 2022 16:57 UTC

52 points

2 comments9 min readEA link

PIBBSS Fellowship 2025: Bounties and Cooperative AI Track Announcement

Dušan D. Nešić (Dushan)9 Jan 2025 14:23 UTC

18 points

0 comments1 min readEA link

Announcing the CLR Foundations Course and CLR S-Risk Seminars

James Faville19 Nov 2024 1:18 UTC

52 points

2 comments3 min readEA link

Apply to the Cooperative AI PhD Fellowship by November 16th!

Lewis Hammond1 Nov 2025 12:15 UTC

23 points

0 comments1 min readEA link

No comments.