Props for thinking outside the box, although there’s a few challenges that jump immediately to mind:
• It’s not easy to make an event prestigious. There’s a chicken and egg problem. Companies won’t want to enter unless it’s prestigious and it won’t be prestigious unless they enter. • Companies racing each other for a trillion-dollar prize might not want to distract themselves by entering even if it were prestigious. • The subjectivity of the competition may reduce the value that people place on this competition.
Executive summary: The author proposes establishing a biennial Human-centred AGI Olympiad tournament where teams compete with general purpose AI agents on real-world tasks designed to test whether AIs adhere to core human values like being helpful, honest, and harmless.
Key points:
An AGI Olympiad could influence incentives driving AGI development towards safety, provide visibility into AI capabilities and risks, and open opportunities to shape policy.
Careful design and implementation focused on beneficial tasks is needed to achieve positive impacts without enabling dual use capabilities.
The proposal aims to leverage positive incentives through competition prestige and corporate sponsorship rather than just relying on punitive measures.
Tasks could span healthcare, legal services, government operations, and business management. The author cautions against areas like science, engineering and forensics that may enable systemic risks.
Success requires significant funding, partnerships with reputable organizations, participation from leading AI labs willing to accept constraints, and targeting 2026 for the first event.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
Props for thinking outside the box, although there’s a few challenges that jump immediately to mind:
• It’s not easy to make an event prestigious. There’s a chicken and egg problem. Companies won’t want to enter unless it’s prestigious and it won’t be prestigious unless they enter.
• Companies racing each other for a trillion-dollar prize might not want to distract themselves by entering even if it were prestigious.
• The subjectivity of the competition may reduce the value that people place on this competition.
Executive summary: The author proposes establishing a biennial Human-centred AGI Olympiad tournament where teams compete with general purpose AI agents on real-world tasks designed to test whether AIs adhere to core human values like being helpful, honest, and harmless.
Key points:
An AGI Olympiad could influence incentives driving AGI development towards safety, provide visibility into AI capabilities and risks, and open opportunities to shape policy.
Careful design and implementation focused on beneficial tasks is needed to achieve positive impacts without enabling dual use capabilities.
The proposal aims to leverage positive incentives through competition prestige and corporate sponsorship rather than just relying on punitive measures.
Tasks could span healthcare, legal services, government operations, and business management. The author cautions against areas like science, engineering and forensics that may enable systemic risks.
Success requires significant funding, partnerships with reputable organizations, participation from leading AI labs willing to accept constraints, and targeting 2026 for the first event.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.