Executive summary: This document compiles a list of research agendas and questions in AI governance that could be important for reducing existential risk, covering topics such as regulation, compute governance, corporate governance, international agreements, misuse, evaluation, China, information security, strategy and forecasting, and post-TAI governance.
Key points:
AI regulation questions focus on estimating the effects of regulatory interventions, and exploring specific issues in the UK, US, and EU.
Compute governance questions examine failure modes and alternative bottlenecks to AI development besides compute.
Corporate governance questions look at the track record of governance tools like ethics boards, responsible scaling policies, and industry standards.
International governance questions explore distinguishing features of effective treaties and drivers of regulatory arbitrage.
Misuse questions focus on empirical evidence for the likelihood of existential risk from AI misuse by malicious actors.
Evaluation questions consider how to ensure the integrity of AI system assessments and how to integrate governance.
China questions examine the relative AI capabilities of China vs. the US/allies and the implications for racing dynamics and governance.
Information security questions look at how infosec concerns affect firm behavior, deterrence of cyattacks, and offensive/defensive dynamics.
Strategy and forecasting questions aim to anticipate AI progress while grounding predictions in solid empirical and theoretical methodology.
Post-TAI governance questions explore issues like the legal status of digital minds and the threat of AI-enabled misinformation to democracy.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
Executive summary: This document compiles a list of research agendas and questions in AI governance that could be important for reducing existential risk, covering topics such as regulation, compute governance, corporate governance, international agreements, misuse, evaluation, China, information security, strategy and forecasting, and post-TAI governance.
Key points:
AI regulation questions focus on estimating the effects of regulatory interventions, and exploring specific issues in the UK, US, and EU.
Compute governance questions examine failure modes and alternative bottlenecks to AI development besides compute.
Corporate governance questions look at the track record of governance tools like ethics boards, responsible scaling policies, and industry standards.
International governance questions explore distinguishing features of effective treaties and drivers of regulatory arbitrage.
Misuse questions focus on empirical evidence for the likelihood of existential risk from AI misuse by malicious actors.
Evaluation questions consider how to ensure the integrity of AI system assessments and how to integrate governance.
China questions examine the relative AI capabilities of China vs. the US/allies and the implications for racing dynamics and governance.
Information security questions look at how infosec concerns affect firm behavior, deterrence of cyattacks, and offensive/defensive dynamics.
Strategy and forecasting questions aim to anticipate AI progress while grounding predictions in solid empirical and theoretical methodology.
Post-TAI governance questions explore issues like the legal status of digital minds and the threat of AI-enabled misinformation to democracy.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.