Engaging seriously with the (nontechnical arguments) for AI Risk: One person’s core recommended reading list (I saw this list in a private message from a more well-read EA than me and wanted to write it up, it’s not my list since I haven’t read most of these, but I thought it was better to have it be public than not):
Intro to ML Safety (lots of examples of AI safety work being done in a modern ML paradigm. There’s debate about exactly how much is relevant to existential safety)
Engaging seriously with the (nontechnical arguments) for AI Risk: One person’s core recommended reading list
(I saw this list in a private message from a more well-read EA than me and wanted to write it up, it’s not my list since I haven’t read most of these, but I thought it was better to have it be public than not):
Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover—Ajeya Cotra
AI Timelines: Where the Arguments, and the “Experts,” Stand—Holden Karnofsky
Many AI experts think there’s a non-negligible chance AI will lead to outcomes as bad as extinction (section in Preventing an AI-related catastrophe 80,000 Hours Problem Profile)
Is Power-Seeking AI an Existential Risk? - Report by Joe Carlsmith
If still unconvinced, might recommend (as examples of arguments most uncorrelated with the above)
AGI Ruin: A List of Lethalities—Eliezer Yudkowsky
Where I agree and disagree with Eliezer—Paul Christiano (partly in response to the above)
Unsolved Problems in ML Safety—Dan Hendrycks
For going deeper:
Intro to ML Safety (lots of examples of AI safety work being done in a modern ML paradigm. There’s debate about exactly how much is relevant to existential safety)
AGI Safety Fundamentals Course
Might as well put a list of skilling up possibilities (probably this has been done before)
Correct me if there are mistakes here
Cambridge’s AGI Safety Fundamentals Course
Curriculum written by Richard Ngo
Machine Learning Safety Scholars
Dan Hendrycks
9 week program for undergrads
SERI / MATS
3-4 weeks
Get matched with a mentor
Do actual alignment research
Ryan Kidd and Victor Warlop
Machine Learning Alignment Bootcamp
Get people interested in alignment really good at machine learning
Run through Redwood
Buck Schlegeris and Max Nadeau
AI Safety Camp
Get matched with a mentor
Several months long
Doing research
Center for AI Safety’s Philosophy Fellowship
PIBBSS Summer Research Fellowship
For huge range of academic backgrounds