As a result of AGI SF readings and other sporadic AI safety readings…
… I feel more confident asking questions of people who know more than I do
I feel like I know the vocabulary, main threat scenarios, and rough approaches to solving the problem, such that I can situate new facts into existing taxonomies
… I’m better able to tell when prominent people disagree / things have more texture
Some (self-)critiques
Honestly thought content for some of the weeks was a bit weak if you just wanted an overview of the alignment problem (e.g., adversarial techniques for scalable oversight probably isn’t what you need to understand if you’re trying to assess the risk)
I wish I’d set up strong accountability
Wish it had more counter-arguments to classic AI risk.
Apparently Stanford AI group modified curriculum to have more of these?
Wish I’d been more active in discussion or questions beforehand: give yourself a chance to be wrong!
Tips I’d recommend for learning more about AI risk
Just start! Always feels daunting to dive into but just find a few explainer articles and dive in
Spaced repetition for learning really does go hard
Talk to knowledgeable people who give you space to be wrong
Write up your thoughts and have knowledgeable people poke holes/ show you where you’re missing something
A few resources that I keep coming back to (not from AGI SF):
As a result of AGI SF readings and other sporadic AI safety readings…
… I feel more confident asking questions of people who know more than I do
I feel like I know the vocabulary, main threat scenarios, and rough approaches to solving the problem, such that I can situate new facts into existing taxonomies
… I’m better able to tell when prominent people disagree / things have more texture
Some (self-)critiques
Honestly thought content for some of the weeks was a bit weak if you just wanted an overview of the alignment problem (e.g., adversarial techniques for scalable oversight probably isn’t what you need to understand if you’re trying to assess the risk)
I wish I’d set up strong accountability
Wish it had more counter-arguments to classic AI risk.
Apparently Stanford AI group modified curriculum to have more of these?
Wish I’d been more active in discussion or questions beforehand: give yourself a chance to be wrong!
Tips I’d recommend for learning more about AI risk
Just start! Always feels daunting to dive into but just find a few explainer articles and dive in
Spaced repetition for learning really does go hard
Talk to knowledgeable people who give you space to be wrong
Write up your thoughts and have knowledgeable people poke holes/ show you where you’re missing something
A few resources that I keep coming back to (not from AGI SF):
* Taxonomy of AI risks from CAIS
* Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover
Simulators