Interesting post, thanks!! Excited to see discussion of it.
One thing I wonder if it’s relevant are the sort of “mixed” pictures? Like, maybe you think totalitarian control of AI would be really bad because it increases the risk of human extinction from misalignment, by increasing the success rate of any attempted AI takeover / making it happen sooner in time (bc a totalitarian might e.g. give more control over to an AI they were using to stay in power).
Or, more vaguely, I sorta feel like if world governments / anyone with ambitions to power think that advanced AI will help them, that’s bad news for alignment efforts.
Anyway, that could be something going on for some people I think?
Hi Bella, yes it is a fascinating read, and thank you for your moderate comment. Although, my first thought was slightly different, I went slightly further than “one could become the consequence of the other”...
Wouldn’t extinction, in itself, be an event where suffering would be short then end? Rather than, for instance, enduring extreme concentration of power (building loyalty, distrusting people around oneself, etc.) or living under a superior intelligence (emulating the harmful dynamics humans had with animals).
Interesting post, thanks!! Excited to see discussion of it.
One thing I wonder if it’s relevant are the sort of “mixed” pictures? Like, maybe you think totalitarian control of AI would be really bad because it increases the risk of human extinction from misalignment, by increasing the success rate of any attempted AI takeover / making it happen sooner in time (bc a totalitarian might e.g. give more control over to an AI they were using to stay in power).
Or, more vaguely, I sorta feel like if world governments / anyone with ambitions to power think that advanced AI will help them, that’s bad news for alignment efforts.
Anyway, that could be something going on for some people I think?
Good point, thanks!
Hi Bella, yes it is a fascinating read, and thank you for your moderate comment. Although, my first thought was slightly different, I went slightly further than “one could become the consequence of the other”...
Wouldn’t extinction, in itself, be an event where suffering would be short then end? Rather than, for instance, enduring extreme concentration of power (building loyalty, distrusting people around oneself, etc.) or living under a superior intelligence (emulating the harmful dynamics humans had with animals).
I could see it as a hierarchy of anxieties