“Some people think that the kinds of risks I’m worried about are far off, farfetched or ridiculous.”
If I made the claim that we had 12 months before all of humanity is wiped by an asteroid, you’d rightly ask me for evidence. Have I picked up a distant rock in space using radio telescopes? Some other tangible proof? Or is it a best-guess, since, hey, it’s technically possible that we could be hit with an asteroid on any given year. Then imagine if I advocate we spend two percent of global GDP preparing for this event.
That’s where the state of AGI fear is—all scenarios depend on wild leaps of faith and successive assumptions that build on each other.
I think you put it well when you said:
“Some people think that the kinds of risks I’m worried about are far off, farfetched or ridiculous.”
If I made the claim that we had 12 months before all of humanity is wiped by an asteroid, you’d rightly ask me for evidence. Have I picked up a distant rock in space using radio telescopes? Some other tangible proof? Or is it a best-guess, since, hey, it’s technically possible that we could be hit with an asteroid on any given year. Then imagine if I advocate we spend two percent of global GDP preparing for this event.
That’s where the state of AGI fear is—all scenarios depend on wild leaps of faith and successive assumptions that build on each other.
I’ve attempted to put this all in one place with this post.