There’s “norms” against burying bombs so whoever steps on them gets blown to pieces. “Norms” against anonymously firing artillery shells at an enemy you cannot even see.
Yet humans eagerly participate in these activities in organized ways because governments win and winning is all that matters.
Does developing AGI let you win, yes or no. Do current world powers believe it will let them win all?
I anticipate your answers are : no, yes. Mine are yes, yes.
This is because you are implicitly assuming early AGI systems will escape human control immediately or prepare a grand betrayal. I think that’s science fiction because you can make an AGI that exists only a moment at a time and it has no opportunity to do any of this.
There’s “norms” against burying bombs so whoever steps on them gets blown to pieces. “Norms” against anonymously firing artillery shells at an enemy you cannot even see.
Yet humans eagerly participate in these activities in organized ways because governments win and winning is all that matters.
Does developing AGI let you win, yes or no. Do current world powers believe it will let them win all?
I anticipate your answers are : no, yes. Mine are yes, yes.
This is because you are implicitly assuming early AGI systems will escape human control immediately or prepare a grand betrayal. I think that’s science fiction because you can make an AGI that exists only a moment at a time and it has no opportunity to do any of this.
That’s the crux, right?
This is a highly reductive way of looking at the issue.
I think if true this is a solution to the alignment problem? Why not share the deets on LessWrong or arXiv, it’d be a huge boon for the field.