I have many disagreements, but I’ll focus on one: I think point 2 is in contradiction with points 3 and 4. To put it it plainly: the “selection pressures” go away pretty quickly if we don’t have reliable methods of knowing or controlling what the AI will do, or preventing it from doing noticeably bad stuff. That applies to the obvious stuff like if AI tries to prematurely go skynet, but it also applies to more mundane stuff like getting an AI to act reliably more than 99% of the time.
I believe that if we manage to control AI enough to make widespread rollout feasible, then it’s pretty likely we’ve already solved alignment well enough to prevent extinction.
Hmm right now this seems wrong to me, and also not worth going into in an introductory post. Do you have a sense that your view is commonplace? (eg from talking to many people not involved in AI)
I have many disagreements, but I’ll focus on one: I think point 2 is in contradiction with points 3 and 4. To put it it plainly: the “selection pressures” go away pretty quickly if we don’t have reliable methods of knowing or controlling what the AI will do, or preventing it from doing noticeably bad stuff. That applies to the obvious stuff like if AI tries to prematurely go skynet, but it also applies to more mundane stuff like getting an AI to act reliably more than 99% of the time.
I believe that if we manage to control AI enough to make widespread rollout feasible, then it’s pretty likely we’ve already solved alignment well enough to prevent extinction.
Hmm right now this seems wrong to me, and also not worth going into in an introductory post. Do you have a sense that your view is commonplace? (eg from talking to many people not involved in AI)