Given that technical AI alignment is impossible, we should focus on political solutions, even though they seem impractical. Running any sufficiently powerful computer system should be treated as launching a nuclear weapon. Major military powers can, and should, coordinate to not do this and destroy any private actor who attempts to do it.
This may seem like an unworkable fantasy now, but if takeoff is slow, there will be a ‘Thalidomide moment’ when an unaligned but not super-intelligent AI does something very bad and scary but is ultimately stopped. We should be ready to capitalize on that moment and ride the public wave of techno-phobia to put in sensible ‘AI arms control’ policies.
Given that technical AI alignment is impossible, we should focus on political solutions, even though they seem impractical. Running any sufficiently powerful computer system should be treated as launching a nuclear weapon. Major military powers can, and should, coordinate to not do this and destroy any private actor who attempts to do it.
This may seem like an unworkable fantasy now, but if takeoff is slow, there will be a ‘Thalidomide moment’ when an unaligned but not super-intelligent AI does something very bad and scary but is ultimately stopped. We should be ready to capitalize on that moment and ride the public wave of techno-phobia to put in sensible ‘AI arms control’ policies.
Technical AI alignment isn’t impossible, we just don’t currently know how to do it. (And it looks hard.)