I believe the end-goal isn’t a world ruled by a benevolent global elite that owns all the robots. The goal isn’t to create a ‘techno-leviathan’ for people to ride. The goal is to find a benevolent God in mind design space. One we would be happy to give up sovereignty to. That’s I think what AI alignment is about.
Either way, I think we’re going to need some serious ‘first principles’ work at the intersection of AI alignment and political philosophy. “What is the nature of a just political and economic order when humans are economically useless and authority lies with a superhuman AI?” “What institution would even have the legitimacy to ask this question, let alone answer it?”
I believe the end-goal isn’t a world ruled by a benevolent global elite that owns all the robots. The goal isn’t to create a ‘techno-leviathan’ for people to ride. The goal is to find a benevolent God in mind design space. One we would be happy to give up sovereignty to. That’s I think what AI alignment is about.
(A related discussion on LW.)
Either way, I think we’re going to need some serious ‘first principles’ work at the intersection of AI alignment and political philosophy. “What is the nature of a just political and economic order when humans are economically useless and authority lies with a superhuman AI?” “What institution would even have the legitimacy to ask this question, let alone answer it?”