Loved the language in the post! To the point without having to use unnecessary jargon.
There are two things I’d like you to elaborate on if possible:
> “the challenge is getting AIs to do what it says on the tin—to reliably do whatever a human operator tells them to do.”
If I understand correctly you imply that there is still a human operator to a superhuman AGI, do you think this is the way that alignment will work out? What I see is that humans have flaws, do we really want to give a “genie” / extremely powerful tool to humans that even already struggle with the powerful tools that they have? At least right now these powerful tools are in the hands of the more responsible few, but if it becomes more widely accessible that’s very different.
What do you think of going the direction of developing a “Guardian AI”, which would still solve the alignment problem using the tools of ML, but involving humans giving up control of the alignment?
The second one is more practical, which action do you think one should take. I’ve of course read the recommendations that other people have put out there so far, but would be curious to hear your take on this.
Loved the language in the post! To the point without having to use unnecessary jargon.
There are two things I’d like you to elaborate on if possible:
> “the challenge is getting AIs to do what it says on the tin—to reliably do whatever a human operator tells them to do.”
If I understand correctly you imply that there is still a human operator to a superhuman AGI, do you think this is the way that alignment will work out? What I see is that humans have flaws, do we really want to give a “genie” / extremely powerful tool to humans that even already struggle with the powerful tools that they have? At least right now these powerful tools are in the hands of the more responsible few, but if it becomes more widely accessible that’s very different.
What do you think of going the direction of developing a “Guardian AI”, which would still solve the alignment problem using the tools of ML, but involving humans giving up control of the alignment?
The second one is more practical, which action do you think one should take. I’ve of course read the recommendations that other people have put out there so far, but would be curious to hear your take on this.