I share your concern about x-risk from ASI, that’s why I want safety-aligned people in these roles as opposed to people who aren’t concerned about the risks.
There are genuine proposals on how to align ASI, so I think it’s possible. I’m not sure what the chances are, but I think it’s possible. I think the most promising proposals involve using advanced AI to assist with oversight, interpretability, and recursive alignment tasks—eventually building a feedback loop where aligned systems help align more powerful successors.
I share your concern about x-risk from ASI, that’s why I want safety-aligned people in these roles as opposed to people who aren’t concerned about the risks.
There are genuine proposals on how to align ASI, so I think it’s possible. I’m not sure what the chances are, but I think it’s possible. I think the most promising proposals involve using advanced AI to assist with oversight, interpretability, and recursive alignment tasks—eventually building a feedback loop where aligned systems help align more powerful successors.
I don’t agree that benefits are speculative by the way. DeepMind has already won the Nobel prize for Chemistry for their work on protein folding.
EDIT: 80,000 Hours also doesn’t seem to promote all roles, only those which contribute to safety, which seems reasonable to me.