JackM—these alleged ‘tremendous’ benefits are all hypothetical and speculative.
Whereas the likely X risk from ASI have been examined in detail by thousands of serious people, and polls show that most people, both inside and outside the AI industry, are deeply concerned by them.
This is why I think it’s deeply unethical for 80k Hours to post jobs to work on ASI within AI companies.
I share your concern about x-risk from ASI, that’s why I want safety-aligned people in these roles as opposed to people who aren’t concerned about the risks.
There are genuine proposals on how to align ASI, so I think it’s possible. I’m not sure what the chances are, but I think it’s possible. I think the most promising proposals involve using advanced AI to assist with oversight, interpretability, and recursive alignment tasks—eventually building a feedback loop where aligned systems help align more powerful successors.
JackM—these alleged ‘tremendous’ benefits are all hypothetical and speculative.
Whereas the likely X risk from ASI have been examined in detail by thousands of serious people, and polls show that most people, both inside and outside the AI industry, are deeply concerned by them.
This is why I think it’s deeply unethical for 80k Hours to post jobs to work on ASI within AI companies.
I share your concern about x-risk from ASI, that’s why I want safety-aligned people in these roles as opposed to people who aren’t concerned about the risks.
There are genuine proposals on how to align ASI, so I think it’s possible. I’m not sure what the chances are, but I think it’s possible. I think the most promising proposals involve using advanced AI to assist with oversight, interpretability, and recursive alignment tasks—eventually building a feedback loop where aligned systems help align more powerful successors.
I don’t agree that benefits are speculative by the way. DeepMind has already won the Nobel prize for Chemistry for their work on protein folding.
EDIT: 80,000 Hours also doesn’t seem to promote all roles, only those which contribute to safety, which seems reasonable to me.