S-risk worries (including animal suffering) mostly don’t imply different actions. At best they perhaps imply prioritizing corrigibility, or potentially imply that you should actively want worse alignment and higher capabilities, in hopes of AI merely killing everyone. The second would of course be a big shift in actions, but I doubt most are willing to bite that bullet. I am skeptical that there’s much alignment work that preferentially addresses S-risks. If there was, then I would agree that it should be prioritized.
I think agent foundations generally is neglected relative to everything else. On my view, RL is very dangerous and interp probably insufficient. You might think agent foundations is too intractable to become useful, but frankly it looks like barely anyone is trying.
The main drivers of my answers:
S-risk worries (including animal suffering) mostly don’t imply different actions. At best they perhaps imply prioritizing corrigibility, or potentially imply that you should actively want worse alignment and higher capabilities, in hopes of AI merely killing everyone. The second would of course be a big shift in actions, but I doubt most are willing to bite that bullet. I am skeptical that there’s much alignment work that preferentially addresses S-risks. If there was, then I would agree that it should be prioritized.
I think agent foundations generally is neglected relative to everything else. On my view, RL is very dangerous and interp probably insufficient. You might think agent foundations is too intractable to become useful, but frankly it looks like barely anyone is trying.