She co-authored a piece a few months back about finding AI safety emotionally compelling. I’d be interested in her thoughts on the following two questions related to that!
How worried should we be about suspicious convergence between AI safety being one of the most interesting/emotionally compelling questions to think about and it being the most pressing problem? There used to be a lot of discussion around 2015 about how it seemed like people were working on AI safety because it’s really fun and interesting to think about, rather than because it’s actually that pressing. I think that argument is pretty clearly false, but I’d be curious how she views this post as interacting with those concerns.
It seems a bit like the post doesn’t draw a clean distinction between capabilities and safety. I agree that, to some extent, they’re inseparable (the people building transformative AI should care about making it safe), but how does she view the downside risks of, e.g., some of the most compelling parts of AI work being capabilities-related? More generally, how worried should we be, as a community, about how interconnected safety and capabilities work are?
Somewhat related: As Patrick Collison puts it, people working on making more effective engineered viruses aren’t high-status among people working on pandemic prevention, so why are capabilities researchers high-status among safety researchers?
(I have a decent sense of different answers within the community – this is not really a top concern of mine – but I’d nonetheless be interested in her take! My sense is that (1) the distinction isn’t nearly as clean since you want to build AI and make it go safely and (2) it’s good for capabilities work to be more safety-geared than the counterfactual.)
She co-authored a piece a few months back about finding AI safety emotionally compelling. I’d be interested in her thoughts on the following two questions related to that!
How worried should we be about suspicious convergence between AI safety being one of the most interesting/emotionally compelling questions to think about and it being the most pressing problem? There used to be a lot of discussion around 2015 about how it seemed like people were working on AI safety because it’s really fun and interesting to think about, rather than because it’s actually that pressing. I think that argument is pretty clearly false, but I’d be curious how she views this post as interacting with those concerns.
It seems a bit like the post doesn’t draw a clean distinction between capabilities and safety. I agree that, to some extent, they’re inseparable (the people building transformative AI should care about making it safe), but how does she view the downside risks of, e.g., some of the most compelling parts of AI work being capabilities-related? More generally, how worried should we be, as a community, about how interconnected safety and capabilities work are?
Somewhat related: As Patrick Collison puts it, people working on making more effective engineered viruses aren’t high-status among people working on pandemic prevention, so why are capabilities researchers high-status among safety researchers?
(I have a decent sense of different answers within the community – this is not really a top concern of mine – but I’d nonetheless be interested in her take! My sense is that (1) the distinction isn’t nearly as clean since you want to build AI and make it go safely and (2) it’s good for capabilities work to be more safety-geared than the counterfactual.)
Great questions! Can you clarify a little more on how safety and capabilities work is interconnected?