Was going to write a longer comment but I basically agree with Buck’s take here.
It’s a little hard to evaluate the counterfactuals here, but I’d much rather have the contributions from EA safety than from non EA safety over the last ten years.
I wanted to endorse this in particular.
On the actual argument:
1. EA safety is small, even relative to a single academic subfield.
2. There is overlap between capabilities and short-term safety work.
3. There is overlap between short-term safety work and long-term safety work.
4. So AI safety is less neglected than the opening quotes imply.
5. Also, on present trends, there’s a good chance that academia will do more safety over time, eventually dwarfing the contribution of EA.
I agree with 1, 2, and 3 (though perhaps disagree with the magnitude of 2 and 3, e.g. you list a bunch of related areas and for most of them I’d be surprised if they mattered much for AGI alignment).
I agree 4 is literally true, but I’m not sure it necessarily matters, as this sort of thing can be said for ~any field (as Ben Todd notes). It would be weird to say that animal welfare is not neglected because of the huge field of academia studying animals, even though those fields are relevant to questions of e.g. sentience or farmed animal welfare.
I strongly agree with 5 (if we replace “academia” with “academia + industry”, it’s plausible to me academia never gets involved while industry does), and when I argue that “work will be done by non-EAs”, I’m talking about future work, not current work.
Was going to write a longer comment but I basically agree with Buck’s take here.
I wanted to endorse this in particular.
On the actual argument:
I agree with 1, 2, and 3 (though perhaps disagree with the magnitude of 2 and 3, e.g. you list a bunch of related areas and for most of them I’d be surprised if they mattered much for AGI alignment).
I agree 4 is literally true, but I’m not sure it necessarily matters, as this sort of thing can be said for ~any field (as Ben Todd notes). It would be weird to say that animal welfare is not neglected because of the huge field of academia studying animals, even though those fields are relevant to questions of e.g. sentience or farmed animal welfare.
I strongly agree with 5 (if we replace “academia” with “academia + industry”, it’s plausible to me academia never gets involved while industry does), and when I argue that “work will be done by non-EAs”, I’m talking about future work, not current work.