some think the right number of people to work on [AI risk] is quite a bit above zero.
I reckon you could safely use a stronger word than “some” there! EAs can risk boxing themselves into an unfortunate rhetorical corner with our talk of picking the single best cause or marginal action. This talk can obscure the fact that plenty of those who don’t think any specific AI-focused charity is the best place for them to donate to on the margin would want to have some funding go to AI safety if they were hypothetically deciding the distribution of all the funding in the world.
That’s right. It would seem extremely unlikely that one should have a multi-billion dollar industry with no-one thinking about what happens if it succeeds at its aim.
It’s very important for EAs to recognise that there probably isn’t a single best cause (and that even if there is, the uncertainties are too big to allow us to identify it). Even if there was an identifiable best cause, it is likely to change, so it’s bad for EAs to identify too strongly with any one cause.
There’s a broader risk in focusing on marginal cost-effectiveness—that it leads to local rather than global optimisation. It’s a good heuristic, but bad to rely on too much.
I reckon you could safely use a stronger word than “some” there! EAs can risk boxing themselves into an unfortunate rhetorical corner with our talk of picking the single best cause or marginal action. This talk can obscure the fact that plenty of those who don’t think any specific AI-focused charity is the best place for them to donate to on the margin would want to have some funding go to AI safety if they were hypothetically deciding the distribution of all the funding in the world.
That’s right. It would seem extremely unlikely that one should have a multi-billion dollar industry with no-one thinking about what happens if it succeeds at its aim.
It’s very important for EAs to recognise that there probably isn’t a single best cause (and that even if there is, the uncertainties are too big to allow us to identify it). Even if there was an identifiable best cause, it is likely to change, so it’s bad for EAs to identify too strongly with any one cause.
There’s a broader risk in focusing on marginal cost-effectiveness—that it leads to local rather than global optimisation. It’s a good heuristic, but bad to rely on too much.