Surely having people work hard to reduce existential risks lowers existential risks relative to them not doing that in expectation. Especially when there are just a few people working on some of the biggest risks.
Why would that be the case? I could work from my hut in Uganda on trying to shoot down Russian drones over Ukraine and it would not achieve anything. Obviously that’s a dumb strawmanb but just because people work on something doesn’t mean they will achieve anything.
And apart from the modal return being probably ~0 (which might not be a problem if funding lots of useless stuff guarantees sufficiently useful projects get funded), it is also possible for longtermist work to be actively counterproductive, with significant uncertainty around the direction as well as the magnitude of the impact of many longtermist activities[1]
Indeed many longtermist arguments strongly imply that work of other longtermists is counterproductive (it is hard to see how Anthropic and Pause AI can both be right about frontier models)
And even you’re certain you’ve picked the correct side of a particular risk-mitigation debate, political donations and campaigning are particularly prone to provoke responses from opponents. This is true of a lot of campaigning, but “AI is very dangerous” is more easily inadvertently motivating accelerationists to act to militarise it first than “this suffering could be stopped” is pivoted into an argument for more cages or fewer bednets
there are probably exceptions to this; it’s difficult to imagine that an asteroid early warning system will hurt us, even if it turns out to be useless
Surely having people work hard to reduce existential risks lowers existential risks relative to them not doing that in expectation. Especially when there are just a few people working on some of the biggest risks.
Why would that be the case? I could work from my hut in Uganda on trying to shoot down Russian drones over Ukraine and it would not achieve anything. Obviously that’s a dumb strawmanb but just because people work on something doesn’t mean they will achieve anything.
And apart from the modal return being probably ~0 (which might not be a problem if funding lots of useless stuff guarantees sufficiently useful projects get funded), it is also possible for longtermist work to be actively counterproductive, with significant uncertainty around the direction as well as the magnitude of the impact of many longtermist activities[1]
Indeed many longtermist arguments strongly imply that work of other longtermists is counterproductive (it is hard to see how Anthropic and Pause AI can both be right about frontier models)
And even you’re certain you’ve picked the correct side of a particular risk-mitigation debate, political donations and campaigning are particularly prone to provoke responses from opponents. This is true of a lot of campaigning, but “AI is very dangerous” is more easily inadvertently motivating accelerationists to act to militarise it first than “this suffering could be stopped” is pivoted into an argument for more cages or fewer bednets
there are probably exceptions to this; it’s difficult to imagine that an asteroid early warning system will hurt us, even if it turns out to be useless