And apart from the modal return being probably ~0 (which might not be a problem if funding lots of useless stuff guarantees sufficiently useful projects get funded), it is also possible for longtermist work to be actively counterproductive, with significant uncertainty around the direction as well as the magnitude of the impact of many longtermist activities[1]
Indeed many longtermist arguments strongly imply that work of other longtermists is counterproductive (it is hard to see how Anthropic and Pause AI can both be right about frontier models)
And even you’re certain you’ve picked the correct side of a particular risk-mitigation debate, political donations and campaigning are particularly prone to provoke responses from opponents. This is true of a lot of campaigning, but “AI is very dangerous” is more easily inadvertently motivating accelerationists to act to militarise it first than “this suffering could be stopped” is pivoted into an argument for more cages or fewer bednets
- ^
there are probably exceptions to this; it’s difficult to imagine that an asteroid early warning system will hurt us, even if it turns out to be useless
Feels like most of these except the kidney are fairly easily faked or not that relevant. Community engagement and working for your own non-profit are about the first thing anyone seeking to grift will do, as well as people who are sincere (including as others have pointed out, the people who are sincerely wrong)
And Sam Bankman Fried was a vegan who apparently started off with a completely sincere desire to work for animal nonprofits. Treating these as signals and the people complaining about his behaviour and some of the things he said as noise was a bad move.