I feel like the motivating example here (of the prisoner’s dilemma between a utilitarian and an environmentalist) is relying a lot on the specific numbers going into the example. In particular, it’s relying on the assumption that cause areas don’t differ dramatically in impact.
If you believe that (a) the best ideosyncratic giving opportunities are very much better than the best consensus giving opportunities, and (b) other people’s best ideosyncratic giving opportunities are much less bad (by your values) than your own best ideosyncratic giving opportunities are good, then the “non-co-operative” altruistic equilibrium will be better by your values overall than the “co-operative” equilibrium.
This seems to be true in the example given. Near-term conservation work might be bad from a total utilitarian perspective (I’m not sure if this is true, but it’s plausible to me), but it seems much less bad than, say, AI safety work is good. If so, the cost of having the environmentalist work on conservation instead of climate change is well worth paying in exchange for being able to work on AI safety.
Ditto for most major “mainstream” charitable causes: from a longtermist perspective I’d say that, whatever their sign, their magnitude tends to be drastically smaller than that of the most promising EA causes. So foregoing large gains from cause prioritisation to work better with groups advocating these causes might simply not be worth it.
Conversely, if you’re dealing with a fairly powerful optimiser with a very different value set from yours (e.g. a committed total utilitarian negotiating with a committed negative utilitarian), their best options might be very negative from your perspective, so co-operation is more important for both of you.
I feel like the motivating example here (of the prisoner’s dilemma between a utilitarian and an environmentalist) is relying a lot on the specific numbers going into the example. In particular, it’s relying on the assumption that cause areas don’t differ dramatically in impact.
If you believe that (a) the best ideosyncratic giving opportunities are very much better than the best consensus giving opportunities, and (b) other people’s best ideosyncratic giving opportunities are much less bad (by your values) than your own best ideosyncratic giving opportunities are good, then the “non-co-operative” altruistic equilibrium will be better by your values overall than the “co-operative” equilibrium.
This seems to be true in the example given. Near-term conservation work might be bad from a total utilitarian perspective (I’m not sure if this is true, but it’s plausible to me), but it seems much less bad than, say, AI safety work is good. If so, the cost of having the environmentalist work on conservation instead of climate change is well worth paying in exchange for being able to work on AI safety.
Ditto for most major “mainstream” charitable causes: from a longtermist perspective I’d say that, whatever their sign, their magnitude tends to be drastically smaller than that of the most promising EA causes. So foregoing large gains from cause prioritisation to work better with groups advocating these causes might simply not be worth it.
Conversely, if you’re dealing with a fairly powerful optimiser with a very different value set from yours (e.g. a committed total utilitarian negotiating with a committed negative utilitarian), their best options might be very negative from your perspective, so co-operation is more important for both of you.
Whoops, sorry, I wrote this yesterday and then forgot to post it until today, and in the meantime Ben Todd made the same point in one of his comments.