Personally, I made the mitigation of existential risk from AI my life mission, but I’m not a longtermist and not sure I’m even an “effective altruist”. I think that utilitarianism is at best a good tool for collective decision making under some circumstances, not a sound moral philosophy. When you expand it from living people to future people, it’s not even that.
My values prioritize me and people around me far above random strangers. I do care about strangers (including animals) and even hypothetical future people more than zero, but I would not make the radical sacrifices demanded by utilitarianism for their sake, without additional incentives. On the other hand, I am strongly committed to following a cooperative strategy, both for reputational reasons and for acausal reasons. And, I am strongly in favor of societal norms that incentivize making the world at large better (because this is in everyone’s interest). I’m even open to acausal trade with hypothetical future people, if there’s a valid case for it. But, this is not the philosophy of EA as commonly understood, certainly not longtermism.
The main case for preventing AI risk is not longtermism. Rather, it’s just that otherwise we are all going to die (and even going by conservative-within-reason timelines, it’s at least a threat to our children or grandchildren).
I’m certainly hoping to recruit people to work with me, and I’m not going to focus solely on EAs. I won’t necessarily even focus on people who care about AI risk: as long as they are talented, and motivated to work on the problems for one reason or the other (e.g. “it’s math and it’s interesting”), I would take them in.
I strongly disagree that utilitarianism isn’t a sound moral philosophy, and don’t understand the black and white distinction between longtermism and us not all dying. I might be missing something there is surely at least some overlap betwen those two reasons for preventing AI risk.
But although I disagree I think you made your points pretty well :).
Out of interest, if you aren’t an effective altruist, nor a longermist then what do you call yourself?
I strongly disagree that utilitarianism isn’t a sound moral philosophy, and don’t understand the black and white distinction between longtermism and us not all dying. I might be missing something there is surely at least some overlap betwen those two reasons for preventing AI risk.
I don’t know if it’s a “black and white distinction”, but surely there’s a difference between:
Existential risk is bad because the future could have a zillion people, so their combined moral weight dominates all other considerations.
Existential risk is bad because (i) I personally am going to die (ii) my children are going to die (iii) everyone I love are going to die (iv) everyone I know are going to die, and also (v) humanity is not going to have a future (regardless of the number of people in it).
For example, something that “only” kills 99.99% of the population would be comparably bad by my standards (because i-iv still apply), whereas it would be way less bad by longtermism standards. Even something that “only” kills (say) everyone I know and everyone they know would be comparably bad for me, whereas utilitarianism would judge it a mere blip in comparison to human extinction.
Out of interest, if you aren’t an effective altruist, nor a longermist then what do you call yourself?
I call myself “Vanessa” :) Keep your identity small and all that. If you mean, do I have a name for my moral philosophy then… not really. We can call it “antirealist contractarianism”, I guess? I’m not that good at academic philosophy.
Strongly agreed.
Personally, I made the mitigation of existential risk from AI my life mission, but I’m not a longtermist and not sure I’m even an “effective altruist”. I think that utilitarianism is at best a good tool for collective decision making under some circumstances, not a sound moral philosophy. When you expand it from living people to future people, it’s not even that.
My values prioritize me and people around me far above random strangers. I do care about strangers (including animals) and even hypothetical future people more than zero, but I would not make the radical sacrifices demanded by utilitarianism for their sake, without additional incentives. On the other hand, I am strongly committed to following a cooperative strategy, both for reputational reasons and for acausal reasons. And, I am strongly in favor of societal norms that incentivize making the world at large better (because this is in everyone’s interest). I’m even open to acausal trade with hypothetical future people, if there’s a valid case for it. But, this is not the philosophy of EA as commonly understood, certainly not longtermism.
The main case for preventing AI risk is not longtermism. Rather, it’s just that otherwise we are all going to die (and even going by conservative-within-reason timelines, it’s at least a threat to our children or grandchildren).
I’m certainly hoping to recruit people to work with me, and I’m not going to focus solely on EAs. I won’t necessarily even focus on people who care about AI risk: as long as they are talented, and motivated to work on the problems for one reason or the other (e.g. “it’s math and it’s interesting”), I would take them in.
I strongly disagree that utilitarianism isn’t a sound moral philosophy, and don’t understand the black and white distinction between longtermism and us not all dying. I might be missing something there is surely at least some overlap betwen those two reasons for preventing AI risk.
But although I disagree I think you made your points pretty well :).
Out of interest, if you aren’t an effective altruist, nor a longermist then what do you call yourself?
I don’t know if it’s a “black and white distinction”, but surely there’s a difference between:
Existential risk is bad because the future could have a zillion people, so their combined moral weight dominates all other considerations.
Existential risk is bad because (i) I personally am going to die (ii) my children are going to die (iii) everyone I love are going to die (iv) everyone I know are going to die, and also (v) humanity is not going to have a future (regardless of the number of people in it).
For example, something that “only” kills 99.99% of the population would be comparably bad by my standards (because i-iv still apply), whereas it would be way less bad by longtermism standards. Even something that “only” kills (say) everyone I know and everyone they know would be comparably bad for me, whereas utilitarianism would judge it a mere blip in comparison to human extinction.
I call myself “Vanessa” :) Keep your identity small and all that. If you mean, do I have a name for my moral philosophy then… not really. We can call it “antirealist contractarianism”, I guess? I’m not that good at academic philosophy.