I’m a bit late to the party here, I know. But I really enjoyed this post. I thought I’d add my two cents worth. Although I have a long term perspective on risk and mitigation, and have long term sympathies, I don’t consider myself a strong longtermist. That said, I wouldn’t like to see anyone (eg from policy circles) walk away from this debate with the view that it is not worth investing resources in existential risk mitigation. I’m not saying that’s what necessarily comes through, but I think there is important middle ground (and this middle ground may actually instrumentally lead to the outcomes that strong longtermists favour, without the need to accept the strong longtermist position).
I think it is just obvious that we should care about the welfare of people here and now. However, the worst thing that can happen to people existing now is for all of them to be killed. So it seems clear that funnelling some resources into x-risk mitigation, here and now, is important. And the primary focus should always be those x-risks that are most threatening in the near term (and the target risks will no doubt change with time, eg I would say it is biotechnology in the next 5-10 years, then perhaps climate or nuclear, and then AI, followed by rarer natural risks, or emerging technological risks, etc while all the while building cross-cutting defences such as institutions and resilience). As you note, every generation becomes the present generation and every x-risk will have it’s time. We can’t ignore future x-risks, for this very reason. Each future risk ‘era’ will become present and we had better be ready. So resources should be invested in future x-risks, or at least in understanding their timing.
The issue I have with strong-longtermism lies in the utility calculations. The Greaves/MacAskill paper presents a table of future human lives that is based on the carrying capacity of the Earth, solar system, etc. However, even here today we do not advocate some imperative that humans must reproduce right up to the carrying capacity of the Earth. In fact many of us think this would be wrong for a number of reasons. To factor ‘quadrillions’ or any definite number at all into the calculations is to miss the point that we (the moral agents) get to determine (morally speaking) the right number of future people, and we might not know how many this is yet. Uncertainty about moral progress means that we cannot know what the morally correct number is, because theory and argument might evolve across time (and yes, it’s probably obvious but I don’t accept that non-actual, and never-actual people can be harmed, and I don’t accept that non-existence is a harm).
However, there seems to be value in SOME humans persisting in order that these projects might be continued and hopefully resolved. Therefore, I don’t think we should be putting speculative utilities into our ‘in expectation’ calculations. There are independent arguments for preventing x-risk than strong-longtermism, and the emotional response it generates from many, potentially including aversive policymakers makes it a risky strategy to push. Even if EA is to be motivated by strong-longtermism, it may be useful to advocate an ‘instrumental’ theory of value in order to achieve the strong-longtermist agenda. There is a possibility that some of EA’s views can themselves be an information hazard. Being right is not always being effective, and therefore not always altruistic.
Hi Vaden,
I’m a bit late to the party here, I know. But I really enjoyed this post. I thought I’d add my two cents worth. Although I have a long term perspective on risk and mitigation, and have long term sympathies, I don’t consider myself a strong longtermist. That said, I wouldn’t like to see anyone (eg from policy circles) walk away from this debate with the view that it is not worth investing resources in existential risk mitigation. I’m not saying that’s what necessarily comes through, but I think there is important middle ground (and this middle ground may actually instrumentally lead to the outcomes that strong longtermists favour, without the need to accept the strong longtermist position).
I think it is just obvious that we should care about the welfare of people here and now. However, the worst thing that can happen to people existing now is for all of them to be killed. So it seems clear that funnelling some resources into x-risk mitigation, here and now, is important. And the primary focus should always be those x-risks that are most threatening in the near term (and the target risks will no doubt change with time, eg I would say it is biotechnology in the next 5-10 years, then perhaps climate or nuclear, and then AI, followed by rarer natural risks, or emerging technological risks, etc while all the while building cross-cutting defences such as institutions and resilience). As you note, every generation becomes the present generation and every x-risk will have it’s time. We can’t ignore future x-risks, for this very reason. Each future risk ‘era’ will become present and we had better be ready. So resources should be invested in future x-risks, or at least in understanding their timing.
The issue I have with strong-longtermism lies in the utility calculations. The Greaves/MacAskill paper presents a table of future human lives that is based on the carrying capacity of the Earth, solar system, etc. However, even here today we do not advocate some imperative that humans must reproduce right up to the carrying capacity of the Earth. In fact many of us think this would be wrong for a number of reasons. To factor ‘quadrillions’ or any definite number at all into the calculations is to miss the point that we (the moral agents) get to determine (morally speaking) the right number of future people, and we might not know how many this is yet. Uncertainty about moral progress means that we cannot know what the morally correct number is, because theory and argument might evolve across time (and yes, it’s probably obvious but I don’t accept that non-actual, and never-actual people can be harmed, and I don’t accept that non-existence is a harm).
However, there seems to be value in SOME humans persisting in order that these projects might be continued and hopefully resolved. Therefore, I don’t think we should be putting speculative utilities into our ‘in expectation’ calculations. There are independent arguments for preventing x-risk than strong-longtermism, and the emotional response it generates from many, potentially including aversive policymakers makes it a risky strategy to push. Even if EA is to be motivated by strong-longtermism, it may be useful to advocate an ‘instrumental’ theory of value in order to achieve the strong-longtermist agenda. There is a possibility that some of EA’s views can themselves be an information hazard. Being right is not always being effective, and therefore not always altruistic.
**