Thanks for laying out this response! It was really interesting, and I think probably a good reason to not take animals as seriously as I suggest you ought to, if you hold these beliefs.
I think something interesting is that this, and the other objections that have been presented to my piece have brought out is that to avoid focusing exclusively on animals in longtermist projects, you have to have some level of faith in these science-fiction scenarios happening. I don’t necessarily think that is a bad thing, but it isn’t something that’s been made explicit in past discussions of long-termism (at least, in the academic literature), and perhaps ought to be explicit?
A few comments on your two arguments:
Claim: Our descendants may wish to optimize for positive moral goods.
I think this is a precondition for EAs and do-gooders in general “winning”, so I almost treat the possibility of this as a tautology.
This isn’t usually assumed in the longtermist literature. It seems more like the argument is made on the basis of future human lives being net-positive, and therefore good that there will be many of them. I think the expected value of your argument (A) hinges on this claim, so it seems like accepting it as a tautology, or something similar, is actually really risky. If you think this is basically 100% likely to be true, of course your conclusion might be true. But if you don’t, it seems plausible that that, like you mention, priority possibly ought to be on s-risks.
In general, a way to summarize this argument, and others given here could be something like, “there is a non-zero chance that we can make loads and loads of digital welfare in the future (more than exists now), so we should focus on reducing existential risk in order to ensure that future can happen”. This raises a question—when will that claim not be true / the argument you’re making not be relevant? It seems plausible that this kind of argument is a justification to work on existential risk reduction until basically the end of the universe (unless we somehow solve it with 100% certainty, etc.), because we might always assume future people will be better at producing welfare than us.
I assume people have discussed the above, and I’m not well read in the area, but it strikes me as odd that the primary justification in these sci-fi scenarios for working on the future is just a claim that can always be made, instead of working directly on making lives with good welfare (but maybe this is a consideration with longtermism in general, and not just this argument).
I guess part of the issue here is you could have an incredibly tiny credence in a very specific number of things being true (the present being at the hinge of history, various things about future sci-fi scenarios), and having those credences would always justify deferral of action.
I’m not totally sure what to make of this, but I do think it gives me pause. But, I admit I haven’t really thought about any of the above much, and don’t read in this area at all.
tl;dr: My above comment relies on longtermism + total utilitarianism (but I attempted to be neutral on the exact moral goods that compose the abstraction of “utility”). With those two strong assumptions + a bunch of other more reasonable ones, I think you can’t escape thinking about science-fictiony scenarios. I think you may not need to care as much about science-fictiony scenarios with moderate probabilities (but extremely high payoffs in expected utility) if your views are primarily non-consequentialist, or if you’re a consequentialist but the aggregative function is not additive.
I also appreciate your thoughtful post and responses!
This isn’t usually assumed in the longtermist literature
Having read relatively little of it, my understanding is that the point of the academic literature (which do not usually assume total utilitarian views?) on longtermism is to show that longtermism is compatible (in some cases required) under a broad scope of moral views that are considered respectable within the academic literature.
So they don’t talk about science-fictiony stuff, since their claim is that longtermism is robustly true (or compatible) with reasonable academic views in moral philosophy.
The point of my comment is that longtermism + total utilitarianism must lead you to think about these science-fictiony scenarios that have significant probability, rather than that longtermism itself must lead you to consider them.
I guess part of the issue here is you could have an incredibly tiny credence in a very specific number of things being true (the present being at the hinge of history, various things about future sci-fi scenarios), and having those credences would always justify deferral of action.
I think if the credence is sufficiently low, either moral uncertainty (since most people aren’t total utilitarians with 100% probability) or model uncertainty will get you to do different actions.
At very low probabilities, you run into issues like Pascal’s Wager and Pascal’s Mugging, but right now the future is so hazy that I think it’s too hubristic to say anything super-concrete about the future. I’m reasonably confident that I’m willing to defend that all of the claims I’ve made above has percentage points of probability[1], which I think is well above the threshold for “not willing to get mugged by low probabilities.”
I suspect that longetermism + moral axiologies that are less demanding/less tail-driven than total utilitarianism will rely less on the speculative/weird/science-fictiony stuff. I hadn’t thought about them in detail.
To demonstrate what I roughly mean, I made up two imaginary axilogies (I think I can understand other nonhedonic total utilitarian views well enough to represent them faithfully, but I’m not well-read enough on non-total utilitarian views, so I made up fake ones rather than accidentally strawman existing views):
1. An example of an axiology that is long-termist but not utilitarian is that you want to maximize the *probability* that the long-term future will be a “just” world, where “justice” is a binary variable (rather than a moral good that can be maximized). If you have some naive prior that this will have 50-50 chance of happening by default, then you might want to care somewhat about extremely good outcomes for justice (eg, creating a world which can never backslide into an unjust one), and extremely bad outcomes (avoiding a dictatorial lock-in that will necessitate permanent unjust society).
But your decisions are by default going to focus more on the median outcomes rather than the tails. Depending on whether you think animals are treated justly right now, this may entail doing substantial work on farmed animals (and whether to work on wild animal welfare depends on positive vs negative conceptions of justice).
2. An example of an axiology that is long-termist and arguably utilitarian but not total utilitarian is if you have something like instead of doing Sum(Utility across all beings), you instead have Average(Log(Utility per being)). In such a world, the tails dominate if and only if you can have a plausible case for why the tails will have extremely large outcomes even on a log scale. I think this is still technically possible, but you need much stronger assumptions or better arguments than the ones I outlined above.
I’d actually be excited for you or someone else with non-total utilitarian views to look into what other moral philosophies (that people actually believe) + longtermism will entail.
[1] A way to rephrase “there’s less than a 1% probability that our descendants will wish to optimize for moral goods” is that “I’m over 99% confident that our descendants wouldn’t care about moral goods, or care very little about them.” And I just don’t think we know enough about the longterm future to be that confident about anything like that.
Having read relatively little of it, my understanding is that the point of the academic literature (which do not usually assume total utilitarian views?) on longtermism is to show that longtermism is compatible (in some cases required) under a broad scope of moral views that are considered respectable within the academic literature.
So they don’t talk about science-fictiony stuff, since their claim is that longtermism is robustly true (or compatible) with reasonable academic views in moral philosophy.
This is also my impression of some of Toby Ord’s work in The Precipice (particularly chapter 2) and some of the work of GPI, at least. I’m not sure how much it applies more widely to academic work that’s explicitly on longtermism, as I haven’t read a great deal of it yet.
On the other hand, many of Bostrom’s seminal works on existential risks very explicitly refer to such “science-fictiony” scenarios.. And these works effectively seem like seminal works for longtermism too, even if they didn’t yet use that term. E.g., Bostrom writes:
Another estimate, which assumes that future minds will be mainly implemented in computational hardware instead of biological neuronal wetware, produces a lower bound of 10^54 human-brain-emulation subjective life-years (or 10^71 basic computational operations) (Bostrom 2003). If we make the less conservative assumption that future civilizations could eventually press close to the absolute bounds of known physics (using some as yet unimagined technology), we get radically higher estimates of the amount of computation and memory storage that is achievable and thus of the number of years of subjective experience that could be realized.
Indeed, in the same paper, he even suggests that not ending up in such scenarios could count as an existential catastrophe in itself:
Permanent stagnation is instantiated if humanity survives but never reaches technological maturity — that is, the attainment of capabilities affording a level of economic productivity and control over nature that is close to the maximum that could feasibly be achieved (in the fullness of time and in the absence of catastrophic defeaters). For instance, a technologically mature civilization could (presumably) engage in large-scale space colonization through the use of automated self-replicating “von Neumann probes.” (Freitas 1980; Moravec 1988; Tipler 1980) It would also be able to modify and enhance human biology — say, through the use of advanced biotechnology or molecular nanotechnology (Freitas 1999 and 2003). Further, it could construct extremely powerful computational hardware and use it to create whole-brain emulations and entirely artificial types of sentient, superintelligent minds (Sandberg and Bostrom 2008). It might have many additional capabilities, some of which may not be fully imaginable from our current vantage point.
This is also relevant to some other claims of Abraham’s in the post or comments, such as “it seems worth noting that much the literature on longtermism, outside Foundation Research Institute, isn’t making claims explicitly about digital minds as the primary holders of future welfare, but just focuses on the future organic human populations (Greaves and MacAskill’s paper, for example), and similar sized populations to the present day human population at that.” I think this may well be true for the academic literature that’s explicitly about “longtermism”, but I’m less confident it’s true for the wider literature on “longtermism”, or the academic literature that seems effectively longtermist.
It also seems worth noting that, to the extent that a desire to appear respectable/conservative explains why academic work on longtermism shies away from discussing things like digital minds, it may also explain why such literature makes relatively little mention of nonhuman animals. I think a substantial concern for the suffering of wild animals would be seen as similarly “wacky” to many audiences, perhaps even more so than a belief that most “humans” in the future may be digital minds. So it may not be the case that, “behind closed doors”, people from e.g. GPI wouldn’t think about the relevance of animals to far future stuff.
(Personally, I’d prefer it if people could just state all such beliefs pretty openly, but I can understand strategic reasons to refrain from doing so in some settings, unfortunately.)
Also, interestingly, Bostrom does appear to note wild animal suffering in the same paper (though only in one footnote):
We might also have responsibilities to nonhuman beings, such as terrestrial (and possible extraterrestrial) animals. Although we are not currently doing much to help them, we have the opportunity to do so in the future. If rendering aid to suffering nonhuman animals in the natural environment is an important value, then achieving technological maturity in a manner that fails to produce such aid could count as flawed realization. Cf. McMahan 2010; Pearce 2004.
Thanks for this. I think for me the major lessons from comments / conversations here is that many longtermists have much stronger beliefs in the possibility of future digital minds than I thought, and I definitely see how that belief could lead one to think that future digital minds are of overwhelming importance. However, I do think that for utilitarian longtermists, animal considerations might dominate in possible futures where digital minds don’t happen or spread massively, so to some extent one’s credence in my argument / concern for future animals ought to be defined by how much you believe in or disbelieve in the possibility and importance of future digital minds.
As someone who is not particularly familiar with longtermist literature, outside a pretty light review done for this piece, and a general sense of this topic from having spent time in the EA community, I’d say I did not really have the impression that the longtermist community was concerned with future digital minds (outside EA Foundation, etc). Though that just may have been bad luck.
Thanks for laying out this response! It was really interesting, and I think probably a good reason to not take animals as seriously as I suggest you ought to, if you hold these beliefs.
I think something interesting is that this, and the other objections that have been presented to my piece have brought out is that to avoid focusing exclusively on animals in longtermist projects, you have to have some level of faith in these science-fiction scenarios happening. I don’t necessarily think that is a bad thing, but it isn’t something that’s been made explicit in past discussions of long-termism (at least, in the academic literature), and perhaps ought to be explicit?
A few comments on your two arguments:
This isn’t usually assumed in the longtermist literature. It seems more like the argument is made on the basis of future human lives being net-positive, and therefore good that there will be many of them. I think the expected value of your argument (A) hinges on this claim, so it seems like accepting it as a tautology, or something similar, is actually really risky. If you think this is basically 100% likely to be true, of course your conclusion might be true. But if you don’t, it seems plausible that that, like you mention, priority possibly ought to be on s-risks.
In general, a way to summarize this argument, and others given here could be something like, “there is a non-zero chance that we can make loads and loads of digital welfare in the future (more than exists now), so we should focus on reducing existential risk in order to ensure that future can happen”. This raises a question—when will that claim not be true / the argument you’re making not be relevant? It seems plausible that this kind of argument is a justification to work on existential risk reduction until basically the end of the universe (unless we somehow solve it with 100% certainty, etc.), because we might always assume future people will be better at producing welfare than us.
I assume people have discussed the above, and I’m not well read in the area, but it strikes me as odd that the primary justification in these sci-fi scenarios for working on the future is just a claim that can always be made, instead of working directly on making lives with good welfare (but maybe this is a consideration with longtermism in general, and not just this argument).
I guess part of the issue here is you could have an incredibly tiny credence in a very specific number of things being true (the present being at the hinge of history, various things about future sci-fi scenarios), and having those credences would always justify deferral of action.
I’m not totally sure what to make of this, but I do think it gives me pause. But, I admit I haven’t really thought about any of the above much, and don’t read in this area at all.
Thanks again for the response!
tl;dr: My above comment relies on longtermism + total utilitarianism (but I attempted to be neutral on the exact moral goods that compose the abstraction of “utility”). With those two strong assumptions + a bunch of other more reasonable ones, I think you can’t escape thinking about science-fictiony scenarios. I think you may not need to care as much about science-fictiony scenarios with moderate probabilities (but extremely high payoffs in expected utility) if your views are primarily non-consequentialist, or if you’re a consequentialist but the aggregative function is not additive.
I also appreciate your thoughtful post and responses!
Having read relatively little of it, my understanding is that the point of the academic literature (which do not usually assume total utilitarian views?) on longtermism is to show that longtermism is compatible (in some cases required) under a broad scope of moral views that are considered respectable within the academic literature.
So they don’t talk about science-fictiony stuff, since their claim is that longtermism is robustly true (or compatible) with reasonable academic views in moral philosophy.
The point of my comment is that longtermism + total utilitarianism must lead you to think about these science-fictiony scenarios that have significant probability, rather than that longtermism itself must lead you to consider them.
I think if the credence is sufficiently low, either moral uncertainty (since most people aren’t total utilitarians with 100% probability) or model uncertainty will get you to do different actions.
At very low probabilities, you run into issues like Pascal’s Wager and Pascal’s Mugging, but right now the future is so hazy that I think it’s too hubristic to say anything super-concrete about the future. I’m reasonably confident that I’m willing to defend that all of the claims I’ve made above has percentage points of probability[1], which I think is well above the threshold for “not willing to get mugged by low probabilities.”
I suspect that longetermism + moral axiologies that are less demanding/less tail-driven than total utilitarianism will rely less on the speculative/weird/science-fictiony stuff. I hadn’t thought about them in detail.
To demonstrate what I roughly mean, I made up two imaginary axilogies (I think I can understand other nonhedonic total utilitarian views well enough to represent them faithfully, but I’m not well-read enough on non-total utilitarian views, so I made up fake ones rather than accidentally strawman existing views):
1. An example of an axiology that is long-termist but not utilitarian is that you want to maximize the *probability* that the long-term future will be a “just” world, where “justice” is a binary variable (rather than a moral good that can be maximized). If you have some naive prior that this will have 50-50 chance of happening by default, then you might want to care somewhat about extremely good outcomes for justice (eg, creating a world which can never backslide into an unjust one), and extremely bad outcomes (avoiding a dictatorial lock-in that will necessitate permanent unjust society).
But your decisions are by default going to focus more on the median outcomes rather than the tails. Depending on whether you think animals are treated justly right now, this may entail doing substantial work on farmed animals (and whether to work on wild animal welfare depends on positive vs negative conceptions of justice).
2. An example of an axiology that is long-termist and arguably utilitarian but not total utilitarian is if you have something like instead of doing Sum(Utility across all beings), you instead have Average(Log(Utility per being)). In such a world, the tails dominate if and only if you can have a plausible case for why the tails will have extremely large outcomes even on a log scale. I think this is still technically possible, but you need much stronger assumptions or better arguments than the ones I outlined above.
I’d actually be excited for you or someone else with non-total utilitarian views to look into what other moral philosophies (that people actually believe) + longtermism will entail.
[1] A way to rephrase “there’s less than a 1% probability that our descendants will wish to optimize for moral goods” is that “I’m over 99% confident that our descendants wouldn’t care about moral goods, or care very little about them.” And I just don’t think we know enough about the longterm future to be that confident about anything like that.
Yeah, the idea of looking into longtermism for nonutilitarians is interesting to me. Thanks for the suggestion!
I think regardless, this helped clarify a lot of things for me about particular beliefs longtermists might hold (to various degrees). Thanks!
Yeah I think that’d be useful to do.
I’m glad it was helpful!
This is also my impression of some of Toby Ord’s work in The Precipice (particularly chapter 2) and some of the work of GPI, at least. I’m not sure how much it applies more widely to academic work that’s explicitly on longtermism, as I haven’t read a great deal of it yet.
On the other hand, many of Bostrom’s seminal works on existential risks very explicitly refer to such “science-fictiony” scenarios.. And these works effectively seem like seminal works for longtermism too, even if they didn’t yet use that term. E.g., Bostrom writes:
Indeed, in the same paper, he even suggests that not ending up in such scenarios could count as an existential catastrophe in itself:
This is also relevant to some other claims of Abraham’s in the post or comments, such as “it seems worth noting that much the literature on longtermism, outside Foundation Research Institute, isn’t making claims explicitly about digital minds as the primary holders of future welfare, but just focuses on the future organic human populations (Greaves and MacAskill’s paper, for example), and similar sized populations to the present day human population at that.” I think this may well be true for the academic literature that’s explicitly about “longtermism”, but I’m less confident it’s true for the wider literature on “longtermism”, or the academic literature that seems effectively longtermist.
It also seems worth noting that, to the extent that a desire to appear respectable/conservative explains why academic work on longtermism shies away from discussing things like digital minds, it may also explain why such literature makes relatively little mention of nonhuman animals. I think a substantial concern for the suffering of wild animals would be seen as similarly “wacky” to many audiences, perhaps even more so than a belief that most “humans” in the future may be digital minds. So it may not be the case that, “behind closed doors”, people from e.g. GPI wouldn’t think about the relevance of animals to far future stuff.
(Personally, I’d prefer it if people could just state all such beliefs pretty openly, but I can understand strategic reasons to refrain from doing so in some settings, unfortunately.)
Also, interestingly, Bostrom does appear to note wild animal suffering in the same paper (though only in one footnote):
Thanks for this. I think for me the major lessons from comments / conversations here is that many longtermists have much stronger beliefs in the possibility of future digital minds than I thought, and I definitely see how that belief could lead one to think that future digital minds are of overwhelming importance. However, I do think that for utilitarian longtermists, animal considerations might dominate in possible futures where digital minds don’t happen or spread massively, so to some extent one’s credence in my argument / concern for future animals ought to be defined by how much you believe in or disbelieve in the possibility and importance of future digital minds.
As someone who is not particularly familiar with longtermist literature, outside a pretty light review done for this piece, and a general sense of this topic from having spent time in the EA community, I’d say I did not really have the impression that the longtermist community was concerned with future digital minds (outside EA Foundation, etc). Though that just may have been bad luck.