I show how a standard argument for advancing progress is extremely sensitive to how humanity’s story eventually ends. Whether advancing progress is ultimately good or bad depends crucially on whether it also advances the end of humanity. Because we know so little about the answer to this crucial question, the case for advancing progress is undermined. I suggest we must either overcome this objection through improving our understanding of these connections between progress and human extinction or switch our focus to advancing certain kinds of progress relative to others — changing where we are going, rather than just how soon we get there.[1]
Things are getting better. While there are substantial ups and downs, long-term progress in science, technology, and values have tended to make people’s lives longer, freer, and more prosperous. We could represent this as a graph of quality of life over time, giving a curve that generally trends upwards.
What would happen if we were to advance all kinds of progress by a year? Imagine a burst of faster progress, where after a short period, all forms of progress end up a year ahead of where they would have been. We might think of the future trajectory of quality of life as being primarily driven by science, technology, the economy, population, culture, societal norms, moral norms, and so forth. We’re considering what would happen if we could move all of these features a year ahead of where they would have been.
While the burst of faster progress may be temporary, we should expect its effect of getting a year ahead to endure.[2] If we’d only advanced some domains of progress, we might expect further progress in those areas to be held back by the domains that didn’t advance — but here we’re imagining moving the entire internal clock of civilisation forward a year.
If we were to advance progress in this way, we’d be shifting the curve of quality of life a year to the left. Since the curve is generally increasing, this would mean the new trajectory of our future is generally higher than the old one. So the value of advancing progress isn’t just a matter of impatience — wanting to get to the good bits sooner — but of overall improvement in people’s quality of life across the future.
Figure 1. Sooner is better. The solid green curve is the default trajectory of quality of life over time, while the dashed curve is the trajectory if progress were uniformly advanced by one year (shifting the default curve to the left). Because the trajectories trend upwards, quality of life is generally higher under the advanced curve. To help see this, I’ve shaded the improvements to quality of life green and the worsenings red.
That’s a standard story within economics: progress in science, technology, and values has been making the world a better place, so a burst of faster progress that brought this all forward by a year would provide a lasting benefit for humanity.
But this story is missing a crucial piece.
The trajectory of humanity’s future is not literally infinite. One day it will come to an end. This might be a global civilisation dying of old age, lumbering under the weight of accumulated bureaucracy or decadence. It might be a civilisation running out of resources: either using them up prematurely or enduring until the sun itself burns out. It might be a civilisation that ends in sudden catastrophe — a natural calamity or one of its own making.
If the trajectory must come to an end, what happens to the previous story of an advancement in progress being a permanent uplifting of the quality of life? The answer depends on the nature of the end time. There are two very natural possibilities.
One is that the end time is fixed. It is a particular calendar date when humanity would come to an end, which would be unchanged if we advanced progress. Economists would call this an exogenous end time. On this model, the standard story still works — what you are effectively doing when you advance progress by a year is skipping a year at current quality of life and getting an extra year at the final quality of life, which could be much higher.
Figure 2. Exogenous end time. Advancing progress with an unchanging end time. Note that the diagram is not to scale (our future is hopefully much longer).
The other natural model is that when advancing progress, the end time is also brought forward by a year (call it an endogenous end time). A simple example would be if there were a powerful technology which once discovered, inevitably led to human extinction shortly thereafter — perhaps something like nuclear weapons, but more accessible and even more destructive. If so, then advancing progress would bring forward this final invention and our own extinction.
Figure 3. Endogenous end time. Advancing progress also brings forward the end time.
On this model, there is a final year which is much worse under the trajectory where progress is advanced by a year. Whether we assume the curve suddenly drops to zero at the end, or whether it declines more smoothly, there is a time where the upwards march of quality-of-life peaks and then goes down. And bringing the trajectory forward by a year makes the period after the peak worse, rather than better. So we have a good effect of improving the quality of life for many years combined with a bad effect of bringing our demise a year sooner.
It turns out that on this endogenous model, the bad effect always outweighs the good: producing lower total and average quality of life for the period from now until the original end time. This is because the region under the advanced curve is just a shifted version of the same shape as the region under the default curve — except that it is missing the first slice of value that would have occurred in the first year.[3] Here advancing progress involves skipping a year at the current quality of life and never replacing it with anything else. It is like skipping a good early track on an album whose quality steadily improves — you get to the great stuff sooner, but ultimately just hear the same music minus one good song.
So when we take account of the fact that this trajectory of humanity is not infinite, we find that the value of uniformly advancing progress is extremely sensitive to whether the end of humanity is driven by an external clock or an internal one.
Both possibilities are very plausible. Consider the ways our trajectory might play out that we listed earlier. We can categorise these by whether the end point is exogenous or endogenous.
Exogenous end points:
enduring until some external end like the sun burning out
a natural catastrophe like a supervolcanic eruption or asteroid impact
Endogenous end points:
accumulation of bad features like bureaucracy or decadence
prematurely running out of resources
anthropogenic catastrophe such as nuclear war
Whether advancing all kinds of progress by a short period of time is good or bad, is thus extremely sensitive to whether humanity’s ultimate end is caused by something on the first list or the second. If we knew that our end was driven by something on the second list, then advancing all progress would be a bad thing.
I am not going to attempt to adjudicate which kind of ending is more likely or whether uniformly advancing progress is good or bad. Importantly — and I want to be clear about this — I am not arguing that advancing progress is a bad thing.
Instead, the point is that whether advancing progress is good or bad depends crucially on an issue that is rarely even mentioned in discussions on the value of progress. Historians of progress have built a detailed case that (on average) progress has greatly improved people’s quality of life. But all this rigorous historical research does not bear on the question of whether it will also eventually bring forward our end. All the historical data could thus be entirely moot if it turns out that advancing progress has also silently shortened our future lifespan.
One upshot of this is that questions about what will ultimately cause human extinction[4] — which are normally seen as extremely esoteric — are in fact central to determining whether uniform advancements of progress are good or bad. These questions would thus need to appear explicitly in economics and social science papers and books about the value of progress. While we cannot hope to know about our future extinction with certainty, there is substantial room for clarifying the key questions, modelling how they affect the value of progress, and showing which parameter values would be required for advancing all progress to be a good thing.
Examples of such clarification and modelling would include:
Considerations of the amount of value at stake in each case.[5]
Modelling existential risk over time, such as with a time-varying hazard rate.[6]
Allowing the end time (or hazard curve) to be neither purely exogenous nor purely endogenous.[7]
Note that my point here isn’t the increasingly well-known debate about whether work aimed at preventing existential risks could be even more important than work aimed at advancing progress.[8] It is that considerations about existential risk undermine a promising argument for advancing progress and call into question whether advancing progress is good at all.
Why hasn’t this been noticed before? The issue is usually hidden behind a modelling assumption of progress continuing over an infinite future. In this never-ending model, when you pull this curve sideways, there is always more curve revealed. It is only when you say that it is of a finite (if unknown) length that the question of what happens when we run out of curve arises. There is something about this that I find troublingly reminiscent of a Ponzi scheme: in the never-ending model, advancing progress keeps bringing forward the benefits we would have had in year t+1 to pay the people in year t, with the resulting debt being passed infinitely far into the future and so never coming due.
Furthermore, the realisation that humanity won’t last for ever is surprisingly recent — especially the possibility that its lifespan might depend on our actions.[9] And the ability to broach this in academic papers without substantial pushback is even more recent. So perhaps it shouldn’t be surprising that this was only noticed now.
The issue raised by this paper has also been masked in many economic analyses by an assumption of pure time-preference: that society should have a discount rate on value itself. If we use that assumption, we end up with a somewhat different argument for advancing progress — one based on impatience; on merely getting to the good stuff sooner, even if that means getting less of it.
Even then, the considerations I’ve raised would undermine this argument. For if it does turn out that advancing progress across the board is bad from a patient perspective, then we’d be left with an argument that ‘advancing progress is good, but only due to fundamental societal impatience and the way it neglects future losses’. The rationale for advancing progress would be fundamentally about robbing tomorrow to pay for today, in a way that is justified only because society doesn’t (or shouldn’t) care much about the people at the end of the chain when the debt comes due. This strikes me as a very troubling position and far from the full-throated endorsement of progress that its advocates seek.
Could attempts to advance progress have some other kind of predictable effect on the shape of the curve of quality of life over time?[10] They might. For instance, they might be able to lengthen our future by bringing forward the moment we develop technologies to protect us from natural threats to our survival. This is an intriguing idea, but there are challenges to getting it to work: especially since there isn’t actually much natural extinction risk for progress to reduce, whereas progress appears to be introducing larger anthropogenic risks.[11] Even if the case could be made, it would be a different case to the one we started with — it would be a case based on progress increasing the duration of lived experience rather than its quality. I’d be delighted if such a revised case can be made, but we can’t just assume it.[12]
If we were to advance some kinds of progress relative to others, we could certainly make large changes to the shape of the curve, or to how long humanity lasts.[13] But this is not even a different kind of argument for the value of progress; it is a different kind of intervention, which we could call differential progress.[14] This would not be an argument that progress is generally good, but that certain parts of it are. Again, I’m open to this argument, but it is not the argument for progress writ large that its proponents desire.
And it may have some uncomfortable consequences. If advancing all progress would turn out to be bad, but advancing some parts of it would be good, then it is likely that advancing the remaining parts would be even more bad. Since some kinds of progress are more plausibly linked to bringing about an earlier demise (e.g. nuclear weapons, climate change, and large-scale resource depletion only became possible because of technological, economic, and scientific progress) these parts may not fare so well in such an analysis. So it may really be an argument for differentially boosting other kinds of progress, such as moral progress or institutional progress, and perhaps even for delaying technological, economic, and scientific progress.
It is important to stress that through all of this we’ve only been exploring the case for advancing progress — comparing the default path of progress to one that is shifted earlier. There might be other claims about the value of progress that are unaffected. For example, these considerations don’t directly undermine the argument that progress is better than stasis, and thus that if progress is fragile, we need to protect it. That may be true even if humanity does eventually bring about its own end, and even if our progress brings it about sooner. Or one could understand ‘progress’ to refer only to those cases where the outcomes are genuinely improved, such that it is by definition a good thing. But the arguments we’ve explored show that these other arguments in favour of progress wouldn’t have many of the policy implications that advocates of progress typically favour — as whether it is good or bad to advance scientific research or economic productivity will still depend very sensitively on esoteric questions about how humanity ends.
I find all this troubling. I’m a natural optimist and an advocate for progress. I don’t want to see the case for advancing progress undermined. But when I tried to model it clearly to better understand its value, I found a substantial gap in the argument. I want to draw people’s attention to that gap in the hope that we can close it. There may be good arguments to settle this issue one way or the other, but they will need to be made. Alternatively, if we do find that uniform advancements of progress are in fact bad, we will need to adjust our advocacy to champion the right kinds of differential progress.
References
Aschenbrenner, Leopold; and Trammell, Philip. (2024). Existential Risk and Growth. GPI Working Paper No. 13-2024, (Global Priorities Institute and Department of Economics, University of Oxford).
Beckstead, Nick. (2013). On the Overwhelming Importance of Shaping the Far Future. PhD Thesis. Department of Philosophy, Rutgers University.
Bostrom, Nick. (2002). Existential Risks: Analyzing Human Extinction, Journal of Evolution and Technology 9(1).
Snyder-Beattie, Andrew E.; Ord, Toby; and Bonsall, Michael B. (2019). An upper bound for the background rate of human extinction. Scientific Reports 9:11054, 1–9.
I’d like to thank Fin Moorhouse, Kevin Kuruc, and Chad Jones for insightful discussion and comments, though of course this doesn’t imply that they endorse the arguments in this paper.
The arguments in this paper would also apply if it were instead a temporary acceleration leading to a permanent speed-up of progress. In that case an endogenous end time would mean a temporal compression of our trajectory up to that point: achieving all the same quality levels, but having less time to enjoy each one, then prematurely ending.
This is proved in more detail, addressing several complications in Ord (forthcoming). Several complexities arise from different approaches to population ethics, but the basic argument holds for any approach to population ethics that considers a future with lower total and average wellbeing to be worse. If your approach to population ethics avoids this argument, then note that whether advancing progress is good or bad is now highly sensitive to the theory of population ethics — another contentious and esoteric topic.
One could generalise this argument beyond human extinction. First, one could consider other kinds of existential risk. For instance, if a global totalitarian regime arises which permanently reduces quality of life to a lower level than it is today, then what matters is whether the date of its onset is endogenous. Alternatively, we could broaden this argument to include decision-makers who are partial to a particular country or culture. In that case, what matters is whether advancing progress would advance the end of that group.
For example, if final years are 10 times as good as present years, then we would only need a 10% chance that humanity’s end was exogenous for advancing progress to be good in expectation. I think this is a promising strategy for saving the expected value of progress.
In this setting, the former idea of an endogenous end time roughly corresponds to the assumption that advancing progress by a year also shifts humanity’s survival curve left by a year, while an exogenous end time corresponds to the assumption that the survival curve stays fixed. A third salient possibility is that it is the corresponding hazard curve which shifts left by a year. This possibility is mentioned in Ord (forthcoming, pp. 17 & 30). The first and third of these correspond to what Bostrom (2014) calls transition risk and state risk.
For example, one could decompose the overall hazard curve into an exogenous hazard curve plus an endogenous one, where only the latter gets shifted to the left. Alternatively, Aschenbrenner & Trammell (2024) present a promising parameterisation that interpolates between state risk and transition risk (and beyond).
Any substantial change to the future, including advancing progress by a year, would probably have large chaotic effects on the future in the manner of the butterfly effect. But only a systematic change to the expected shape or duration of the curve, could help save the initial case for advancing progress.
The fossil record suggests natural extinction risk is lower than 1 in 1,000 per century — after all humanity has already survived 2,000 centuries and typical mammalian species survive 10,000 centuries. See Snyder-Beattie et al. (2019) and Ord (2020, pp. 80–87).
Another kind of case that is often made is that the sheer rate of progress could change the future trajectory. A common version is that the world is more stable with steady 3–5% economic growth: if we don’t keep moving forward, we will fall over. This is plausible but gives quite a different kind of conclusion. It suggests it is good to advance economic progress when growth is at 2% but may be bad to advance it when growth is 5%. Moreover, unless the central point about progress bringing forward humanity’s end is dealt with head-on, the best this steady-growth argument can achieve is to say that ‘yes, advancing progress may cut short our future, but unfortunately we need to do it anyway to avoid collapse’.
On the Value of Advancing Progress
(PDF available)
I show how a standard argument for advancing progress is extremely sensitive to how humanity’s story eventually ends. Whether advancing progress is ultimately good or bad depends crucially on whether it also advances the end of humanity. Because we know so little about the answer to this crucial question, the case for advancing progress is undermined. I suggest we must either overcome this objection through improving our understanding of these connections between progress and human extinction or switch our focus to advancing certain kinds of progress relative to others — changing where we are going, rather than just how soon we get there.[1]
Things are getting better. While there are substantial ups and downs, long-term progress in science, technology, and values have tended to make people’s lives longer, freer, and more prosperous. We could represent this as a graph of quality of life over time, giving a curve that generally trends upwards.
What would happen if we were to advance all kinds of progress by a year? Imagine a burst of faster progress, where after a short period, all forms of progress end up a year ahead of where they would have been. We might think of the future trajectory of quality of life as being primarily driven by science, technology, the economy, population, culture, societal norms, moral norms, and so forth. We’re considering what would happen if we could move all of these features a year ahead of where they would have been.
While the burst of faster progress may be temporary, we should expect its effect of getting a year ahead to endure.[2] If we’d only advanced some domains of progress, we might expect further progress in those areas to be held back by the domains that didn’t advance — but here we’re imagining moving the entire internal clock of civilisation forward a year.
If we were to advance progress in this way, we’d be shifting the curve of quality of life a year to the left. Since the curve is generally increasing, this would mean the new trajectory of our future is generally higher than the old one. So the value of advancing progress isn’t just a matter of impatience — wanting to get to the good bits sooner — but of overall improvement in people’s quality of life across the future.
Figure 1. Sooner is better. The solid green curve is the default trajectory of quality of life over time, while the dashed curve is the trajectory if progress were uniformly advanced by one year (shifting the default curve to the left). Because the trajectories trend upwards, quality of life is generally higher under the advanced curve. To help see this, I’ve shaded the improvements to quality of life green and the worsenings red.
That’s a standard story within economics: progress in science, technology, and values has been making the world a better place, so a burst of faster progress that brought this all forward by a year would provide a lasting benefit for humanity.
But this story is missing a crucial piece.
The trajectory of humanity’s future is not literally infinite. One day it will come to an end. This might be a global civilisation dying of old age, lumbering under the weight of accumulated bureaucracy or decadence. It might be a civilisation running out of resources: either using them up prematurely or enduring until the sun itself burns out. It might be a civilisation that ends in sudden catastrophe — a natural calamity or one of its own making.
If the trajectory must come to an end, what happens to the previous story of an advancement in progress being a permanent uplifting of the quality of life? The answer depends on the nature of the end time. There are two very natural possibilities.
One is that the end time is fixed. It is a particular calendar date when humanity would come to an end, which would be unchanged if we advanced progress. Economists would call this an exogenous end time. On this model, the standard story still works — what you are effectively doing when you advance progress by a year is skipping a year at current quality of life and getting an extra year at the final quality of life, which could be much higher.
Figure 2. Exogenous end time. Advancing progress with an unchanging end time. Note that the diagram is not to scale (our future is hopefully much longer).
The other natural model is that when advancing progress, the end time is also brought forward by a year (call it an endogenous end time). A simple example would be if there were a powerful technology which once discovered, inevitably led to human extinction shortly thereafter — perhaps something like nuclear weapons, but more accessible and even more destructive. If so, then advancing progress would bring forward this final invention and our own extinction.
Figure 3. Endogenous end time. Advancing progress also brings forward the end time.
On this model, there is a final year which is much worse under the trajectory where progress is advanced by a year. Whether we assume the curve suddenly drops to zero at the end, or whether it declines more smoothly, there is a time where the upwards march of quality-of-life peaks and then goes down. And bringing the trajectory forward by a year makes the period after the peak worse, rather than better. So we have a good effect of improving the quality of life for many years combined with a bad effect of bringing our demise a year sooner.
It turns out that on this endogenous model, the bad effect always outweighs the good: producing lower total and average quality of life for the period from now until the original end time. This is because the region under the advanced curve is just a shifted version of the same shape as the region under the default curve — except that it is missing the first slice of value that would have occurred in the first year.[3] Here advancing progress involves skipping a year at the current quality of life and never replacing it with anything else. It is like skipping a good early track on an album whose quality steadily improves — you get to the great stuff sooner, but ultimately just hear the same music minus one good song.
So when we take account of the fact that this trajectory of humanity is not infinite, we find that the value of uniformly advancing progress is extremely sensitive to whether the end of humanity is driven by an external clock or an internal one.
Both possibilities are very plausible. Consider the ways our trajectory might play out that we listed earlier. We can categorise these by whether the end point is exogenous or endogenous.
Exogenous end points:
enduring until some external end like the sun burning out
a natural catastrophe like a supervolcanic eruption or asteroid impact
Endogenous end points:
accumulation of bad features like bureaucracy or decadence
prematurely running out of resources
anthropogenic catastrophe such as nuclear war
Whether advancing all kinds of progress by a short period of time is good or bad, is thus extremely sensitive to whether humanity’s ultimate end is caused by something on the first list or the second. If we knew that our end was driven by something on the second list, then advancing all progress would be a bad thing.
I am not going to attempt to adjudicate which kind of ending is more likely or whether uniformly advancing progress is good or bad. Importantly — and I want to be clear about this — I am not arguing that advancing progress is a bad thing.
Instead, the point is that whether advancing progress is good or bad depends crucially on an issue that is rarely even mentioned in discussions on the value of progress. Historians of progress have built a detailed case that (on average) progress has greatly improved people’s quality of life. But all this rigorous historical research does not bear on the question of whether it will also eventually bring forward our end. All the historical data could thus be entirely moot if it turns out that advancing progress has also silently shortened our future lifespan.
One upshot of this is that questions about what will ultimately cause human extinction[4] — which are normally seen as extremely esoteric — are in fact central to determining whether uniform advancements of progress are good or bad. These questions would thus need to appear explicitly in economics and social science papers and books about the value of progress. While we cannot hope to know about our future extinction with certainty, there is substantial room for clarifying the key questions, modelling how they affect the value of progress, and showing which parameter values would be required for advancing all progress to be a good thing.
Examples of such clarification and modelling would include:
Considerations of the amount of value at stake in each case.[5]
Modelling existential risk over time, such as with a time-varying hazard rate.[6]
Allowing the end time (or hazard curve) to be neither purely exogenous nor purely endogenous.[7]
Note that my point here isn’t the increasingly well-known debate about whether work aimed at preventing existential risks could be even more important than work aimed at advancing progress.[8] It is that considerations about existential risk undermine a promising argument for advancing progress and call into question whether advancing progress is good at all.
Why hasn’t this been noticed before? The issue is usually hidden behind a modelling assumption of progress continuing over an infinite future. In this never-ending model, when you pull this curve sideways, there is always more curve revealed. It is only when you say that it is of a finite (if unknown) length that the question of what happens when we run out of curve arises. There is something about this that I find troublingly reminiscent of a Ponzi scheme: in the never-ending model, advancing progress keeps bringing forward the benefits we would have had in year t+1 to pay the people in year t, with the resulting debt being passed infinitely far into the future and so never coming due.
Furthermore, the realisation that humanity won’t last for ever is surprisingly recent — especially the possibility that its lifespan might depend on our actions.[9] And the ability to broach this in academic papers without substantial pushback is even more recent. So perhaps it shouldn’t be surprising that this was only noticed now.
The issue raised by this paper has also been masked in many economic analyses by an assumption of pure time-preference: that society should have a discount rate on value itself. If we use that assumption, we end up with a somewhat different argument for advancing progress — one based on impatience; on merely getting to the good stuff sooner, even if that means getting less of it.
Even then, the considerations I’ve raised would undermine this argument. For if it does turn out that advancing progress across the board is bad from a patient perspective, then we’d be left with an argument that ‘advancing progress is good, but only due to fundamental societal impatience and the way it neglects future losses’. The rationale for advancing progress would be fundamentally about robbing tomorrow to pay for today, in a way that is justified only because society doesn’t (or shouldn’t) care much about the people at the end of the chain when the debt comes due. This strikes me as a very troubling position and far from the full-throated endorsement of progress that its advocates seek.
Could attempts to advance progress have some other kind of predictable effect on the shape of the curve of quality of life over time?[10] They might. For instance, they might be able to lengthen our future by bringing forward the moment we develop technologies to protect us from natural threats to our survival. This is an intriguing idea, but there are challenges to getting it to work: especially since there isn’t actually much natural extinction risk for progress to reduce, whereas progress appears to be introducing larger anthropogenic risks.[11] Even if the case could be made, it would be a different case to the one we started with — it would be a case based on progress increasing the duration of lived experience rather than its quality. I’d be delighted if such a revised case can be made, but we can’t just assume it.[12]
If we were to advance some kinds of progress relative to others, we could certainly make large changes to the shape of the curve, or to how long humanity lasts.[13] But this is not even a different kind of argument for the value of progress; it is a different kind of intervention, which we could call differential progress.[14] This would not be an argument that progress is generally good, but that certain parts of it are. Again, I’m open to this argument, but it is not the argument for progress writ large that its proponents desire.
And it may have some uncomfortable consequences. If advancing all progress would turn out to be bad, but advancing some parts of it would be good, then it is likely that advancing the remaining parts would be even more bad. Since some kinds of progress are more plausibly linked to bringing about an earlier demise (e.g. nuclear weapons, climate change, and large-scale resource depletion only became possible because of technological, economic, and scientific progress) these parts may not fare so well in such an analysis. So it may really be an argument for differentially boosting other kinds of progress, such as moral progress or institutional progress, and perhaps even for delaying technological, economic, and scientific progress.
It is important to stress that through all of this we’ve only been exploring the case for advancing progress — comparing the default path of progress to one that is shifted earlier. There might be other claims about the value of progress that are unaffected. For example, these considerations don’t directly undermine the argument that progress is better than stasis, and thus that if progress is fragile, we need to protect it. That may be true even if humanity does eventually bring about its own end, and even if our progress brings it about sooner. Or one could understand ‘progress’ to refer only to those cases where the outcomes are genuinely improved, such that it is by definition a good thing. But the arguments we’ve explored show that these other arguments in favour of progress wouldn’t have many of the policy implications that advocates of progress typically favour — as whether it is good or bad to advance scientific research or economic productivity will still depend very sensitively on esoteric questions about how humanity ends.
I find all this troubling. I’m a natural optimist and an advocate for progress. I don’t want to see the case for advancing progress undermined. But when I tried to model it clearly to better understand its value, I found a substantial gap in the argument. I want to draw people’s attention to that gap in the hope that we can close it. There may be good arguments to settle this issue one way or the other, but they will need to be made. Alternatively, if we do find that uniform advancements of progress are in fact bad, we will need to adjust our advocacy to champion the right kinds of differential progress.
References
Aschenbrenner, Leopold; and Trammell, Philip. (2024). Existential Risk and Growth. GPI Working Paper No. 13-2024, (Global Priorities Institute and Department of Economics, University of Oxford).
Beckstead, Nick. (2013). On the Overwhelming Importance of Shaping the Far Future. PhD Thesis. Department of Philosophy, Rutgers University.
Bostrom, Nick. (2002). Existential Risks: Analyzing Human Extinction, Journal of Evolution and Technology 9(1).
Bostrom, Nick. (2014). Superintelligence: Paths, dangers, strategies, (Oxford: OUP).
Moynihan, Thomas. (2020). X-Risk: How Humanity Discovered its Own Extinction, (Urbanomic).
Ord, Toby. (2020). The Precipice: Existential risk and the Future of Humanity, (London: Bloomsbury).
Ord, Toby. (Forthcoming). Shaping humanity’s longterm trajectory. In Barrett et al (eds.) Essays on Longtermism, (Oxford: OUP).
Snyder-Beattie, Andrew E.; Ord, Toby; and Bonsall, Michael B. (2019). An upper bound for the background rate of human extinction. Scientific Reports 9:11054, 1–9.
I’d like to thank Fin Moorhouse, Kevin Kuruc, and Chad Jones for insightful discussion and comments, though of course this doesn’t imply that they endorse the arguments in this paper.
The arguments in this paper would also apply if it were instead a temporary acceleration leading to a permanent speed-up of progress. In that case an endogenous end time would mean a temporal compression of our trajectory up to that point: achieving all the same quality levels, but having less time to enjoy each one, then prematurely ending.
This is proved in more detail, addressing several complications in Ord (forthcoming). Several complexities arise from different approaches to population ethics, but the basic argument holds for any approach to population ethics that considers a future with lower total and average wellbeing to be worse. If your approach to population ethics avoids this argument, then note that whether advancing progress is good or bad is now highly sensitive to the theory of population ethics — another contentious and esoteric topic.
One could generalise this argument beyond human extinction. First, one could consider other kinds of existential risk. For instance, if a global totalitarian regime arises which permanently reduces quality of life to a lower level than it is today, then what matters is whether the date of its onset is endogenous. Alternatively, we could broaden this argument to include decision-makers who are partial to a particular country or culture. In that case, what matters is whether advancing progress would advance the end of that group.
For example, if final years are 10 times as good as present years, then we would only need a 10% chance that humanity’s end was exogenous for advancing progress to be good in expectation. I think this is a promising strategy for saving the expected value of progress.
In this setting, the former idea of an endogenous end time roughly corresponds to the assumption that advancing progress by a year also shifts humanity’s survival curve left by a year, while an exogenous end time corresponds to the assumption that the survival curve stays fixed. A third salient possibility is that it is the corresponding hazard curve which shifts left by a year. This possibility is mentioned in Ord (forthcoming, pp. 17 & 30). The first and third of these correspond to what Bostrom (2014) calls transition risk and state risk.
For example, one could decompose the overall hazard curve into an exogenous hazard curve plus an endogenous one, where only the latter gets shifted to the left. Alternatively, Aschenbrenner & Trammell (2024) present a promising parameterisation that interpolates between state risk and transition risk (and beyond).
See, for example, my book on existential risk (Ord 2020), or the final section of Ord (forthcoming).
See Moynihan (2020).
Any substantial change to the future, including advancing progress by a year, would probably have large chaotic effects on the future in the manner of the butterfly effect. But only a systematic change to the expected shape or duration of the curve, could help save the initial case for advancing progress.
The fossil record suggests natural extinction risk is lower than 1 in 1,000 per century — after all humanity has already survived 2,000 centuries and typical mammalian species survive 10,000 centuries. See Snyder-Beattie et al. (2019) and Ord (2020, pp. 80–87).
Another kind of case that is often made is that the sheer rate of progress could change the future trajectory. A common version is that the world is more stable with steady 3–5% economic growth: if we don’t keep moving forward, we will fall over. This is plausible but gives quite a different kind of conclusion. It suggests it is good to advance economic progress when growth is at 2% but may be bad to advance it when growth is 5%. Moreover, unless the central point about progress bringing forward humanity’s end is dealt with head-on, the best this steady-growth argument can achieve is to say that ‘yes, advancing progress may cut short our future, but unfortunately we need to do it anyway to avoid collapse’.
Beckstead calls this kind of change to where we’re going, rather than just how quickly we get there a ‘trajectory change’ (Beckstead 2013).
Bostrom’s concept of differential technological development (Bostrom 2002) is a special case.