I agree that it’s important to look in detail at models to see what is going on. We can’t settle debates about value from the armchair.
I’ll try to type up some thoughts in a few edits, since I want to make sure to think about what to say.
Population growth: It’s definitely possible to decompose the components of the Ord/Adamczewski/Thorstad model into their macroeconomic determinants (population, capital, technology, etc.). Economists like to do this. For example, Leopold does this.
It can also be helpful to decompose the model in other ways. Sociologists might want to split up things like v and r into a more fine-grained model of their social/political determinants, for example.
I tend to think that population growth is not going to be enough to substantially reverse the conclusions of the model, although I’d be really interested to see if you wanted to work through the conclusions here. For example, my impression with Leopold’s model is that if you literally cut population growth out of the model, the long-term qualitative behavior and conclusions wouldn’t change that much: it would still be the case that the driving assumptions are (a) research in safety technologies exponentially decreases risk, (b) the rate at which safety research exponentially decreases risk is “fast”, in the sense specified by Leopold’s main theorem.
(I didn’t check this explicitly. I’d be curious if someone wanted to verify this).
I also think it is important to think about some ways in which population growth can be bad for the value of existential risk mitigation. For example, the economist Maya Eden is looking at non-Malthusian models of population growth on which there is room for the human population to grow quite substantially for quite some time. She thinks that these models can often make it very important to do things that will kickstart population growth now, like staying at home to raise children or investing in economic growth. Insofar as investments and research into existential risk mitigation take us away from those activities, it might turn out that existential risk mitigation is relatively less valuable—quite substantially so, on some of Maya’s models. We just managed to talk Maya into contributing a paper to an anthology on longtermism that GPI is putting together, so I hope that some of these models will be in print within 1-2 years.
In the meantime, maybe some people would like to play around with population in Leopold’s model and report back on what happens?
First edit: Constant risk and guaranteed extinction: A nitpick: constant risk doesn’t assume guaranteed extinction, but rather extinction with probability 1. Probability 0 events are maximally unlikely, but not impossible. (Example: for every point on 3-D space there’s probability 0 that a dart lands centered around that point. But it does land centered around one).
More to the point, it’s not quite the assumption of constant risk that gives probability 1 to (eventual) extinction. To get probability <1 of eventual extinction, in most models you something stronger like (a) indefinitely sustained decay in extinction risk of exponential or higher speed, and (b) a “fast” rate of decay in your chosen functional form. This is pretty clear, for example, in Leopold’s model.
I think that claims like (a) and (b) can definitely be true, but that it is important to think carefully about why they are true. For example, Leopold generates (a) explicitly through a specified mechanism: producing safety technologies exponentially decreases risk. I’ve argued that (a) might not be true. Leopold doesn’t generate (b). Leopold just put (b) as part of his conclusion: if (b) isn’t true, we’re toast.
Second edit: Allocating resources towards safety: I think it would be very valuable to do more explicit modelling about the effect of resources on safety.
Leopold’s model discusses one way to do this. If you assume that safety research drives down risk exponentially at a reasonably quick rate, then we can just spend a lot more on safety as we get richer. I discuss this model in Section 5.
That’s not the only mechanism you could propose for using resources to provide safety. Did you have another mechanism in mind?
I think it would be super-useful to have an alternative to Leopold’s model. More models are (almost) always better.
In the spirit of our shared belief in the importance of models, I would encourage you to write down your model of choice explicitly and study it a bit. I think it could be very valuable for readers to work through the kinds of modelling assumptions needed to get a Time of Perils Hypothesis of the needed form, and why we might believe/disbelieve these assumptions.
Third edit: Constant value: It would definitely be bad to assume that all centuries have constant value. I think the exact word I used for that assumption was “bonkers”! My intention with the value growth models in Section 3 was to show that the conclusions are robust to some models of value growth (linear, quadratic) and in another comment I’ve sketched some calculations to suggest that my the model’s conclusions might even be robust to cubic, quartic (!) or maybe even quintic (!!) growth.
I definitely didn’t consider logistic growth explicitly. It could be quite interesting to think about the case of logistic growth. Do you want to write this up and see what happens?
I don’t think that the warming on short-termist cause areas should rely on an assumption of constant v. This is the case for a number of reasons. (a) Anything we can do to decrease the value of longtermist causes translates into a warming in our (relative) attitudes towards short-termist cause areas; (b) Short-termist causes can often have quite good longtermist implications, as for example in the Maya Eden models I mentioned earlier, or in some of Tyler Cowen’s work on economic growth. (c) As we saw in the model, probabilities matter. Even if you think there is an astronomical amount of value out there to be gained, if you think we’re sufficiently unlikely to gain it then short-termist causes can look relatively more attractive.
A concluding thought: I do very much like your emphasis on concrete models as a substitute for disgareement. Perhaps I could interest you in working out some of the models you raised in more detail, for example by modeling the effects of population growth using a standard macroeconomic model (Leopold uses a Solow-style model, but you could use another, and maybe go more endogenous)? I’d be curious to see what you find!
Thanks Mark! This is extremely helpful.
I agree that it’s important to look in detail at models to see what is going on. We can’t settle debates about value from the armchair.
I’ll try to type up some thoughts in a few edits, since I want to make sure to think about what to say.
Population growth: It’s definitely possible to decompose the components of the Ord/Adamczewski/Thorstad model into their macroeconomic determinants (population, capital, technology, etc.). Economists like to do this. For example, Leopold does this.
It can also be helpful to decompose the model in other ways. Sociologists might want to split up things like v and r into a more fine-grained model of their social/political determinants, for example.
I tend to think that population growth is not going to be enough to substantially reverse the conclusions of the model, although I’d be really interested to see if you wanted to work through the conclusions here. For example, my impression with Leopold’s model is that if you literally cut population growth out of the model, the long-term qualitative behavior and conclusions wouldn’t change that much: it would still be the case that the driving assumptions are (a) research in safety technologies exponentially decreases risk, (b) the rate at which safety research exponentially decreases risk is “fast”, in the sense specified by Leopold’s main theorem.
(I didn’t check this explicitly. I’d be curious if someone wanted to verify this).
I also think it is important to think about some ways in which population growth can be bad for the value of existential risk mitigation. For example, the economist Maya Eden is looking at non-Malthusian models of population growth on which there is room for the human population to grow quite substantially for quite some time. She thinks that these models can often make it very important to do things that will kickstart population growth now, like staying at home to raise children or investing in economic growth. Insofar as investments and research into existential risk mitigation take us away from those activities, it might turn out that existential risk mitigation is relatively less valuable—quite substantially so, on some of Maya’s models. We just managed to talk Maya into contributing a paper to an anthology on longtermism that GPI is putting together, so I hope that some of these models will be in print within 1-2 years.
In the meantime, maybe some people would like to play around with population in Leopold’s model and report back on what happens?
First edit: Constant risk and guaranteed extinction: A nitpick: constant risk doesn’t assume guaranteed extinction, but rather extinction with probability 1. Probability 0 events are maximally unlikely, but not impossible. (Example: for every point on 3-D space there’s probability 0 that a dart lands centered around that point. But it does land centered around one).
More to the point, it’s not quite the assumption of constant risk that gives probability 1 to (eventual) extinction. To get probability <1 of eventual extinction, in most models you something stronger like (a) indefinitely sustained decay in extinction risk of exponential or higher speed, and (b) a “fast” rate of decay in your chosen functional form. This is pretty clear, for example, in Leopold’s model.
I think that claims like (a) and (b) can definitely be true, but that it is important to think carefully about why they are true. For example, Leopold generates (a) explicitly through a specified mechanism: producing safety technologies exponentially decreases risk. I’ve argued that (a) might not be true. Leopold doesn’t generate (b). Leopold just put (b) as part of his conclusion: if (b) isn’t true, we’re toast.
Second edit: Allocating resources towards safety: I think it would be very valuable to do more explicit modelling about the effect of resources on safety.
Leopold’s model discusses one way to do this. If you assume that safety research drives down risk exponentially at a reasonably quick rate, then we can just spend a lot more on safety as we get richer. I discuss this model in Section 5.
That’s not the only mechanism you could propose for using resources to provide safety. Did you have another mechanism in mind?
I think it would be super-useful to have an alternative to Leopold’s model. More models are (almost) always better.
In the spirit of our shared belief in the importance of models, I would encourage you to write down your model of choice explicitly and study it a bit. I think it could be very valuable for readers to work through the kinds of modelling assumptions needed to get a Time of Perils Hypothesis of the needed form, and why we might believe/disbelieve these assumptions.
Third edit: Constant value: It would definitely be bad to assume that all centuries have constant value. I think the exact word I used for that assumption was “bonkers”! My intention with the value growth models in Section 3 was to show that the conclusions are robust to some models of value growth (linear, quadratic) and in another comment I’ve sketched some calculations to suggest that my the model’s conclusions might even be robust to cubic, quartic (!) or maybe even quintic (!!) growth.
I definitely didn’t consider logistic growth explicitly. It could be quite interesting to think about the case of logistic growth. Do you want to write this up and see what happens?
I don’t think that the warming on short-termist cause areas should rely on an assumption of constant v. This is the case for a number of reasons. (a) Anything we can do to decrease the value of longtermist causes translates into a warming in our (relative) attitudes towards short-termist cause areas; (b) Short-termist causes can often have quite good longtermist implications, as for example in the Maya Eden models I mentioned earlier, or in some of Tyler Cowen’s work on economic growth. (c) As we saw in the model, probabilities matter. Even if you think there is an astronomical amount of value out there to be gained, if you think we’re sufficiently unlikely to gain it then short-termist causes can look relatively more attractive.
A concluding thought: I do very much like your emphasis on concrete models as a substitute for disgareement. Perhaps I could interest you in working out some of the models you raised in more detail, for example by modeling the effects of population growth using a standard macroeconomic model (Leopold uses a Solow-style model, but you could use another, and maybe go more endogenous)? I’d be curious to see what you find!