I think this was a fantastic post—interesting and somehow, among all that maths, quite fun to read :P I’m about to start working on a related research project, so if you’d be willing to talk through this in person I’d love to arrange something. I’ll PM you my contact details.
A couple of stylistic suggestions:
Change the title! My immediate response was ‘oh god, another doom and gloom post’, and one friend already mentioned to me he was going to put off reading it ‘until he was in too good a mood’. I think that’s doing it an injustice.
Label the axes tables explicitly—I found them hard to parse, even having felt like I grasped the surrounding discussion.
Substantively, my main disagreement is on the space side. Firstly I agree with weeatquince that in 20 centuries we could get way beyond one offworld colony. Elon Musk’s goal last I heard was to get a self-sustaining Mars settlement by the 2050s, and if we get that far I’d expect the ‘forcing function’ he/Zubrin describe to incentivise faster colonisation of other habitats. Even assuming Musk is underestimating the challenge by a factor of 2-3, such that it will take a century then if there aren’t any hard limits like insuperable health problems associated with microgravity, a colonisation rate of 1 new self-sustaining colony per colony per century, or 2^n total colonies, where n is number of centuries from now seems quite plausible to me! - at least up to the point where we’ve started using up all the rocky bodies in the solar system (but we’ll hopefully be sending colonisation missions to other star systems by then—which would necessarily be capable of self-sufficiency even if our whole system is gone by the time they get there).
Secondly, I think even short term it’s a better defence than you give it credit for. Engineered pandemics would have a much tougher time both reaching Mars without advance warning from everyone dying on Earth, and then spreading around a world which is presumably going to be heavily airlocked, and generally have much more active monitoring and control of its environment. Obviously it’s hard to say much about ‘other/unforeseen anthropogenic risks’, but we should presumably discount some of their risk for similar reasons. More importantly IMO, the ‘green’ probability estimates you cite are for those things directly wiping out humanity, not the risk of them causing some major catastrophe, the latter of which I would guess is in total much higher than the risk of catastrophe from the red box. And IMO the EA movement tends to overestimate the probability that we can fully rebuild (and then reach the stars) from civilisational collapse. If you put that probability at 90-99% then such catastrophes are essentially a 0.01-0.1x multiplier on the loss of value of collapse—so they could still end up being a major x-risk factor if the triggering events are sufficiently likely (this is the focus of the project I mentioned).
Thanks! Super-interesting. Please do PM me your contact details.
I’ll try to type up a proper reply within a couple of hours. I’m currently being chased out of my house by the 90+-degree heat in England. Not good with no A/C. I think I’ll be able to add something to the discussion once I’ve gone somewhere cool and downed a pint of ice cream.
Edit: Okay, semi-proper response below.
The presentational suggestions are extremely helpful. Many thanks!
1. How far could we go?: I’m really curious to hear more about how quickly you think we might settle the galaxy, and how we could go about doing that. I’ve heard similar views to yours from some people at GPI, for example Tomi Francis.
I guess I tend to think that settling $2^n$ planets in $n$ centuries might be a bit fast, especially in the short-term. This would put us at just over a million planets in twenty centuries. And while that might not sound like a lot, let’s remember that the closest nearby star (Alpha Centauri) is more than four light years away. So if we’re going to make it to even a few dozen planets in 2,000 years, we need to travel several light years, and drop enough people and supplies to make the settlement viable and worthwhile. And to make it to a million planets, we’d have to rinse and repeat many times.
I certainly wouldn’t want to rule out that possibility in principle. And I have to admit that as a philosopher, I’m a bit under-qualified to pronounce on the future of space travel. But do you have some thoughts about how we might get this far in the near future?
2. Short-term: This is really worth thinking more about. Thanks! I think you must be right to think that settling space would help a great deal with global catastrophic risks, which might be quite bad even if they fall short of existential risks. So for example, there’s a pretty good chance that COVID would not have made it to Mars.
I also tend to agree with you that people are sometimes a bit pessimistic about the possibility of recovery from global catastrophe. That’s one of the reasons I would not consider myself a Pessimist about existential risk in the sense set out in this post.
I think it might be a bit harder to make the case that settling Mars would lead to a substantial drop in existential risk (rather than catastrophic risk), if you have the Pessimist’s view that most existential risk comes from things in the red box. So for example, the Pessimist isn’t thinking about pandemics that start infecting and killing people in a visible manner over a course of weeks or months. Those wouldn’t kill everybody, since even people on earth would have advanced warning. She’s thinking about a bit more science-y scenarios like “someone creates a sleeper virus and spreads that virus to every living person on earth, and only then activates the virus”. I can’t say I place quite as much probability on such scenarios. But if we’re really considering the possibility of infecting every man, woman and child on earth with a deadly sleeper virus, how sure are we that they couldn’t send a few folks with the virus to Mars? It’s not a sure bet that they would manage that, but lots of things in this sleeper virus story fall a fair bit short of sure bets.
Looking forward to chatting more about this in person!
I had another thought on why you might be underrating space settlement. The threat of an engineered pandemic, nuclear war etc constitute a certain type of constantish risk per year per colony. So we can agree that colonies reduce risk on nuclear war, and disagree for now on biorisk.
AIs seem like a separate class of one-off risk. At some point, we’ll create (or become) a superhuman AI, and at that point it will either wipe us out or not. If so, that I agree that even multiple colonies in the solar system, and perhaps even in other stars wouldn’t afford much protection—though they might afford some. But if not, it becomes much harder to envisage another AI coming along and doing what the first one didn’t, since now we presumably have an intelligence that can match it (and had a head start).
On this view, AI has its own built-in time-of-perilsness, and if in the scenario where it doesn’t wipe us all out it doesn’t also permanently fix all our problems, space colonisation now reduces the risk of the remaining threats by a much larger proportion.
Re 1, we needn’t be talking about planets. In principle, any decently sized rocky body in the asteroid belt can be a colony, or you could just build O’Neill cylinders. They might not be long-term sustainable without importing resources, but doing so wouldn’t be a problem in a lot of catastrophic scenarios, eg where some major shock destroyed civilisation on planets and left behind most of the minerals. In this scenario ‘self-sustainability’ is more like a scale than a distinct property, and having more sustainable-ish colonies seems like it would still dramatically increase resilience.
At some point you’ll still hit a physical limit of matter in the system, so such a growth rate wouldn’t last that long, but for this discussion it wouldn’t need to. Even just having colonies on the rocky planets and major moons would reduce the probability of any event that didn’t intentionally target all outposts getting them all would be much closer to zero. At 2^n growth rate (which actually seems very conservative to me in the absence of major catastrophes—Earth alone seems like it could hit that growth rate for a few centuries) I feel like you’d have reduced the risk of non-targeted catastrophes to effectively zero by the time you had maybe 10 colonies?
Re 2, I think we’re disagreeing where you say we’re agreeing :P—I think the EA movement probably overestimates the probability of ‘recovery’ from global catastrophe, esp where ‘recovery’ really means ‘get all the way to the glorious Virgo supercluster future’. If I’m right then they’re effectively existential risks with a 0.1 multiplier, or whatever you think the probability of non-recovery is.
In scenarios such as the sleeper virus, it seems like more colonies would still provide resilience. Presumably if it’s possible to create such a virus it’s possible to detect and neutralise it before its activation, and the probability of doing so is some function of time—which more colonies would give you more of, if you couldn’t activate it til it had infected everyone. I feel like this principle would generalise to almost any technological threat that was in principle reversible.
I think this was a fantastic post—interesting and somehow, among all that maths, quite fun to read :P I’m about to start working on a related research project, so if you’d be willing to talk through this in person I’d love to arrange something. I’ll PM you my contact details.
A couple of stylistic suggestions:
Change the title! My immediate response was ‘oh god, another doom and gloom post’, and one friend already mentioned to me he was going to put off reading it ‘until he was in too good a mood’. I think that’s doing it an injustice.
Label the axes tables explicitly—I found them hard to parse, even having felt like I grasped the surrounding discussion.
Substantively, my main disagreement is on the space side. Firstly I agree with weeatquince that in 20 centuries we could get way beyond one offworld colony. Elon Musk’s goal last I heard was to get a self-sustaining Mars settlement by the 2050s, and if we get that far I’d expect the ‘forcing function’ he/Zubrin describe to incentivise faster colonisation of other habitats. Even assuming Musk is underestimating the challenge by a factor of 2-3, such that it will take a century then if there aren’t any hard limits like insuperable health problems associated with microgravity, a colonisation rate of 1 new self-sustaining colony per colony per century, or 2^n total colonies, where n is number of centuries from now seems quite plausible to me! - at least up to the point where we’ve started using up all the rocky bodies in the solar system (but we’ll hopefully be sending colonisation missions to other star systems by then—which would necessarily be capable of self-sufficiency even if our whole system is gone by the time they get there).
Secondly, I think even short term it’s a better defence than you give it credit for. Engineered pandemics would have a much tougher time both reaching Mars without advance warning from everyone dying on Earth, and then spreading around a world which is presumably going to be heavily airlocked, and generally have much more active monitoring and control of its environment. Obviously it’s hard to say much about ‘other/unforeseen anthropogenic risks’, but we should presumably discount some of their risk for similar reasons. More importantly IMO, the ‘green’ probability estimates you cite are for those things directly wiping out humanity, not the risk of them causing some major catastrophe, the latter of which I would guess is in total much higher than the risk of catastrophe from the red box. And IMO the EA movement tends to overestimate the probability that we can fully rebuild (and then reach the stars) from civilisational collapse. If you put that probability at 90-99% then such catastrophes are essentially a 0.01-0.1x multiplier on the loss of value of collapse—so they could still end up being a major x-risk factor if the triggering events are sufficiently likely (this is the focus of the project I mentioned).
Thanks! Super-interesting. Please do PM me your contact details.
I’ll try to type up a proper reply within a couple of hours. I’m currently being chased out of my house by the 90+-degree heat in England. Not good with no A/C. I think I’ll be able to add something to the discussion once I’ve gone somewhere cool and downed a pint of ice cream.
Edit: Okay, semi-proper response below.
The presentational suggestions are extremely helpful. Many thanks!
1. How far could we go?: I’m really curious to hear more about how quickly you think we might settle the galaxy, and how we could go about doing that. I’ve heard similar views to yours from some people at GPI, for example Tomi Francis.
I guess I tend to think that settling $2^n$ planets in $n$ centuries might be a bit fast, especially in the short-term. This would put us at just over a million planets in twenty centuries. And while that might not sound like a lot, let’s remember that the closest nearby star (Alpha Centauri) is more than four light years away. So if we’re going to make it to even a few dozen planets in 2,000 years, we need to travel several light years, and drop enough people and supplies to make the settlement viable and worthwhile. And to make it to a million planets, we’d have to rinse and repeat many times.
I certainly wouldn’t want to rule out that possibility in principle. And I have to admit that as a philosopher, I’m a bit under-qualified to pronounce on the future of space travel. But do you have some thoughts about how we might get this far in the near future?
2. Short-term: This is really worth thinking more about. Thanks! I think you must be right to think that settling space would help a great deal with global catastrophic risks, which might be quite bad even if they fall short of existential risks. So for example, there’s a pretty good chance that COVID would not have made it to Mars.
I also tend to agree with you that people are sometimes a bit pessimistic about the possibility of recovery from global catastrophe. That’s one of the reasons I would not consider myself a Pessimist about existential risk in the sense set out in this post.
I think it might be a bit harder to make the case that settling Mars would lead to a substantial drop in existential risk (rather than catastrophic risk), if you have the Pessimist’s view that most existential risk comes from things in the red box. So for example, the Pessimist isn’t thinking about pandemics that start infecting and killing people in a visible manner over a course of weeks or months. Those wouldn’t kill everybody, since even people on earth would have advanced warning. She’s thinking about a bit more science-y scenarios like “someone creates a sleeper virus and spreads that virus to every living person on earth, and only then activates the virus”. I can’t say I place quite as much probability on such scenarios. But if we’re really considering the possibility of infecting every man, woman and child on earth with a deadly sleeper virus, how sure are we that they couldn’t send a few folks with the virus to Mars? It’s not a sure bet that they would manage that, but lots of things in this sleeper virus story fall a fair bit short of sure bets.
Looking forward to chatting more about this in person!
I had another thought on why you might be underrating space settlement. The threat of an engineered pandemic, nuclear war etc constitute a certain type of constantish risk per year per colony. So we can agree that colonies reduce risk on nuclear war, and disagree for now on biorisk.
AIs seem like a separate class of one-off risk. At some point, we’ll create (or become) a superhuman AI, and at that point it will either wipe us out or not. If so, that I agree that even multiple colonies in the solar system, and perhaps even in other stars wouldn’t afford much protection—though they might afford some. But if not, it becomes much harder to envisage another AI coming along and doing what the first one didn’t, since now we presumably have an intelligence that can match it (and had a head start).
On this view, AI has its own built-in time-of-perilsness, and if in the scenario where it doesn’t wipe us all out it doesn’t also permanently fix all our problems, space colonisation now reduces the risk of the remaining threats by a much larger proportion.
Hey :)
Re 1, we needn’t be talking about planets. In principle, any decently sized rocky body in the asteroid belt can be a colony, or you could just build O’Neill cylinders. They might not be long-term sustainable without importing resources, but doing so wouldn’t be a problem in a lot of catastrophic scenarios, eg where some major shock destroyed civilisation on planets and left behind most of the minerals. In this scenario ‘self-sustainability’ is more like a scale than a distinct property, and having more sustainable-ish colonies seems like it would still dramatically increase resilience.
At some point you’ll still hit a physical limit of matter in the system, so such a growth rate wouldn’t last that long, but for this discussion it wouldn’t need to. Even just having colonies on the rocky planets and major moons would reduce the probability of any event that didn’t intentionally target all outposts getting them all would be much closer to zero. At 2^n growth rate (which actually seems very conservative to me in the absence of major catastrophes—Earth alone seems like it could hit that growth rate for a few centuries) I feel like you’d have reduced the risk of non-targeted catastrophes to effectively zero by the time you had maybe 10 colonies?
Re 2, I think we’re disagreeing where you say we’re agreeing :P—I think the EA movement probably overestimates the probability of ‘recovery’ from global catastrophe, esp where ‘recovery’ really means ‘get all the way to the glorious Virgo supercluster future’. If I’m right then they’re effectively existential risks with a 0.1 multiplier, or whatever you think the probability of non-recovery is.
In scenarios such as the sleeper virus, it seems like more colonies would still provide resilience. Presumably if it’s possible to create such a virus it’s possible to detect and neutralise it before its activation, and the probability of doing so is some function of time—which more colonies would give you more of, if you couldn’t activate it til it had infected everyone. I feel like this principle would generalise to almost any technological threat that was in principle reversible.