(Here I define utopia to be an interplanetary+ civilization with no wars, no poverty, no disease, no animal farming, etc.).
Given you had to choose one of these 3 scenarios, which one is preferable?
Scenario A: Utopia is guaranteed by the year 2150, but humanity has a 50% risk of extinction on the way there.
Scenario B: Utopia is guaranteed by the year 2250, but humanity has a 10% risk of extinction on the way there.
Scenario C: Utopia is guaranteed by the year 2350, but humanity has a 1% risk of extinction on the way there.
(The question ultimately boils down to “Would you sacrifice speed of progress for reducing existential risk?”).
I would easily pick C, and I base that choice on my intuition, an EV analysis, and a decision theory analysis.
First, intuition. It seems correct to me to delay progress for 2 centuries (or more) to practically ensure humanity achieves utopia, given that when we do, it would probably last thousands, if not millions, if not billions of years. A few centuries of slowed down progress are more than worth uncountable years of bliss for far, far, far more humans than have been alive throughout history. Intuition won’t get us too far though, so let’s go straight to the EV analysis.
EV Analysis
How many people, and how many years, would be needed to justify choosing Scenario C?
I made a graph in Desmos that plots out the expected utility of the 3 scenarios. The x-axis is the amount of years, while the y-axis is the expected utility. You can also change the population amount (p) and make the current world worse (n) such that n=1 means the present is as bad as a utopia would be good, n=2 means the present is twice as bad as a utopia would be good, n=-1 means the present is as good as a utopia, etc.
One thing to notice is the amount of years required to make up for the “utility debt” of not living in utopia is unaffected by the population size, but I thought I should still include it to make the graph more complete.
Assuming our present is as bad as a utopia would be good (n=1), to justify picking Scenario C over Scenario A would require that utopia last at least ~612 years. To justify picking C over B would require utopia to last at least ~2111 years. I think it’s safe to assume utopia will last far longer than that if achieved. With that said, let’s move on to the decision theory analysis.
Decision Theory Analysis
There are 3 scenarios with 2 outcomes each, giving us 6 possibilities. Let’s assume utopia will last for 10000 years, and just like in the EV analysis, n=1 (meaning the current world is as bad as utopia is good). We then get this:
Forgive the ugly drawing, it’s just to prove a point.
I can’t imagine anyone picking A over C, but I also find it quite hard to see someone pick B over C. The difference in utility from C to B is imperceptible, and while the loss is a fair chunk smaller (~33%), it’s also 10x more likely.
Just wanted to quickly write this post up since I see some people that seem to almost be more concerned with progress than existential risk. I think that’s misguided, since progress can always be made at a later time, but the extinction only happens once.
Why Rate Of Progress Is Basically Pointless
Link post
(Here I define utopia to be an interplanetary+ civilization with no wars, no poverty, no disease, no animal farming, etc.).
Given you had to choose one of these 3 scenarios, which one is preferable?
Scenario A: Utopia is guaranteed by the year 2150, but humanity has a 50% risk of extinction on the way there.
Scenario B: Utopia is guaranteed by the year 2250, but humanity has a 10% risk of extinction on the way there.
Scenario C: Utopia is guaranteed by the year 2350, but humanity has a 1% risk of extinction on the way there.
(The question ultimately boils down to “Would you sacrifice speed of progress for reducing existential risk?”).
I would easily pick C, and I base that choice on my intuition, an EV analysis, and a decision theory analysis.
First, intuition. It seems correct to me to delay progress for 2 centuries (or more) to practically ensure humanity achieves utopia, given that when we do, it would probably last thousands, if not millions, if not billions of years. A few centuries of slowed down progress are more than worth uncountable years of bliss for far, far, far more humans than have been alive throughout history. Intuition won’t get us too far though, so let’s go straight to the EV analysis.
EV Analysis
How many people, and how many years, would be needed to justify choosing Scenario C?
I made a graph in Desmos that plots out the expected utility of the 3 scenarios. The x-axis is the amount of years, while the y-axis is the expected utility. You can also change the population amount (p) and make the current world worse (n) such that n=1 means the present is as bad as a utopia would be good, n=2 means the present is twice as bad as a utopia would be good, n=-1 means the present is as good as a utopia, etc.
https://www.desmos.com/calculator/rpalpofavm?lang=it
One thing to notice is the amount of years required to make up for the “utility debt” of not living in utopia is unaffected by the population size, but I thought I should still include it to make the graph more complete.
Assuming our present is as bad as a utopia would be good (n=1), to justify picking Scenario C over Scenario A would require that utopia last at least ~612 years. To justify picking C over B would require utopia to last at least ~2111 years. I think it’s safe to assume utopia will last far longer than that if achieved. With that said, let’s move on to the decision theory analysis.
Decision Theory Analysis
There are 3 scenarios with 2 outcomes each, giving us 6 possibilities. Let’s assume utopia will last for 10000 years, and just like in the EV analysis, n=1 (meaning the current world is as bad as utopia is good). We then get this:
I can’t imagine anyone picking A over C, but I also find it quite hard to see someone pick B over C. The difference in utility from C to B is imperceptible, and while the loss is a fair chunk smaller (~33%), it’s also 10x more likely.
Just wanted to quickly write this post up since I see some people that seem to almost be more concerned with progress than existential risk. I think that’s misguided, since progress can always be made at a later time, but the extinction only happens once.