My understanding is that, at a high level, this effect is counterbalanced by the fact that a high rate of extinction risk means the expected value of the future is lower. In this example, we only reduce the risk this century to 10%, but next century it will be 20%, and the one after that it will be 20% and so on. So the risk is 10x higher than in the 2% to 1% scenario. And in general, higher risk lowers the expected value of the future.
In this simple model, these two effects perfectly counterbalance each other for proportional reductions of existential risk. In fact, in this simple model the value of reducing risk is determined entirely by the proportion of the risk reduced and the value of future centuries. (This model is very simplified, and Thorstad explores more complex scenarios in the paper).
Yes, essentially preventing extinction “pays off” more in the low risk situation because the effects ripple on for longer.
Mathematically, if the value of one century is v, the “standard” chance of extinction is r, and the rate of extinction just for this century is d, then the expected value of the remaining world will be
v(1−d)+v(1−d)(1−r)+v(1−d)(1−r)2+...
= v(1−d)/r (using geometric sums).
In the world where background risk is 20%, but we reduce this century risk from 20% to 10%, the total value goes from 4*v to 4.5*v.
In the world where background risk is 2%, but we reduce this century risk from 20% to 10%, the total value goes from 49*v to 49.5*v.
In both cases, our intervention has added 0.5v to the total value.
My understanding is that, at a high level, this effect is counterbalanced by the fact that a high rate of extinction risk means the expected value of the future is lower. In this example, we only reduce the risk this century to 10%, but next century it will be 20%, and the one after that it will be 20% and so on. So the risk is 10x higher than in the 2% to 1% scenario. And in general, higher risk lowers the expected value of the future.
In this simple model, these two effects perfectly counterbalance each other for proportional reductions of existential risk. In fact, in this simple model the value of reducing risk is determined entirely by the proportion of the risk reduced and the value of future centuries. (This model is very simplified, and Thorstad explores more complex scenarios in the paper).
Yes, essentially preventing extinction “pays off” more in the low risk situation because the effects ripple on for longer.
Mathematically, if the value of one century is v, the “standard” chance of extinction is r, and the rate of extinction just for this century is d, then the expected value of the remaining world will be
v(1−d)+v(1−d)(1−r)+v(1−d)(1−r)2+...
= v(1−d)/r (using geometric sums).
In the world where background risk is 20%, but we reduce this century risk from 20% to 10%, the total value goes from 4*v to 4.5*v.
In the world where background risk is 2%, but we reduce this century risk from 20% to 10%, the total value goes from 49*v to 49.5*v.
In both cases, our intervention has added 0.5v to the total value.
Alright, that makes sense; thank you!