I thought this post to be really fun and useful, thanks! Some notes are below.
Optimism helps: If you’re an optimist about current levels of existential risk, you should think it’s even more important to reduce existential risk than if you’re a Pessimist.
This is not true for a short Time of Perils and low post-perils x-risk. Under the Time of Perils hypothesis, the derivative of the expected increase in value of the post-peril periods with respect to the existial risk per 100 year is:
(1 - r)^(N − 2) * (1 - N r) * f * V_SAFE.
This is positive for N < r^-1. Consequently, assuming r = 1⁄6, the length of the Time of Perils has to be shorter than 600 year for the value of reducing x-risk to increase with x-risk.
All of the models in this paper strongly suggest that this reaction [lower x-risk implies less value of reducing x-risk] misses the mark.
The models you present do not estimate the length of the Time of Perils nor the post-perils x-risk, so they do not present evidence against the possibility that, under the Time of Perild hypothesis, higher x-risk may imply increased value of reducing x-risk (see just above).
The Time of Perils Hypothesis is very important: Pessimists should think that settling the truth value of the Time of Perils Hypothesis is one of the most crucial outstanding research questions facing humanity today.
I agree the Time of Perils Hypothesis (TOP) is very important, and think this post is really useful to understand that. However, it is worth noting that even a small probability (e.g. 1 %) of TOP being true would imply that mitigating x-risk is quite good in terms of expected valuable. So, since it would not be reasonable to assign a very low probability to TOP being true (e.g. << 1 %), mitigating x-risk is still quite valuable. Of course, further research on TOP could still be useful.
For example, Toby Ord (2020) puts the risk of existential catastrophe by 2100 at 1⁄6
Ords estimate of 1⁄6 is for the next 100 years following 2020, i.e. until 2120 instead of 2100.
In the 2nd expression for V[WX]here, v should multiply the 1st fraction inside the square brackets.
Even if post-peril risk drops by 2,000% to 1% per century
Going from 20 % to 1 % is a reduction of 95 % (= 1 − 0.01/0.2), not 2 k%.
We need risk to drop very low, towards something like 0.01%. Note that for the Pessimist, this is a reduction of 200,000%!
Going from 20 % to 0.01 % is a reduction of 99.95 % (= 1 − 0.0001/0.2), not 200 k%.
For example, Toby Ord estimates natural risk in the next century at 1⁄10,000, but overall risk at 1⁄6.
For clarity, I would replace “century” by “100 years”, which is what Ord’s estimates refer to.
I thought this post to be really fun and useful, thanks! Some notes are below.
This is not true for a short Time of Perils and low post-perils x-risk. Under the Time of Perils hypothesis, the derivative of the expected increase in value of the post-peril periods with respect to the existial risk per 100 year is:
(1 - r)^(N − 2) * (1 - N r) * f * V_SAFE.
This is positive for N < r^-1. Consequently, assuming r = 1⁄6, the length of the Time of Perils has to be shorter than 600 year for the value of reducing x-risk to increase with x-risk.
The models you present do not estimate the length of the Time of Perils nor the post-perils x-risk, so they do not present evidence against the possibility that, under the Time of Perild hypothesis, higher x-risk may imply increased value of reducing x-risk (see just above).
I agree the Time of Perils Hypothesis (TOP) is very important, and think this post is really useful to understand that. However, it is worth noting that even a small probability (e.g. 1 %) of TOP being true would imply that mitigating x-risk is quite good in terms of expected valuable. So, since it would not be reasonable to assign a very low probability to TOP being true (e.g. << 1 %), mitigating x-risk is still quite valuable. Of course, further research on TOP could still be useful.
Ords estimate of 1⁄6 is for the next 100 years following 2020, i.e. until 2120 instead of 2100.
In the 2nd expression for V[WX] here, v should multiply the 1st fraction inside the square brackets.
Going from 20 % to 1 % is a reduction of 95 % (= 1 − 0.01/0.2), not 2 k%.
Going from 20 % to 0.01 % is a reduction of 99.95 % (= 1 − 0.0001/0.2), not 200 k%.
For clarity, I would replace “century” by “100 years”, which is what Ord’s estimates refer to.