Causing extinction (or even some planet scale catastrophe with thousand-year plus consequences that falls short of extinction) would be an example of this wouldnāt it?
Didnāt Stanislav Petrov have the opportunity to meaningfully change expected utility for more than a few centuries?
In my mind, not meaningfully so. Based on my adjustments to the mortality rates of cooling events by CEARCH, the expected annual mortality rate from nuclear winter is 7.32*10^-6. So, given a 1.31 % annual probability of a nuclear weapon being detonated as an act of war, the mortality rate from nuclear winter conditional on at least 1 nuclear detonation is 0.0559 % (= 7.32*10^-6/ā0.0131). I estimated the direct deaths would be 1.16 times as large as the ones from the climatic effects, and I think CEARCHās mortality rates only account for these. So the total mortality rate conditional on at least 1 nuclear detonation would be 0.121 % (= 5.59*10^-4*(1 + 1.16)), corresponding to 9.68 M (= 0.00121*8*10^9) deaths for todayās population. Preventing millions of deaths is absolutely great, but arguably not enough to meaningfully improve the longterm future?
9.89 M people died from cancer in 2021, but eliminating cancer deaths over 1 year would not meaningully improve the future in e.g. 1 k years?
The Black Death killed like half of Europeās population, i.e. 413 (= 0.5/ā0.00121) times my estimate for the mortality rate conditional on a nuclear detonation, but will the world be meaningfully worse 300 years from now (i.e. 1 k years after the Black Death) because of it?
Even if the world is non-infinitesimaly different in 1 k years as a result of the above, would the counterfactual improvements in the world value after 1 k years be negligible in comparison with that before 1 k years?
Why not simply assessing the value of preventing wars and pandemics based on standard cost-effectiveness analyses, for which we can rely on empirically informed estimates like the above?
Sure, but the question is what do we change by speeding progress up.
The way I think about it is that interventions improve the world for a few years to centuries, and then after that practically nothing changes (in the sense the expected benefits coming from changing the longterm are negligible). If I understand correctly, frameworks like Tobyās or yours that suggest that improving the world now may be outweighted by making the world worse in the future suppose we can reliably change the value the world will have longterm, and that this is may be the driver of the overall effect.
Thanks for the detailed reply on that! Youāve clearly thought about this a lot, and Iām very happy to believe youāre right on the impact of nuclear war, but It sounds like you are more or less opting for what I called option 1? In which case, just substitute nuclear war for a threat that would literally cause extinction with high probability (say release of a carefully engineered pathogen with high fatality rate, long incubation period, and high infectiousness). Wouldnāt that meaningfully affect utility for more than a few centuries? Because there would be literally no one left, and that effect is guaranteed to be persistent! Even if it ājustā reduced the population by 99%, that seems like it would very plausibly have effects for thousands of years into the future.
It seems to me that to avoid this, you have to either say that causing extinction (or near extinction level catastrophe) is virtually impossible, through any means, (what I was describing as option 1) or go the other extreme and say that it is virtually guaranteed in the short term anyway, so that counterfactual impact disappears quickly (what I was describing as option 2). Just so I understand what youāre saying, are you claiming one of these two things? Or is there another way out that Iām missing?
Thanks for the kind words. I was actually unsure whether I should have followed up given my comments in this thread had been downvoted (all else equal, I do not want to annoy readers!), so it is good to get some information.
Thanks for the detailed reply on that! Youāve clearly thought about this a lot, and Iām very happy to believe youāre right on the impact of nuclear war, but It sounds like you are more or less opting for what I called option 1? In which case, just substitute nuclear war for a threat that would literally cause extinction with high probability (say release of a carefully engineered pathogen with high fatality rate, long incubation period, and high infectiousness). Wouldnāt that meaningfully affect utility for more than a few centuries? Because there would be literally no one left, and that effect is guaranteed to be persistent! Even if it ājustā reduced the population by 99%, that seems like it would very plausibly have effects for thousands of years into the future.
I think the effect of the intervention will still decrease to practically 0 in at most a few centuries in that case, such that reducing the nearterm risk of human extinction is not astronomically cost-effective. I guess you are imagining that humans either go extinct or have a long future where they go on to realise lots of value. However, this is overly binary in my view. I elaborate on this in the post I linked to at the start of this paragraph, and its comments.
It seems to me that to avoid this, you have to either say that causing extinction (or near extinction level catastrophe) is virtually impossible, through any means, (what I was describing as option 1) or go the other extreme and say that it is virtually guaranteed in the short term anyway, so that counterfactual impact disappears quickly (what I was describing as option 2). Just so I understand what youāre saying, are you claiming one of these two things? Or is there another way out that Iām missing?
I guess the probability of human extinction in the next 10 years is around 10^-7, i.e. very unlikely, but far from impossible.
Iām at least finding it useful figuring out exactly where we disagree. Please stop replying if itās taking too much of your time, but not because of the downvotes!
I guess you are imagining that humans either go extinct or have a long future where they go on to realise lots of value.
This isnāt quite what Iām saying, depending on what you mean by ālotsā and ālongā. For your āimpossible for an intervention to have counterfactual effects for more than a few centuriesā claim to be false, we only need the future of humanity to have a non-tiny chance of being longer than a few centuries (not that long), and for there to be conceivable interventions which have a non-tiny chance of very quickly causing extinction. These interventions would then meaningfully affect counterfactual utility for more than a few centuries.
To be more concrete and less binary, suppose we are considering an intervention that has a risk p of almost immediately leading to extinction, and otherwise does nothing. Let U be the expected utility generated in a year, in 500 years time, absent any intervention. If you decide to make this intervention, that has the effect of changing U to (1-p)U, and so the utility generated in that far future year has been changed by pU.
For this to be tiny/ānon-meaningful, we either need p to be tiny, or U to be tiny (or both).
Are you saying:
There are no concievable interventions someone could make with p non-tiny.
U, expected utility in a year in 500 years time, is approximately 0.
Something elseā¦ my setup of the situation is wrong, or unrealistic..?
There are no concievable interventions someone could make with p non-tiny.
U, expected utility in a year in 500 years time, is approximately 0.
Something elseā¦ my setup of the situation is wrong, or unrealistic..?
1, in the sense I think the change in the immediate risk of human extinction per cost is astronomically low for any conceivable intervention. Relatedly, you may want to check my discussion with Larks in the post I linked to.
I do not think so.
In my mind, not meaningfully so. Based on my adjustments to the mortality rates of cooling events by CEARCH, the expected annual mortality rate from nuclear winter is 7.32*10^-6. So, given a 1.31 % annual probability of a nuclear weapon being detonated as an act of war, the mortality rate from nuclear winter conditional on at least 1 nuclear detonation is 0.0559 % (= 7.32*10^-6/ā0.0131). I estimated the direct deaths would be 1.16 times as large as the ones from the climatic effects, and I think CEARCHās mortality rates only account for these. So the total mortality rate conditional on at least 1 nuclear detonation would be 0.121 % (= 5.59*10^-4*(1 + 1.16)), corresponding to 9.68 M (= 0.00121*8*10^9) deaths for todayās population. Preventing millions of deaths is absolutely great, but arguably not enough to meaningfully improve the longterm future?
9.89 M people died from cancer in 2021, but eliminating cancer deaths over 1 year would not meaningully improve the future in e.g. 1 k years?
The Black Death killed like half of Europeās population, i.e. 413 (= 0.5/ā0.00121) times my estimate for the mortality rate conditional on a nuclear detonation, but will the world be meaningfully worse 300 years from now (i.e. 1 k years after the Black Death) because of it?
Even if the world is non-infinitesimaly different in 1 k years as a result of the above, would the counterfactual improvements in the world value after 1 k years be negligible in comparison with that before 1 k years?
Why not simply assessing the value of preventing wars and pandemics based on standard cost-effectiveness analyses, for which we can rely on empirically informed estimates like the above?
The way I think about it is that interventions improve the world for a few years to centuries, and then after that practically nothing changes (in the sense the expected benefits coming from changing the longterm are negligible). If I understand correctly, frameworks like Tobyās or yours that suggest that improving the world now may be outweighted by making the world worse in the future suppose we can reliably change the value the world will have longterm, and that this is may be the driver of the overall effect.
Thanks for the detailed reply on that! Youāve clearly thought about this a lot, and Iām very happy to believe youāre right on the impact of nuclear war, but It sounds like you are more or less opting for what I called option 1? In which case, just substitute nuclear war for a threat that would literally cause extinction with high probability (say release of a carefully engineered pathogen with high fatality rate, long incubation period, and high infectiousness). Wouldnāt that meaningfully affect utility for more than a few centuries? Because there would be literally no one left, and that effect is guaranteed to be persistent! Even if it ājustā reduced the population by 99%, that seems like it would very plausibly have effects for thousands of years into the future.
It seems to me that to avoid this, you have to either say that causing extinction (or near extinction level catastrophe) is virtually impossible, through any means, (what I was describing as option 1) or go the other extreme and say that it is virtually guaranteed in the short term anyway, so that counterfactual impact disappears quickly (what I was describing as option 2). Just so I understand what youāre saying, are you claiming one of these two things? Or is there another way out that Iām missing?
Thanks for the kind words. I was actually unsure whether I should have followed up given my comments in this thread had been downvoted (all else equal, I do not want to annoy readers!), so it is good to get some information.
I think the effect of the intervention will still decrease to practically 0 in at most a few centuries in that case, such that reducing the nearterm risk of human extinction is not astronomically cost-effective. I guess you are imagining that humans either go extinct or have a long future where they go on to realise lots of value. However, this is overly binary in my view. I elaborate on this in the post I linked to at the start of this paragraph, and its comments.
I guess the probability of human extinction in the next 10 years is around 10^-7, i.e. very unlikely, but far from impossible.
Iām at least finding it useful figuring out exactly where we disagree. Please stop replying if itās taking too much of your time, but not because of the downvotes!
This isnāt quite what Iām saying, depending on what you mean by ālotsā and ālongā. For your āimpossible for an intervention to have counterfactual effects for more than a few centuriesā claim to be false, we only need the future of humanity to have a non-tiny chance of being longer than a few centuries (not that long), and for there to be conceivable interventions which have a non-tiny chance of very quickly causing extinction. These interventions would then meaningfully affect counterfactual utility for more than a few centuries.
To be more concrete and less binary, suppose we are considering an intervention that has a risk p of almost immediately leading to extinction, and otherwise does nothing. Let U be the expected utility generated in a year, in 500 years time, absent any intervention. If you decide to make this intervention, that has the effect of changing U to (1-p)U, and so the utility generated in that far future year has been changed by pU.
For this to be tiny/ānon-meaningful, we either need p to be tiny, or U to be tiny (or both).
Are you saying:
There are no concievable interventions someone could make with p non-tiny.
U, expected utility in a year in 500 years time, is approximately 0.
Something elseā¦ my setup of the situation is wrong, or unrealistic..?
1, in the sense I think the change in the immediate risk of human extinction per cost is astronomically low for any conceivable intervention. Relatedly, you may want to check my discussion with Larks in the post I linked to.