Imagine graphing out percentiles for your credence distribution over values the entire future can take.
I personally find it more intuitive to plot the PDF of the value of the future, instead of its cumulative distribution function as you did, but I like your graphs too!
The closest thing to a ‘classic’ story in my head looks like below: on which (i) the long-run future is basically biomodal, between ruin and near-best futures, and (ii) the main effect of extinction mitigation is to make near-best futures more likely.
I do not know whether the value of the future is bimodal. However, even if so, I would guess reducing the nearterm risk of human extinction only infinitesimaly increases the probability of astronomically valuable worlds. Below is an illustration. The normal and dashed lines are the PDFs of the value of the future before and after the intervention. I suppose the neutral worlds would be made slightly less likely (dashed line below the normal line), and the slightly negative and positive worlds would me made slightly more likely (dashed line above the normal line), but the astronomically negative and positive worlds would only be made infinitesimaly more likely (basically no difference between the normal and dashed line).
Are there any interventions whose estimates of (posterior) counterfactual impact, in terms of expected total hedonistic utility (not e.g. preventing the extinction of a random species), do not decay to 0 in at most a few centuries? From my perspective, their absence establishes a strong prior against persistent longterm effects.
I’ll also add that, in the case of AI risk, I think that framing literal human extinction as the main test of whether the future will be good seems like a mistake, in particular because I think literal human extinction is much less likely than worlds where things go badly for other reasons.
I wonder whether the probability of things going badly is proportional to the nearterm risk of human extinction. In any case, I am not sure the size of the risk affects the possibility of some interventions having astronomical cost-effectiveness. I guess this mainly depends on how fast the (posterior) counterfactual impact decreases with the value of the future. If exponentially, based on my Fermi estimate in the post, the counterfactual impact linked to making astronomically valuable worlds more likely will arguably be negligible.
I would be curious to know how low would the risk of human extinction in the next 10 years (or other period) have to be for you to mostly:
Donate to animal welfare interventions (i.e. donate at least 50 % of your annual donations to animal welfare interventions).
Work on animal welfare (i.e. spend at least 50 % of your working time on animal welfare).
Thanks for the comment, Fin! Strongly upvoted.
I personally find it more intuitive to plot the PDF of the value of the future, instead of its cumulative distribution function as you did, but I like your graphs too!
I do not know whether the value of the future is bimodal. However, even if so, I would guess reducing the nearterm risk of human extinction only infinitesimaly increases the probability of astronomically valuable worlds. Below is an illustration. The normal and dashed lines are the PDFs of the value of the future before and after the intervention. I suppose the neutral worlds would be made slightly less likely (dashed line below the normal line), and the slightly negative and positive worlds would me made slightly more likely (dashed line above the normal line), but the astronomically negative and positive worlds would only be made infinitesimaly more likely (basically no difference between the normal and dashed line).
Are there any interventions whose estimates of (posterior) counterfactual impact, in terms of expected total hedonistic utility (not e.g. preventing the extinction of a random species), do not decay to 0 in at most a few centuries? From my perspective, their absence establishes a strong prior against persistent longterm effects.
I wonder whether the probability of things going badly is proportional to the nearterm risk of human extinction. In any case, I am not sure the size of the risk affects the possibility of some interventions having astronomical cost-effectiveness. I guess this mainly depends on how fast the (posterior) counterfactual impact decreases with the value of the future. If exponentially, based on my Fermi estimate in the post, the counterfactual impact linked to making astronomically valuable worlds more likely will arguably be negligible.
I would be curious to know how low would the risk of human extinction in the next 10 years (or other period) have to be for you to mostly:
Donate to animal welfare interventions (i.e. donate at least 50 % of your annual donations to animal welfare interventions).
Work on animal welfare (i.e. spend at least 50 % of your working time on animal welfare).