Sorry, I don’t find this is really speaking to my question?
I totally believe that people make mistakes in thinking about this stuff for reasons along the lines of the biases you discuss. But I also think that you’re making some strong assumptions about things essentially cancelling out that I think are unjustified, and talking about mistakes that other people are making doesn’t (it seems to me) work as a justification.
Sorry, I don’t find this is really speaking to my question?
I do not think the difficulty of decreasing a risk is independent of the value at stake. It is harder to decrease a risk when a larger value is at stake. So, in my mind, decreasing the nearterm risk of human extinction is astronomically easier than decreasing the risk of not achieving 10^50 lives of value, such that decreasing the former by e.g. 10^-10 leads to a relative increase in the latter much smaller than 10^-10.
I also think that you’re making some strong assumptions about things essentially cancelling out
Could you elaborate on why you think I am making a strong assumption in terms of questioning the following?
In light of the above, I expect what David Thorstad calls rapid diminution. I see the difference between the PDF after and before an intervention reducing the nearterm risk of human extinction as quickly decaying to 0, thus making the increase in the expected value of the astronomically valuable worlds negligible. For instance:
If the difference between the PDF after and before the intervention decays exponentially with the value of the future v, the increase in the value density caused by the intervention will be proportional to v*e^-v[4].
The above rapidly goes to 0 as v increases. For a value of the future equal to my expected value of 1.40*10^52 human lives, the increase in value density will multiply a factor of 1.40*10^52*e^(-1.40*10^52) = 10^(log10(1.40)*52 - log10(e)*1.40*10^52) = 10^(-6.08*10^51), i.e. it will be basically 0.
Do you think I am overestimating how fast the difference between the PDF after and before the intervention decays? As far as I can tell, the (posterior) counterfactual impact of interventions whose effects can be accurately measured, like ones in global health and development, decays to 0 as time goes by. I do not have a strong view on the particular shape of the difference, but exponential decay is quite typical in many contexts.
I do not think the difficulty of decreasing a risk is independent of the value at stake. It is harder to decrease a risk when a larger value is at stake.
This makes sense as a kind of general prior to come in with. Although note:
It’s surely observational, not causal—there’s no magic at play which means if you keep a scenario fixed except for changing the value at stake, this should impact the difficulty
One of the plausible generating mechanisms is having a broad altruistic market which takes the best opportunities, leaving no free lunches—but for some of the cases we’re discussing it’s unclear the market could have made it efficient
So, in my mind, decreasing the nearterm risk of human extinction is astronomically easier than decreasing the risk of not achieving 10^50 lives of value, such that decreasing the former by e.g. 10^-10 leads to a relative increase in the latter much smaller than 10^-10.
Now it looks to me as though you’re dogmatically sticking with the prior. Having come across the (kinda striking) observation which says “if there’s a realistic chance of spreading to the stars, then premature human extinction would forgo astronomical value”, it seems like you’re saying “well that would mean that the prior was wrong, so that observation can’t be quite right”, and then reasoning from your prior to try to draw conclusions about the causal relationships there.
Whereas I feel that the prior reasonably justifies more scepticism in cases where more lives are at stake (and indeed, I do put a bunch of probability on “averting near-term extinction doesn’t save astronomical value for some reason or another”, though the reasons tend to be ones where we never actually had a shot of an astronomically big future in the first place, and I think that that’s sort of the appropriate target for scepticism), but doesn’t give you anything strong enough to be confident about things.
(I certainly wouldn’t be surprised if I’m somehow misunderstanding what you’re doing; I’m just responding to the picture I’m getting from what you’ve written.)
Now it looks to me as though you’re dogmatically sticking with the prior.
Are there any interventions whose estimates of (posterior) counterfactual impact do not decay to 0 in at most a few centuries? From my perspective, their absence establishes a strong prior against persistent longterm effects.
I do put a bunch of probability on “averting near-term extinction doesn’t save astronomical value for some reason or another”, though the reasons tend to be ones where we never actually had a shot of an astronomically big future in the first place, and I think that that’s sort of the appropriate target for scepticism
In general our ability to measure long term effects is kind of lousy. But if I wanted to look for interventions which don’t have that decay pattern it would be most natural to think of conservation work saving species from extinction. Once we’ve lost biodiversity, it’s essentially gone (maybe taking millions of years to build up again naturally). Conservation work can stop that. And with rises in conservation work over time it’s quite plausible that early saving species won’t just lead to them going extinct slightly later, but being preserved indefinitely.
I was not clear above, but I meant (posterior) counterfactual impact under expectedtotalhedonisticutilitarianism. Even if a species is counterfactually preserved indefinitely due to actions now, which I think would be very hard, I do not see how it would permanently increase wellbeing. In addition, I meant to ask for actual empirical evidence as opposed to hypothetical examples (e.g. of one species being saved and making an immortal conservationist happy indefinitely).
I think this is something where our ability to measure is just pretty bad, and in particular our ability to empirically detect whether the type of things that plausibly have long lasting counterfactual impacts actually do is pretty terrible.
I respond to that by saying “ok I guess empirics aren’t super helpful for the big picture question let’s try to build mechanistic understanding of things grounded wherever possible in empirics, as well as priors about what types of distributions occur when various different generating mechanisms are at play”, whereas it sounds like you’re responding by saying something like “well as a prior we’ll just use the parts of the distribution we can actually measure, and assume that generalizes unless we get contradictory data”?
I respond to that by saying “ok I guess empirics aren’t super helpful for the big picture question let’s try to build mechanistic understanding of things grounded wherever possible in empirics, as well as priors about what types of distributions occur when various different generating mechanisms are at play”, whereas it sounds like you’re responding by saying something like “well as a prior we’ll just use the parts of the distribution we can actually measure, and assume that generalizes unless we get contradictory data”?
Yes, that would be my reply. Thanks for clarifying.
Yeah, so I basically think that that response feels “spiritually frequentist”, and is more likely to lead you to large errors than the approach I outlined (which feels more “spiritually Bayesian”), especially in cases like this where we’re trying to extrapolate significantly beyond the data we’ve been able to gather.
Sorry, I don’t find this is really speaking to my question?
I totally believe that people make mistakes in thinking about this stuff for reasons along the lines of the biases you discuss. But I also think that you’re making some strong assumptions about things essentially cancelling out that I think are unjustified, and talking about mistakes that other people are making doesn’t (it seems to me) work as a justification.
I do not think the difficulty of decreasing a risk is independent of the value at stake. It is harder to decrease a risk when a larger value is at stake. So, in my mind, decreasing the nearterm risk of human extinction is astronomically easier than decreasing the risk of not achieving 10^50 lives of value, such that decreasing the former by e.g. 10^-10 leads to a relative increase in the latter much smaller than 10^-10.
Could you elaborate on why you think I am making a strong assumption in terms of questioning the following?
Do you think I am overestimating how fast the difference between the PDF after and before the intervention decays? As far as I can tell, the (posterior) counterfactual impact of interventions whose effects can be accurately measured, like ones in global health and development, decays to 0 as time goes by. I do not have a strong view on the particular shape of the difference, but exponential decay is quite typical in many contexts.
This makes sense as a kind of general prior to come in with. Although note:
It’s surely observational, not causal—there’s no magic at play which means if you keep a scenario fixed except for changing the value at stake, this should impact the difficulty
One of the plausible generating mechanisms is having a broad altruistic market which takes the best opportunities, leaving no free lunches—but for some of the cases we’re discussing it’s unclear the market could have made it efficient
Now it looks to me as though you’re dogmatically sticking with the prior. Having come across the (kinda striking) observation which says “if there’s a realistic chance of spreading to the stars, then premature human extinction would forgo astronomical value”, it seems like you’re saying “well that would mean that the prior was wrong, so that observation can’t be quite right”, and then reasoning from your prior to try to draw conclusions about the causal relationships there.
Whereas I feel that the prior reasonably justifies more scepticism in cases where more lives are at stake (and indeed, I do put a bunch of probability on “averting near-term extinction doesn’t save astronomical value for some reason or another”, though the reasons tend to be ones where we never actually had a shot of an astronomically big future in the first place, and I think that that’s sort of the appropriate target for scepticism), but doesn’t give you anything strong enough to be confident about things.
(I certainly wouldn’t be surprised if I’m somehow misunderstanding what you’re doing; I’m just responding to the picture I’m getting from what you’ve written.)
Are there any interventions whose estimates of (posterior) counterfactual impact do not decay to 0 in at most a few centuries? From my perspective, their absence establishes a strong prior against persistent longterm effects.
This makes a lot of sense to me too.
In general our ability to measure long term effects is kind of lousy. But if I wanted to look for interventions which don’t have that decay pattern it would be most natural to think of conservation work saving species from extinction. Once we’ve lost biodiversity, it’s essentially gone (maybe taking millions of years to build up again naturally). Conservation work can stop that. And with rises in conservation work over time it’s quite plausible that early saving species won’t just lead to them going extinct slightly later, but being preserved indefinitely.
I was not clear above, but I meant (posterior) counterfactual impact under expected total hedonistic utilitarianism. Even if a species is counterfactually preserved indefinitely due to actions now, which I think would be very hard, I do not see how it would permanently increase wellbeing. In addition, I meant to ask for actual empirical evidence as opposed to hypothetical examples (e.g. of one species being saved and making an immortal conservationist happy indefinitely).
I think this is something where our ability to measure is just pretty bad, and in particular our ability to empirically detect whether the type of things that plausibly have long lasting counterfactual impacts actually do is pretty terrible.
I respond to that by saying “ok I guess empirics aren’t super helpful for the big picture question let’s try to build mechanistic understanding of things grounded wherever possible in empirics, as well as priors about what types of distributions occur when various different generating mechanisms are at play”, whereas it sounds like you’re responding by saying something like “well as a prior we’ll just use the parts of the distribution we can actually measure, and assume that generalizes unless we get contradictory data”?
Yes, that would be my reply. Thanks for clarifying.
Yeah, so I basically think that that response feels “spiritually frequentist”, and is more likely to lead you to large errors than the approach I outlined (which feels more “spiritually Bayesian”), especially in cases like this where we’re trying to extrapolate significantly beyond the data we’ve been able to gather.