Thanks for writing this up! It’s great to formalize intuitions, and this had a bunch of links I’m interested in following up on.
One simplifying assumption that got made was that both interventions cash out in constant amounts of utility for the duration of their relevance. You spoke at the end about the ways in which conclusions would change by changing assumptions; this seems like an important one! If utility increases over time, you have additional juice in that part of the race.
Is this basically addressed by you saying you weren’t assuming the bigness of the universe (since if there’s more people, presumably some good intervention will have more impact) + leaving aside attractor states? I think not quite, since attractor states mostly seem good by limiting the unpredictability rather than by increasing the impact, and I can imagine ways besides there being more people that a good intervention will increase in utility generation over time (snowball effects, allowing other good things to happen on top, etc).
But maybe it just doesn’t add a lot to the central idea? The question is simply one of comparing integrals, and we can construct more complicated integrands and model a bunch of different possibilities / hopefully test them empirically and that will tell us a lot about how to proceed.
You’re right that the constant predicted benefits for each intervention is an important simplifying assumption. However, as you mention, it would be relatively easy to change the integrand to allow for different shapes of signalled benefits. For example, a signal that suggests increasing benefits as we increase the time horizon might increase the relative value of the longtermist intervention.
It quickly becomes an empirical question what the predicted-benefit function looks like, and so it will depend on the exact intervention we are looking at, along with various other empirical predictions. An important one is indeed whether we think the “size”/”scale” of the future will be much larger in value terms (e.g. if the number of individuals increases continuously in the future, the predicted benefits of L could plausibly increase over time).
About attractor states, you say:
attractor states mostly seem good by limiting the unpredictability rather than by increasing the impact
I think that’s basically true, although we need to be careful here about what we mean by “impact”. Even if the “impact” at any one time of being in a good attractor state vs a bad attractor state may be relatively small, the overall “impact” of getting into that attractor state may be large because it persists for so long.
Thanks for writing this up! It’s great to formalize intuitions, and this had a bunch of links I’m interested in following up on.
One simplifying assumption that got made was that both interventions cash out in constant amounts of utility for the duration of their relevance. You spoke at the end about the ways in which conclusions would change by changing assumptions; this seems like an important one! If utility increases over time, you have additional juice in that part of the race.
Is this basically addressed by you saying you weren’t assuming the bigness of the universe (since if there’s more people, presumably some good intervention will have more impact) + leaving aside attractor states? I think not quite, since attractor states mostly seem good by limiting the unpredictability rather than by increasing the impact, and I can imagine ways besides there being more people that a good intervention will increase in utility generation over time (snowball effects, allowing other good things to happen on top, etc).
But maybe it just doesn’t add a lot to the central idea? The question is simply one of comparing integrals, and we can construct more complicated integrands and model a bunch of different possibilities / hopefully test them empirically and that will tell us a lot about how to proceed.
Thanks for this, and would love your thoughts!
You’re right that the constant predicted benefits for each intervention is an important simplifying assumption. However, as you mention, it would be relatively easy to change the integrand to allow for different shapes of signalled benefits. For example, a signal that suggests increasing benefits as we increase the time horizon might increase the relative value of the longtermist intervention.
It quickly becomes an empirical question what the predicted-benefit function looks like, and so it will depend on the exact intervention we are looking at, along with various other empirical predictions. An important one is indeed whether we think the “size”/”scale” of the future will be much larger in value terms (e.g. if the number of individuals increases continuously in the future, the predicted benefits of L could plausibly increase over time).
About attractor states, you say:
I think that’s basically true, although we need to be careful here about what we mean by “impact”. Even if the “impact” at any one time of being in a good attractor state vs a bad attractor state may be relatively small, the overall “impact” of getting into that attractor state may be large because it persists for so long.