Interesting post. I think I have a couple of thoughts, please forgive the uneditted nature.
One issue is whether more than one person can get credit for the same event. If this is the case, then both the climber girl and the parents can get credit for her surviving the climb (after all, both their actions were sufficient). Similarly, both we and the future people can get credit for saving the world.
If not, then only one person can get the credit for every instance of world saving. Either we can harvest them now, or we can leave them for other people to get. But the latter strategy involves the risk that they will remain unharvested, leading to a reduction in the total quantity of creditworthiness mankind accrues. So from the point of view of an impartial maximiser of humanity’s creditworthiness, we should seize as many as we can, leaving as little as possible for the future.
Secondly, as a new parent I see the appeal of the invisible robots of deliverance! I am keen to let the sproglet explore and stake out her own achievements, but I don’t think she loses much when I keep her from dying. She can get plenty of moral achievement from ascending to new heights, even if I have sealed off the depths.
Finally, there is of course the numerical consideration that even if facing a 1% risk of extinction carried some inherent moral glory, it would also reduce the value of all subsequent things by 1% (in expectation). Unless you think the benefit from our children, rather than us, overcoming that risk is large compared to the total value of the future of humanity, it seems like we should probably deny them it.
Thanks, this all makes sense to me. Just one quick comment:
So from the point of view of an impartial maximiser of humanity’s creditworthiness, we should seize as many as we can, leaving as little as possible for the future.
If I understand you correctly, your argument for this conclusion assumed that the total number of world-saving instances is fixed independently of anyone’s actions. But I think in practice this is wrong, i.e. the number of world-saving opportunities is endogenous to people’s actions including in particular whether they reap current world-saving opportunities.
Oversimplified example: perhaps currently there is one world-saving instance per year from Petrov-style incidents, i.e. countries not launching a nuclear strike in response to a false alarm of a nuclear attack. But if there was a breakthrough in nuclear disarmament that reduced nuclear stockpiles to zero this would also eliminate these future world-saving opportunities.
[Oversimplified b/c in fact a nuclear exchange isn’t clearly an x-risk.]
Hey, yes—I would count that nuclear disarmament breakthrough as being equal to the sum of those annual world-saving instances. So you’re right that the number of events isn’t fixed, but their measure (as in the % of the future of humanity saved) is bounded.
Interesting post. I think I have a couple of thoughts, please forgive the uneditted nature.
One issue is whether more than one person can get credit for the same event. If this is the case, then both the climber girl and the parents can get credit for her surviving the climb (after all, both their actions were sufficient). Similarly, both we and the future people can get credit for saving the world.
If not, then only one person can get the credit for every instance of world saving. Either we can harvest them now, or we can leave them for other people to get. But the latter strategy involves the risk that they will remain unharvested, leading to a reduction in the total quantity of creditworthiness mankind accrues. So from the point of view of an impartial maximiser of humanity’s creditworthiness, we should seize as many as we can, leaving as little as possible for the future.
Secondly, as a new parent I see the appeal of the invisible robots of deliverance! I am keen to let the sproglet explore and stake out her own achievements, but I don’t think she loses much when I keep her from dying. She can get plenty of moral achievement from ascending to new heights, even if I have sealed off the depths.
Finally, there is of course the numerical consideration that even if facing a 1% risk of extinction carried some inherent moral glory, it would also reduce the value of all subsequent things by 1% (in expectation). Unless you think the benefit from our children, rather than us, overcoming that risk is large compared to the total value of the future of humanity, it seems like we should probably deny them it.
Thanks, this all makes sense to me. Just one quick comment:
If I understand you correctly, your argument for this conclusion assumed that the total number of world-saving instances is fixed independently of anyone’s actions. But I think in practice this is wrong, i.e. the number of world-saving opportunities is endogenous to people’s actions including in particular whether they reap current world-saving opportunities.
Oversimplified example: perhaps currently there is one world-saving instance per year from Petrov-style incidents, i.e. countries not launching a nuclear strike in response to a false alarm of a nuclear attack. But if there was a breakthrough in nuclear disarmament that reduced nuclear stockpiles to zero this would also eliminate these future world-saving opportunities.
[Oversimplified b/c in fact a nuclear exchange isn’t clearly an x-risk.]
Hey, yes—I would count that nuclear disarmament breakthrough as being equal to the sum of those annual world-saving instances. So you’re right that the number of events isn’t fixed, but their measure (as in the % of the future of humanity saved) is bounded.