Thanks, this all makes sense to me. Just one quick comment:
So from the point of view of an impartial maximiser of humanity’s creditworthiness, we should seize as many as we can, leaving as little as possible for the future.
If I understand you correctly, your argument for this conclusion assumed that the total number of world-saving instances is fixed independently of anyone’s actions. But I think in practice this is wrong, i.e. the number of world-saving opportunities is endogenous to people’s actions including in particular whether they reap current world-saving opportunities.
Oversimplified example: perhaps currently there is one world-saving instance per year from Petrov-style incidents, i.e. countries not launching a nuclear strike in response to a false alarm of a nuclear attack. But if there was a breakthrough in nuclear disarmament that reduced nuclear stockpiles to zero this would also eliminate these future world-saving opportunities.
[Oversimplified b/c in fact a nuclear exchange isn’t clearly an x-risk.]
Hey, yes—I would count that nuclear disarmament breakthrough as being equal to the sum of those annual world-saving instances. So you’re right that the number of events isn’t fixed, but their measure (as in the % of the future of humanity saved) is bounded.
Thanks, this all makes sense to me. Just one quick comment:
If I understand you correctly, your argument for this conclusion assumed that the total number of world-saving instances is fixed independently of anyone’s actions. But I think in practice this is wrong, i.e. the number of world-saving opportunities is endogenous to people’s actions including in particular whether they reap current world-saving opportunities.
Oversimplified example: perhaps currently there is one world-saving instance per year from Petrov-style incidents, i.e. countries not launching a nuclear strike in response to a false alarm of a nuclear attack. But if there was a breakthrough in nuclear disarmament that reduced nuclear stockpiles to zero this would also eliminate these future world-saving opportunities.
[Oversimplified b/c in fact a nuclear exchange isn’t clearly an x-risk.]
Hey, yes—I would count that nuclear disarmament breakthrough as being equal to the sum of those annual world-saving instances. So you’re right that the number of events isn’t fixed, but their measure (as in the % of the future of humanity saved) is bounded.