Elaborating on the last paragraph: when considering the value of the set-back society, we’re conditioning on the fact that it got set back. On one hand, (as you say) this could be evidence that society was (up to the point of catastrophe) more liberal and decentralised than it could have been, since many global catastrophes are less likely to occur under the control of a world government. Since I think the future looks brighter if society is more liberal on the dawn of AGI, then I think that’s evidence the current “run” is worth preserving over the next roll we’d get; even if we’re absolutely confident civilisation would survive another run after being set back (assuming a catastrophe would re-roll the dice on how well things are going). That’s not saying anything about whether the world is currently looking surprisingly liberal — just that interventions to prevent pre-AGI catastrophes plausibly move probability mass from liberal/decentralised civilisations to illberal/centralised ones. And maybe that’s the main effect of preventing pre-AGI catastrophes.
Elaborating on the last paragraph: when considering the value of the set-back society, we’re conditioning on the fact that it got set back. On one hand, (as you say) this could be evidence that society was (up to the point of catastrophe) more liberal and decentralised than it could have been, since many global catastrophes are less likely to occur under the control of a world government. Since I think the future looks brighter if society is more liberal on the dawn of AGI, then I think that’s evidence the current “run” is worth preserving over the next roll we’d get; even if we’re absolutely confident civilisation would survive another run after being set back (assuming a catastrophe would re-roll the dice on how well things are going). That’s not saying anything about whether the world is currently looking surprisingly liberal — just that interventions to prevent pre-AGI catastrophes plausibly move probability mass from liberal/decentralised civilisations to illberal/centralised ones. And maybe that’s the main effect of preventing pre-AGI catastrophes.