Leopold Aschenbrenner returns to X-risk and growth

[nickwhitaker]

“I argue that the opposite is the case. It is not safe stagnation and risky growth that we must choose between; rather, it is stagnation that is risky and it is growth that leads to safety.”

More here

[Michael_Wiebe]

What if the dot (representing where we are) was further to the left? In that case, it’s not so clear that we want to speed through and incur the increased risk.

[Greg_Colbourn]

The idea is that by speeding through you increase risk initially, but the total risk is lower—i.e. smaller area under the grey curve here:

I think this probably breaks down if the peak is high enough though (here I’m thinking of AGI x-risk). Aschenbrenner gives the example of:

On the other extreme, humanity is extremely fragile. No matter how high a fraction of our resources we dedicate to safety, we cannot prevent an unrecoverable catastrophe. ..there is nothing we can do regardless. An existential catastrophe is inevitable, and it is impossible for us to survive to reach a grand future.

And argues that

even if there is some probability we do live in this world, to maximize the moral value of the future, we should act as if we live in the other scenarios where a long and flourishing future is possible.

I’m not sure if this applies if there is some possibility of “pulling the curve sideways” to flatten it—i.e. increase the fraction of resources spent on safety whilst keeping consumption (or growth) constant. This seems to be what those concerned with x-risk are doing for the most part (rather than trying to slow down growth).

[Michael_Wiebe]

That graph represents the comparison of increased growth (gray line) to some baseline (black).

Instead, I’m talking about the case where the dot is at the far left side of the graph: we’re currently at a low level of risk, and continuing at the baseline growth rate means ‘climbing the risk mountain’ before we get to the other side with lower risk. If the peak is very high (relative to the dot), then it’s not clear that continued (or accelerated) growth is optimal; stagnation might be better.