On your second point on overestimating stagnation I also had a few issues:
Understating effect of AI (not AGI):
This section does not acknowledge that AI (narrow, not AGI or superintelligence) is likely going to be a significant productivity booster on innovation.
AlphaFold is not AGI, won’t cause any catastrophes, but will likely contribute to the productivity of researchers in a variety of fields.
AI enables better imaging and control, possibly allowing some breakthroughs in plasma control thus harnessing fusion, as well as breakthroughs in areas where traditional control is just not good enough.
A language model focussed on formal mathematics might speed up all sorts of mathematical research.
It’s kind of pointless to wait for an “aligned AGI” to speed up scientific progress. We already have very very powerful specialised (thus aligned) tools for that.
economic growth vs development of technological capabilities
To a large extent, economic growth is driven by consumption, not necessarily technological or scientific progress that is useful for humanity. The type of innovation that drives the economy is about new flavours of candies, how to sell more candies, how to advertise more candies, How to build apps where candies can be advertised more.
Thus, when people say we might have to slow down economic growth as part of solving some problems, they typically mostly mean halting the pointless growth in some consumer product sales, not halting scientific progress leading to intellectual stagnation.
On your second point on overestimating stagnation I also had a few issues:
Understating effect of AI (not AGI):
This section does not acknowledge that AI (narrow, not AGI or superintelligence) is likely going to be a significant productivity booster on innovation.
AlphaFold is not AGI, won’t cause any catastrophes, but will likely contribute to the productivity of researchers in a variety of fields.
AI enables better imaging and control, possibly allowing some breakthroughs in plasma control thus harnessing fusion, as well as breakthroughs in areas where traditional control is just not good enough.
A language model focussed on formal mathematics might speed up all sorts of mathematical research.
It’s kind of pointless to wait for an “aligned AGI” to speed up scientific progress. We already have very very powerful specialised (thus aligned) tools for that.
economic growth vs development of technological capabilities
To a large extent, economic growth is driven by consumption, not necessarily technological or scientific progress that is useful for humanity. The type of innovation that drives the economy is about new flavours of candies, how to sell more candies, how to advertise more candies, How to build apps where candies can be advertised more.
Thus, when people say we might have to slow down economic growth as part of solving some problems, they typically mostly mean halting the pointless growth in some consumer product sales, not halting scientific progress leading to intellectual stagnation.