Thanks for the quibble, seems big if true! And agreed it is not something that I was tracking when writing the article.
A few thoughts:
I am fairly unsure if the economies of scale point is actually right. Some reasons for doubt:
Partly I’m thinking of Drexler’s CAIS arguments and intuitions that ecosystems of different specialised systems will outcompete monoculture
Partly I’m looking at AI development today
Partly the form of the economies of scale argument seems to be ‘one constraint on human economies of scale is coordination costs between humans. So if those are removed, economies of scale will go to infinity!’ But there may well be other trade offs that you reach at higher levels. For example, I’d expect that you lose out on things like creativity/innovation, and that you run higher risks of correlated failures, vulnerabilities etc.
Assuming it is true, it doesn’t seem like the most important argument within economic dominance to me:
The most natural way of thinking about it for me is that AGI increasing economies of scale is a subset of outgrowing the world (where the general class is ‘having better AI enables growing to most of the economy’, and the economies of scale sub-class is ‘doing that via using copies of literally the same AI, such that you get more economies of scale’
Put another way, I think the economies of scale thing only leads to extreme power concentration in combination with a big capabilities gap. If lots of people have similarly powerful AI systems, and can use them to figure out that they’d be best off by using a single system to do everything, then I don’t see any reason why one country would dominate. So it doesn’t seem like an independent route to me, it’s a particular form of a route that is causally driven by another factor.
Nice!
One quibble: IMO, the most important argument within ‘economic dominance,’ which doesn’t appear in your list (nor really in the body of your text), is Wei Dai’s ‘AGI will drastically increase economies of scale’.
Thanks for the quibble, seems big if true! And agreed it is not something that I was tracking when writing the article.
A few thoughts:
I am fairly unsure if the economies of scale point is actually right. Some reasons for doubt:
Partly I’m thinking of Drexler’s CAIS arguments and intuitions that ecosystems of different specialised systems will outcompete monoculture
Partly I’m looking at AI development today
Partly the form of the economies of scale argument seems to be ‘one constraint on human economies of scale is coordination costs between humans. So if those are removed, economies of scale will go to infinity!’ But there may well be other trade offs that you reach at higher levels. For example, I’d expect that you lose out on things like creativity/innovation, and that you run higher risks of correlated failures, vulnerabilities etc.
Assuming it is true, it doesn’t seem like the most important argument within economic dominance to me:
The most natural way of thinking about it for me is that AGI increasing economies of scale is a subset of outgrowing the world (where the general class is ‘having better AI enables growing to most of the economy’, and the economies of scale sub-class is ‘doing that via using copies of literally the same AI, such that you get more economies of scale’
Put another way, I think the economies of scale thing only leads to extreme power concentration in combination with a big capabilities gap. If lots of people have similarly powerful AI systems, and can use them to figure out that they’d be best off by using a single system to do everything, then I don’t see any reason why one country would dominate. So it doesn’t seem like an independent route to me, it’s a particular form of a route that is causally driven by another factor.
Interested in your takes here!