Gwern on creating your own AI race and China’s Fast Follower strategy.

[Larks]

Link post

Gwern recently wrote a very interesting thread about Chinese AI strategy and the downsides of US AI racing. It’s both quite short and hard to excerpt so here is almost the entire thing with very minimal editing:

Hsu is a long-time China hawk and has been talking up the scientific & technological capabilities of the CCP for a long time, saying they were going to surpass the West any moment now, so I found this interesting when Hsu explains that:

1. the scientific culture of China is ‘mafia’ like (Hsu’s term, not mine) and focused on legible easily-cited incremental research, and is against making any daring research leaps or controversial breakthroughs...

   but is capable of extremely high quality world-class followup and large scientific investments given a clear objective target and government marching orders

2. there is no interest or investment in an AI arms race, in part because of a “quiet confidence” (ie. apathy/​laying-flat) that if anything important happens, China can just catch up a few years later and win the real race. They just aren’t doing it. There is no Chinese Manhattan Project. There is no race. They aren’t dumping the money into it, and other things, like chips and Taiwan and demographics, are the big concerns which have the focus from the top of the government, and no one is interested in sticking their necks out for wacky things like ‘spending a billion dollars on a single training run’ without explicit enthusiastic endorsement from the very top.

   Let the crazy Americans with their fantasies of AGI in a few years race ahead and knock themselves out, and China will stroll along, and scoop up the results, and scale it all out cost-effectively and outcompete any Western AGI-related stuff (ie. be the BYD to the Tesla). The Westerners may make the history books, but the Chinese will make the huge bucks.

So, this raises an important question for the arms race people: if you believe it’s OK to race, because even if your race winds up creating the very race you claimed you were trying to avoid, you are still going to beat China to AGI (which is highly plausible, inasmuch as it is easy to win a race when only one side is racing), and you have AGI a year (or two at the most) before China and you supposedly “win”… Then what?

1. race to AGI and win

2. trigger a bunch of other countries racing to their own AGI (now that they know it’s doable, increasingly much about how to do it, can borrow/​steal/​imitate the first AGI, and have to do so “before it’s too late”)

3. ???

4. profit!

What does winning look like? What do you do next? How do you “bury the body”? You get AGI and you show it off publicly, Xi blows his stack as he realizes how badly he screwed up strategically and declares a national emergency and the CCP starts racing towards its own AGI in a year, and… then what? What do you do in this 1 year period, while you still enjoy AGI supremacy? You have millions of AGIs which can do… stuff. What is this stuff? Are you going to start massive weaponized hacking to subvert CCP AI programs as much as possible short of nuclear war? Lobby the UN to ban rival AGIs and approve US carrier group air strikes on the Chinese mainland? License it to the CCP to buy them off? Just… do nothing and enjoy 10%+ GDP growth for one year before the rival CCP AGIs all start getting deployed? Do you have any idea at all? If you don’t, what is the point of ‘winning the race’?

[Benevolent_Rain]

Strong upvote. I feel like the China-US AI race debate is laden with ideology and confused lack of specificity. It is like how “capitalism” is used. People throw the term around and it can mean hundred different things. Every mention of China in relation to AI should be specific. Are we talking about AI-enabled cyberattacks against civilian infrastructure in the West? Are we talking about some weird pathway where they will create a communist super-bot that will both rule China and the rest of the world? Are we talking about only the GDP impact? Or how increased GDP will allow them to build a larger military? Or some subset of these multitude of ways in which “China winning on AI” is important?

Currently, without the lack of specificity I feel that the China AI debate is less about any specific threat and more about creating some appealing, overly simplified narrative that can lubricate the bureaucratic machine, get buy-in from a range of private sector stakeholders and fabricate some sense of urgency for people to push through various agendas. My fear is that this sounds eerily similar to “threat of the communists” during the Cold war which can be argued led to the disastrous outcome of thousands of nuclear warheads still being pointed at some of the most dense clusters of civilians around the world.

I have read far from everything about AI, so if someone has pointers to material on why China as a one-word concept is useful to point to I would be grateful. I see this issue has been raised many times on the Forum and have not read everything but decided to comment any way as I think signal boosting here is important, especially as the China one-worder is being quite casually thrown around by people with lots of influence.

[Benevolent_Rain]

Sorry for posting it here too, but a FLI podcast just dropped that seems relevant, it mentions 24 minutes in some push by several actors to use China to motivate action.

[1a3orn]

I think that Gwern is acting as somewhat lossy reflection of what Hsu actually said. (https://​​www.manifold1.com/​​episodes/​​letter-from-shanghai-reflections-on-china-in-2024-73/​​transcript):

While I was in Beijing, I also met with some top venture capitalists and technologists. I again can’t say too much about it. I just want to say that there’s quiet confidence throughout all, among all the people in China, whether it was academic scientists, technologists, investors, venture capitalists, business people, just quiet confidence that nothing the outside world, specifically the U. S., can do is really going to stop the rise of China.

And in particular, a lot of conversation was about AI and the chip war. And there’s a sense of quiet confidence here that China’s going to get the AI training done that it needs to do. It’s not going to fall way behind in the race for AGI or ASI. There are government national level plans in place to build the data centers, to produce domestically the chips necessary to run those data centers, to power those data centers, and to stay abreast of developments in AI and also in frontier chip manufacturing.

Let’s just say that there’s quiet confidence here. That, you know, they may not fully catch up. They may not get their EUV machine for some number of years, but they’re not really worried. And so, and many people have said to me that the very stupid Biden Jake Sullivan chip war against China has only helped Chinese companies. This is something I’ve discussed in other podcasts, when the U. S. cuts off access for Chinese companies to key products and technologies used in the semiconductor supply chain from the U. S. and say Dutch companies like ASML, Japanese companies as well. When the U. S. starts to threaten that, it only causes a coalescence of effort here in China. It creates a necessary coordination of effort here that then lets the Chinese supply chain ecosystem for semiconductors advance very rapidly.

And so it was, it was a stupid policy by the Biden administration. And it was also based on a miscalibrated estimate of how fast we were going to get to AGI. They thought, Oh, if we just, if we just kneecap the Chinese right now, since we’re AGI is right around the corner, this will let America get to super AGI and the Chinese will be behind and then they’ll be screwed. And it doesn’t look like it’s playing out that way. Let’s just put it that way.

I can’t say much more about the details of what I learned on this trip.

But I think quiet confidence and a sense of inevitability in that sector, but across all sectors here.

[Garrison]

Hsu clarified his position on my thread here:

“Clarifications:

1. The mafia tendencies (careerist groups working together out of self-interest and not to advance science itself) are present in the West as well these days. In fact the term was first used in this way by Italian academics.

2. They’re not against big breakthroughs in PRC, esp. obvious ones. The bureaucracy bases promotions, raises, etc. on metrics like publications in top journals, cititations, … However there are very obvious wins that they will go after in a coordinated way—including AI, semiconductors, new energy tech, etc.

3. I could be described as a China hawk in that I’ve been pointing to a US-China competition as unavoidable for over a decade. But I think I have more realistic views about what is happening in PRC than most China hawks. I also try to focus on simple descriptive analysis rather than getting distracted by normative midwit stuff.

4. There is coordinated planning btw govt and industry in PRC to stay at the frontier in AI/​AGI/​ASI. They are less susceptible to “visionaries” (ie grifters) so you’ll find fewer doomers or singularitarians, etc. Certainly not in the top govt positions. The quiet confidence I mentioned extends to AI, not just semiconductors and other key technologies.”