If leading AI scientists all agree not to build advanced AI, then it does not get built. The question is whether a non-binding commitment will work. There have been similar successes in the past, especially in genetics with voluntary moratoriums on human cloning and human genetic engineering.
It may be worth trying, but I donât think a voluntary moratorium for AI would work.
The Asilomar Conference was preceded by a seven-month voluntary pause on dangerous DNA splicing research, but I think this would be much harder to do for AI.
Some differences:
1. The molecular biology community in 1975 was small and mostly concentrated in the US, but the AI research community today is massive and globally dispersed.
2. Despite the pause being effective in the West, it didnât stop the USSRâs DNA splicing research in its secret biological weapons program.
3. The voluntary DNA splicing pause was only for a few months, and the scientists believed their research would resume after the Asilomar Conference. An effective AI pause would ideally be much longer than that, probably without a defined end date.
4. FLI already called for a six-month pause in 2023 - it was ignored.
5. There are far greater incentives for TAI than recombinant DNA research.
I havenât looked into other historical voluntary moratoriums in depth but I donât think the bottom line would be different.
Thanks for this post, I think itâs great! Just adding my perspective on this part since Iâve researched this topic before.
Actually, if this trend continues, AI could be doing tasks that take humans a month in less than a year. Claude Mythos Preview (early) has a task length of âlikely at least 16 hoursâ on the METR graph. Assuming its task length is 16 hours and extrapolating the 3-month doubling time you mention:
16 hours = 960 minutes. For a working month (~160 hours = 9,600 min), thatâs 10Ă further â logâ(10) â 3.32 doublings â ~10 months.
Iâll also point out that AI is already doing some tasks that take humans a month. E.g. GPT-5.4 Pro solved this open problem from the FrontierMath benchmark, which they estimate would take a human expert 1-3 months to solve.