A variant on your proposal could be a moratorium on training new large models (e.g. OpenAI would be forbidden from training GPT-5, for example).
That would be more enforceable, because you need lots of compute to train a new model. I don’t know how we would stop an academic thinking up new ideas on how to structure AI models better, and even if we could, it would be hard to disentangle this from alignment research.
It would probably achieve most of what you want. For someone who’s worried about short timelines, reducing the scope for the scaling hypothesis to apply is probably pretty powerful, at least in the short term
Interesting, yes such moratorium on training new LLMs could help. But we also need to make the research morally unacceptable too—I think stigmatisation of AGI capabilities research could go a long way. No one is working on human genetic enhancement or cloning, mainly because of the taboos around them. It’s not like there is a lot of underground research there. (I’m thinking this is needed, because any limits on compute that are imposed could easily be got around).
A limit on compute designed to constrain OpenAI, Anthropic, or Google from training a new model sounds like a very high bar. I don’t understand why that could easily be got around?
Spoofing accounts to combine multiple of them together (as in the Clippy story linked, but I’m imagining humans doing it). The kind of bending of the rules that happens when something is merely regulated but not taboo. It’s not just Microsoft and Google we need to worry about. If the techniques and code are out there (open source models are not far behind cutting edge research), many actors will be trying to run them at scale.
These will still be massive, and massively expensive, training runs though—big operations that will constitute very big strategic decisions only available to the best-resourced actors.
In the post-AutoGPT world, this seems like it will no longer be the case. There is enough fervour by AGI accelerationists that the required resources could be quickly amassed by crowdfunding (cf. crypto projects raising similar amounts to those needed).
A variant on your proposal could be a moratorium on training new large models (e.g. OpenAI would be forbidden from training GPT-5, for example).
That would be more enforceable, because you need lots of compute to train a new model. I don’t know how we would stop an academic thinking up new ideas on how to structure AI models better, and even if we could, it would be hard to disentangle this from alignment research.
It would probably achieve most of what you want. For someone who’s worried about short timelines, reducing the scope for the scaling hypothesis to apply is probably pretty powerful, at least in the short term
Interesting, yes such moratorium on training new LLMs could help. But we also need to make the research morally unacceptable too—I think stigmatisation of AGI capabilities research could go a long way. No one is working on human genetic enhancement or cloning, mainly because of the taboos around them. It’s not like there is a lot of underground research there. (I’m thinking this is needed, because any limits on compute that are imposed could easily be got around).
A limit on compute designed to constrain OpenAI, Anthropic, or Google from training a new model sounds like a very high bar. I don’t understand why that could easily be got around?
Spoofing accounts to combine multiple of them together (as in the Clippy story linked, but I’m imagining humans doing it). The kind of bending of the rules that happens when something is merely regulated but not taboo. It’s not just Microsoft and Google we need to worry about. If the techniques and code are out there (open source models are not far behind cutting edge research), many actors will be trying to run them at scale.
These will still be massive, and massively expensive, training runs though—big operations that will constitute very big strategic decisions only available to the best-resourced actors.
In the post-AutoGPT world, this seems like it will no longer be the case. There is enough fervour by AGI accelerationists that the required resources could be quickly amassed by crowdfunding (cf. crypto projects raising similar amounts to those needed).
Yes, but they will become increasingly cheaper. A taboo is far stronger than regulation.