Cap the model size and sophistication somewhere near where it is now? Seems like there’s easily a decade worth of alignment research that could be done on current models (and other theoretical work), which should be done before capabilities are advanced further. A moratorium would help bridge that gap. Demis Hassabis has talked about hitting the pause button as we get closer to the “grey zone”. Now is the time!
A variant on your proposal could be a moratorium on training new large models (e.g. OpenAI would be forbidden from training GPT-5, for example).
That would be more enforceable, because you need lots of compute to train a new model. I don’t know how we would stop an academic thinking up new ideas on how to structure AI models better, and even if we could, it would be hard to disentangle this from alignment research.
It would probably achieve most of what you want. For someone who’s worried about short timelines, reducing the scope for the scaling hypothesis to apply is probably pretty powerful, at least in the short term
Interesting, yes such moratorium on training new LLMs could help. But we also need to make the research morally unacceptable too—I think stigmatisation of AGI capabilities research could go a long way. No one is working on human genetic enhancement or cloning, mainly because of the taboos around them. It’s not like there is a lot of underground research there. (I’m thinking this is needed, because any limits on compute that are imposed could easily be got around).
A limit on compute designed to constrain OpenAI, Anthropic, or Google from training a new model sounds like a very high bar. I don’t understand why that could easily be got around?
Spoofing accounts to combine multiple of them together (as in the Clippy story linked, but I’m imagining humans doing it). The kind of bending of the rules that happens when something is merely regulated but not taboo. It’s not just Microsoft and Google we need to worry about. If the techniques and code are out there (open source models are not far behind cutting edge research), many actors will be trying to run them at scale.
These will still be massive, and massively expensive, training runs though—big operations that will constitute very big strategic decisions only available to the best-resourced actors.
In the post-AutoGPT world, this seems like it will no longer be the case. There is enough fervour by AGI accelerationists that the required resources could be quickly amassed by crowdfunding (cf. crypto projects raising similar amounts to those needed).
I don’t see how we could implement a moratorium on AGI research that does stop capabilities research but doesn’t stop alignment research?
Cap the model size and sophistication somewhere near where it is now? Seems like there’s easily a decade worth of alignment research that could be done on current models (and other theoretical work), which should be done before capabilities are advanced further. A moratorium would help bridge that gap. Demis Hassabis has talked about hitting the pause button as we get closer to the “grey zone”. Now is the time!
A variant on your proposal could be a moratorium on training new large models (e.g. OpenAI would be forbidden from training GPT-5, for example).
That would be more enforceable, because you need lots of compute to train a new model. I don’t know how we would stop an academic thinking up new ideas on how to structure AI models better, and even if we could, it would be hard to disentangle this from alignment research.
It would probably achieve most of what you want. For someone who’s worried about short timelines, reducing the scope for the scaling hypothesis to apply is probably pretty powerful, at least in the short term
Interesting, yes such moratorium on training new LLMs could help. But we also need to make the research morally unacceptable too—I think stigmatisation of AGI capabilities research could go a long way. No one is working on human genetic enhancement or cloning, mainly because of the taboos around them. It’s not like there is a lot of underground research there. (I’m thinking this is needed, because any limits on compute that are imposed could easily be got around).
A limit on compute designed to constrain OpenAI, Anthropic, or Google from training a new model sounds like a very high bar. I don’t understand why that could easily be got around?
Spoofing accounts to combine multiple of them together (as in the Clippy story linked, but I’m imagining humans doing it). The kind of bending of the rules that happens when something is merely regulated but not taboo. It’s not just Microsoft and Google we need to worry about. If the techniques and code are out there (open source models are not far behind cutting edge research), many actors will be trying to run them at scale.
These will still be massive, and massively expensive, training runs though—big operations that will constitute very big strategic decisions only available to the best-resourced actors.
In the post-AutoGPT world, this seems like it will no longer be the case. There is enough fervour by AGI accelerationists that the required resources could be quickly amassed by crowdfunding (cf. crypto projects raising similar amounts to those needed).
Yes, but they will become increasingly cheaper. A taboo is far stronger than regulation.