My understanding of your main claim: If AGI is not a magic problem-solving oracle and is instead limited by needing to be unhobbled and integrated with complex infrastructure, it will be relatively safe for model weights to be available to foreign adversaries. Or at least key national security decision makers will believe that’s the case.
Please correct me if I’m wrong. My thoughts on the above:
Where is this relative safety coming from? Is it from expecting that adversaries aren’t going to be able to figure out how to do unhobbling or steal the necessary secrets to do unhobbling? Is it from expecting the unhobbling and building infrastrucure around AIs to be a really hard endeavor?
The way I’m viewing this picture, AI that can integrate all across the economy, even if that takes substantial effort, is a major threat to global stability and US dominance.
I guess you can think about the AI-for-productive-purposes supply chain as having two components: Develop the powerful AI model (Initial development), and unhobble it / integrate it in workflows / etc. (Unhobbling/Integration). And you’re arguing that the second of these will be an acceptable place to focus restrictions. My intuition says we will want restrictions on both, but more on the part that is most expensive or excludable (e.g., AI chips being concentrated is a point for initial development). It’s not clear to me what the cost of both supply chain steps is: Currently, it looks like pre-training costs are higher than fine-tuning costs (point for initial development); but actually integrating AIs across the economy seems very expensive to do, the economy is really big (point for unhobbling/integration) (this depends a lot on the systems at the time and how easy they are to work with).
My understanding of your main claim: If AGI is not a magic problem-solving oracle and is instead limited by needing to be unhobbled and integrated with complex infrastructure, it will be relatively safe for model weights to be available to foreign adversaries. Or at least key national security decision makers will believe that’s the case.
Please correct me if I’m wrong. My thoughts on the above:
Where is this relative safety coming from? Is it from expecting that adversaries aren’t going to be able to figure out how to do unhobbling or steal the necessary secrets to do unhobbling? Is it from expecting the unhobbling and building infrastrucure around AIs to be a really hard endeavor?
The way I’m viewing this picture, AI that can integrate all across the economy, even if that takes substantial effort, is a major threat to global stability and US dominance.
I guess you can think about the AI-for-productive-purposes supply chain as having two components: Develop the powerful AI model (Initial development), and unhobble it / integrate it in workflows / etc. (Unhobbling/Integration). And you’re arguing that the second of these will be an acceptable place to focus restrictions. My intuition says we will want restrictions on both, but more on the part that is most expensive or excludable (e.g., AI chips being concentrated is a point for initial development). It’s not clear to me what the cost of both supply chain steps is: Currently, it looks like pre-training costs are higher than fine-tuning costs (point for initial development); but actually integrating AIs across the economy seems very expensive to do, the economy is really big (point for unhobbling/integration) (this depends a lot on the systems at the time and how easy they are to work with).