That’s an interesting way to connect these. I suppose one way to view your model is as making clear the point that you can’t cost-effectively use models on tasks that much longer than their 50% horizons — even if you are willing to try multiple times — and that trend of dramatic price improvements over time isn’t enough to help with this. Instead you need the continuation of the METR trend of exponentially growing horizons. Moreover, you give a nice intuitive explanation of why that is.
One thing to watch out for is Gus Hamilton’s recent study suggesting that there isn’t a constant hazard rate. I share my thoughts on it here, but my basic conclusion is that he is probably right. In particular, he has a functional form estimating how their success probability declines. You could add this to your model (it is basically 1 minus the CDF of a Weibull distribution with K=0.6). I think this survival function tail is a power law rather than an exponential, making the ‘just run it heaps of times’ thing slightly more tenable. It may mean that it is the cost of human verification that gets you, rather than it being untenable even on AI costs alone.
Thank you for this thoughtful reply, this comment is basically the reason this update exists. You were right that Hamilton is probably right.
I have written a longer update incorporating Hamilton’s reanalysis and extending the economics in two directions: a quantitative treatment of verification as the binding constraint, and a systematic look at the economic conditions under which a genuinely dangerous autonomous agent actually gets to run.
Curious whether you think the analysis holds up, and whether there are important considerations I have missed!
That’s an interesting way to connect these. I suppose one way to view your model is as making clear the point that you can’t cost-effectively use models on tasks that much longer than their 50% horizons — even if you are willing to try multiple times — and that trend of dramatic price improvements over time isn’t enough to help with this. Instead you need the continuation of the METR trend of exponentially growing horizons. Moreover, you give a nice intuitive explanation of why that is.
One thing to watch out for is Gus Hamilton’s recent study suggesting that there isn’t a constant hazard rate. I share my thoughts on it here, but my basic conclusion is that he is probably right. In particular, he has a functional form estimating how their success probability declines. You could add this to your model (it is basically 1 minus the CDF of a Weibull distribution with K=0.6). I think this survival function tail is a power law rather than an exponential, making the ‘just run it heaps of times’ thing slightly more tenable. It may mean that it is the cost of human verification that gets you, rather than it being untenable even on AI costs alone.
Thank you for this thoughtful reply, this comment is basically the reason this update exists. You were right that Hamilton is probably right.
I have written a longer update incorporating Hamilton’s reanalysis and extending the economics in two directions: a quantitative treatment of verification as the binding constraint, and a systematic look at the economic conditions under which a genuinely dangerous autonomous agent actually gets to run.
Curious whether you think the analysis holds up, and whether there are important considerations I have missed!