Some musings on meta science here, kind of tangential and half baked. capabilities is a function of intelligence only once you pick a specific technology and you fix a utility function over the tech, otherwise it’s not well defined.
I haven’t done formal probability in long enough to have the exact words, but essentially you can think of building a technology as a markov chain of decisions. Both the path you take and the chance of success at each path are (partly) a function of your (intelligence). Capabilities is taking a utility function defined over the tech tree and multiplying it by the current probabilities/expected number of steps to different parts of the tree.
imagine you are trying to build a spear. You have 2 choices of sticks, 2 binding agents (glue or chords), 2 choices of stones, and only 1 of each choice is going to work. I guess you can think of this in p or bits but essentially you have 1⁄8. chance of a random walk working, given we fix p(success per task) at 100% for simplicity (https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ also it already has its own discussion).
Ok so the maximum intelligence in any situation is getting to the absorbing node/final tech in the minimum number of steps (with 100% chance). For any specific technology there does exist some (not necc unique) pathway s.t. there does not exist any other pathway with less (difficulty weighted) steps. So capability is always bounded on a specific tech.
The divergence (ratio of difficulty weighted steps) between the random walk and the correct path is basically the amount of juice on that tech intelligence can give you. This is basically unbounded in theory. Also p(success per task) might be a function of the same underlying architecture as p(correct path taken at node n) idk.
Now multiply a utility function over the tech tree. The increase of capabilities is the delta in utility per step between the two intelligences.
It also gets much more complicated in reality. Paths can be self correcting if when the agent goes down a wrong path there is some probability > 0 that they can realize this, which depends both on the intelligence of the agent and the nodes in the tech path itself. Similarly, some paths are smooth and monotonic among all the reasonable choices and others will punish greedy algorithms. but I think the core intuition is what I said earlier.
Some musings on meta science here, kind of tangential and half baked. capabilities is a function of intelligence only once you pick a specific technology and you fix a utility function over the tech, otherwise it’s not well defined.
I haven’t done formal probability in long enough to have the exact words, but essentially you can think of building a technology as a markov chain of decisions. Both the path you take and the chance of success at each path are (partly) a function of your (intelligence). Capabilities is taking a utility function defined over the tech tree and multiplying it by the current probabilities/expected number of steps to different parts of the tree.
imagine you are trying to build a spear. You have 2 choices of sticks, 2 binding agents (glue or chords), 2 choices of stones, and only 1 of each choice is going to work. I guess you can think of this in p or bits but essentially you have 1⁄8. chance of a random walk working, given we fix p(success per task) at 100% for simplicity (https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ also it already has its own discussion).
Ok so the maximum intelligence in any situation is getting to the absorbing node/final tech in the minimum number of steps (with 100% chance). For any specific technology there does exist some (not necc unique) pathway s.t. there does not exist any other pathway with less (difficulty weighted) steps. So capability is always bounded on a specific tech.
The divergence (ratio of difficulty weighted steps) between the random walk and the correct path is basically the amount of juice on that tech intelligence can give you. This is basically unbounded in theory. Also p(success per task) might be a function of the same underlying architecture as p(correct path taken at node n) idk.
Now multiply a utility function over the tech tree. The increase of capabilities is the delta in utility per step between the two intelligences.
It also gets much more complicated in reality. Paths can be self correcting if when the agent goes down a wrong path there is some probability > 0 that they can realize this, which depends both on the intelligence of the agent and the nodes in the tech path itself. Similarly, some paths are smooth and monotonic among all the reasonable choices and others will punish greedy algorithms. but I think the core intuition is what I said earlier.