Problems vary on three axes: u’(R), R, u_total. You’re expressing this in the basis u’(R), R, u_total. The ITN framework uses the basis I, T, N = u_total, u’(R) * R * 1/ u_total, 1/R. The basis is arbitrary: we could just as easily use some crazy basis like X, Y, Z = u_total^2, R^5, sqrt(u’(R)). But we want to use a basis that’s useful in practice, which means variables with intuitive meanings, hence ITN.
But why is tractability roughly constant with neglectedness in practice? Equivalently, why are there logarithmic returns to many problems? I don’t think it’s related to your (1) or (2) because those are about comparing different problems, whereas the mystery is the relationship between u’(R) * R * 1/ u_total and 1/R for a given problem. One model that suggests log returns is if we have to surpass some unknown resource threshold r* (the “difficulty”) to solve the problem, and r* ranges over many orders of magnitude with an approximately log-uniform distribution [1]. Owen C-B has empirical evidence and some theoretical justification for why this might happen in practice. When it does, my post then derives that tractability doesn’t vary dramatically between (most) problems.
Note that sometimes we know r* or have a different prior, and then our problem stops being logarithmic, like in the second section of the post. This is exactly when tractability can vary dramatically between problems. In your AI alignment subproblem example, we know that alignment takes 20 years, which means a strong update away from the logarithmic prior.
[1]: log-uniform distributions over the reals don’t exist, so I mean something like “doesn’t vary by more than ~1.5x every doubling in the fat part of the distribution”.
The entire time I’ve been thinking about this, I’ve been thinking of utility curves as logarithmic, so you don’t have to sell me on that. I think my original comment here is another way of understanding why tractability perhaps doesn’t vary much between problems, not within a problem.
Problems vary on three axes: u’(R), R, u_total. You’re expressing this in the basis u’(R), R, u_total. The ITN framework uses the basis I, T, N = u_total, u’(R) * R * 1/ u_total, 1/R. The basis is arbitrary: we could just as easily use some crazy basis like X, Y, Z = u_total^2, R^5, sqrt(u’(R)). But we want to use a basis that’s useful in practice, which means variables with intuitive meanings, hence ITN.
But why is tractability roughly constant with neglectedness in practice? Equivalently, why are there logarithmic returns to many problems? I don’t think it’s related to your (1) or (2) because those are about comparing different problems, whereas the mystery is the relationship between u’(R) * R * 1/ u_total and 1/R for a given problem. One model that suggests log returns is if we have to surpass some unknown resource threshold r* (the “difficulty”) to solve the problem, and r* ranges over many orders of magnitude with an approximately log-uniform distribution [1]. Owen C-B has empirical evidence and some theoretical justification for why this might happen in practice. When it does, my post then derives that tractability doesn’t vary dramatically between (most) problems.
Note that sometimes we know r* or have a different prior, and then our problem stops being logarithmic, like in the second section of the post. This is exactly when tractability can vary dramatically between problems. In your AI alignment subproblem example, we know that alignment takes 20 years, which means a strong update away from the logarithmic prior.
[1]: log-uniform distributions over the reals don’t exist, so I mean something like “doesn’t vary by more than ~1.5x every doubling in the fat part of the distribution”.
The entire time I’ve been thinking about this, I’ve been thinking of utility curves as logarithmic, so you don’t have to sell me on that. I think my original comment here is another way of understanding why tractability perhaps doesn’t vary much between problems, not within a problem.
I don’t see why logarithmic utility iff tractability doesn’t change with neglectedness.