Thanks for this Halstead—thoughtful article.
I have a one push-back, and one question about your preferred process for applying the ITN framework.
1. After explaining the 80K formalisation of ITN you say
Thus, once we have information on importance, tractability and neglectedness (thus defined), then we can produce an estimate of marginal cost-effectiveness.
The problem with this is: if we can do this, then why would we calculate these three terms separately in the first place?
I think the answer is that in some contexts it’s easier to calculate each term separately and then combine them in a later step, than to calculate the cost-effectiveness directly. It’s also easier to sanity check that each term looks sensible separately, as our intuitions are often more reliable for the separate terms than for the marginal cost effectiveness.
Take technical AI safety research as an example. I’d have trouble directly estimating “How much good would we do by spending $1000 in this area”, or sanity checking the result. I’d also have trouble with “What % of this problem would we solve by spending another $100?” (your preferred definition of tractability). I’d feel at least somewhat more confident making and eye-balling estimates for
“How good would it be to solve technical AI safety?”
“How much of the problem would we solve by doubling the amount of money/researchers in this area (or increasing it by 10%)?”
“How much is being spent in the area?”
I do think the tractability estimate is the hardest to construct and assess in this case, but I think it’s better than the alternatives. And if we assume diminishing marginal returns we can make the tractability estimate easier by replacing it with “How many resources would be needed to completely solve this problem?”
So I think the 80K formalisation is useful in at least some contexts, e.g. AI safety.
2. In the alternative ITN framework of the Founders Pledge, neglectedness is just one input to tractability. But then you score each cause on i) the ratio importance/neglectedness, and ii) all the factors bearing on tractability except neglectedness. To me, it feels like (ii) would be quite hard to score, as you have to pretend you don’t know things that you do know (neglectedness).
Wouldn’t it be easier to simply score each cause on importance and tractability, using neglectedness as one input to the tractability score? This has the added benefit of not assuming diminishing marginal returns, as you can weight neglectedness less strongly when you don’t think there are DMR.
Agreed—the framework can be applied to things other than AGI.