Importance, Tractibility and Neglectedness should not have equal weight.
TL;Dr, Neglectedness is a useful tiebreaker and gives you information about tractability but the relatively common matrix approach of scoring possible ideas on ITN and then ranking based on the sum of the scores overweights it.
If you’re using the formal mathematical definitions of the terms from this section of the 80,000 Hours article, then their product (before taking logs) has an interpretation in natural units, as good done / extra person or $, so if you reweight, this interpretation for the product will be lost. Are you interpreting the ITN terms differently?
Yes, or at least I think the way they are often interpreted is different. I actually have no issue with 80k’s formal definition, but qualitative use in practice (not by 80k) has often put both both of 80k’s last two points in the tractability metric, then there’s this other nebulous factor called ‘Neglectedness’ which ends up being counted again. The key metric is how much good can be done by one marginal extra person or dollar, and I’ve seen a few cases of people estimating that (which will clearly be affected by diminishing marginal returns), then adding a Neglectedness score on as well, which seems wrong.
I haven’t written this up yet as I don’t think it’s hugely important- it’s typically a feature of naïve/rough work, and there’s definitely a chance that some of this kind of work is actually using a framework modelled on 80k but just not exposing that well. Most high quality research is just done by an actual CEA rather than by ITN framework, so there’s obviously no issue there.
Importance, Tractibility and Neglectedness should not have equal weight.
TL;Dr, Neglectedness is a useful tiebreaker and gives you information about tractability but the relatively common matrix approach of scoring possible ideas on ITN and then ranking based on the sum of the scores overweights it.
If you’re using the formal mathematical definitions of the terms from this section of the 80,000 Hours article, then their product (before taking logs) has an interpretation in natural units, as good done / extra person or $, so if you reweight, this interpretation for the product will be lost. Are you interpreting the ITN terms differently?
Yes, or at least I think the way they are often interpreted is different. I actually have no issue with 80k’s formal definition, but qualitative use in practice (not by 80k) has often put both both of 80k’s last two points in the tractability metric, then there’s this other nebulous factor called ‘Neglectedness’ which ends up being counted again. The key metric is how much good can be done by one marginal extra person or dollar, and I’ve seen a few cases of people estimating that (which will clearly be affected by diminishing marginal returns), then adding a Neglectedness score on as well, which seems wrong.
I haven’t written this up yet as I don’t think it’s hugely important- it’s typically a feature of naïve/rough work, and there’s definitely a chance that some of this kind of work is actually using a framework modelled on 80k but just not exposing that well. Most high quality research is just done by an actual CEA rather than by ITN framework, so there’s obviously no issue there.
Ok, makes sense!
In case you haven’t seen it, this might be helpful to see what other critiques are out there already.