The ITN (Impact, Tractability, Neglectedness) framework is widely used in EA to do cause prioritisation . Which could be other metrics that we might be missing out on? Is the ITN framework exhaustive?
[Question] Factors other than ITN?
I’m not sure if it directly answers your question, but this question did finally lead me to write the post about the stock issues framework (which seems to be listed in the pingbacks). I hope that is relevant to your question!
INT = good per dollar by definition (when used in the quantitative rather than heuristic way), so in that sense it’s exhaustive, though in practice, people often miss some factors that are not as naturally captured by the framework:
People often assess ‘importance’ just based on one yardstick (e.g. QALYs) when there are effects on other relevant metrics (e.g. x-risk reduction, economic growth) (and the choice of which yardsticks to focus on in the first place is where a lot of the action is).
Value of information—can be included in either I or N, but often missed.
Coordination considerations e.g. portfolio approach, comparative advantage, trade with people with other values.
Cross-cutting epistemic considerations such as regression to the mean & epistemic humility & how to deal with unmeasured factors – partially covered in my recent podcast. People often only report their ‘unadjusted’ estimates and don’t account for these.
Movement building effects e.g. promoting one cause might bring people into others.
Funging e.g. if you solve one problem, it might free up resources to work on another.
I find it useful to try to make a ‘direct’ estimate using INT, and then to have a separate ‘all considered’ estimate that aims to take account of all the above.
The application of INT also gets more complicated depending on whether you’re interested in the resources spent in a certain year or over all time. Likewise, there are issues like complementarities between different forms of resources (e.g. funding vs. labour) that can mean the analysis is different for different resources.
You could also have another category of timing considerations, such as these. Toby’s soon, sharp, sudden framework helps to capture some of timing factors as well, or you could think of them as guides to what’s most important.
As an alternative, I think it’s also useful to think of cost-effectiveness analysis of specific interventions as a separate framework that provides a different perspective.
INT is also only about how pressing causes are in general. In practice if you’re making a real decision, you also need to consider your personal fit, career capital, the quality of the specific opportunity etc. as well as other moral considerations besides good done.
I personally don’t find the ITN framework useful and agree with most of John Halstead’s criticism of the framework here. Cost-effectiveness seems better if you want to make something numerical, within a particular cause area.
In fact, for me, I think both cost-effectiveness and ITN as intellectual frameworks fall down because they mask what I see as a fundamentally philosophical set of questions, if you’re trying to do something like compare mental health with health interventions with animal welfare with more longtermist interventions.
If there’s an underlying question: how does one prioritise causes? From my own perspective, I just gravitated towards a more longtermist cause prioritisation over time as I found the arguments quite convincing, but I agree that many other areas are extremely worthwhile.
Other factors
If you are thinking about what cause you should work on, you may also consider personal fit. 80,000 hours explain why they didn’t include it as a factor here. Also, none of us is 100% altruistic, so you may also want to consider the personal benefit of working on the cause, although I guess that can go under personal fit as well.
Is it exhaustive?
You could say that there are two ITN frameworks: informal and quantitative. It’s easier to talk about the quantitative framework, so that’s what I will talk about, even though people usually use the informal one.
The quantitative framework cancels out to Good done / extra person or $. If you are a pure consequentialist, I think that this is exhaustive by definition. It doesn’t capture non-consequentialist concerns. E.g., maybe making cost-effective progress on this cause would involve morally questionable means like lying or blackmail. However, you can incorporate these by redefining tractability to something like “tractable with only using means I am comfortable with”. Or you can just assume that in the long run, using these means is bad from a consequentialist point of view anyway, which usually seems to be the case.
Also, it’s exhaustive only if Good done includes all things you intrinsically value like personal benefit, equity, etc. Usually when people use the framework, they assume pure utilitarianism and don’t include these.
I feel I should also mention that I personally find the framework unnecessary and limiting. My opinion is that we don’t need any framework here. I find that it’s easier and more productive to simply think about what actions I can take and what consequences those actions will lead to. But this is a bit off-topic and I will explain my view in full another time.
Toby Ord writes about prioritizing risks that are soon, sudden and sharp in the Precipice.
Also see a criticism of the current ITN framework and a proposal for a new one by John Halstead.
If I can be forgiven for tooting my own horn, I also wrote a forum post about the framework around the same time as John posted his. EAs have often talked about “cause prioritisation” as being distinct from “intervention evaluation”: the former is done in terms of ITN, the latter in term of cost-effectiveness. I agree with Ben Todd’s suggestion the best way to understand ITN is as three factors that combine to a calculation of cost-effectiveness (aka “good done per dollar”). One result of this that I think it’s confused to think that “cause prioritisation” and “intervention evaluation” are two different things. I discuss some implications of this.
Given a set of values, I see there as being multiple layers of heuristics, which are all useful to consider and make comparisons based on:
Yardsticks (e.g. x-risk, qualys)
Causes (e.g. AI alignment)
Interventions (e.g. research into the deployment problem)
Specific jobs /orgs (e.g. working at FHI)
Comparisons at all levels are all ultimately about finding proxies for expected value relative to your values.
The cause level abstraction seems to be especially useful for career planning (and grantmaking) since it helps you get career capital that builds up in a useful area. Intervention selection usually seems too brittle. Yardsticks are too broad. This post is pretty old but tries to give some more detail: https://80000hours.org/2013/12/why-pick-a-cause/
I think it’s probably the case that good heuristics for making career decisions are different than good heuristics for making donation decisions. We shouldn’t necessarily expect a framework (ITN or otherwise) to be ideal for both.
If someone today decides to work on a certain cause, they strengthen the pipeline of good funding opportunities in that cause. But there’s a time lag. Pivoting to work on biosecurity might be a great career decision right now. However funding a person to do that work might not be a great donation until a few years down the road, when they’ve gained the skills and credentials needed to make an impact.
I’ve seen people suggest Urgency as an additional dimension. I wonder if anyone has tried to integrate it into an ITN evaluation
Other perspectives that are arguably missing or extensions that can be done are:
Side effect analysis and modelling the entire future trajectory (generalizations of these can be created):
https://www.lesswrong.com/posts/BfKQGYJBwdHfik4Kd/fai-research-constraints-and-agi-side-effects
https://forum.effectivealtruism.org/posts/TCxik4KvTgGzMowP9/state-space-of-x-risk-trajectories
Refining our estimates of value given uncertainties about the future:
https://forum.effectivealtruism.org/posts/4oGYbvcy2SRHTWgWk/improving-the-future-by-influencing-actors-benevolence
Specialization for a particular problem, with ensuing accuracy improvements can be done by causal analysis, Fermi modelling, Bayesian modelling, and inspired by https://forum.effectivealtruism.org/posts/zdAst6ezi45cChRi6/list-of-ways-in-which-cost-effectiveness-estimates-can-be
The value to action chain idea can be used to substantiate value more clearly: https://forum.effectivealtruism.org/posts/Ekzvat8FbHRiPLn9Z/the-values-to-actions-decision-chain-a-lens-for-improving
One can even try to derive things directly from expected utility all the way to actions but the mathematics gets pretty complex quickly (I did some work on this in 2015 and am sitting on a draft paper that someday somewhere I’ll publish. It is beyond my current ability to efficiently explain clearly to a large audience.)
Here also is an additional post analyzing the ITN framework: https://forum.effectivealtruism.org/posts/fR55cjoph2wwiSk8R/formalizing-the-cause-prioritization-framework