The ITN framework, cost-effectiveness, and cause prioritisation

From read­ing EA ma­te­rial, one might get the im­pres­sion that the Im­por­tance, Tractabil­ity and Ne­glect­ed­ness (ITN) frame­work is the (1) only, or (2) best way to pri­ori­tise causes. For ex­am­ple, in EA con­cepts’ two en­tries on cause pri­ori­ti­sa­tion, the ITN frame­work is put for­ward as the only or lead­ing way to pri­ori­tise causes. Will MacAskill’s re­cent TedTalk leaned heav­ily on the ITN frame­work as the way to make cause pri­ori­ti­sa­tion de­ci­sions. Open Philan­thropy Pro­ject ex­plic­itly pri­ori­tises causes us­ing an in­for­mal ver­sion of the ITN frame­work.

In this post, I ar­gue that:

  1. Ex­tant ver­sions of the ITN frame­work are sub­ject to con­cep­tual prob­lems.

  2. A new ver­sion of the ITN frame­work, de­vel­oped here, is prefer­able to ex­tant ver­sions.

  3. Non-ITN cost-effec­tive­ness anal­y­sis is, when work­able, su­pe­rior to ITN anal­y­sis for the pur­poses of cause pri­ori­ti­sa­tion.

  4. This is be­cause:

    1. Marginal cost-effec­tive­ness is what we ul­ti­mately care about.

    2. If we can es­ti­mate the marginal cost-effec­tive­ness of work on a cause with­out es­ti­mat­ing the to­tal scale of a prob­lem or its ne­glect­ed­ness, then we should do that, in or­der to save time.

    3. Marginal cost-effec­tive­ness anal­y­sis does not re­quire the as­sump­tion of diminish­ing marginal re­turns, which may not char­ac­ter­ise all prob­lems.

  5. ITN anal­y­sis may be use­ful when it is difficult to pro­duce in­tu­itions about the marginal cost-effec­tive­ness of work on a prob­lem. In that case, we can make progress by zoom­ing out and car­ry­ing out an ITN anal­y­sis.

  6. In difficult high stakes cause pri­ori­ti­sa­tion de­ci­sions, we have to get into the weeds and con­sider in-depth the ar­gu­ments for and against differ­ent prob­lems be­ing cost-effec­tive to work on. We can­not by­pass this pro­cess through sim­ple mechanis­tic scor­ing and ag­gre­ga­tion of the three ITN fac­tors.

  7. For this rea­son, the EA move­ment has thus far sig­nifi­cantly over-re­lied on the ITN frame­work as a way to pri­ori­tise causes. For high stakes cause pri­ori­ti­sa­tion de­ci­sions, we should move to­wards in-depth anal­y­sis of marginal cost-effec­tive­ness.

[up­date—my foot­notes didn’t trans­fer from the google­doc, so I am adding them now]

1. Out­lin­ing the ITN frame­work

Im­por­tance, tractabil­ity and ne­glect­ed­ness are three fac­tors which are widely held to be cor­re­lated with cost-effec­tive­ness; if one cause is more im­por­tant, tractable and ne­glected than an­other, then it is likely to be more cost-effec­tive to work on, on the mar­gin. ITN analy­ses are meant to be use­ful when it is difficult to es­ti­mate di­rectly the cost-effec­tive­ness of work on differ­ent causes.

In­for­mal and for­mal ver­sions of the ITN frame­work tend to define im­por­tance and ne­glect­ed­ness in the same way. As we will see be­low, they differ on how to define tractabil­ity.

Im­por­tance or scale = the over­all bad­ness of a prob­lem, or cor­re­spond­ingly, how good it would be to solve it. So for ex­am­ple, the im­por­tance of malaria is given by the to­tal health bur­den it im­poses, which you could mea­sure in terms of a health or welfare met­ric like DALYs.

Ne­glect­ed­ness = the to­tal amount of re­sources or at­ten­tion a prob­lem cur­rently re­ceives. So for ex­am­ple, a good proxy for the ne­glect­ed­ness of malaria is the to­tal amount of money that cur­rently goes to­wards deal­ing with the dis­ease.[^1]

Ex­tant in­for­mal defi­ni­tions of tractability

Tractabil­ity is harder to define and harder to quan­tify than im­por­tance and ne­glect­ed­ness. In in­for­mal ver­sions of the frame­work, tractabil­ity is some­times defined in terms of cost-effec­tive­ness. How­ever, this does not make that much sense be­cause, as men­tioned, the ITN frame­work is meant to be most use­ful when it is difficult to es­ti­mate the marginal cost-effec­tive­ness of work on a par­tic­u­lar cause. There would be no rea­son to calcu­late ne­glect­ed­ness if we already knew tractabil­ity, thus defined.

Other in­for­mal ver­sions of the ITN frame­work of­ten use in­tu­itive defi­ni­tions such as “tractable causes are those in which it is easy to make progress”. This defi­ni­tion seems to sug­gest that tractabil­ity is defined as how much of a prob­lem you can solve with a given amount of fund­ing. How­ever, if you knew this, there would be no point in calcu­lat­ing ne­glect­ed­ness, since with im­por­tance and tractabil­ity alone, you could calcu­late the marginal cost-effec­tive­ness of work on a prob­lem, which is ul­ti­mately what we care about. This defi­ni­tion ren­ders the Ne­glect­ed­ness part of the anal­y­sis un­nec­es­sary, or at least sug­gests that we would only calcu­late ne­glect­ed­ness as one fac­tor that bears on tractabil­ity, rather than as three dis­tinct quan­tities that can be ag­gre­gated and scored.

Thus, ex­tant in­for­mal ver­sions of the ITN frame­work have some con­cep­tual difficul­ties.

Ex­tant for­mal defi­ni­tions of tractabil­ity

80,000 Hours de­vel­ops a more for­mal ver­sion of the ITN frame­work which ad­vances a differ­ent defi­ni­tion of tractabil­ity:

“% of prob­lem solved /​ % in­crease in re­sources”[^2]

The terms in the 80k ITN defi­ni­tions can­cel out as fol­lows:

  • Im­por­tance = good done /​ % of prob­lem solved

  • Tractabil­ity = % of prob­lem solved /​ % in­crease in resources

  • Ne­glect­ed­ness = % in­crease in re­sources /​ ex­tra $

Thus, once we have in­for­ma­tion on im­por­tance, tractabil­ity and ne­glect­ed­ness (thus defined), then we can pro­duce an es­ti­mate of marginal cost-effec­tive­ness.

The prob­lem with this is: if we can do this, then why would we calcu­late these three terms sep­a­rately in the first place? The ITN is sup­posed to be use­ful as a heuris­tic when we lack in­for­ma­tion on cost-effec­tive­ness, but on these defi­ni­tions, we must already have in­for­ma­tion on cost-effec­tive­ness. On these defi­ni­tions, there is no rea­son to calcu­late ne­glect­ed­ness.

To be as clear as pos­si­ble, on the 80k frame­work, if we know the ITN es­ti­mates, then we know the differ­ence that an ad­di­tional $1m (say) will make on solv­ing a prob­lem. So, we do not nec­es­sar­ily have to calcu­late the ne­glect­ed­ness of a prob­lem in or­der to pri­ori­tise causes.

It is im­por­tant to bear in mind, but easy to for­get, that cause pri­ori­ti­sa­tion in terms of the ITN crite­ria thus defined in­volve judge­ments about cost-effec­tive­ness. For ex­am­ple, all of 80,000 Hours’ cause pri­ori­ti­sa­tion rests on judge­ments about all ITN fac­tors thus defined, and so we must be able to de­duce from them marginal cost-effec­tive­ness es­ti­mates for work on AI, biorisk, nu­clear se­cu­rity and cli­mate change, and so on.

An al­ter­na­tive ITN framework

Ex­ist­ing ver­sions of the ITN frame­work seem to have some con­cep­tual prob­lems. Nev­er­the­less, the ITN frame­work in some form of­ten seems a use­ful heuris­tic. The ques­tion there­fore is: how should we define tractabil­ity in a con­cep­tu­ally co­her­ent way such that the ITN frame­work re­mains use­ful?

The re­search team at Founders Pledge has de­vel­oped a frame­work which at­tempts to meet these crite­ria. We define tractabil­ity in the fol­low­ing way:

Tractabil­ity = % of prob­lem solved per marginal re­source.

On this defi­ni­tion, ne­glect­ed­ness is just one among many de­ter­mi­nants of tractabil­ity. Im­por­tance and ne­glect­ed­ness can be quan­tified quite eas­ily, but the other fac­tors, aside from ne­glect­ed­ness, that bear on tractabil­ity are harder to quan­tify. As­sum­ing diminish­ing re­turns, the con­cep­tual re­la­tion­ship be­tween the fac­tors can be rep­re­sented as fol­lows:


[I can’t get the textboxes to show here—they are la­bels of ‘good done’ for the y-axis, and ‘re­sources’ for the x-axis.]

The scale im­por­tance of a prob­lem is the max­i­mal point that the curve meets on the y-axis—the higher up the y-axis you can go, the bet­ter it is. Ne­glect­ed­ness tells you where you are on the x-axis at pre­sent. The other fac­tors that bear on tractabil­ity tell you the over­all shape of the curve. In­tractable prob­lems will have flat­ter curves, such that mov­ing along the x-axis (putting more re­sources in) doesn’t take you far up the y-axis (solve much of the prob­lem). Cor­re­spond­ingly, eas­ily solv­able prob­lems will have steep curves.

When we are ini­tially eval­u­at­ing a prob­lem, it is of­ten difficult to know the shape of the re­turns to re­sources curve, but easy to calcu­late how big a prob­lem is and how ne­glected it is. This is why ITN anal­y­sis comes into its own when it is difficult to gather in­for­ma­tion about cost-effec­tive­ness. Thus, when we are car­ry­ing out ITN anal­y­sis in this new for­mat, the pro­cess would be:

  1. We quan­tify im­por­tance to ne­glect­ed­ness ra­tios for differ­ent prob­lems.

  2. We eval­u­ate the other fac­tors (aside from ne­glect­ed­ness) that bear on the tractabil­ity of a prob­lem.

  3. We make a judge­ment about whether the differ­ences in tractabil­ity could be suffi­cient to over­come the ini­tial im­por­tance/​ne­glect­ed­ness rank­ing.

For step 1, prob­lems with higher im­por­tance/​ne­glect­ed­ness ra­tios should be a higher pri­or­ity, other things equal. That is, we should pre­fer to work on huge but ne­glected prob­lems than small crowded ones, other things equal.

For step 2, we would have to find a way to ab­stract from the cur­rent ne­glect­ed­ness of differ­ent prob­lems.[^3] One way to do this would be to try to eval­u­ate the av­er­age tractabil­ity of two differ­ent prob­lems. Another way would be to eval­u­ate the two prob­lems imag­in­ing that they were at the same level of ne­glect­ed­ness. When we are as­sess­ing tractabil­ity, con­trol­ling for ne­glect­ed­ness, we would con­sider fac­tors such as:

  • The level of op­po­si­tion to work­ing on a problem

  • The strength of the poli­ti­cal or eco­nomic in­cen­tives to solve a problem

  • The co­or­di­na­tion re­quired to solve a prob­lem

For step 3, once we have the in­for­ma­tion of the other fac­tors (aside from ne­glect­ed­ness) bear­ing on tractabil­ity, we then have to de­cide how these af­fect our ini­tial step 1 rank­ing. One op­tion would be to give prob­lems differ­ent very rough scores on tractabil­ity per­haps us­ing a check­list of the fac­tors above. Some prob­lems will dom­i­nate oth­ers in terms of the three ITN crite­ria, and pri­ori­ti­sa­tion will then be straight­for­ward. In more difficult cases, some prob­lems will be highly ne­glected but much less tractable than oth­ers (eg in cli­mate change, nu­clear power is much more ne­glected than re­new­ables but also ar­guably more un­pop­u­lar at all lev­els of ne­glect­ed­ness), or the tractabil­ity of work of a prob­lem will be very un­clear. In these cases, we have to make judge­ment calls about whether any of the differ­ences in the other fac­tors bear­ing on tractabil­ity are suffi­cient to change our ini­tial step 1 rank­ing. That is, we have to make rough as­sump­tions claims about the shape of the re­turns curve for differ­ent prob­lems.

On this ver­sion of the frame­work, it is not pos­si­ble to mechanis­ti­cally ag­gre­gate ITN scores be­tween prob­lems to pro­duce an over­all cause rank­ing. This ver­sion of the ITN frame­work pro­duces rank­ings be­tween prob­lems that are quite low re­s­olu­tion: it will of­ten be difficult to know the over­all rank­ing of differ­ent causes, analysed in this way. This is what we should ex­pect from the ITN frame­work. The ITN frame­work is use­ful pre­cisely when it is difficult to have in­tu­itions about cost-effec­tive­ness.

The ad­van­tage of this ver­sion of the frame­work is that it is more con­cep­tu­ally co­her­ent than ex­tant ver­sions of the frame­work.

The dis­ad­van­tages of this ver­sion of the frame­work are:

  1. It re­lies on the as­sump­tion of diminish­ing re­turns, which may not char­ac­ter­ise all prob­lems.

  2. ITN anal­y­sis is in some cases in­fe­rior to cost-effec­tive­ness anal­y­sis as a cause pri­ori­ti­sa­tion tool.

To these two points, I now turn.

2. Cost-effec­tive­ness anal­y­sis with­out ITN anal­y­sis

We have seen that on some ver­sions of the frame­work, ITN analy­ses nec­es­sar­ily give us the in­for­ma­tion for a marginal cost-effec­tive­ness es­ti­mate. How­ever, it is pos­si­ble to calcu­late the marginal cost-effec­tive­ness of work on a cause with­out car­ry­ing out an ITN anal­y­sis. There are two main ways in which cost-effec­tive­ness anal­y­sis could differ from an ITN anal­y­sis:

  1. Calcu­lat­ing the size of the whole prob­lem

ITN anal­y­sis in­volves es­ti­mat­ing the size of a whole prob­lem. For ex­am­ple, when es­ti­mat­ing the im­por­tance of malaria, one would quan­tify the to­tal scale of the prob­lem of malaria in DALYs. But if you are do­ing cost-effec­tive­ness anal­y­sis, it would not always be nec­es­sary to quan­tify the to­tal scale of the whole prob­lem. Rather, you could es­ti­mate di­rectly how good it is to solve x% of a prob­lem with y re­sources.

  1. Calcu­lat­ing neglectedness

Cost-effec­tive­ness analy­ses do not nec­es­sar­ily have to calcu­late the ne­glect­ed­ness of a prob­lem. It is some­times pos­si­ble to di­rectly calcu­late how much good an ex­tra x re­sources will do, which does not nec­es­sar­ily re­quire you to as­sess how many re­sources a prob­lem cur­rently re­ceives in to­tal. This is be­cause ne­glect­ed­ness is just one de­ter­mi­nant of tractabil­ity (un­der­stood as % of prob­lem solved/​$) among oth­ers, and it may be pos­si­ble to es­ti­mate how tractable a prob­lem is with­out es­ti­mat­ing any one de­ter­mi­nant of tractabil­ity, whether that be ne­glect­ed­ness, level of poli­ti­cal op­po­si­tion, de­gree of co­or­di­na­tion re­quired, or what­ever.

Non-ITN cost-effec­tive­ness es­ti­mates have two main ad­van­tages over ITN analy­ses.

  1. Marginal cost-effec­tive­ness is what we ul­ti­mately care about. If we can pro­duce an es­ti­mate of that with­out hav­ing to go through the ex­tra steps of quan­tify­ing the whole prob­lem or calcu­lat­ing ne­glect­ed­ness, then we should do that, purely to save time.

  2. Avoid­ing the­o­ret­i­cal re­li­ance on calcu­lat­ing ne­glect­ed­ness avoids re­li­ance on the as­sump­tion of diminish­ing marginal re­turns, which may not char­ac­ter­ise ev­ery prob­lem.[^4]

To illus­trate the pos­si­bil­ity, and ad­van­tages, of non-ITN cost-effec­tive­ness anal­y­sis, ex­am­ples fol­low.

Giv­ing What We Can on global health

Giv­ing What We Can ar­gued that donat­ing to the best global health char­i­ties is bet­ter than donat­ing do­mes­ti­cally.

“The UK’s Na­tional Health Ser­vice con­sid­ers it cost-effec­tive to spend up to £20,000 (about $25,000) for a sin­gle year of healthy life added.
By con­trast, be­cause of their poverty many de­vel­op­ing coun­tries are still plagued by dis­eases which would cost the de­vel­oped world com­par­a­tively tiny sums to con­trol. For ex­am­ple, GiveWell es­ti­mates that the cost per child life saved through an LLIN dis­tri­bu­tion funded by the Against Malaria Foun­da­tion is about $7,500. The NHS would spend this amount to add about four months of healthy life to a pa­tient.”

This ar­gu­ment uses a cost-effec­tive­ness es­ti­mate to ar­gue for fo­cus­ing on the best global health in­ter­ven­tions rather than donat­ing in a high-in­come coun­try.

It is true that Giv­ing What We Can ap­peals here to ne­glect­ed­ness as a way to ex­plain why the cost-effec­tive­ness of health spend­ing differs be­tween the UK and the best global poverty char­i­ties. But the ar­gu­ment from the di­rect cost-effec­tive­ness es­ti­mate alone is suffi­cient to get to the con­clu­sion: if the cost-effec­tive­ness of health spend­ing is ac­tu­ally higher in the UK than poor coun­tries, then the point about ne­glect­ed­ness would be moot. This illus­trates the re­la­tion ne­glect­ed­ness has to cost-effec­tive­ness anal­y­sis, and how ne­glect­ed­ness anal­y­sis is not always nec­es­sary for cause com­par­i­sons.

Lant Pritch­ett on eco­nomic growth

In his pa­per ‘Alle­vi­at­ing Global Poverty: La­bor Mo­bil­ity, Direct As­sis­tance, and Eco­nomic Growth’, Lant Pritch­ett ar­gues that re­search on, and ad­vo­cacy for, eco­nomic growth is a bet­ter bet than di­rect ‘ev­i­dence-based de­vel­op­ment’ (eg, dis­tribut­ing bed­nets, cash trans­fers and de­worm­ing pills).

Here he lays out the po­ten­tial benefits of one form of ev­i­dence-based de­vel­op­ment, the ‘Grad­u­a­tion ap­proach’:

“Sup­pose the im­pact of the Grad­u­a­tion pro­gram in Ethiopia was what it was on av­er­age for the five coun­tries and gen­er­ated $1,720 in NPV for each $1000 in­vested.” (p25)

Thus, one gets a 1.7x re­turn from the Grad­u­a­tion ap­proach. Here he lays out the benefits of re­search and ad­vo­cacy for growth:

“The mem­ber­ship of the Amer­i­can Eco­nomics As­so­ci­a­tion is about 20,000 and sup­pose the global to­tal num­ber of economists is twice that and the in­clu­sive cost to some­one of an economist per year is $150,000 on av­er­age. Then the cost of all economists in the world is about 6 billion dol­lars. Sup­pose this was con­stant for 50 years and hence cost 300 billion to sus­tain the eco­nomics pro­fes­sion from 1960 to 2010. Sup­pose the only im­pact of all economists in all these 50 years was to be even a mod­est part of the many fac­tors that per­suaded the Chi­nese lead­er­ship to switch eco­nomic strat­egy and pro­duce 14 trillion dol­lars in cu­mu­la­tive ad­di­tional out­put.” (p24)

Even if the to­tal im­pact of all economists in the world for 50 years was only to in­crease by 4% (in ab­solute terms) the prob­a­bil­ity of the change in course in Chi­nese policy, it would still have greater ex­pected value than di­rectly fund­ing the grad­u­a­tion ap­proach. Since de­vel­op­ment economists likely did much more than this, re­search and ad­vo­cacy for growth-friendly poli­cies is bet­ter than ev­i­dence-based de­vel­op­ment. Pritch­ett con­tinues:

“For in­stance, the World Bank’s in­ter­nal ex­pen­di­tures (BB bud­get) on all of Devel­op­ment Eco­nomics (of which re­search is just a por­tion) in FY2016 was about 50 mil­lion dol­lars. The gains in NPV of GDP from just the In­dian 2002 growth ac­cel­er­a­tion of 2.5 trillion are 50,000 times larger. The losses in NPV from Brazil’s 1980 growth de­cel­er­a­tion are 150,000 times larger. So even if by do­ing decades of re­search on what ac­cel­er­ates growth (or avoids losses) and even if that only as a small chance of suc­cess in chang­ing poli­cies this still could have just enor­mous re­turns—be­cause the policy or other changes that cre­ate growth in­duces coun­try-wide gains in A (which are, eco­nom­i­cally, free) and in­duces vol­un­tary in­vest­ments that have no di­rect fis­cal cost (or con­versely causes those to dis­ap­pear).” (p25)

This, again, is a way of plac­ing a lower bound on a cost-effec­tive­ness es­ti­mate of re­search and ad­vo­cacy for growth, as against di­rect in­ter­ven­tions.

Try­ing to bend the rea­son­ing here into an ITN anal­y­sis would add un­nec­es­sary com­plex­ity to Pritch­ett’s ar­gu­ment. This illus­trates the ad­van­tage of non-ITN cost-effec­tive­ness anal­y­sis:

  1. Calcu­lat­ing the scale of the benefits of eco­nomic growth

What is the to­tal scale of the prob­lem that eco­nomic growth is try­ing to solve? Eco­nomic growth can ar­guably pro­duce ar­bi­trar­ily large benefits, so should we use a dis­count rate of some sort? Which one should we use? Etc. We can avoid these ques­tions by fo­cus­ing on the limited benefits of par­tic­u­lar growth epi­sodes a la Lant.

  1. Calcu­lat­ing the ne­glect­ed­ness of eco­nomics research

At no point does Pritch­ett ap­peal to the ne­glect­ed­ness of re­search and ad­vo­cacy for growth rel­a­tive to ev­i­dence-based de­vel­op­ment. Do­ing so is un­nec­es­sary to get to his con­clu­sion.

Bostrom on ex­is­ten­tial risk

In his pa­per, ‘Ex­is­ten­tial Risk Preven­tion as Global Pri­or­ity’, Nick Bostrom defends the view that re­duc­ing ex­is­ten­tial risk should be a top pri­or­ity for our civil­i­sa­tion, and ar­gues:

“Even if we give this allegedly lower bound on the cu­mu­la­tive out­put po­ten­tial of a tech­nolog­i­cally ma­ture civ­i­liza­tion a mere 1% chance of be­ing cor­rect, we find that the ex­pected value of re­duc­ing ex­is­ten­tial risk by a mere one billionth of one billionth of one per­centage point is worth a hun­dred billion times as much as a billion hu­man lives.
One might con­se­quently ar­gue that even the tiniest re­duc­tion of ex­is­ten­tial risk has an ex­pected value greater than that of the definite pro­vi­sion of any “or­di­nary” good, such as the di­rect benefit of sav­ing 1 billion lives.”

This is an ar­gu­ment in favour of fo­cus­ing on ex­is­ten­tial risk that pro­vides a lower bound cost-effec­tive­ness es­ti­mate for the ex­pected value of ex­is­ten­tial risk re­duc­tion. The ar­gu­ment is: plau­si­ble work on ex­is­ten­tial risk is likely to make even very small re­duc­tions in ex risk, which will have greater ex­pected value than any ac­tion one could plau­si­bly take to im­prove (eg) global poverty or health. Es­ti­mat­ing the to­tal ne­glect­ed­ness of the prob­lem of ex­is­ten­tial risk is un­nec­es­sary here. You just have to know the lower bound of how big an effect a state could ex­pect to have on ex­is­ten­tial risk. That is, you have to know the lower bound of tractabil­ity, un­der­stood as ‘% of prob­lem solved /​$’. Ne­glect­ed­ness is one de­ter­mi­nant of tractabil­ity (thus defined), and it is not nec­es­sary to eval­u­ate all de­ter­mi­nants of tractabil­ity when eval­u­at­ing tractabil­ity.

Ma­theny on ex­is­ten­tial risk and as­ter­oid protection

In his ‘Re­duc­ing the Risk of Hu­man Ex­tinc­tion’, Ja­son Ma­theny ar­gues that ex­is­ten­tial risk re­duc­tion has very high ex­pected value, and uses the ex­am­ple of as­ter­oid pro­tec­tion to illus­trate this. He con­cludes that an as­ter­oid de­tect and deflect sys­tem cost­ing $20 billion would pro­duce benefits equiv­a­lent to sav­ing a life for $2.50.[^5]

Since this is much lower than what you can get from al­most all global poverty and health in­ter­ven­tions, as­ter­oid pro­tec­tion is bet­ter than global poverty and health. One might add that since that most ex­perts think that work on other prob­lems such as AI, biose­cu­rity and nu­clear se­cu­rity is much more cost-effec­tive than as­ter­oid pro­tec­tion (from a long-ter­mist point of view), work­ing on these prob­lems must (per epistemic mod­esty) a for­tiori be even bet­ter than global poverty and health. If there were re­li­able con­ver­sions be­tween the best an­i­mal welfare in­ter­ven­tions and global poverty and health in­ter­ven­tions, then this would also en­able us to choose be­tween the causes of ex­is­ten­tial risk, global poverty and an­i­mal welfare.

In this case, one does not need to es­ti­mate ne­glect­ed­ness be­cause we can already quan­tify what a par­tic­u­lar amount of re­sources will achieve in re­duc­ing the prob­lem of ex­is­ten­tial risk.

3. Conclusions

In this post, I have ar­gued that:

  1. Ex­tant ver­sions of the ITN frame­work are sub­ject to con­cep­tual prob­lems.

  2. A new ver­sion of the ITN frame­work, de­vel­oped here, is prefer­able to ex­tant ver­sions.

  3. Non-ITN cost-effec­tive­ness anal­y­sis is, when work­able, su­pe­rior to ITN anal­y­sis for the pur­poses of cause pri­ori­ti­sa­tion.

  4. This is be­cause:

    1. Marginal cost-effec­tive­ness is what we ul­ti­mately care about.

    2. If we can es­ti­mate the marginal cost-effec­tive­ness of work on a cause with­out es­ti­mat­ing the to­tal scale of a prob­lem or its ne­glect­ed­ness, then we should do that, in or­der to save time.

    3. Marginal cost-effec­tive­ness anal­y­sis does not re­quire the as­sump­tion of diminish­ing marginal re­turns, which may not char­ac­ter­ise all prob­lems.

The ITN frame­work may be prefer­able to cost-effec­tive­ness anal­y­sis when:

  • At cur­rent lev­els of in­for­ma­tion, it is difficult to pro­duce in­tu­itions about the effect that a marginal amount of re­sources will have on a prob­lem. In that case, it may be eas­ier to zoom out and get some lower re­s­olu­tion in­for­ma­tion on the to­tal scale of a prob­lem and on its ne­glect­ed­ness, and then to try to weigh up the other fac­tors (aside from ne­glect­ed­ness) bear­ing on tractabil­ity. This can be a good way to economise time on cause pri­ori­ti­sa­tion de­ci­sions.

Often, as we have seen, this will some­times leave us un­cer­tain about which cause is best. This is what we should ex­pect from ITN anal­y­sis. We should not ex­pect ITN anal­y­sis to re­solve all of our difficult cause se­lec­tion de­ci­sions. We can re­solve this un­cer­tainty by gath­er­ing more in­for­ma­tion about the fac­tors bear­ing on the cost-effec­tive­ness of work­ing on a prob­lem. This is difficult work that must go far be­yond a sim­ple mechanis­tic pro­cess of quan­tify­ing and ag­gre­gat­ing three scores.

For ex­am­ple, sup­pose we are de­cid­ing how to pri­ori­tise global poverty and cli­mate change. This is a high stakes de­ci­sion for the EA com­mu­nity, as it could af­fect the al­lo­ca­tion of tens of mil­lions of dol­lars. While it may be rel­a­tively easy to bound the im­por­tance and ne­glect­ed­ness of these prob­lems, that still leaves out a lot of in­for­ma­tion on the other mul­ti­tudi­nous fac­tors that bear on how cost-effec­tive these causes are to work on. To re­ally be con­fi­dent in our choice be­tween these causes, we would have to con­sider fac­tors in­clud­ing:

  • The best way to make progress on global poverty. Should we fo­cus on health or on growth? How do we in­crease growth? Etc etc

  • What are the in­di­rect effects of growth? Does it make peo­ple more tol­er­ant and liberal? To what ex­tent does it in­crease the dis­cov­ery of civil­i­sa­tion-threat­en­ing tech­nolo­gies? Etc etc.

  • Plau­si­ble es­ti­mates of the so­cial cost of car­bon. How should we quan­tify the po­ten­tial mass mi­gra­tion that could re­sult from ex­treme cli­mate change? How should we dis­count fu­ture benefits? How rich will peo­ple be in 100 years? Etc etc.

  • Ways to con­vert global poverty re­duc­tion benefits into cli­mate change re­duc­tion benefits.

  • What are the best lev­ers to pull on in cli­mate change. How good would a car­bon tax be and how tractable is it? Can re­new­ables take over the elec­tric­ity sup­ply? Is in­no­va­tion the way for­ward? What are the prospects of suc­cess for nu­clear in­no­va­tion and en­hanced geother­mal? Do any non-prof­its stand a chance of af­fect­ing in­no­va­tion? Does in­creas­ing nu­clear power lead to weapons pro­lifer­a­tion? etc etc.

Th­ese are all difficult ques­tions, and we need to an­swer them in or­der to make rea­son­able judge­ments about cause pri­ori­ti­sa­tion. We should not ex­pect a sim­ple three fac­tor ag­gre­ga­tion pro­cess to solve difficult cause pri­ori­ti­sa­tion de­ci­sions such as these. The more we look in de­tail at par­tic­u­lar causes, the fur­ther we get from low re­s­olu­tion ITN anal­y­sis, and the closer we get to pro­duc­ing a di­rect marginal cost-effec­tive­ness es­ti­mate of work on these prob­lems.

To have con­fi­dence in our high stakes cause pri­ori­ti­sa­tion de­ci­sions, the EA com­mu­nity should move away from ITN anal­y­sis, and move to­wards in-depth marginal cost-effec­tive­ness anal­y­sis.

Thanks to Ste­fan Schu­bert and Mar­tijn Kaag for helpful com­ments and sug­ges­tions.


[^1] Plau­si­bly, we should ac­tu­ally con­sider the to­tal all-time re­sources that will go to a prob­lem over time, but that is the sub­ject for an­other post.

[^2] For math­e­mat­i­cal ease, we can make the de­nom­i­na­tor 1 here and so calcu­late the good pro­duced by a dou­bling of re­sources from the cur­rent level

[^3] Rob Wiblin ‘The Im­por­tant/​Ne­glected/​Tractable frame­work needs to be ap­plied with care’ (2016)

[^4] On this, see Arepo (Sasha Cooper), ‘Against ne­glect­ed­ness’, EA Fo­rum (Nov 2017); sbehmer, ‘Is Ne­glect­ed­ness a Strong Pre­dic­tor of Marginal Im­pact?’, EA Fo­rum (Nov 2018). See also Owen Cot­ton-Bar­rat ‘The law of diminish­ing re­turns’, FHI (2014).

[^5] For similar, see Piers Millett and An­drew Sny­der-Beat­tie, ‘Ex­is­ten­tial Risk and Cost-Effec­tive Biose­cu­rity’, Health Se­cu­rity 15, no. 4 (1 Au­gust 2017): 373–83, https://​​doi.org/​​10.1089/​​hs.2017.0028.