Five Ways to Handle Flow-Through Effects

As effec­tive al­tru­ists, we are in­ter­ested in find­ing op­por­tu­ni­ties to pro­duce the high­est ex­pected moral value per unit of cost. This is hard enough even when just look­ing at short-term effects—good giv­ing op­por­tu­ni­ties are scarce,good sci­ence is difficult, er­rors can go un­cor­rected for awhile, and it’s hard to even figure out what kind of met­ric we re­ally want to be im­prov­ing.

But even if we were to some­how figure out an ac­cu­rate view of the short-term effects of a par­tic­u­lar op­por­tu­nity, we have no guaran­tee that effects other than the di­rect, mea­surable short-term effects will also work out in our fa­vor. Th­ese ad­di­tional effects—such as medium-term effects, long-term effects, and other in­di­rect effects not cap­tured in the ini­tial im­pact anal­y­sis—are col­lec­tively referred to here as “flow-through effects”—and can sig­nifi­cantly change how we think about a par­tic­u­lar cause. (This ter­minol­ogy may be sub­par , but I use it any­way be­cause it is what is com­monly used in the EA move­ment right now and there’s no agreed upon re­place­ment.)

For ex­am­ple, even while AMF likely has very valuable short-term effects, could it be cre­at­ing harm­ful over­pop­u­la­tion in the medium-term ( likely no)? Or could it be cre­at­ing a net pop­u­la­tion loss, thereby de­creas­ing the amount of happy peo­ple in the world ( pos­si­bly yes)? And in the long-term, could the eco­nomic de­vel­op­ment caused by AMF just be mak­ing it more likely we die in some tech­nolog­i­cal catas­tro­phe ? Or is eco­nomic de­vel­op­ment pos­i­tive for the world in the long-run?

Ad­di­tion­ally, some have ar­gued that peo­ple and non­hu­man an­i­mals who will live in the far fu­ture (but don’t yet ex­ist) still have sig­nifi­cant moral weight. Given that they will (hope­fully) ex­ist in much more mas­sive quan­tities than those cur­rently al­ive, what­ever im­pact we have on those in the far fu­ture, how­ever minis­cule now, may add up to a much larger over­all effect than our effect on those al­ive now . This would also sug­gest we need in­creased at­ten­tion to long-term effects.

Deal­ing with these po­ten­tial prob­lems is a very difficult challenge be­cause these prob­lems are of­ten not well re­solved by di­rect em­piri­cal anal­y­sis (you can’t just eas­ily wait 100 years to see how things turn out) and be­cause such cru­cial con­sid­er­a­tions can wildly af­fect our choices, mak­ing AMF ei­ther highly benefi­cial or even net nega­tive.

While flow-through effects will still re­quire sig­nifi­cant anal­y­sis, it is im­por­tant to make sure they do not par­a­lyze our de­ci­sions. Thus, in the mean­time, I sug­gest five non-ex­clu­sive po­ten­tial ways to han­dle flow-through effects go­ing for­ward. This isn’t in­tended to be a fully com­pre­hen­sive eval­u­a­tion, but hope­fully offers an ini­tial per­spec­tive that can start dis­cus­sion about how to bet­ter han­dle flow-through effects, which I think is still a de­vel­op­ing part of EA dis­course.

Ig­nor­ing flow-through effects

The sim­plest strat­egy and pos­si­bly the most com­mon one in the char­ity sec­tor is to not take any flow-through effects into ac­count when mak­ing de­ci­sions. This could be be­cause of a be­lief that flow-through effects are too com­pli­cated and in­tractable to be worth work­ing with and/​or be­cause hav­ing a good short-term effect is taken as the best proxy for hav­ing good other effects. One might also think that the flow-through effects are likely to be much smaller in mag­ni­tude than the di­rect, short-term effects, per­haps be­cause these effects diffuse quickly. Prox­i­mate con­se­quen­tial­ism, a ver­sion of con­se­quen­tial­ism that looks only at im­me­di­ate effects, ba­si­cally cod­ifies this “ig­nore” rea­son­ing into moral philos­o­phy.

Pros

  • Very quick and easy

  • Avoids adding very spec­u­la­tive el­e­ments into your im­pact calculations

Cons

  • Very likely to miss large, clear, and im­por­tant effects that greatly af­fect ex­pected value

Qual­i­ta­tive evaluation

Is im­prov­ing eco­nomic de­vel­op­ment net nega­tive? While differ­en­tial tech­nolog­i­cal progress may be a con­cern, GiveWell has offered a few rea­sons to think eco­nomic de­vel­op­ment is pos­i­tive. This ap­proach in­volves mak­ing a best guess about how a par­tic­u­lar flow-through effect will ad­just your im­pact calcu­la­tion and ad­just­ing ac­cord­ingly, per­haps us­ing a “many weak ar­gu­ments” frame­work. Mak­ing such a guess can out­perform ig­nor­ing with­out re­quiring a huge amount of effort on a po­ten­tially in­tractable is­sue.

Pros

  • Rel­a­tively quick to perform

  • In­tu­itively appealing

Cons

  • Prob­a­bly less ac­cu­rate than more re­fined methods

  • Guesses may provide no real value but give a false sense of confidence

  • Can lead to only tak­ing into ac­count the most no­tice­able flow-through effects

  • May in­vite bias

Weighted quan­ti­ta­tive modeling

Similar to guess­ing, one can im­ple­ment a weighted model based on our con­fi­dence in the di­rec­tion (pos­i­tive vs. nega­tive) and mag­ni­tude of each po­ten­tial flow-through effect. Ba­si­cally it en­tails weigh­ing the flow-through effects based on not only how im­por­tant they are but also how much ev­i­dence or con­fi­dence one can have in them.

To use some fic­ti­tious num­bers, imag­ine that AMF has a short-term im­pact of +1, a medium-im­pact of +10, and po­ten­tial long-term im­pacts of ei­ther +100 or −100 de­pend­ing on whether eco­nomic de­vel­op­ment is good or bad. If we’re 60% sure eco­nomic de­vel­op­ment is net good, we can trans­form the long-term im­pact as (0.6)(100) + (0.4)(-100) = 20. Ad­di­tion­ally, if we’re 10x more con­fi­dent in the short-term pro­jec­tions as the medium-term pro­jec­tions and 2x more con­fi­dent in the medium-term pro­jec­tions as the long-term pro­jec­tions, we could ad­just by this to cre­ate a to­tal im­pact of +1 + (1/​10)(+10) + (1/​10)(1/​2)(+20) = +3.

Of course, these two ad­just­ments are very sim­plis­tic and more time should be put into turn­ing this into some sort of model, per­haps based on pri­ors and Bayesian ad­just­ment. There are many such ways to make these ad­just­ments and it is not clear yet which way is the most cor­rect.

Pros

  • Very spec­u­la­tive flow-through effects do not dom­i­nate the cost-effec­tive­ness calculation

  • Takes a medium amount of time rel­a­tive to other options

  • Seems like a good trade off be­tween time and results

Cons

  • Could mis­tak­enly min­i­mize the effect of a very im­por­tant flow-through effect

  • May lead to es­ti­mates based on bi­ased, wish­ful thinking

Con­duct­ing research

Another ap­proach would be to list the po­ten­tial flow-through effects and try to re­search them as much as is fea­si­ble, even if the ques­tions risk be­ing in­tractable. For ex­am­ple, those wor­ried about over­pop­u­la­tion could ex­pand upon GiveWell’s re­view. Other ques­tions may be much harder to re­search, such as differ­en­tial tech­nolog­i­cal progress and eco­nomic de­vel­op­ment, though GiveWell did come up with an ini­tial frame­work for think­ing about it , and one could po­ten­tially try to do more care­ful his­tor­i­cal anal­y­sis on similar situ­a­tions (e.g., im­prove­ment in nu­clear ca­pa­bil­ities vs. im­prove­ment in nu­clear safety /​ disar­ma­ment, im­prove­ments in DNA re­search vs. safe­guards against ge­net­i­cally en­g­ineered pan­demics). Carl Shul­man has offered nu­mer­ous proxy vari­ables we could at­tempt to mea­sure and con­nect causally with var­i­ous in­ter­ven­tions. There is also the po­ten­tial for meta-re­search into ways oth­ers have dealt with flow-through effects.

Pros

  • More em­piri­cally grounded than guesswork

  • More ver­ifi­able and spe­cific than guesswork

  • Can po­ten­tially make use of ex­ist­ing em­piri­cal research

Cons

  • Very slow

  • Very costly, both in time and money

  • May be prac­ti­cally im­pos­si­ble for some (or even most) questions

  • May not pro­duce an an­swer that sig­nifi­cantly im­proves upon guesswork

Pri­ori­tiz­ing robustness

Fi­nally, an­other ap­proach to han­dling flow-through effects is to pick a cause that is be­lieved to be net pos­i­tive , even if a wide va­ri­ety of flow-through effects end up be­ing im­por­tant. For ex­am­ple, it seems much less likely that work spent to im­prove de­vel­oped world ed­u­ca­tion, in­ter­na­tional co­op­er­a­tion, or philo­soph­i­cal thought­ful­ness will be net nega­tive (even if they do have a siz­able risk of be­ing net zero).

Pros

  • Safer from many differ­ent value and episte­molog­i­cal stances

  • Least likely to end up do­ing mas­sive harm

Cons

  • It’s hard to de­ter­mine what is a ro­bust cause

  • We don’t yet have a good frame­work for nav­i­gat­ing trade-offs be­tween cause effec­tive­ness and robustness

  • Does not take into ac­count rel­a­tive scale of flow-through effects

  • Might lead to over­all worse cost-effec­tive­ness if we trade away di­rect im­pact to avoid the risk of nega­tive flow-through effects. For ex­am­ple, many de­vel­oped world ed­u­ca­tion ini­ti­a­tives end up be­ing very ex­pen­sive and pro­duce lit­tle to no re­sults, even if they may have re­duced risk of nega­tive flow-through effects.