Thoughts on the “Meta Trap”

Cross-posted to my blog. Thanks to Ajeya Co­tra and Jeff Kauf­man for feed­back on a draft of this post. Any re­main­ing er­rors are my own. Com­ments on this post may be copied to my blog.

Last year, Peter Hur­ford wrote a post ti­tled ‘EA risks fal­ling into a “meta trap”. But we can avoid it.’ Ben Todd wrote a fol­lowup that clar­ified a few points. I have a few more meta traps to add to the list.

What do I mean by “meta”?

In the origi­nal two posts, I be­lieve “meta” means “pro­mot­ing effec­tive al­tru­ism in ab­stract, with the hope that peo­ple do good ob­ject level pro­jects in the fu­ture”. This does not in­clude cause pri­ori­ti­za­tion re­search, which is typ­i­cally also lumped un­der “meta”. In this post, I’ll use “meta” in much the same way—I’m talk­ing about work that is try­ing to pro­mote effec­tive al­tru­ism in the ab­stract and/​or fundrais­ing for effec­tive char­i­ties. Ex­am­ple or­ga­ni­za­tions in­clude most of the Cen­tre for Effec­tive Altru­ism (Giv­ing What We Can, EA Outreach, EA Global, Chap­ters team), 80,000 Hours, most of .im­pact, Rais­ing for Effec­tive Giv­ing, Char­ity Science, Cen­ter for Ap­plied Ra­tion­al­ity, En­vi­sion, Stu­dents for High Im­pact Char­ity and lo­cal EA groups. How­ever, it does not in­clude GiveWell or the Foun­da­tional Re­search In­sti­tute.

What is this post not about?

I am not sug­gest­ing that we should put less money into meta; ac­tu­ally I think most meta-char­i­ties are do­ing great work and should get more money than they have. I am more wor­ried that in the fu­ture, meta or­ga­ni­za­tions will be funded more than they should be be­cause of lev­er­age ra­tio con­sid­er­a­tions that don’t take into ac­count all of the po­ten­tial down­sides.

Po­ten­tial Biases

Most of the work I do falls un­der “meta”—I help co­or­di­nate across lo­cal EA groups, and I run a lo­cal group my­self. As a re­sult, I’ve be­come pretty fa­mil­iar with the land­scape of cer­tain meta or­ga­ni­za­tions, which means that I’ve thought much more about meta than other cause ar­eas. I ex­pect that if I spent more time look­ing into other cause ar­eas, I would find is­sues with them as well that I don’t cur­rently know about.

I have be­come most fa­mil­iar with lo­cal EA groups, the CEA chap­ters team, Giv­ing What We Can, and 80,000 Hours, so most of my ex­am­ples fo­cus on those or­ga­ni­za­tions. I’m not try­ing to sin­gle them out or sug­gest that they suffer most from the is­sues I’m high­light­ing—it’s just eas­iest for me to use them as ex­am­ples since I know them well.

Sum­mary of con­sid­er­a­tions raised before

In Peter Hur­ford’s origi­nal post, there were sev­eral points made:

  1. Meta Orgs Risk Not Ac­tu­ally Hav­ing an Im­pact. Since meta or­ga­ni­za­tions are of­ten sev­eral lev­els re­moved from di­rect im­pact, if even one of the “chains of im­pact” be­tween lev­els fails to ma­te­ri­al­ize, the meta org will not have any im­pact.

  2. Meta Orgs Risk Curl­ing In On Them­selves. This is the failure mode where meta or­ga­ni­za­tions are op­ti­mized to spread the meta-move­ment but fail to ac­tu­ally have ob­ject-level im­pact.

  3. You Can’t Use Meta as an Ex­cuse for Cause In­de­ci­sive­ness. Donat­ing to meta or­ga­ni­za­tions al­lows you to skip the work of de­cid­ing which ob­ject-level cause is best, which is not good for the health of the move­ment.

  4. At Some Point, You Have to Stop Be­ing Meta. This is the worry that we do too much meta work and don’t get around to do­ing ob­ject-level work soon enough.

  5. Some­times, Well Ex­e­cuted Ob­ject-Level Ac­tion is What Best Grows the Move­ment. For ex­am­ple, GiveWell built an ex­cel­lent re­search product with­out fo­cus­ing on out­reach, but still ended up grow­ing the move­ment.

It seems to me that meta trap #4 is sim­ply a con­se­quence of sev­eral other traps—there are many meta traps that will cause us to over­es­ti­mate the value of meta work, and as a re­sult we may do more meta work than is war­ranted, and not do enough ob­ject-level work. So, I will be ig­nor­ing meta trap #4 for the rest of this post.

I’m also not wor­ried about meta trap #2, be­cause I think that meta or­ga­ni­za­tions will always have a “chain of im­pact” that bot­toms out with ob­ject-level im­pact. (Either you start a meta or­ga­ni­za­tion that helps an ob­ject-level or­ga­ni­za­tion, or you start a meta or­ga­ni­za­tion that helps some other or­ga­ni­za­tion that in­duc­tively even­tu­ally helps an ob­ject-level or­ga­ni­za­tion.) To fall into meta trap #2, at some point in the chain one of the or­ga­ni­za­tions would have to change di­rec­tion fairly dras­ti­cally to not have any ob­ject-level im­pact at all. How­ever, the mag­ni­tude of the ob­ject-level im­pact may be much smaller than we ex­pect. In par­tic­u­lar, this “chain of im­pact” is what causes meta trap #1.

The Chain of Impact

Let’s take the ex­am­ple of the Chap­ters team at the Cen­tre for Effec­tive Altru­ism (CEA), which is one of the most meta or­ga­ni­za­tions—they are aimed at helping and co­or­di­nat­ing lo­cal EA groups, which are helping to spread EA, which is both en­courag­ing more dona­tions to effec­tive char­i­ties and en­courag­ing more effec­tive ca­reer choice, which ul­ti­mately causes ob­ject-level im­pact. At each layer, there are sev­eral good met­rics which are pretty clear in­di­ca­tors that ob­ject-level im­pact is hap­pen­ing:

  • The Chap­ters team could use “num­ber of large lo­cal EA groups” or “to­tal GWWC pledges/​sig­nifi­cant plan changes from lo­cal groups”

  • Each lo­cal group could use “num­ber of GWWC pledges”, “num­ber of sig­nifi­cant plan changes”, and “num­ber of grad­u­ated mem­bers work­ing at EA or­ga­ni­za­tions”

  • GWWC and 80,000 Hours them­selves have met­rics to value a pledge and a sig­nifi­cant plan change, re­spec­tively.

  • The ob­ject-level char­i­ties that GWWC recom­mends have their own met­rics to eval­u­ate their ob­ject-level im­pact.

Like I said above, this “chain of im­pact” makes it pretty un­likely that meta or­ga­ni­za­tions will end up curl­ing in on them­selves and be­come de­tached from the ob­ject level. How­ever, a long chain does mean that there is a high risk of hav­ing no im­pact (meta trap #1). In fact, I would gen­er­al­ize meta trap #1:

Meta Trap #1. Meta orgs am­plify bad things too

The whole point of meta or­ga­ni­za­tions is to have large lev­er­age ra­tios through a “chain of im­pact”, which is usu­ally op­er­a­tional­ized us­ing a “chain of met­rics”. How­ever, this sort of chain can lead to other am­plifi­ca­tions as well:

Meta Trap #1a. Prob­a­bil­ity of not hav­ing an impact

This is the point made in the origi­nal post—the more meta an or­ga­ni­za­tion is, the more likely that one of the links in the chain fails and you don’t have any im­pact at all.

One coun­ter­ar­gu­ment is that of­ten, meta or­ga­ni­za­tions help many or­ga­ni­za­tions one level be­low them (eg. GWWC). In this case, it is ex­tremely un­likely that none of these or­ga­ni­za­tions have any im­pact, and so the risk is limited to just the risk that the meta or­ga­ni­za­tion it­self fails at in­tro­duc­ing enough ad­di­tional effi­ciency to jus­tify its costs.

Meta Trap #1b. Over­es­ti­mat­ing impact

It is plau­si­ble that or­ga­ni­za­tions sys­tem­at­i­cally over­es­ti­mate their im­pact (some­thing like the over­con­fi­dence effect). If most or all of the or­ga­ni­za­tions along the chain over­es­ti­mate their im­pact, then the or­ga­ni­za­tion at the top of the chain will have a vastly over­es­ti­mated ob­ject-level im­pact. Note that if an or­ga­ni­za­tion phrases its im­pact in terms of its im­pact on the or­ga­ni­za­tion di­rectly be­low them in the chain, this does not ap­ply to that num­ber. It only ap­plies to the to­tal es­ti­mated ob­ject-level im­pact of the or­ga­ni­za­tion.

You can get similar prob­lems with se­lec­tion effects, where the peo­ple who start meta or­ga­ni­za­tions are more likely to think that meta or­ga­ni­za­tions are worth­while. Each se­lec­tion effect leads to more over­con­fi­dence in the ap­proach, and once again the nega­tive effects of the bias grow as you go fur­ther up the chain.

Meta Trap #1c. Is­sues with metrics

A com­monly noted is­sue with met­rics is that an or­ga­ni­za­tion will op­ti­mize to do well on its met­rics, rather than what we ac­tu­ally care about. In this way, badly cho­sen met­rics can make us far less effec­tive.

As a con­crete ex­am­ple, let’s look at the CEA Chap­ters Team chain of met­rics:

  • One of many plau­si­ble met­rics for the CEA Chap­ters team is “num­ber of large lo­cal groups”. (I’m not sure what met­rics they ac­tu­ally use.) They may fo­cus on get­ting smaller lo­cal groups to grow, but this may re­sult in lo­cal groups hav­ing large speaker events and count­ing vaguely in­ter­ested stu­dents as “mem­bers” that re­sults in a clas­sifi­ca­tion of “large” even though not much has ac­tu­ally changed with that lo­cal group.

  • Lo­cal groups them­selves of­ten use “num­ber of GWWC pledges” as a met­ric. They may start to pro­mote the pledge very widely with ex­ter­nal in­cen­tives (eg. free food for event at­ten­dees, fol­lowed by so­cial pres­sure to sign the pledge). As a re­sult, more stu­dents may take the pledge, but they may be much more likely to drop out quickly.

  • GWWC has three met­rics on its home page—num­ber of pledges, amount of money already donated, and amount of money pledged. They may prefer­en­tially build a com­mu­nity for pledge tak­ers who already make money, since they will donate sooner and in­crease the amount of money already donated. Stu­dents may lose mo­ti­va­tion due to the lack of com­mu­nity and so would for­get about the pledge, and wouldn’t donate when the time came. GWWC would still be happy to have such pledges, be­cause they in­crease both the num­ber of pledges and the amount of money pledged.

  • The var­i­ous ob­ject-level or­ga­ni­za­tions that GWWC mem­bers donate to could have similar prob­lems. For ex­am­ple, per­haps the Against Malaria Foun­da­tion would fo­cus on ar­eas where bed­nets can be bought and dis­tributed cheaply, giv­ing a bet­ter “cost per net” met­ric, rather than spend­ing slightly more to dis­tribute nets in re­gions with much higher malaria bur­dens, where the money would save more lives.

If all of these were true, you’d have to ques­tion whether the CEA Chap­ters team is hav­ing much of an im­pact at all. On the other hand, even though the met­rics for AMF lead to sub­op­ti­mal be­hav­ior, you can still be quite con­fi­dent that they have a sig­nifi­cant im­pact.

I don’t think that these are true, but I could be­lieve weaker ver­sions (in par­tic­u­lar, I worry that GWWC pledges from lo­cal groups are not as good as the av­er­age GWWC pledge).

In ad­di­tion to the 5 traps that Peter Hur­ford men­tions, I be­lieve there are other meta traps that tend to make us over­es­ti­mate the im­pact of meta work:

Meta Trap #6. Marginal im­pact may be much lower than av­er­age im­pact.

The GiveWell top char­i­ties gen­er­ally fo­cus on im­ple­ment­ing a sin­gle in­ter­ven­tion. As a re­sult, we can roughly ex­pect that the marginal im­pact of a dol­lar donated is about the same as the av­er­age im­pact of a dol­lar, since both dol­lars go to the same in­ter­ven­tion. It’s still likely lower (for ex­am­ple, AMF may start fund­ing bed­nets in ar­eas with lower malaria bur­den, re­duc­ing marginal im­pact), but not that much lower.

How­ever, meta or­ga­ni­za­tions typ­i­cally have many dis­tinct ac­tivi­ties for the same goal. Th­ese ac­tivi­ties can have very differ­ent cost-effec­tive­ness. The marginal dol­lar will typ­i­cally fund the ac­tivity with the low­est (es­ti­mated) cost-effec­tive­ness, and so will likely be sig­nifi­cantly less im­pact­ful than the av­er­age dol­lar.

Note that this as­sumes that the ac­tivi­ties are not sym­biotic—that is, the ar­gu­ment only works if stop­ping one of the ac­tivi­ties would not sig­nifi­cantly af­fect the cost-effec­tive­ness of other ac­tivi­ties. As a con­crete ex­am­ple of such ac­tivi­ties in a meta or­ga­ni­za­tion, see this com­ment about all the ac­tivi­ties that get pledges for GWWC.

The num­bers that are typ­i­cally pub­li­cized are for av­er­age im­pact. For ex­am­ple, on av­er­age, GWWC claims a 104:1 mul­ti­plier, or 6:1 for their pes­simistic calcu­la­tion, and claims that the av­er­age pledge is worth $73,000. On av­er­age the cost to 80,000 Hours of a sig­nifi­cant plan change is £1,667. (Th­ese num­bers are out­dated and will prob­a­bly be re­placed by newer ones soon.) What about the marginal im­pact of the next dol­lar? I have no idea, ex­cept that it’s prob­a­bly quite a bit worse than the av­er­age. I would not be sur­prised if the marginal dol­lar didn’t even achieve a 1:1 mul­ti­plier un­der GWWC’s as­sump­tions for their pes­simistic im­pact calcu­la­tion. (But their pes­simistic im­pact calcu­la­tion is re­ally pes­simistic.) To be fair to meta or­ga­ni­za­tions, marginal im­pact is a lot harder to as­sess than av­er­age im­pact. I my­self fo­cus on av­er­age im­pact when mak­ing the case for lo­cal EA groups be­cause I ac­tu­ally have a rea­son­able es­ti­mate for those num­bers.

One coun­ter­ar­gu­ment against this gen­eral ar­gu­ment is that each ad­di­tional ac­tivity fur­ther shares fixed costs, mak­ing ev­ery­thing more cost-effec­tive. For ex­am­ple, GWWC has to main­tain a web­site re­gard­less of which ac­tivi­ties it runs. If it adds an­other ac­tivity to get more pledges that re­lies on the web­site, then the cost of main­tain­ing the web­site is spread over the new ac­tivity as well, mak­ing the other ac­tivi­ties more cost effec­tive. Similar con­sid­er­a­tions could ap­ply to office space costs, le­gal fees, etc. I would guess that this is much less im­por­tant than the in­her­ent differ­ence in cost effec­tive­ness of differ­ent ac­tivi­ties.

Meta Trap #7. Meta suffers more from co­or­di­na­tion prob­lems.

As the move­ment grows and more peo­ple and meta or­ga­ni­za­tions be­come a part of it, it be­comes more im­por­tant to con­sider the group as a whole, rather than the im­pact of just your ac­tions. This 80,000 Hours post ap­plies this idea to five differ­ent situ­a­tions.

Espe­cially in the area of pro­mot­ing effec­tive al­tru­ism in the ab­stract, there are a lot of or­ga­ni­za­tions work­ing to­ward the same goal and tar­get­ing the same peo­ple. For ex­am­ple, per­haps Alice learns about EA from a lo­cal EA group, goes to a CFAR work­shop, starts a com­pany be­cause of 80,000 Hours and takes the Founder’s Pledge and GWWC pledge. We now have five differ­ent or­ga­ni­za­tions that can each claim credit for Alice’s im­pact, not to men­tion Alice her­self. In ad­di­tion, from a “sin­gle player coun­ter­fac­tual anal­y­sis”, it is rea­son­able for all the or­ga­ni­za­tions to at­tribute nearly all the im­pact to them­selves—if Alice was talk­ing to these or­ga­ni­za­tions while mak­ing her de­ci­sion, each or­ga­ni­za­tion could sep­a­rately con­clude that Alice would not have made these life changes with­out them, and so coun­ter­fac­tu­ally they get all the credit. (And this could be a rea­son­able con­clu­sion by each or­ga­ni­za­tion.) How­ever, the to­tal im­pact caused would then be smaller than the sum of the im­pacts each or­ga­ni­za­tion thinks they had.

Imag­ine that Alice will now have an ad­di­tional $2,000 of im­pact, and each or­ga­ni­za­tion spent $1,000 to ac­com­plish this. Then each or­ga­ni­za­tion would (cor­rectly) claim a lev­er­age ra­tio of 2:1, but the ag­gre­gate out­come is that we spent $5,000 to get $2,000 of benefit, which is clearly sub­op­ti­mal. Th­ese num­bers are com­pletely made up for ped­a­gog­i­cal pur­poses and not meant to be ac­tual es­ti­mates. In re­al­ity, even in this sce­nario I sus­pect that the ra­tio would be bet­ter than 1:1, though it would be smaller than the ra­tio each or­ga­ni­za­tion would com­pute for it­self.

Note that the re­cent changes at CEA have helped with this prob­lem, but it still mat­ters.

Meta Trap #8. Coun­ter­fac­tu­als are harder to as­sess.

It’s very un­clear what would hap­pen in the ab­sence of meta or­ga­ni­za­tions—I would ex­pect the EA move­ment to grow any­way sim­ply by spread­ing through word of mouth, but I don’t know how much. If Giv­ing What We Can didn’t ex­ist, per­haps EA would grow at the same rate and EAs would pub­li­cize their dona­tions on the EA Hub. If 80,000 Hours didn’t ex­ist, per­haps some peo­ple would still make effec­tive ca­reer choices by talk­ing to other EAs about their ca­reers. It is hard to prop­erly es­ti­mate the coun­ter­fac­tual for im­pact calcu­la­tions—for ex­am­ple, GWWC asks pledge tak­ers to self-re­port their coun­ter­fac­tual dona­tions, which is fraught with un­cer­tain­ties and bi­ases, and as far as I know 80,000 Hours does not try to es­ti­mate the im­pact that a per­son would have had be­fore their plan change.

This isn’t a trap in and of it­self—it be­comes a trap when you com­bine it with bi­ases that lead us to over­es­ti­mate how much coun­ter­fac­tual im­pact we have by pro­ject­ing the coun­ter­fac­tual as worse than it ac­tu­ally would have been. We should care about this for the same rea­sons that we care about ro­bust­ness of ev­i­dence.

I think that with most ob­ject-level causes this is less of an is­sue. When RCTs are con­ducted, they elimi­nate the prob­lem, at least in the­ory (though you do run into prob­lems when try­ing to gen­er­al­ize from RCTs to other en­vi­ron­ments). I think that this is a prob­lem in far fu­ture ar­eas (would the ex­is­ten­tial risk have hap­pened, or would it have been solved any­way?), but peo­ple are aware of the prob­lem and tackle it (re­search into the prob­a­bil­ities of var­i­ous ex­is­ten­tial risks, look­ing for par­tic­u­larly ne­glected ex­is­ten­tial risks such as AI risk). I haven’t seen any­thing similar for meta or­ga­ni­za­tions.

What should we do?

Like I men­tioned be­fore, I’m not wor­ried about meta traps #2 and #4. I agree that meta traps #3 (cause in­de­ci­sive­ness) and #5 (ob­ject-level ac­tion as a meta strat­egy) are im­por­tant but I don’t have con­crete sug­ges­tions for them, other than mak­ing sure that peo­ple are aware of them.

For meta trap #1, I agree with Peter’s sug­ges­tion: The more steps away from im­pact an EA plan is, the more ad­di­tional scrutiny it should get. In ad­di­tion, I like 80,000 Hours’ policy of pub­lish­ing some spe­cific, in­di­vi­d­ual sig­nifi­cant plan changes that they have caused. Look­ing at the de­tails of in­di­vi­d­ual cases makes it clear what sort of im­pact the or­ga­ni­za­tion has un­der­neath all the met­rics, and ideally would di­rectly show the ob­ject-level im­pact that the or­ga­ni­za­tion is caus­ing (even if the mag­ni­tude is un­clear). It seems to me that most meta or­ga­ni­za­tions can do some ver­sion of this.

I worry most about meta trap #6 (marginal vs. av­er­age im­pact). It also ap­plies to an­i­mal welfare or­ga­ni­za­tions, but I’m less wor­ried there be­cause An­i­mal Char­ity Eval­u­a­tors (ACE) does a good job of think­ing about that con­sid­er­a­tion deeply, cre­at­ing cost-effec­tive­ness es­ti­mates for each ac­tivity, and bas­ing its recom­men­da­tions on that. We could cre­ate “Meta Char­ity Eval­u­a­tors”, but I’m not con­fi­dent that this is ac­tu­ally worth­while, given that there are rel­a­tively few meta or­ga­ni­za­tions, and not much fund­ing flows to them. How­ever, this is similar to the case for ACE, so there’s some rea­son to do this. This could also help with meta trap #7 (co­or­di­na­tion prob­lems), if it took on co­or­di­na­tion as an ex­plicit goal.

I would guess that we don’t need to do any­thing about meta trap #8 (hard coun­ter­fac­tu­als) now. I think most meta or­ga­ni­za­tions have fairly strong cases for large coun­ter­fac­tual im­pact. I would guess that we could make good progress by re­search into what the coun­ter­fac­tu­als are, but again since there is not much fund­ing for meta or­ga­ni­za­tions now, this does not seem par­tic­u­larly valuable.