2017 LEAN Impact Assessment: Evaluation & Strategic Conclusions

This is the third post in the LEAN Im­pact Assess­ment se­ries.

Quan­ti­ta­tive Findings

Qual­i­ta­tive Findings

Eval­u­a­tion & Strate­gic Conclusions

Methodology

In the pre­vi­ous Quan­ti­ta­tive and Qual­i­ta­tive re­ports we aimed to de­scribe our find­ings with min­i­mal com­men­tary in or­der to al­low the data to ‘speak for it­self’ and al­low read­ers to draw their own im­pres­sions with­out be­ing in­fluenced by our own in­ter­pre­ta­tions and con­clu­sions. In this re­port we aim to offer more in­ter­pre­ta­tion and anal­y­sis of the im­pli­ca­tions of our find­ings for LEAN strat­egy and for EA lo­cal groups.

The LEAN Im­pact Assess­ment has aimed to provide in­sight both into the sta­tus and value of EA lo­cal groups as a whole, and the effi­cacy of efforts (by LEAN and oth­ers) to sup­port these groups. Th­ese top­ics oc­cupy the first two sec­tions of this re­port, re­spec­tively, and in the fi­nal sec­tion we out­line LEAN’s strate­gic plans formed in the light of these find­ings.

Our find­ings rep­re­sent only a first step in re­search­ing EA Lo­cal Groups and we plan to con­duct fur­ther, more spe­cific, re­search into lo­cal groups and LEAN’s ser­vices in the fu­ture.

Ex­ec­u­tive Summary

Group Size and Activity

EA Sur­vey Data

Age of Groups

Impact

Is it all the largest groups?

Fur­ther Ev­i­dence of Group Impact

Value of ex­ist­ing pro­gramme ser­vices and resources

Alter­na­tive An­a­lytic Strategies

Eval­u­a­tions of Group Sup­port and Services

Tech­ni­cal support

Per­sonal Sup­port and Expertise

Group Communication

Group Calls

EA Group Newsletter

EA Men­tor­ing Programme

Strate­gic Summary

Acknowledgments

Ex­ec­u­tive Summary

  • EA groups re­port pro­duc­ing sig­nifi­cant im­pact e.g. coun­ter­fac­tual pledges and dona­tions influenced

  • Most mem­bers re­port that groups are a large or very large fac­tor in their en­gage­ment with EA

  • Or­ganisers ex­press a strong de­mand for per­sonal sup­port and feed­back, writ­ten guides and re­search on group impact

Group Size and Activity

The LEAN Im­pact Assess­ment offers the most com­pre­hen­sive em­piri­cal in­ves­ti­ga­tion into EA lo­cal groups to date. Prior to the LEAN Im­pact Assess­ment there was lit­tle, if any, sys­tem­atic data about EA lo­cal groups as a whole, though min­i­mal re­lated data was gath­ered through the EA Sur­vey (also run by Re­think Char­ity). As such, the first stage was to provide ba­sic in­for­ma­tion about the num­ber and size of EA groups and their ac­tivi­ties.

Our sam­ple (dis­sem­i­nated with help from CEA and EAF) in­cluded 98 dis­crete lo­cal groups and a larger num­ber of or­ganisers and mem­bers. This sam­ple likely does not in­clude all EA groups. Nor did re­spon­dents an­swer ev­ery ques­tion. How­ever, it seems rea­son­able to as­sume that non-re­spon­dents were, on the less, dis­pro­por­tionately less ac­tive groups or groups who had less im­pact to re­port for par­tic­u­lar met­rics.

Re­ported group size ranged from 1 to 1350, with a me­dian of 10. This means that the re­ported group size num­bers are heav­ily dom­i­nated by a small num­ber of very large groups. Of a to­tal 4280 re­ported group mem­bers, across all groups, more than half the re­ported mem­bers come from the largest 3 groups, al­most 70% from the largest 5 groups and al­most 78% from the largest 10 groups. By con­trast, more than 50% of groups con­tained 10 mem­bers or fewer and slightly more than 76% re­ported 20 mem­bers or fewer.

As noted pre­vi­ously, the ex­tent to which a small num­ber of very large groups ac­count for al­most all EA lo­cal group mem­bers is likely some­what over­stated due to differ­ences in how group “mem­bers” were counted by differ­ent or­ganisers. For ex­am­ple, the num­ber of group mem­bers re­ported by the largest group in our sam­ple cor­re­sponds to the num­ber of group mem­bers they have in their Face­book group, but their re­ported num­ber of peo­ple who have at­tended mul­ti­ple of their events is much lower (>1300 and 160, re­spec­tively). While it is pos­si­ble that smaller groups may also be ‘over-re­port­ing’ their num­bers in this way, it does not seem pos­si­ble that the smaller groups could be in­flat­ing their num­bers to the same ex­tent (un­less we sup­pose that the ma­jor­ity of groups have 1 mem­ber or less).

Ac­count­ing for this, the gap be­tween the typ­i­cal group and the very largest groups is less as­tro­nom­i­cal, though still sub­stan­tial. Of course, the num­ber of group mem­bers is not in it­self im­por­tant, but rather the im­pact that each group can have, which fol­lows later in the re­port.

EA Sur­vey Data

It is worth briefly com­par­ing this data with the re­sults of the EA Sur­vey on lo­cal group mem­ber­ship, to see whether they are plau­si­ble, and to gain in­sight from a dis­tinct source on the scale and scope of EA lo­cal groups. We would not ex­pect these num­bers to match up par­tic­u­larly closely, given the differ­ent ques­tions asked and very differ­ent sam­pling tech­niques (the EA Sur­vey was dis­tributed widely and an­swered by EAs and non-EAs al­ike and had sev­eral thou­sand re­sponses), the Lo­cal Groups Sur­vey was dis­tributed di­rectly to group or­ganisers (as well as posted in rele­vant face­book groups) and was speci­fi­cally for lo­cal group or­ganisers and mem­bers. Nev­er­the­less it seems worth­while to com­pare the re­sults across these sur­veys to see whether EAs in a broader sam­ple re­port be­ing in­fluenced by EA groups, rather than in a sam­ple speci­fi­cally tar­get­ing group or­ganisers and mem­bers.

The EA Sur­vey found 469 EA re­spon­dents in­di­cat­ing that “Yes” they were a mem­ber of a lo­cal group, com­pared to 953 an­swer­ing “No” and 430 non-re­sponses for that ques­tion. This means slightly more than 25% of EA re­spon­dents (in­clud­ing those who did not an­swer that ques­tion at all in or­der to give a more con­ser­va­tive es­ti­mate) re­ported be­ing in an EA group.

We can­not as­sume this pro­por­tion ap­plies across the whole EA pop­u­la­tion though, since it seems plau­si­ble that those EAs who an­swer the sur­vey may be more likely to be in an EA group (as­sum­ing they are gen­er­ally more ac­tive and con­nected). On the other hand, the ab­solute num­ber of EAs in lo­cal groups is likely to be higher than the ab­solute count re­ported here, since the EA Sur­vey doubtless did not in­clude all EAs. Re­gret­tably, no-one knows pre­cisely how many EAs (by any defi­ni­tion) there ac­tu­ally are in the EA pop­u­la­tion (or whether or how far EA is grow­ing). Nev­er­the­less, al­most 500 self-re­ported EA lo­cal group mem­bers from the EA Sur­vey sam­ple of ~1800 seems an ap­pre­cia­ble num­ber.

The EA Sur­vey also pro­vided data on whether in­di­vi­d­u­als re­ported that they would at­tend an EA group if there were one near their home. In ad­di­tion to the 469 EAs who said they do at­tend an EA lo­cal group, a fur­ther 271 re­ported that they would if there was one near their home and a fur­ther 310 who do not at­tend sug­gested that they were “un­sure” whether they would at­tend or not. This is sug­ges­tive of fairly sig­nifi­cant de­mand for EA lo­cal groups from in­di­vi­d­u­als who presently can­not at­tend one due to lack­ing one nearby.

The EA Sur­vey also offers data re­gard­ing the im­por­tance of EA lo­cal groups in get­ting EAs into and more in­volved in EA. 3.6% of the sam­ple re­ported first hear­ing about EA from a lo­cal group, slightly ex­ceed­ing Do­ing Good Bet­ter (3.4%) and Face­book (2.8%). Due to the wide va­ri­ety of differ­ent an­swers in­di­cated in the sur­vey few ‘routes’ re­ceived very high per­centages (the high­est were Per­sonal Con­tact and LessWrong on 15.5% and 15.3% re­spec­tively, fol­lowed by ‘Other blog post’ and SlateS­tarCodex on 9.4% and 7.4%).


A sur­vey sam­ple drawn speci­fi­cally from the EA Face­book group (us­ing differ­ent method­ol­ogy) re­turned higher num­bers re­port­ing that they first heard about EA from an EA lo­cal group. Here 7% re­ported hear­ing about EA first from a lo­cal group, beaten only by Friends (15%) Peter Singer/​TLYCS (13%) and 80,000 Hours (13%). In­clud­ing the 5% who re­ported first hear­ing about EA from a uni­ver­sity group, the to­tal % hear­ing about EA from groups was 12% (much closer to the top op­tions) and com­fortably beat­ing the other op­tions, in­clud­ing Face­book, SlateS­tarCodex, LessWrong and Do­ing Good Bet­ter.

Th­ese find­ings are sug­ges­tive of EA lo­cal groups play­ing a sig­nifi­cant role as the first place that many EAs en­counter EA, albeit as one among a wide va­ri­ety of differ­ent sources each mak­ing up a minor­ity of the whole.

The EA Sur­vey also offers data re­gard­ing how many EAs thought that EA groups were im­por­tant in get­ting them into or more in­volved with EA. Here re­spon­dents could se­lect mul­ti­ple op­tions as each be­ing im­por­tant. 261 re­spon­dents in­di­cated that lo­cal groups were im­por­tant for get­ting them more in­volved in EA, whereas the most com­monly cited fac­tors GiveWell and ‘Book or blog’ were cited 532 and 519 times. 261 EAs in the sam­ple get­ting more in­volved in EA seems like a sig­nifi­cant source of po­ten­tial im­pact. It is sig­nifi­cantly smaller than the num­ber cit­ing other fac­tors, but this is pre­sum­ably some­what due to the fact that al­most ev­ery EA has prob­a­bly en­coun­tered GiveWell or an EA book or blog, whereas likely fewer than 50% of EAs have en­coun­tered a lo­cal group.

Over­all, we take this find­ings from the EA Sur­vey to offer re­as­surance that EA groups are reach­ing and in­fluenc­ing a sub­stan­tial num­ber of EAs, even in a broader sam­ple not speci­fi­cally tar­get­ing group or­ganisers.

Age of Groups

Read­ers may be con­cerned by the fact that so many in­di­vi­d­ual groups ap­pear to be so small (for ex­am­ple, 985 con­tain fewer than 4 mem­bers). Like­wise, look­ing at var­i­ous met­rics of im­pact, it is re­li­ably the case that a num­ber of groups re­port lit­tle im­pact. If many EA groups are largely quies­cent or in­effi­ca­cious, then this may be a mat­ter for con­cern.

To try to un­der­stand this bet­ter, we looked at how long groups had been run­ning. A large num­ber of EA groups re­ported be­ing very new.

This may be re­as­sur­ing con­text when con­sid­er­ing why some groups have not had much of an im­pact. They may not have had much im­pact yet, but many groups have not even seen a com­plete aca­demic year or two sea­sonal pledge drives. This may be a ne­glected fea­ture to con­sider in EA move­ment build­ing dis­cus­sions: a large num­ber of EA groups (see the left hand side of the chart above) are still in their in­fancy and in the com­ing years may be com­ing into their own.

If we look at the num­ber of groups of cer­tain sizes at differ­ent ages, we see that while a ma­jor­ity (68%) of groups founded less than a year ago have fewer than 10 mem­bers, this pro­por­tion de­clines sub­stan­tially for older groups (38% for groups 1-2 years old, 33% for groups 2-3 years old, and 14% for groups 3-4 years old).

Similarly, the me­dian group size in­creases with group age (we have ex­cluded the 5-7 year old groups from this graph be­cause with only 2 mem­bers in these cat­e­gories, the me­dian is fairly un­indica­tive):

A con­se­quence of this is that even though most groups are younger groups (58% less than 2 years old, >80% less than 3 years old), most mem­bers are in older groups.

Another pos­si­bil­ity we con­sid­ered is that EA group size and ac­tivity might be quite vari­able across time, with groups, for ex­am­ple, be­ing mod­er­ately ac­tive with a num­ber of mem­bers, then fal­ling away when a core or­ganiser leaves or in be­tween aca­demic years, and then gain­ing more mem­bers once an aca­demic year has started. If so, then while at any one time (such as the snap­shot our sur­vey pro­vides), a num­ber of groups may ap­pear to be all but non-ex­is­tent, the same groups may range across time through pe­ri­ods of ac­tivity and in­ac­tivity. Our qual­i­ta­tive data also in­cluded city-based group or­ganisers re­port­ing that they had high turnover of young pro­fes­sional mem­bers who would stay in a city only a short pe­riod of time be­fore mov­ing. This is partly sug­gested by our data on the per­ceived ‘pre­car­ity’ of groups, with a sig­nifi­cant minor­ity (30/​89) of or­ganisers think­ing their group was un­likely or very un­likely to con­tinue to func­tion af­ter the cur­rent or­ganisers left. This is a trope which we heard from a num­ber of or­ganisers, where groups ac­tu­ally fell into abeyance af­ter pre­vi­ously be­ing very ac­tive. We presently lack data on how com­mon this is, but this, and how to han­dle tran­si­tions bet­ter, as well as how to keep groups could be kept ac­tive, is a po­ten­tial topic for fu­ture re­search.

Impact

Calcu­lat­ing the im­pact of lo­cal groups is ex­ceed­ingly difficult in a num­ber of ways. EA groups aim to have an im­pact in a va­ri­ety of differ­ent ways: at­tract­ing new mem­bers to EA, en­courag­ing mem­bers to be­come more en­gaged with EA (through pro­vid­ing so­cial con­nec­tion, mo­ti­va­tion and in­for­ma­tion) and (di­rectly and in­di­rectly) en­courag­ing mem­bers to take more effec­tive ac­tions (such as donat­ing to effec­tive char­i­ties, tak­ing the pledge, con­sid­er­ing their ca­reer based on EA prin­ci­ples), and in­creas­ing mem­ber re­ten­tion.

Some of these things are in­trin­si­cally difficult to mea­sure and quan­tify. For ex­am­ple, many EAs re­port that be­ing in a lo­cal group in­creases their mo­ti­va­tion and en­gage­ment with EA. Some im­pacts which we think EA groups are likely to have, are all but im­pos­si­ble to mea­sure. It seems plau­si­ble that EA groups con­tribute to re­ten­tion of EAs, as some EAs re­port that they likely would have left the move­ment but for their in­volve­ment in a lo­cal group: but EAs who do leave the move­ment and their rea­sons for leav­ing are typ­i­cally in­ac­cessible to EA data-gath­er­ing, as, a for­tiori, are peo­ple who never join the move­ment but would have, coun­ter­fac­tu­ally, had they en­coun­tered a lo­cal group. As such, much of the true im­pact of lo­cal groups may be ex­cluded from our anal­y­sis.

Fur­ther, it is ex­ceed­ingly difficult to dis­cern the coun­ter­fac­tu­al­ity of each of these out­puts. While we can ask mem­bers and or­ganisers for their self-re­ported es­ti­ma­tions of what they would have done were they not in a group, it will of­ten be hard for them to gen­uinely know. Many re­port that they would not have taken cer­tain ac­tions were they not in a lo­cal group, but coun­ter­fac­tu­ally, it is pos­si­ble that they would have been moved by some­thing else to, for ex­am­ple, take the pledge had it been im­pos­si­ble for them to join a group. Con­versely, an in­di­vi­d­ual in a group may feel con­fi­dent that they would have con­tinued to take effec­tive ac­tions if they weren’t in­volved in a lo­cal group, but it is pos­si­ble that they may be mis­taken about this. This is par­tic­u­larly difficult given the wide va­ri­ety of in­fluences on any given EA’s ac­tions: their in­ter­ac­tions with EAs face to face in a lo­cal group, their con­tact with other EAs on­line, their read­ing EA liter­a­ture, their at­ten­dance at an EA con­fer­ence, may all in­fluence their EA ac­tions and at­ti­tudes and which of these fac­tors are im­por­tant may vary sub­stan­tially across differ­ent in­di­vi­d­u­als and the in­fluence of all of these fac­tors may not be trans­par­ent even to the in­di­vi­d­ual in ques­tion.

Like­wise, EAs may rea­son­ably dis­agree quite rad­i­cally about how much value to at­tach to each of these things (for ex­am­ple, the value of a given per­son tak­ing the Giv­ing What We Can pledge) or get­ting an ad­di­tional per­son in­volved in the EA move­ment. We do not aim to set­tle such fun­da­men­tal de­bates here, but rather re­port figures for a va­ri­ety of met­rics which will be con­sid­ered of value to a wide range of EAs with differ­ent val­ues.

Below we re­port figures for the col­lec­tive ‘out­puts’, for var­i­ous mea­sures, across all the groups in our sam­ple:

To­tal mem­bers:

4280

To­tal events, mee­tups, etc.:

1835

To­tal new event at­ten­dees (un­fa­mil­iar with EA):

5994

To­tal coun­ter­fac­tual ac­tive com­mit­ments to EA:

617

To­tal coun­ter­fac­tual pledges:

227.5

To­tal EA in­formed ca­reer de­ci­sions:

517.5

To­tal funds raised (group fundraisers):

$51,495.36

To­tal coun­ter­fac­tual pri­vate dona­tions in­fluenced:

$704,390.92

Clearly these mea­sures can­not cap­ture all the im­pact of EA groups, since many po­ten­tial effects of EA groups were not or could not be mea­sured by the sur­vey e.g. less tan­gible benefits re­sult­ing from re­duc­ing EA at­tri­tion, value drift or of con­nect­ing EA group or­ganisers to valuable op­por­tu­ni­ties.

Even look­ing solely at the es­ti­mated (by group or­ganisers) coun­ter­fac­tual im­pact of lo­cal EA groups, these figures seem quite sub­stan­tial. As, at pre­sent, most EA groups are run by un­paid vol­un­teers with lit­tle or no fund­ing, the di­rect costs as­so­ci­ated with lo­cal groups are quite low. The costs as­so­ci­ated with vol­un­teer or paid staff time, are likely to be larger, but here there will be higher var­i­ance. For many or­ganisers the op­por­tu­nity costs of their time spent run­ning an EA group may be low and there may even be net per­sonal gains from their in­volve­ment in terms of ex­pe­rience, ca­reer cap­i­tal, boosts to mo­ti­va­tion and con­nec­tion with oth­ers in EA. Of course, for other EAs, with higher im­pact el­se­where, run­ning a group would be a net loss. We ex­pect that, for the most part, in­di­vi­d­ual or­ganisers may be best placed to make these judge­ments them­selves. We do, how­ever, think that fur­ther re­search into the ac­tivi­ties and op­por­tu­nity costs of EA group or­ganisers would be valuable.

Hav­ing looked at the to­tal out­puts across all EA lo­cal groups it is, of course, im­por­tant to try to look at this more at an in­di­vi­d­ual group level. The mean im­pact per group con­tinues to ap­pear to be quite high, but these num­bers are driven up­wards by a small num­ber of very suc­cess­ful groups.

There­fore we pre­sent the me­dian figures across a va­ri­ety of met­rics:

Me­dian num­ber of mem­bers:

10

Me­dian new event at­ten­dees (un­fa­mil­iar with EA):

30

Me­dian coun­ter­fac­tual ac­tive com­mit­ments to EA:

5

Me­dian coun­ter­fac­tual pledges:

3

Me­dian EA in­formed ca­reer de­ci­sions:

4

Me­dian funds raised col­lec­tively (group fundrais­ing):

$232.11

Me­dian coun­ter­fac­tual pri­vate dona­tions in­fluenced:

$1386.4

While these figures are much lower than the figures re­ported by the most suc­cess­ful groups on each met­ric or the mean figures, they still seem plau­si­bly to rep­re­sent sig­nifi­cant im­pact. As noted, it is im­pos­si­ble to at­tach a defini­tive value to these var­i­ous out­puts, but 3 coun­ter­fac­tual GWWC pledges, and 5 peo­ple be­com­ing coun­ter­fac­tu­ally ac­tively com­mit­ted to EA may rep­re­sent sig­nifi­cant im­pact.

Of course, look­ing at the me­dian figures across differ­ent met­rics only cap­tures one facet of the dis­tri­bu­tion of im­pact across differ­ent in­di­vi­d­ual groups. If one wants to get a pic­ture of how much im­pact a ‘typ­i­cal’ ac­tive EA group’s im­pact, one might think that the me­dian figures un­der-rep­re­sent this, given that it in­cludes a num­ber of groups who are very new or quies­cent (hav­ing one or­ganiser, but no group). For ex­am­ple, if you want to know how much groups typ­i­cally raise through group fundraisers, you might want to ex­clude from these figures the many groups which didn’t even at­tempt to run a fundraiser. By con­trast, if you want to es­ti­mate the ex­pected value of start­ing a group, this might be a more ap­pro­pri­ate met­ric; or you might want to iden­tify a more spe­cific refer­ence class by look­ing at groups in com­pa­rable situ­a­tions (e.g. in cities which you judge to be similar) and draw in­fer­ences from that. We are wary about dig­ging into these differ­ent ways of rep­re­sent­ing the data too much, be­cause it in­tro­duces too much free­dom for ju­di­cious se­lec­tion flat­ter­ing re­sults, and we recom­mend that the reader looks at the spe­cific re­sults we re­port in pre­vi­ous sec­tions.

Is it all the largest groups?

One of the trends that ap­peared most clearly in the re­sults we posted in the quan­ti­ta­tive data re­port, was how far the to­tal figures (e.g. for mem­bers or money raised) ap­peared to be dom­i­nated by a small num­ber of ‘su­per groups.’ Much of our data ex­hibited a very strong power law dis­tri­bu­tion.

We think this is likely to lead to the im­pres­sion that since it seems al­most all the im­pact from lo­cal groups, as a whole, is com­ing from a small num­ber of ‘top’ groups, this is where at­ten­tion and in­vest­ment should be fo­cused. This is worth con­sid­er­ing fur­ther.

Firstly, it should be noted that strate­gi­cally this doesn’t nec­es­sar­ily fol­low. If we can, at very low cost (for ex­am­ple, time val­ued at $500), cause a smaller group to bring about 3 coun­ter­fac­tual pledges (val­ued at, say, 3 x $10,000 each), this may rep­re­sent a bet­ter in­vest­ment than try­ing to help a large group pro­duc­ing 100 pledges pro­duce yet more pledges, as even though their mean im­pact may great, pro­duc­ing ex­tra marginal im­pact may be very costly. Even if we sup­pose that the num­ber of mem­bers in a group is all that mat­ters (for pro­duc­ing im­pact), it may be that it is most effec­tive to fo­cus in­vest­ment in the smaller groups, in or­der to in­crease their size to that of the larger groups. This may hold true for other met­rics as well. Sup­pose we are in­ter­ested in re­cruit­ing top EA tal­ent. It may be more likely that tal­ented in­di­vi­d­u­als will come from larger groups (since there are more mem­bers), but it may be that such in­di­vi­d­u­als would likely be found any­way (due to already be­ing sur­rounded by and con­nected with many EAs in an EA Hub). It might there­fore be more im­por­tant to sup­port smaller lo­cal groups in con­nect­ing EAs with promis­ing op­por­tu­ni­ties, as they are more likely to be missed. None of this is in­tended as an ar­gu­ment that smaller groups ac­tu­ally are more im­por­tant to in­vest in, merely that it does not au­to­mat­i­cally fol­low that if cer­tain groups pro­duce most im­pact, it is more im­por­tant to at­tend to and in­vest in these groups rel­a­tive to the more nu­mer­ous smaller groups.

How­ever, there is also more to be said in terms of un­der­stand­ing the data and which groups are pro­duc­ing most im­pact. Given the strik­ing dis­tri­bu­tions of group size and var­i­ous mea­sures of im­pact de­scribes above, it would be tempt­ing to con­clude that it is the largest groups (with the most mem­bers) that are pro­duc­ing most of the im­pact. Fur­ther, one might sus­pect that it is largely sim­ply hav­ing more mem­bers that drives in­creases suc­cess on var­i­ous met­rics (more coun­ter­fac­tual pledges, more dona­tions in­fluenced and so on).

Fur­ther con­sid­er­a­tion of the data sug­gests that this is only some­what true. As noted in the quan­ti­ta­tive re­port, for most mea­sures there was a pos­i­tive cor­re­la­tion be­tween num­ber of group mem­bers and var­i­ous out­puts, how­ever this varied. For ex­am­ple, there was a strong cor­re­la­tion be­tween num­bers of group mem­bers bas­ing their ca­reer on EA prin­ci­ples, but lit­tle re­la­tion­ship be­tween the size of a group and the num­ber of new at­ten­dees to events who were un­fa­mil­iar with EA.

More­over, as we noted in the quan­ti­ta­tive re­port, the data did not sug­gest that larger groups were bet­ter at pro­duc­ing out­puts from their mem­bers (e.g. get­ting mem­bers to make ca­reer choices based on EA) - in fact, there ap­peared to be a weak nega­tive re­la­tion­ship be­tween the size of a group and the pro­por­tion of mem­bers who were bas­ing ca­reer choices on EA or tak­ing the pledge (plau­si­bly ex­plained by the fact that larger groups may con­tain rel­a­tively more new mem­bers and smaller groups may con­tain, as a pro­por­tion of their to­tal size, rel­a­tively more core or­ganisers).

It is also im­por­tant to note that these cor­re­la­tions and var­i­ous pos­i­tive out­puts do not nec­es­sar­ily sug­gest that hav­ing more group mem­bers causes higher im­pact in terms of pledges taken, funds raised etc. It may be that a third fac­tor (group ac­tivity, pro­pi­tious en­vi­ron­ment, good or­gani­sa­tion) re­li­ably causes groups to be larger and to have more pledges, funds raised etc.

To gain more in­sight into these ques­tions we looked at the high­est perform­ing groups across differ­ent met­rics (e.g. num­ber of mem­bers, amount raised in group fundraisers, num­ber of mem­bers be­com­ing ac­tively com­mit­ted to EA, num­ber of pledges, num­ber of ca­reer changes based on EA prin­ci­ples, and num­ber of new at­ten­dees at events who were un­fa­mil­iar with EA). Just be­cause each met­ric seemed to be dom­i­nated by a small num­ber of groups perform­ing ex­ceed­ingly well on that met­ric, it didn’t nec­es­sar­ily fol­low that it was the same groups dom­i­nat­ing across each met­ric: it might be that the groups ac­count­ing for al­most all the to­tal group mem­bers were differ­ent from the groups ac­count­ing for al­most all the pledges, for ex­am­ple.

Our anal­y­sis sug­gested that there was quite a lot of over­lap be­tween the groups dom­i­nat­ing across differ­ent met­rics. We don’t name spe­cific groups be­cause we wish to avoid pub­li­cly rank­ing groups in terms of suc­cess. Of the 5 groups with the high­est mem­ber count, 4 re­curred mul­ti­ple times (3-4) over the other 4 cat­e­gories, with the largest group top­ping the cat­e­gories for ac­tive com­mit­ments, pledges and ca­reer changes. Notably, the other group in the top 5 for size did not re­cur across any of the other cat­e­gories.

How­ever there was scope for di­ver­gence from this pat­tern of dom­i­nance by the ‘top’ groups. Across the ‘top 5’ for the other 5 cat­e­gories, each time at least 2 of the top 5 didn’t ap­pear in the top 5 for any other cat­e­gory (e.g. at least two of the groups re­port­ing the most pledges did not re­port the most mem­bers, most funds raised, most EA ca­reer choices or most new at­ten­dees). More­over, one lo­cal group ap­peared in the top 5 for all but one of the re­main­ing cat­e­gories (funds raised, ac­tive com­mit­ments, pledges and new event at­ten­dees) de­spite not be­ing one of the groups with the most mem­bers (in­deed, they only have 35 mem­bers, plac­ing them out­side the top 10 for size).

Two of the cat­e­gories ex­am­ined were also strik­ing out­liers. The top ‘group fundraisers’ did not in­clude the ‘top’ group across most cat­e­gories and 3 of the 5 places were taken by groups which were not among the largest groups. As this graph shows, rel­a­tively lit­tle of the var­i­ance here was ex­plained by group size:

The num­ber of new event at­ten­dees, for a given group, who were not fa­mil­iar with EA be­fore­hand was even more strik­ing. Here none of the largest groups were among the top groups (though the largest group, tied for 5th place) and only 1 of the 5 groups re­curred across any other cat­e­gories. This is quite note­wor­thy be­cause this met­ric seems like the best mea­sure we have (from this sur­vey) for groups reach­ing out to non-EAs and suc­ceed­ing in at least in­tro­duc­ing them (via an event) to EA, yet it seemed to bear lit­tle con­nec­tion to suc­cess on any other met­rics. One other strik­ing fea­ture of this cat­e­gory is that all of the top groups (ex­cept for the largest group, ty­ing in 5th place, and with only half as many new event at­ten­dees as the top group in this cat­e­gory) were from non-An­glo-Amer­i­can coun­tries. While this is purely spec­u­la­tive, an ex­pla­na­tion for this pat­tern might be that these groups are ag­gres­sively reach­ing out to peo­ple un­fa­mil­iar with EA in their ar­eas, get­ting them to at­tend events, but largely not see­ing suc­cess in trans­fer­ring this into in­creased group mem­ber­ship. This is­sue prob­a­bly bears fur­ther re­search, as it seems plau­si­ble that EA groups out­side of the tra­di­tional ge­o­graph­i­cal ar­eas may face dis­tinct challenges and re­quire more tai­lored sup­port (such as trans­la­tion of ma­te­ri­als).

Th­ese find­ings sug­gest that though a small num­ber of groups tend to be very suc­cess­ful across differ­ent cat­e­gories, there is still clear scope for other groups, which are not par­tic­u­larly large to pro­duce great im­pact, com­pet­i­tive with the largest and high­est perform­ing groups.

It is also no­table that many of the groups in the top 5 for var­i­ous met­rics were not from ob­vi­ously pro­pi­tious en­vi­ron­ments (e.g. elite uni­ver­si­ties or ma­jor cities). The fact that there is a fair amount of var­i­ance in suc­cess across differ­ent met­rics, in­clud­ing suc­cess out­side of the largest groups and out­side of ob­vi­ous EA ‘hubs’, sug­gests that there are in­fluences on group im­pact be­yond size and be­ing in pro­pi­tious lo­ca­tions. Fur­ther re­search seems needed to in­ves­ti­gate how the most im­pact­ful groups at­tain their suc­cess and to see how far this can be prop­a­gated as best prac­tice.

A fi­nal ob­ser­va­tion worth not­ing is that the much larger in­fluence of the biggest group(s) may be a re­sult of par­tic­u­lar, con­tin­gent poli­cies, rather than an in­evitable fea­ture of the way that EA groups’ im­pact will be dis­tributed. For ex­am­ple, EA Lon­don re­li­ably dom­i­nates most met­rics by a sub­stan­tial amount, but is un­usual among groups in hav­ing had a full-time paid or­ganiser, since 2016 (and now two ded­i­cated staff). Look­ing at it from the pre­sent van­tage point, it may ap­pear in­evitable that Lon­don would have grown to the size and in­fluence that it has now. How­ever, de­spite hav­ing been run­ning since 2013, un­til quite re­cently, EA Lon­don was much smaller, av­er­ag­ing around 5 so­cial event at­ten­dees ev­ery other month in 2014 com­pared to slightly more than 50 at the end of 2017. Of course, not all of this growth should be at­tributed to the pres­ence of a funded or­ganiser, and nor does it sug­gest that an or­ganiser would have been equally suc­cess­ful in a differ­ent city, but it does some­what count against the view that cer­tain groups were sim­ply in­evitably go­ing to be very large.

Fur­ther Ev­i­dence of Group Impact

Above we noted the me­dian re­sults for var­i­ous met­rics of group im­pact (e.g. pledges, dona­tions etc.). How­ever, as noted, groups also aim to have im­pact in a va­ri­ety of ways which are harder to mea­sure and quan­tify or which sim­ply can’t be trans­lated into a me­dian value per group. We note some of these here:

We asked in­di­vi­d­ual or­ganisers and group mem­bers to in­di­cate how much of a fac­tor be­ing in­volved with an EA group was for their in­volve­ment with EA. As noted in the quan­ti­ta­tive re­port, a ma­jor­ity re­ported that be­ing in­volved in an EA group had been a “large” or “very large” fac­tor for their en­gage­ment with EA. Similarly, 89% of or­ganisers and 78% of mem­bers re­ported that the way they thought about the world and/​or their be­havi­our had changed since be­com­ing a mem­ber of a lo­cal group, with large ma­jori­ties of these re­port­ing that they ex­pected to have more so­cial im­pact as a re­sult of these changes.

Though hard to at­tach a pre­cise value to, and re­li­ant on self-re­ports, this is strongly sug­ges­tive that lo­cal groups are hav­ing a pos­i­tive im­pact on EAs and in­creas­ing their en­gage­ment with EA. While this is not di­rect ev­i­dence of im­pact, it seems likely that in­creas­ing peo­ple’s en­gage­ment with EA may lead to im­pact by, on the whole, mak­ing in­di­vi­d­u­als more mo­ti­vated and pro­mot­ing EA ac­tions as norms.

Our qual­i­ta­tive data also strongly sup­ported this, with many in­di­vi­d­u­als ex­plic­itly re­port­ing the im­por­tance of “per­sonal in­ter­ac­tion” with other EA for their mo­ti­va­tion and en­gage­ment. It seems plau­si­ble that, for at least a sub­set of EAs, face-to-face in­ter­ac­tion with other EAs, rather than only on­line con­tact, may be im­por­tant.

There was also a gen­eral in­di­ca­tor that lo­cal groups are valuable to EAs, as a large ma­jor­ity (93%) of mem­bers rated their group’s ac­tivi­ties as “valuable” or “very valuable.” In­ter­est­ingly, group or­ganisers, while still hav­ing a ma­jor­ity (57%) rat­ing their group’s ac­tivi­ties as “valuable” or “very valuable” and 89% rat­ing them be­tween “mod­er­ately” or “very valuable” were no­tice­ably less pos­i­tive than group mem­bers. Over half (55%) of all mem­bers who re­sponded rated the group’s ac­tivi­ties “very valuable.” We spec­u­late that this may be partly ex­plained by or­ganisers and mem­bers in­ter­pret­ing the ques­tion of how valuable their group’s ac­tivi­ties are slightly differ­ently, with mem­bers tak­ing it to be about how valuable the ac­tivi­ties are to them and or­ganisers tak­ing it to be ask­ing for an eval­u­a­tion of how valuable the group is as a whole. It is also pos­si­ble that or­ganisers felt the need to be more self-crit­i­cal about their group’s ac­tivi­ties and achieve­ments, even though mem­bers near-uniformly found the groups to be ex­tremely valuable.

Groups also in­tro­duced a fairly large num­ber of in­di­vi­d­u­als who were un­fa­mil­iar with EA to EA via events (nearly 6000 in to­tal, and a me­dian of 30 per group- which is quite sub­stan­tial given a me­dian group size of only 10). This can­not be di­rectly iden­ti­fied with im­pact (as we do not know what pro­por­tion if any of these in­di­vi­d­u­als be­came more en­gaged with EA- this would be an av­enue for po­ten­tial fu­ture re­search). How­ever, there is sug­ges­tive ev­i­dence from our find­ings that slightly more than half (138/​254) of our re­spon­dents did not con­sider them­selves an EA prior to join­ing an EA group. This may be sug­ges­tive of non-EAs join­ing groups and com­ing to iden­tify with EA, though it may also re­flect in­di­vi­d­u­als ap­ply­ing a very stringent defi­ni­tion of EA to them­selves and only iden­ti­fy­ing as EAs once they have be­come ac­tively in­volved.

Value of ex­ist­ing pro­gramme ser­vices and resources

Where iden­ti­fy­ing the im­pact of lo­cal groups is, as we note above, ex­ceed­ingly messy, iden­ti­fy­ing the im­pact of efforts to sup­port lo­cal groups is sub­stan­tially messier.

Where lo­cal groups aim to pro­duce im­pact through a wide va­ri­ety of means: in­tro­duc­ing peo­ple to EA, mak­ing EAs more mo­ti­vated and en­gaged, mak­ing them more in­formed and con­nected to op­por­tu­ni­ties, en­courag­ing im­pact­ful ac­tions, in­creas­ing re­ten­tion and re­duc­ing dropout, efforts to sup­port and pro­mote the work of EA groups in a similarly broad set of ways with a broad set of means.

For ex­am­ple, LEAN has aimed to sup­port EA groups in a host of ways:


- Pro­vid­ing tech­ni­cal in­fras­truc­ture ser­vices (e.g. group e-mails, group web­sites, meetup.com etc)

- Pro­vid­ing in­for­ma­tional re­sources (e.g. guides to run­ning groups, the map of EA groups)

- Per­sonal sup­port and feed­back to group leaders

- Sup­port­ing group com­mu­ni­ca­tion (newslet­ter, group calls etc.)

Tak­ing even the more straight­for­ward of these ser­vices—pro­vid­ing a group with e-mails and web­sites—these ser­vices may po­ten­tially have pos­i­tive effects on a host of out­comes via a va­ri­ety of differ­ent mechanisms. They aim to make the group ap­pear more pro­fes­sional, which may at­tract more mem­bers and/​or make the group (or wider EA move­ment) seem more ap­peal­ing, and it may al­low the group to ac­cess op­por­tu­ni­ties (in virtue of as one of our qual­i­ta­tive re­spon­dents put it “show­ing peo­ple that we’re not some fringe thing that’s only lo­cally run”). They may also make or­ganisers and cur­rent mem­bers feel bet­ter about their group, in­creas­ing mo­ti­va­tion. Pro­vid­ing them as a ser­vice may make run­ning the group more con­ve­nient and less costly for or­ganisers (who might oth­er­wise feel they need to set up these solu­tions them­selves), en­courag­ing peo­ple to run groups and in­creas­ing re­ten­tion. (For di­verse other pos­i­tive effects of sup­port offered to lo­cal groups, see the qual­i­ta­tive re­port).

How­ever, for each of these pu­ta­tive mechanisms and met­rics, there are in­nu­mer­able other fac­tors in­fluenc­ing how pro­fes­sional an EA group ap­pears, mak­ing EA groups more or less con­ve­nient or costly to run, and in­fluenc­ing how many peo­ple are at­tracted to or en­gage with the group. And like­wise for ev­ery other means of sup­port­ing groups: pro­vid­ing lead­ers with men­tor­ing or guidance, mak­ing writ­ten guides available, pro­vid­ing video calls for lead­ers to dis­cuss is­sues and so on. Coun­ter­fac­tu­als here are, of course, par­tic­u­larly difficult to dis­cern, given the mu­tual in­fluence of so many differ­ent fac­tors on the same out­comes.

As such, iden­ti­fy­ing the causal im­pact of any par­tic­u­lar ser­vice on the perfor­mance of groups (whose own im­pact is it­self very difficult to iden­tify, as we noted in the pre­vi­ous sec­tion) is all but im­pos­si­ble. As a con­se­quence, in this re­port, we rely on the re­ports of group lead­ers on how use­ful they found differ­ent ser­vices offered to sup­port groups. Th­ese eval­u­a­tions cer­tainly have their limi­ta­tions, re­li­ant as they are on the self-re­ports of or­ganisers, but we think they are among the best ev­i­dence available to us of whether ser­vices are ac­tu­ally helping sup­port groups.

Alter­na­tive An­a­lytic Strategies

It is worth briefly sketch­ing out some al­ter­na­tive means of calcu­lat­ing the im­pact of efforts to sup­port lo­cal groups and why they are ei­ther un­work­able or severely limited:

  • Ran­domised as­sign­ment of lo­cal groups to re­ceive sup­port or not (for ex­am­ple, one set of groups might re­ceive web­sites or per­sonal sup­port, and an­other set would not): this is a gold stan­dard of ex­per­i­men­tal de­sign highly fa­mil­iar to EAs. It may be im­prac­ti­cal, in this case, to aid some groups and not oth­ers though. Given very small sam­ples of highly het­ero­ge­neous groups, it may be very difficult to en­sure that the treat­ment and con­trol groups are com­pa­rable. Fur­ther­more, some ser­vices pro­vided are eas­ily ac­cessible pub­lic goods (e.g. on­line re­sources and guides). Also not treat­ing a ‘con­trol’ group may have its own effects: e.g. severely dis­pirit­ing group or­ganisers who are de­nied as­sis­tance or en­courag­ing or­ganisers to adopt their own er­satz solu­tions (for ex­am­ple, if the treat­ment group are, very visi­bly, be­ing given new web­sites, this may move those as­signed to the con­trol group to set up their own web­site).

  • Com­par­ing pre/​post treat­ment met­rics: for ex­am­ple, see­ing how a group performs be­fore/​af­ter re­ceiv­ing sup­port. We already have some data along these lines: for ex­am­ple, groups re­port­ing in­creases in re­sponse rates af­ter us­ing an offi­cial-look­ing group e-mail ad­dress. We aim to ex­pand this kind of small scale ex­per­i­men­ta­tion, where prac­ti­ca­ble, next year. How­ever, in many case, where there are not dis­crete met­rics (such as e-mail re­sponse rates), this will also be limited, as mul­ti­ple other changes over the course of a year will con­found the changes ob­served af­ter the im­ple­men­ta­tion of a par­tic­u­lar ser­vice.

  • Com­par­i­son of out­comes from ob­ser­va­tional data for differ­ent kinds of groups e.g. CEA groups vs LEAN groups: this par­tic­u­lar sug­ges­tion would be es­sen­tially im­pos­si­ble, be­cause many groups re­ceive sup­port from a num­ber of differ­ent sources, mean­ing there are few, if any, “pure CEA” or “pure LEAN” groups. It may be pos­si­ble to make in­for­ma­tive com­par­i­sons of groups of other types.

  • Com­par­ing EA in­di­vi­d­u­als who are or are not in EA lo­cal groups (based on the EA Sur­vey) to see their im­pact, rates of at­tri­tion etc. This sug­ges­tion seems ham­strung by the fact that these differ­ences would likely be severely con­founded. In­di­vi­d­u­als who are (or are not) in an EA group are likely to sys­tem­at­i­cally differ in other ways due to self-se­lec­tion, among other things. For ex­am­ple, more mo­ti­vated in­di­vi­d­u­als may be more likely to at­tend an EA group. Alter­na­tively peo­ple who are not in an EA group may be more likely to live in iso­lated ar­eas away from an EA hub com­pared to those in a group, which may in­de­pen­dently ex­ert an effect. Efforts could be made in an anal­y­sis to ‘match’ com­pa­rable peo­ple and over­come this in other ways, how­ever we are not sure this would be re­li­able or worth­while. (The EA Sur­vey data is freely available to any­one who wishes to make an at­tempt.)

Eval­u­a­tions of Group Sup­port and Services

Dur­ing the in­ter­views and the Lo­cal Group Sur­vey, we re­ceived feed­back about a range of re­sources and ser­vices, some of which had been pro­vided by LEAN ex­clu­sively and some of which had been jointly offered by LEAN and other EA or­gani­sa­tions or in­de­pen­dent EA in­di­vi­d­u­als. Disag­gre­gat­ing the im­pact of differ­ent or­gani­sa­tions’ ser­vices is there­fore difficult to do sys­tem­at­i­cally, though it is some­times pos­si­ble to iden­tify in­di­vi­d­ual cases of im­pact from the qual­i­ta­tive in­ter­views. In­deed, of­ten group or­ganisers them­selves couldn’t iden­tify which ser­vices were pro­vided by which or­gani­sa­tions. Nev­er­the­less, we think these re­sults offer a re­li­able in­di­ca­tion of the value which or­ganisers at­tach to differ­ent ser­vices and thus, at least to some ex­tent, of the de­gree to which they are likely to help sup­port EA groups’ work.

Tech­ni­cal support

Tech­ni­cal sup­port is the area where we can con­nect re­spon­dent ex­pe­riences most di­rectly to the LEAN’s in­vest­ment. This is be­cause, dur­ing the time frame con­sid­ered, most of these fa­cil­ities were not made available by any other or­gani­sa­tion. Some or­ganisers sup­plied their own web­sites and Meetup.com ac­counts, but these are in a minor­ity. Fur­ther­more, or­ganisers came to LEAN over time and ar­ranged for us to take on host­ing for ex­ist­ing web­sites, or pay­ment for ex­ist­ing Meetup.com ac­counts.

Ser­vices Provided

Do­main name registration

Meetup.com subscription

Web hosting

Cus­tom email addresses

Word­press templates

Ba­sic user support

EA Hub Group Profiles

Map of EAs and EA Groups

Web­site deployment

Mailchimp and Face­book management

Our quan­ti­ta­tive re­port offers rat­ings of the use­ful­ness and im­pact of var­i­ous of the tech sup­port ser­vices. Com­bined with our qual­i­ta­tive data, these gen­er­ally fol­low a pat­tern of ser­vices be­ing found to be highly use­ful by a small num­ber of groups, but not very use­ful or not used at all by oth­ers. That said, “tech­ni­cal sup­port” as a whole “for in­stance, sub­scrip­tions for on­line ser­vices, free web­sites, group email ad­dresses” was rated as use­ful or very use­ful by >70% of re­spon­dents. Qual­i­ta­tive re­sponses bore out a pat­tern which re­curs across our tech ser­vices, which is that these ser­vices were in­valuable to some groups and min­i­mally or not at all valuable to oth­ers, with a large num­ber of groups not us­ing the ser­vice at all.

Web­sites: or­ganisers were asked whether their group web­site “makes a non-triv­ial differ­ence in the effec­tive­ness of your group’s out­reach efforts” and web­sites were rated as “sig­nifi­cantly use­ful” rather than “no more than triv­ially use­ful” by a slim ma­jor­ity of re­spon­dents. Re­sponse rates were low though (39/​98), in line with the fact that rel­a­tively few groups have a web­site.

Meetup.com sub­scrip­tion: We asked or­ganisers to es­ti­mate how much of a coun­ter­fac­tual in­crease (%) in at­ten­dees they had as a re­sult of us­ing meetup.com. There was an ex­ceed­ingly wide range, with es­ti­mates liter­ally rang­ing from 0% to 100%, with a me­dian of 15%, but a small num­ber of groups gain­ing very large in­creases (50-100%) from us­ing the ser­vices.

E-mails: we did not di­rectly sur­vey in­di­vi­d­u­als about the use­ful­ness of group e-mail ad­dresses, but some in­di­ca­tion of in­ter­est came from the fact that a ma­jor­ity (29/​54) re­spon­dents in­di­cated that they would like one for their groups. Qual­i­ta­tive data offered fur­ther con­fir­ma­tion, with a num­ber of or­ganisers not­ing that they preferred hav­ing a group e-mail as it looked more “pro­fes­sional” and that they no­ticed higher re­sponse rates af­ter us­ing them.

We are not sur­prised by the fact that these ser­vices were ex­tremely valuable to some groups and of lit­tle value to oth­ers, since we find that EA groups differ ex­tremely in a wide va­ri­ety of ways. As such, a tai­lored ap­proach may be nec­es­sary to di­rect spe­cific ser­vice to the spe­cific groups which find them use­ful. It does not au­to­mat­i­cally fol­low from the fact that many groups find rel­a­tively lit­tle benefit from us­ing meetup.com, that it is not worth con­tin­u­ing to provide it if a small num­ber of groups gain sub­stan­tial benefits from it. This is es­pe­cially true in cases such as Meetup.com where it is cheap and easy to provide ac­counts to groups who want one, and where the ser­vice al­lows three differ­ent groups to share one sub­scrip­tion.

Per­sonal Sup­port and Expertise

LEAN pro­vides ex­per­tise and sup­port to groups through the fol­low­ing ser­vices and ac­tivi­ties:

  • Per­sonal Feedback

  • Prac­ti­cal sup­port and ideas

  • Writ­ten guides

  • Con­nect­ing and introducing

  • Im­pact as­sess­ment and re­search support

The LEAN Im­pact Assess­ment quan­ti­ta­tive re­port found that group or­ganisers at­tached ex­tremely high value to re­ceiv­ing per­sonal feed­back, prac­ti­cal sup­port and ideas and guidance in the form of writ­ten guides.

Per­sonal feed­back and sup­port: 78% use­ful or very useful

Prac­ti­cal sup­port and new ideas for group ac­tivi­ties: 94% use­ful or very useful

Writ­ten guides: 83% use­ful or very useful

Our qual­i­ta­tive data filled out the de­tails about ex­actly what and why or­ganisers found valuable. A re­cur­ring theme was that some or­ganisers felt in­se­cure about run­ning a group, as to whether or not what they were do­ing was effec­tive, felt iso­lated from the broader com­mu­nity and felt that per­sonal con­tact for feed­back, re­as­surance and ad­vice was benefi­cial for mo­ti­va­tion.

In­creased guidance and pro­vi­sion of EA ma­te­ri­als (for ex­am­ple, stock con­tent which could be posted to their Face­book group) were also cited as things which would make run­ning a lo­cal group eas­ier. Some or­ganisers re­ported that they felt that they were hav­ing to in­vent things them­selves, even though they knew that other groups must have worked on similar prob­lems be­fore.

Other re­lated fac­tors which came up in our qual­i­ta­tive data, but which were not cap­tured in terms of our quan­ti­ta­tive met­rics, in­volved helping con­nect EAs and groups with rele­vant peo­ple (ei­ther di­rectly, through our net­works, or in­di­rectly through the use of the Effec­tive Altru­ism Map.)

A fur­ther re­cur­ring theme was a de­mand from group or­ganisers for more re­search which would in­form how they should be op­ti­mally run­ning their groups. Many EAs wished to know whether they were act­ing effec­tively and hav­ing an im­pact, and wished to see cen­tral­ised data from other groups to bet­ter dis­cern what would be effec­tive.

Group Communication

  • Group Calls

  • EA Group Newsletter

  • Men­tor­ing Programme

LEAN, in some of these cases in col­lab­o­ra­tion with CEA and EAF, sup­port a num­ber of plat­forms for EA group or­ganisers to dis­cuss with each other and with orgs sup­port­ing move­ment build­ing, to share ex­per­tise and in­for­ma­tion. There was sub­stan­tial vari­abil­ity in how use­ful each of these plat­forms was rated as be­ing, as some were new or lit­tle used or or­ganisers were not yet aware they ex­isted, whereas oth­ers were widely val­ued.

Group Calls

Slightly more than half (54.3%) of re­spon­dents rated group calls as be­ing use­ful or very use­ful.

Qual­i­ta­tive data sug­gested that or­ganisers were broadly sup­port­ive of mea­sures to im­prove in­ter­con­nec­tivity be­tween groups and group lead­ers, though there was lit­tle spe­cific men­tion of group calls. Fur­ther­more, some in­ter­vie­wees men­tioned that the one-size-fits all na­ture of group calls meant that they had not found the is­sues dis­cussed par­tic­u­larly rele­vant for their own groups.

EA Group Newsletter

LEAN pub­lished eleven newslet­ters for group or­ganisers be­tween 2015 and May 2016. The group or­ganisers newslet­ter was re­launched as an in­ter-or­ga­ni­za­tional ser­vice dur­ing 2017, and four edi­tions have been pub­lished (note that this is not the same as the gen­eral EA Newslet­ter, which is also pro­vided in col­lab­o­ra­tion be­tween CEA, LEAN and EAF).

Feed­back on the con­cept is limited due to the fact that few re­spon­dents were in re­ceipt of the newslet­ter. 32% in­di­cated that they re­ceive the newslet­ter, whereas 61.6% of re­spon­dents ex­pressed an in­ter­est in be­ing added to the newslet­ter. 53% of re­spon­dents to a ques­tion about the use­ful­ness of the newslet­ter rated it as use­ful or very use­ful, while 40% rated it as nei­ther use­ful nor use­less.

EA Men­tor­ing Programme

A pi­lot men­tor­ing pro­gramme was ini­ti­ated by CEA in late 2015. In 2017 the con­cept was re­booted by LEAN in beta form. As with the re­booted EA Groups Newslet­ter, few par­ti­ci­pants in the sam­ple would have di­rect ex­pe­rience of the Men­tor­ing pro­gramme so far. At of the time of writ­ing, 9 men­tor­ing pairs are in­cluded in the pro­gramme. Given this, it is not sur­pris­ing that the ma­jor­ity (61%) of re­spon­dents rated the pro­gramme as ‘nei­ther use­ful nor use­less,’ though 30.7% deemed the pro­gramme to be ei­ther ‘use­ful’ or ‘very use­ful’. We re­ceived two or three com­ments from sur­vey par­ti­ci­pants ex­plic­itly stat­ing that they had not heard of the Men­tor­ing Pro­gramme and the Newslet­ter, and ask­ing to be sent fur­ther in­for­ma­tion. We will be fol­low­ing up on the pairs of in­di­vi­d­u­als in­volved in the men­tor­ing pro­gram to in­ves­ti­gate how im­pact­ful it is.

Strate­gic Summary

This sec­tion sum­marises the strate­gic plans that LEAN has for 2018 based on these find­ings.

Tech sup­port and group com­mu­ni­ca­tion ser­vices will con­tinue, via more stream­lined tech­niques. Our tech re­sources will be fo­cused on de­vel­op­ing a high qual­ity in­ter­face for writ­ten con­tent and re­sources. In re­sponse to de­mand from or­ganisers for these ser­vices, we will ex­pand our in­vest­ment in offer­ing per­sonal sup­port and feed­back to or­ganisers, and will con­soli­date our on­line re­sources for group lead­ers. We think that these writ­ten guides and per­son­al­ised sup­port will be syn­er­gis­tic with LEAN’s ex­panded work re­search­ing EA groups and how they can best pro­duce im­pact. More broadly, we hope to work more on dis­cern­ing the spe­cific needs of groups in differ­ent con­texts and offer­ing tai­lored sup­port and ad­vice.

Tech sup­port:

Hav­ing found that our tech ser­vices are highly use­ful to a minor­ity of groups we have de­cided to con­tinue, but stream­line our ser­vice pro­vi­sion. While it might seem that ser­vices which are use­ful only to a minor­ity of groups are not worth pro­vid­ing, we have con­cluded that the high level of use­ful­ness to a small num­ber of groups jus­tifies pro­vid­ing these ser­vices. A sig­nifi­cant con­sid­er­a­tion is that, in many cases, if LEAN ceased pro­vid­ing these ser­vices, the fi­nan­cial and time costs would likely sim­ply fall to in­di­vi­d­ual or­ganisers, mak­ing run­ning groups more costly for in­di­vi­d­u­als. We have, how­ever, switched to em­ploy­ing sim­pler, more au­to­mated pro­cesses for pro­vid­ing ser­vices (for ex­am­ple, a static site gen­er­a­tor for EA group web­sites, and an im­proved email host) which re­quires less time and fi­nan­cial in­vest­ment. At the same time we have also in­vested more in ded­i­cated tech staff to en­sure our sys­tems are more re­li­able in the fu­ture.

Tech development

Based on the ev­i­dent pop­u­lar­ity of ex­ist­ing writ­ten con­tent, and the wide­spread wish to see ex­ist­ing con­tent con­soli­dated, we are in­vest­ing in a new web in­ter­face for or­ganisers. This will in­volve edit­ing ex­ist­ing com­mu­nity gen­er­ated con­tent, and as­similat­ing it into a cen­tral, vi­su­ally ap­peal­ing in­ter­face. The new in­ter­face will be based on the EA Hub, which will it­self be mod­ernised and re­struc­tured. Where pos­si­ble, we will fa­cil­i­tate ed­i­to­rial ad­di­tions from the com­mu­nity, in or­der to make the tool a log­i­cal home for shar­ing re­sources.

Group Communication

We will con­tinue to fa­cil­i­tate group calls alongside other or­gani­sa­tions, as there is still de­mand for these, though we will work on pro­mot­ing more tar­geted calls (and, in par­tic­u­lar, offer­ing groups more in­di­vi­d­ual calls where re­quested). We will con­tinue to run the group newslet­ter (which now has a larger num­ber of or­ganisers added) to keep or­ganisers up to date on rele­vant de­vel­op­ment. We will also con­tinue to fa­cil­i­tate the men­tor­ing pro­gram for in­ter­ested or­ganisers.

Per­sonal sup­port and feedback

In re­sponse to very high rat­ings of use­ful­ness of these kinds of ser­vices, we have de­cided to ded­i­cate more at­ten­tion to pro­vid­ing per­sonal sup­port and feed­back to or­ganisers where re­quested. Much of this will take the form of one-to-one calls with in­di­vi­d­ual or­ganisers to talk through challenges their group is fac­ing, con­nect them with rele­vant in­di­vi­d­u­als, in­for­ma­tion and re­sources. We hope this kind of tai­lored sup­port will help guide or­ganisers, and also in­crease their mo­ti­va­tion and con­fi­dence in their ac­tions. This will also al­low a closer de­ter­mi­na­tion of i) how much coun­ter­fac­tual im­pact groups are hav­ing, ii) how far par­tic­u­lar ser­vices are sup­port­ing groups.

Writ­ten guides and resources

Similarly, in re­sponse to a strong in­di­ca­tion of or­ganiser in­ter­est in this, we will ded­i­cate time to con­soli­dat­ing and adding to ex­ist­ing writ­ten re­sources aimed at helping or­ganisers to launch and run groups. This will con­tribute to the new con­tent tool for or­ganisers on the EA Hub, men­tioned above. We aim to up­date these re­sources on an on­go­ing ba­sis us­ing our on­go­ing re­search into how EA groups are func­tion­ing and how they might func­tion bet­ter.

Research

Many or­ganisers ex­pressed an in­ter­est in us­ing met­rics to de­ter­mine their im­pact and in hav­ing ac­cess to ev­i­dence-based re­search on how best to run EA groups. We have also noted, through­out this re­port, var­i­ous ques­tions about EA move­ment build­ing where fur­ther em­piri­cal re­search is needed. Some of this re­search may be con­ducted in con­cert with the EA Sur­vey (also run by Re­think Char­ity). We aim to help in­di­vi­d­ual groups to mea­sure their im­pact, and to pro­duce gen­eral re­search re­gard­ing EA groups for the wider com­mu­nity.

Grants

Recog­nis­ing that funds are im­por­tant for groups’ suc­cess, and hav­ing heard from a num­ber of group or­ganisers that lack of funds was a bot­tle­neck, we are plan­ning to make small grants to groups on a case by case ba­sis to help fa­cil­i­tate group growth and are ex­plor­ing ways to col­lab­o­rate with CEA to di­rect funds to groups. Where ap­pro­pri­ate we will al­ign this with our on­go­ing re­search into EA groups, to try to dis­cern how far tar­geted grants boost EA groups’ im­pact.

Ac­knowl­edg­ments

This re­port was analysed and au­thored by Richenda Herzig and David Moss.

Spe­cial thanks to Tee Bar­nett for ed­i­to­rial feed­back, and for co­or­di­nat­ing those in­volved in the Im­pact Assess­ment, both in­ter­nal and ex­ter­nal. Fur­ther thanks to Peter Hur­ford for in­put as an in­ter­nal ad­vi­sor.

We are very grate­ful to Greg Lewis for his on­go­ing in­put as our ex­ter­nal ad­vi­sor.