EA Survey 2018 Series: Where People First Hear About EA and Influences on Involvement

In a re­ply to our pre­vi­ous post in this se­ries ti­tled ’How do peo­ple get in­volved in EA?, Ben Todd made the fol­low­ing re­quest:

[An] ad­di­tional anal­y­sis which would be great is if you could iden­tify the 20% of the re­spon­dents who seem most in­volved and ded­i­cated, and then re­peat the anal­y­sis by source for this sub-group. This would give us some sense of the qual­ity as well as the scale of the reach of differ­ent sources.

We are happy to oblige!

Select­ing Prox­ies for Involvement

Nat­u­rally, what mea­sure we should use to de­ter­mine the “most in­volved and ded­i­cated” EAs is con­tro­ver­sial. Any proxy we se­lect will also nec­es­sar­ily be im­perfect as there will al­most cer­tainly be ex­cep­tions to the rule.

In this post we ex­am­ine two po­ten­tial prox­ies, which may po­ten­tially each cap­ture two dis­tinct modes of EA in­volve­ment.

The first is EA Fo­rum mem­ber­ship. This seems likely to serve as a rea­son­able proxy for re­spon­dents who are es­pe­cially likely to be in­ter­ested in and fa­mil­iar with in-depth EA dis­cus­sions. As it hap­pens, it also picks out a group that make up al­most ex­actly 20% of re­spon­dents (20.1%).

We also look at the group of re­spon­dents who have taken the Giv­ing What We Can Pledge and re­port that effec­tive al­tru­ism has changed their ca­reer path or they ex­pect that it will. Close to a third of re­spon­dents (32.4%) had taken the GWWC Pledge, similarly 33.9% re­port that EA has shifted their ca­reer path while a slighter larger per­centage re­port that they ex­pect it will shift their ca­reer path (37.7%). Th­ese fac­tors may cap­ture a dis­tinct di­men­sion to in­volve­ment and ded­i­ca­tion (e.g. ac­tu­ally tak­ing the Pledge and chang­ing ca­reer, but not nec­es­sar­ily be­ing deeply in­volved or fa­mil­iar with EA con­cepts or dis­cus­sions).

A slightly higher per­centage of re­spon­dents had both taken the GWWC Pledge and made (or ex­pected to make) a ca­reer change (22.9%) than were EA Fo­rum mem­bers. Notably, how­ever, the GWWC-Ca­reer group and the Fo­rum group were not par­tic­u­larly over­lap­ping. Only about 38% of peo­ple in this group were mem­bers of the Fo­rum, while only 43% of Fo­rum mem­bers had both taken the Pledge and made or ex­pect to make a ca­reer change (though this is a sig­nifi­cantly higher pro­por­tion than that of the sam­ple as a whole).

EA Fo­rum Membership

While we pre­sent a break­down of where EA Fo­rum mem­bers (com­pared to non-mem­bers) first hear about EA be­low, we think these raw figures are likely to be of limited use for dis­cern­ing which routes tend to lead to a higher pro­por­tion of more in­volved mem­bers.

This is largely due to pos­si­ble con­founders, the most ob­vi­ous of which is when an in­di­vi­d­ual first heard of EA. As noted in our pre­vi­ous post, where EAs first hear about EA has changed dra­mat­i­cally over time. For ex­am­ple, a full 25% of EAs hear­ing about EA in the last year, re­port hear­ing about EA from 80,000 Hours, com­pared to less than 5% in ear­lier years. N.B. images can be viewed in full size if opened in a new tab.

In ad­di­tion, many more EAs joined in more re­cent years (apart from in 2018, when the sur­vey was run) than in ear­lier years. This means that 80,000 Hours is heav­ily over­rep­re­sented among those newest to EA.

As we know that re­spon­dents who more re­cently heard of EA are much less likely to be more in­volved in EA (as cap­tured by met­rics such as EA Fo­rum mem­ber­ship- see graph be­low), this means that we would ex­pect those who heard of EA from 80,000 Hours to con­tain many more new and less in­volved EAs.

This means that com­par­ing the per­centages of more in­volved EAs who first heard EA from differ­ent sources will likely be mis­lead­ing, due to pe­nal­is­ing routes which have dis­pro­por­tionately re­cruited EAs more re­cently. That said, it may still serve as an in­di­ca­tor of the effect of differ­ent groups’ re­cruit­ment on the com­po­si­tion of EA so far.

Sure enough, a smaller per­centage of EAs first heard of EA from 80,000 Hours, when look­ing at Fo­rum mem­bers, rather than look­ing at the rest of the sam­ple. The same holds for those hear­ing about it from SlateS­tarCodex. Fo­rum mem­bers were dis­pro­por­tionately more likely to have heard about EA from LessWrong, by con­trast (note that more peo­ple first heard about EA via LessWrong in ear­lier years as well).

For the most part, the raw per­centage point differ­ences be­tween the other cat­e­gories are quite small (though they may re­flect large differ­ences in the pro­por­tion of peo­ple from differ­ent sources who join the Fo­rum as shown in the table be­low) as are the to­tal num­bers of EAs (and Fo­rum mem­bers) com­ing from many of these differ­ent sources.

It is im­por­tant to in­ter­pret the differ­ences in the pro­por­tions of peo­ple from differ­ent sources be­com­ing Fo­rum mem­bers in the con­text of the ab­solute num­ber of Fo­rum mem­bers com­ing from each source. For ex­am­ple, while a much higher per­centage of peo­ple who first hear about EA from EA Global are mem­bers of the Fo­rum, than other sources (such as 80,000 Hours and Per­sonal Con­tact), the to­tal num­ber of Fo­rum mem­bers com­ing from these other sources is much higher. That said, if you were to take the per­centage of EAs re­cruited from a given source who are in­volved as a po­ten­tial in­di­ca­tor of the qual­ity of the re­cruit­ment from that source, then you might care more about groups re­cruit­ing a higher per­centages of in­volved mem­bers, rather than re­cruit­ing higher to­tal num­bers of both more and less in­volved EAs. It is also im­por­tant to note that in prin­ci­ple, high pro­por­tions of in­volved mem­bers could po­ten­tially be pro­duced by a cer­tain route into EA be­ing very bad at en­gag­ing mem­bers (for ex­am­ple, if a group caused dropout in all but its most en­gaged re­cruits, then sur­vivor­ship bias might leave a very small, but ex­clu­sively highly en­gaged group tak­ing the sur­vey). We would cau­tion against putting too much weight on these per­centages as a means to com­pare differ­ent routes into EA in this way, due to the small num­bers in­volved in many of these cat­e­gories and po­ten­tial con­founds men­tioned above. The difficulty of iden­ti­fy­ing pat­terns of clear differ­ences across differ­ent routes into EA and differ­ent years can be dis­cerned by view­ing the fol­low­ing crosstab, which shows the to­tals and per­centages of EA Fo­rum users from each source, within each year.

More­over, when we ex­am­ine the cor­re­la­tion be­tween the num­ber of EAs, in to­tal, re­cruited from differ­ent sources and the num­ber of Fo­rum users re­cruited from each source, we find this to be very highly cor­re­lated (r=.929), sug­gest­ing much of the var­i­ance in to­tal Fo­rum users re­cruited is ex­plained sim­ply by differ­ent sources re­cruit­ing more to­tal EAs. One can see in the scat­ter plot be­low the small num­ber of sources which are above or be­low the line, sug­gest­ing a dis­pro­por­tionately high or low num­ber of Fo­rum mem­bers re­cruited rel­a­tive to to­tal EAs re­cruited.

We at­tempted to model pre­dic­tors of EA Fo­rum mem­ber­ship in a way which could ac­count for where EAs first heard about EA be­ing con­founded by other in­fluences on EA Fo­rum mem­ber­ship by cre­at­ing a ran­dom for­est model and sub­se­quently run­ning a lo­gis­tic re­gres­sion. Nei­ther of these mod­els were in­for­ma­tive. Ex­am­i­na­tion of the con­fu­sion ma­tri­ces in­di­cated that nei­ther model was bet­ter than ran­dom for pre­dict­ing the minor­ity class of be­ing an EA fo­rum mem­ber from fac­tors such as first heard EA, in­volve­ment or mem­ber­ship in other groups. Nei­ther up­sam­pling nor use of syn­thetic data im­proved the pre­dic­tions.

GWWC Pledge and Ca­reer Changes

As with EA Fo­rum mem­ber­ship above, we ex­am­ine the raw per­centages of peo­ple in our tak­ing the GWWC Pledge, mak­ing a ca­reer change or both who hear about EA from differ­ent sources. For the most part, the re­sults do not differ sub­stan­tially from those above for Fo­rum mem­ber­ship. As with EA Fo­rum mem­bers, you can also view the cross-tabs show­ing the to­tal EAs, to­tal GWWC pledg­ing/​ca­reer chang­ing/​both pledg­ing and ca­reer chang­ing, and % pledg­ing/​ca­reer chang­ing/​both pledg­ing and ca­reer chang­ing EAs, within each year from each source of peo­ple first hear­ing about EA.

Run­ning the same analy­ses as for EA Fo­rum mem­ber­ship above, we find again that a dis­pro­por­tionately small num­ber of GWWC pledgers first heard of EA from 80,000 Hours, but a very dis­pro­por­tionately large num­ber re­port first hear­ing about EA from GWWC.

In gen­eral, there are very small raw per­centage differ­ences be­tween ca­reer change and non-ca­reer change groups, across each route into EA. As the table be­low shows, more­over, more than 50% of EAs hear­ing about EA from each source ei­ther have changed their ca­reer plans as a re­sult of EA or ex­pect to. Of note, though, 80,000 Hours now seems to be over-rep­re­sented, rather than un­der-rep­re­sented, among EAs who have or ex­pect to make ca­reer changes.

As with to­tal num­ber of Fo­rum mem­bers, the to­tal num­bers of EAs tak­ing the GWWC Pledge (r=0.970 (p<0.01)), mak­ing (or ex­pect­ing) a ca­reer change (r=0.9888 (p<0.01)) or both (r=0.9501 (p<0.01), were highly cor­re­lated with the to­tal num­bers of EAs be­ing re­cruited from each source.

We also ran an or­di­nal re­gres­sion on com­bined GWWC-ca­reer changes. GWWC and ei­ther an ac­tual or ex­pected ca­reer change were each coded as 1, giv­ing each re­spon­dent a com­bined score of 0-2 (with 2 rep­re­sent­ing an EA who both took the GWWC Pledge and ei­ther changed or ex­pected to change ca­reer). Back­wards elimi­na­tion us­ing AIC pro­duced a fi­nal model with AIC = 4103 and resi­d­ual de­viance of 4021. This com­pared favourably to a full model with AIC = 4160 and resi­d­ual de­viance of 3988 which in­cluded all fac­tors of in­volve­ment, mem­ber­ship, stud­ies, em­ploy­ment, coun­try, poli­tics and ed­u­ca­tion. After step­wise se­lec­tion we found the fol­low­ing sig­nifi­cant marginal effects on level 2. Among “where first heard of EA” op­tions (com­pared to a base cat­e­gory of first hear­ing about EA via Face­book) the largest pos­i­tive effect was for hear­ing about EA from EAF/​FRI/​REG etc. This was fol­lowed by hear­ing about EA from GWWC, 80,000 Hours, a search en­g­ine, a lo­cal group, Per­sonal Con­tact or Other.

Re­port­ing get­ting more in­volved in EA through 80,000 Hours or GWWC were both as­so­ci­ated with mak­ing a ca­reer change or tak­ing the Pledge. This should not be too sur­pris­ing, since mak­ing a ca­reer change based on 80,000 Hours or tak­ing the GWWC pledge could them­selves both be con­sid­ered forms of get­ting more in­volved with EA though each of these or­gani­sa­tions.

Hav­ing taken the TLYCS pledge, and be­ing mem­bers of EA Face­book, the EA Fo­rum and a lo­cal EA group were each pre­dic­tors of hav­ing taken the GWWC Pledge and made a ca­reer change.

Be­ing a stu­dent was pos­i­tively as­so­ci­ated with tak­ing the GWWC pledge and hav­ing made or ex­pected to make a ca­reer change, whereas, un­sur­pris­ingly, be­ing re­tired (and so in lit­tle po­si­tion to make a ca­reer change) was nega­tively as­so­ci­ated.

Study­ing eco­nomics speci­fi­cally seemed to be a pos­i­tive pre­dic­tor of hav­ing both taken the GWWC pledge and made a ca­reer change. Notably, be­ing a veg­e­tar­ian and ve­gan (es­pe­cially a ve­gan) were both sig­nifi­cantly pos­i­tively as­so­ci­ated with hav­ing both taken the GWWC pledge and made a ca­reer change.

Other mea­sures of involvement

We also ex­am­ined some al­ter­na­tive in­di­ca­tors of in­volve­ment. The first was an un­weighted scale of mem­ber­ship across groups. Re­spon­dents were as­signed a ‘score’ out of 6, where be­ing a mem­ber of EA Face­book, the EA Fo­rum, Giv­ing What We Can (hav­ing taken the pledge), Lo­cal Groups, LessWrong or The Life You Can Save (hav­ing taken the Pledge) each counted for 1 point. Nat­u­rally, we would ex­pect many EAs to think that some of these group mem­ber­ships count as bet­ter prox­ies for (valuable) in­volve­ment in EA, which would re­quire com­men­su­rate weight­ing of differ­ent mem­ber­ships in a com­bined in­volve­ment score. How­ever, as we would ex­pect a va­ri­ety of differ­ent views about what those weight­ings should be, which we can­not re­solve here we sim­ply pre­sent an un­weighted score, where higher scores sim­ply re­flect in­volve­ment in a greater num­ber of the listed groups, count­ing each group equally. In line with the other analy­ses above, those who first hear of EA from REG/​EAF/​FRI seem to be more likely to be in­volved in a wider va­ri­ety of groups.

We also ex­am­ine the per­centage of EAs first hear­ing about EA from each source who are in­volved (in at least one of the EA groups listed above) ver­sus not in­volved in any.

Conclusions

We ex­am­ined the num­bers and pro­por­tions of EAs who were mem­bers of the EA Fo­rum and who had both made or ex­pected to make a ca­reer change and taken the GWWC Pledge, across differ­ent routes into EA.

For the most part, we do not think that these re­sults sug­gest sig­nifi­cant differ­ences be­tween differ­ent places that peo­ple first hear of EA in terms of the pro­por­tion of ‘more in­volved’ EAs they re­cruit. The num­ber of more in­volved EAs re­cruited from each source cor­re­lates ex­tremely highly with the to­tal num­ber of EAs re­cruited from each source. There are some pos­si­ble ex­cep­tions: EAG and REG/​FRI/​EAF both seem as­so­ci­ated, across met­rics, with a higher per­centage of highly in­volved mem­bers, though very low num­bers of EAs first hear about EA from these sources over­all.

In fu­ture iter­a­tions of the EA Sur­vey we will con­sider adding fur­ther po­ten­tial prox­ies for higher lev­els of in­volve­ment in EA.

Credits

This post was writ­ten and with anal­y­sis by David Moss, Neil Dul­laghan and Kim Cud­ding­ton.

Thanks to Tee Bar­nett for re­view and edit­ing.

The an­nual EA Sur­vey is a pro­ject of Re­think Char­ity with anal­y­sis and com­men­tary from re­searchers at Re­think Pri­ori­ties.

Sup­port­ing Documents

Ar­ti­cles in the 2018 EA Sur­vey Series

I—Com­mu­nity De­mo­graph­ics & Characteristics

II—Distri­bu­tion & Anal­y­sis Methodology

III—How do peo­ple get in­volved in EA?

IV—Sub­scribers and Identifiers

V—Dona­tion Data

VI—Cause Selection

VII- Group Membership

VIII- Where Peo­ple First Hear About EA and In­fluences on Involvement

IX- Geo­graphic Differ­ences in EA

X- Wel­com­ing­ness- How Wel­com­ing is EA?

Prior EA Sur­veys con­ducted by Re­think Charity

The 2017 Sur­vey of Effec­tive Altruists

The 2015 Sur­vey of Effec­tive Altru­ists: Re­sults and Analysis

The 2014 Sur­vey of Effec­tive Altru­ists: Re­sults and Analysis