EA Survey 2019 Series: Community Information

[David_Moss]

Summary

• More respondents’ level of interest in EA increased over the last year (43%) than decreased (18%).

• The most common reasons for interest increasing were local EA groups (14%), the respondent being new to EA (12%), the local EA community more broadly (10%), or career change (10%).

• The most common reasons for interest decreasing were people being too busy (18%), a perceived mismatch between the person’s cause preferences with that of the overall EA community (12%), or finding diminishing returns from involvement in EA (10%).

• The most commonly cited barriers to further involvement in EA were lack of job opportunities that were a good fit (29%), no close friends in EA (28%), and it being too hard to get an EA job (23%).

• The main ways people wanted to become more involved in EA were an EA-aligned career (58%), donating more (55%), socialising with EAs (44%), and participating in more local EA events (43%).

• The factors most often cited as important for retaining people in EA were personal contacts (43%), local EA groups (27%), and reading SlateStarCodex (21%).

• Among those who knew of someone who had been involved in EA but then became disengaged (27%), the most commonly cited reasons were lack of interest in EA (31%), bad experiences with other EAs (30%), lack of opportunities to implement effective altruism in their lives (28%), and lack of friends or community interested in EA (25%).

• We found significant gender differences and differences between low/​high engagement EAs in what they selected as barriers to further involvement in EA and as important for their retention in EA, however we found few signs of differences between white/​non-white EAs.

Introduction

The 2019 EA Survey included a number of new questions concerning experiences of the EA community, many of which were requested by the Centre for Effective Altruism. Questions about these topics seem particularly likely to raise concerns about the representativeness of the survey sample, since individuals who have had particularly negative experiences of the EA community are plausibly less likely to take the EA Survey. Nevertheless, we think that this data provides important qualitative and quantitative information to the community.

Changes in level of interest in EA

Respondents were asked “In the last year, how would you say that your level of interest in effective altruism has changed?”. 1934 respondents answered this question.

While the modal response by a large margin was “Stayed the same”, on the whole, we can see that more respondents’ level of interest increased (43%) than decreased (18%) over the last year. Naturally, as mentioned above, the fact that people whose interest has decreased may be less likely to take the survey is potentially a significant factor here.

We examine predictors of change in interest in a later section of this post.

Changes in level of interest in EA: Qualitative Data

Respondents were also asked “If your level of interest in effective altruism has changed, please explain why (if possible).” 968 respondents answered this question (out of 1196 who indicated that their level of interest changed).^[1]

Among those who reported that their level of interest decreased, the most common reason given, by a wide margin, was that they had become increasingly busy (17.7%).

This was followed by references to “cause preferences” (11.7%). These often alluded to EA being uninteresting if you didn’t share mainstream EA preferences and/​or to frustration that EA had come to be dominated by a certain cause area.

9.7% were classified as mentioning “diminishing returns.” Such comments often suggested that respondents thought that they had already internalised or acted on the core EA lessons and there wasn’t much more for them to gain from continued engagement.^[2] These responses often overlapped with a concern that EA lacked new content and was just rehashing old ideas (see also the “stagnated” category).

The most commonly cited reason for people’s level of interest increasing was a local EA group (13.9%) and the local EA community (not mentioning a group specifically) was also among the most cited (9.9%). Since responses were only coded to one category each, this means that almost 24% of responses cited either a local group or a local EA community as the reason for their increase in interest in EA.

The clear lesson which seems to follow from these results is the importance of local communities and local groups. The other most often mentioned reasons for increasing interest were characteristics of respondents which seem hard to influence: e.g. people being new to EA or considering career changes.

How excited would you be to recommend EA to a friend?

Respondents were asked “If you had a friend who you thought would agree with the core principles of EA, how excited would you be to introduce them to the EA community?” on a scale from (1) Not at all excited to (10) Extremely excited.

Responses are clearly skewed to the right, with the median response being 7 and the mean response being 7.11.

How exactly this score should be interpreted overall is unclear._ Prima facie_, ⁷⁄₁₀ appears to be a more positive than negative score, with most responses closer to “Extremely excited” than “Not at all excited.”

However, this question is a variant on the scale used for the “Net Promoter Score” which is widely used in industry. The “Net Promoter Score” is calculated by taking the percentage of respondents who gave a response between 1 and 6 (detractors) and subtracting the percentage of respondents who gave a response of 9 or 10 (labelled “Promoters”). (Reicheld, 2003) This gives EA a Net Promoter Score of 4.5 on a −100 to +100 scale. Scores above 0 are, by convention, considered “Good” while scores above 50 are considered “Excellent.” Nevertheless, Net Promoter Scores vary widely across industries, from 2 for internet services to 58 for department stores, so it is very difficult to interpret anything from the EA community’s score without a relevant reference class (Net Promoter Scores are widely used by nonprofits, but we are unaware of any uses for social movement or communities, and its use in this context has been criticised (Schulman and Sargeant, 2013). Moreover, we think there are serious reasons to doubt the validity and reliability of the Net Promoter Score. The method used to calculate the Net Promoter Score based on the 1-10 scale is a highly atypical use of a linear response scale and its superiority over other measures and predictive power has been challenged (Keiningham et al, 2007) As such in our later analyses in this post we simply treat this a straightforward ordinal scale and examine predictors of people being more likely to indicate higher levels of excitement at referring EA to a hypothetical friend.

Barriers to EA Involvement

1756 respondents indicated which were the top barriers to them becoming more involved in EA. This was a multi-select question and the percentages show the proportion of respondents who selected each option.

Two of the most commonly selected reasons concerned lack of EA job opportunities (32%) and it being too hard to get an EA job (26%) respectively. In between these, having no close EA friends was the second most mentioned (30%).

Barriers to Involvement in EA: Other

23.46% of respondents (412) selected the “Other (Please specify)” option (potentially alongside others since it was a multi-select question). We hand-coded these, where possible, into categories that seemed to capture recurring themes. 123 remained uncoded i.e. due to being unclear or not clearly fitting a theme. Below we display the percentages of these 412 responses which were coded into each category, for those categories which contained more than 5 responses. There were many categories which could plausibly have been assigned which would capture fewer than 5 responses.

The most commonly recurring theme was the individual being too “busy” or lacking time (29%), which was followed by other personal issues which precluded further involvement (24%). Clearly there is often some overlap between personal issues and lacking time, though we only coded 5 responses as involving both, where they were each explicitly mentioned separately. As such, were these two combined into a broader “personal circumstances” category, this would account for a much larger part (more than half) of responses.

The next most common category contained responses indicating that the individual felt that they were already involved enough (10%). This seems conceptually similar to the often mentioned sense of “diminishing returns” which was invoked to explain decreased interest in EA, in the analysis mentioned above. This was followed by substantive disagreements with EA (such as disagreement about cause prioritisation) (8%), while geography was also frequently mentioned (6%) i.e. people felt that there was no way for them to get more involved given where they lived, without moving. The view that EA was too demanding (often in specific ways, such as requiring a career change that the person didn’t want to make) was also mentioned fairly frequently (5%).

The category labelled “Not interested in “community”” (3%) referred to individuals who specifically said that they weren’t interested in or didn’t see any reason to be involved in the EA community, but for example, simply wanted to donate and follow the EA philosophy independently. Some explicitly noted that being part of a “community” wasn’t the point of EA. This suggested that at least some respondents interpreted the question’s reference to “becoming more involved in EA” as narrowly referring to becoming more involved in the EA community. The category “Age” matching 2% of comments, refers to comments that specifically mentioned age, e.g. finding it hard to engage with the community due to the preponderance of young people. There were a smaller number of comments that mentioned other forms of demographic diversity or the EA community being too concerned with such issues.

Ways People are Interested in Becoming More Involved in EA

1739 respondents answered this question which was, again, multi-select and the percentages depict the percentages of respondents who selected this option.

In line with what are probably the two main traditional focuses of EA, the top two most commonly mentioned ways people would like to get more involved in EA were through careers or donating more, which were each selected by over half of respondents. These were followed by socialising with EAs more and participating in local events. Of note, participating in _global _events was mentioned significantly less frequently.

These results seem to line up with the previously noted importance of personal contacts and local groups for people’s involvement and the large number of EAs who wish to pursue a career in an EA org.

What Factors are Important for EA Retention

1746 respondents answered this question and were instructed to choose “up to 3” factors. In line with our findings in the post on Getting Involved in EA, personal contacts were the factor EAs most often said was important for retaining them within EA (42.61%), significantly ahead of the other categories. The next most popular was involvement in an EA group (26.80%), which was itself substantially ahead of the next most selected option. The third most selected option was reading SlateStarCodex (20.62%).

Of note, donating to GiveWell specifically was the fourth most selected option (17.64%), far ahead of the other donation options listed, including EA Funds (9.56%) and donating to other places (8.76%). Leading an EA group was the next most selected option (14.03%), which is striking given that only a relatively small number of EAs have led a group. Working on an EA project was also selected by 13.75% of respondents, and again, it seems likely that only a relatively small number of respondents have worked on an EA project.

Another comparison that may be of interest is that substantially more respondents selected reading or writing on the EA Forum (13.69%) than EA Facebook (6.41%). Participating in the EA Discord server was selected by far fewer respondents (1.72%).

What Factors are Important for EA Retention Among Those with Experience of Those Factors

For some of these categories we can go further and examine how many individuals indicated that a factor was important out of the total number that reported some experience with that factor. For example, we can look at what proportion of local group members think that local groups were important and how many EAG attendees think that EAG was important to them. This is important because the numbers of EAs in the total sample who have experience of each of these things are quite different: 907 EAs were members of local groups, compared to 590 having attended an EAG. As such, it might be little surprise that over 26% of respondents said that a local EA group was important for their retention in EA, compared to only 11.51% saying the same for EAG, given that more EAs are members of local groups.^[3]

In fact, examining only those respondents who said that they have attended EAG, are a member of a local group, or are a member of GWWC (have taken the Giving What We Can Pledge), we find that lower percentages reported that EA Global was important in retaining them than local groups, though slightly more than people who had taken the GWWC pledge was important.

For comparison, we also examined the proportions of individuals saying that each of these categories was important for their involvement in EA (see our earlier post on Getting Involved in EA). Plausibly, what factors are important for EAs remaining in the EA Community differs from what helps them get more involved.

Our results seem to suggest this is the case. Giving What We Can now has the _highest _proportion of people saying that it was important for them getting involved in EA, even though the lowest proportion of individuals selected it as being important for retention (perhaps due to its relatively greater importance in earlier years of the EA community, in addition to being more important earlier in EA involvement). However, it remained the case that local groups were more often selected as important by local group members (45%) EAG by EAG attendees (30%).

What Factors are Important for EA Retention: Other data

355 respondents selected “Other (please specify)”, making this the fourth most commonly selected option for this question.

The most commonly mentioned category within these, with 100 mentions, was 80,000 Hours (28.1%). Most of these comments (72%) mentioned the podcast specifically, while 24% mentioned 80,000 Hours articles and 4% mentioned e-mails from 80,000 Hours.

21.6% of comments mentioned respondents’ personal values, often explicitly noting that given their beliefs they couldn’t not be involved in EA. 5.9% mentioned a Newsletter of some kind, while 4.5% of comments remained uncoded.

How many effective altruists do you know well enough to ask a favour?

1580 respondents answered the question “How many individual effective altruists do you think you know well enough to (hypothetically) ask for a personal favor?” Respondents were presented with fixed categories e.g. 0, 1-2, … 9-10, >10 and the distribution across these categories were very non-normally distributed.

As we can see below, there were large contingents who knew very few EAs well enough personally to ask a favour, 26% knowing 0 and 25% knowing 1-2 (accounting for just over half of respondents), while 18% knew more than 10. In between these extremes, there was a clear skew towards the low numbers, with 25% selecting 3-4 or 5-6, and only 6% selecting 7-8 or 9-10. In future iterations of the survey, we will have a better idea of the most relevant categories to use in this question.

Do you know any individuals who were once engaged in EA, but no longer are?

1553 respondents answered this question. Of these, only 27.11% of respondents answered “Yes” while 72.89% answered “No”.

Prima facie, this might appear to be a reassuringly low number, although the number of EAs who know someone who dropped out from EA is probably a poor proxy for attrition rates (for more research on attrition rates in EA see our earlier post). However, it is important to consider, in light of the above question, that many respondents might not know any EAs in the first place and so could not know any who dropped out.

Indeed, we can see that the percentage of EAs who know someone who disengaged from EA differs dramatically depending on the number of EAs they know. Among those who know 0 EAs well enough to ask a favour, 98% don’t know any EAs who dropped out. Conversely, among those who know >10 EAs well enough to ask a favour, a full 72% of EAs know someone who has disengaged from EA.

There is a similar trend across the number of years a respondent has been in EA. While this trend is a bit choppier, due to the larger number of categories and small sample sizes for those EAs who have been in EA the longest, there is still a clear trend for much smaller proportions of those who have only been in EA a year to two to know anyone who was dropped out, while those who have been in EA for several years are much more likely to know someone who has dropped out. Around 40% or more of respondents who have been in EA 5 years or more know someone who has dropped out.^[4]

Reasons people become less engaged

We asked respondents who reported knowing someone who disengaged from EA and specified an engagement level of this individual^[5] “To the best of your knowledge, why do you think they lost interest in effective altruism?” Respondents were offered fixed options and asked to select all that apply.

Respondents selected a mean number of 1.64 reasons with a median of 2 (not including “I don’t know”), despite being able to select any number of options, which does not suggest a wide variety of factors being at play in any given person disengaging from EA (at least so far as the respondents know).

No straightforward picture emerges from these results.

The 3 or 4 most commonly selected options have quite similar frequencies: lack of motivation or interest in the ideas of effective altruism (31%), other bad experiences with other effective altruists, online or in person (30%), lack of concrete opportunities to implement effective altruism in their lives (28%), lack of community of friends interested in EA (25%). Change in life circumstances, in fifth place, was also not far behind (21%).

These each suggest quite different problems and potential solutions. Individuals lacking interest in the ideas of effective altruism or motivation may be an insuperable problem, or may suggest only solutions that would generally make EA more interesting and motivating. Bad experiences with EAs, conversely, would require solutions which aim to address the specific kind of bad experiences which people have been having. More targeted qualitative work to discern what these experiences are may be necessary. Lack of concrete opportunities to implement EA may point to a structural problem, such as the low number of EA jobs (in fields people want to work in and which people are suitable for), which would be hard to address, or it might point to the need to identify and highlight more specific activities for people to do in order to keep them engaged. Lack of community or friends interested in EA may also be difficult to solve for people who are distant from other EA communities, but may point to the need for other ways to engage people in the community, whether online through more regional events, encouraging networking or mentorship.

It is also important to recall that these represent the responses of respondents who were asked about people who they thought would be good fits for the EA community. As such, they do not necessarily tell us about the reasons why most people disengage from EA: much higher percentages of these may include fundamental disagreements with EA, misunderstanding what effective altruism is and lack of interest in EA.

Reasons people become less engaged: qualitative data

We received 99 qualitative responses explaining further details about why the person in question disengaged from EA. Due to the small number we do not present quantitative frequencies resulting from these responses.

Many of these responses recapitulated themes from the fixed categories: changing life circumstances and lack of time, lacking a local community, personal issues, EA being too demanding, different values from EA etc.

Other themes which were frequently recurring included EA lacking left wing politics (e.g. not being Marxist, EA not prioritising minority groups), wanting to pursue a particular career other than a high impact one, and bad experiences with EAs.

Reasons for disengaging from EA across engagement levels

We can also examine how the reasons why respondents thought the person lost interest in EA varied depending on how engaged the respondent thought that person was with EA (based on self-reported levels of engagement from (1) Not at all engaged- (5) Highly engaged). Naturally these third-person estimates of the other person’s engagement level might be less reliable than the first-person estimates of respondents’ own engagement levels.^[6]

Some of the most striking differences here are for people who had the very highest levels of engagement with EA (level 5). Respondents were much more likely to say of the most engaged individuals, that they did not know why they lost interest in EA. They were also much more likely to attribute this to changes in life circumstances or bad experiences with other EAs, than for other categories of respondents (though, of course, the most engaged EAs might have been more likely to have had more experiences with EAs than less engaged individuals). In contrast, respondents were much less likely to say that the most engaged individuals lost interest due to “lack of motivation/​interest” in the ideas of EA, or lack of community or friends interested in EA, which makes sense since the most engaged EAs are likely to have, at least initially, had interest in EA and had access to community or friends interested in EA.

For most other categories of reason and other engagement levels, there were relatively few substantial differences in responses.

Differences Across Groups

Retention in EA

Gender and Retention

We also examined gender differences in which options respondents indicated were important for keeping them involved in EA. As in our previous post, we used a bootstrap approach to generate the appropriate probability distribution, we found a significant difference in the pattern of responses between men and women.

In line with the previous analyses, we found statistically significant differences in the number of women/​men selecting LessWrong and SSC as important (with fewer women selecting these), while more women selected personal contacts as important. There were also significant differences in the proportions of men and women selecting donating to EA funds (with fewer women selecting this relative to men).

When p-values corrected using Bonferroni adjustment, EA discord no longer sig (p=0.5359611)

White/​non-white Respondents and Retention

We only found differences in the proportions of white vs non-white EAs selecting EAGx as important for their retention to be significant, with more non-white EAs (though still a relatively small percentage overall) selecting this category.

Low/​High Engagement EAs and Retention

As in our analysis in this post, we categorised EAs as low or high in engagement based on their self-reported level of engagement on a 1-5 scale.

There were multiple large, significant differences in the proportions of low/​high engagement EAs selecting different factors as important for retaining them in EA. Personal contacts, local groups, being a local group leader, being involved in an EA project, attending EAG or EAGx were all disproportionately more often selected by high engagement EAs. Conversely, GiveWell, donating to EA Funds, donating somewhere else, SSC, LessWrong and Future Perfect were more often selected by those lower in engagement.

To show which factors were more important for highly engaged EAs specifically we show the percentages for this category alone sorted from low to high below.

As this graph makes quite clear, personal contacts were overwhelmingly most important, followed by local groups in the next two categories. Working on an EA project and EAG were the next most common categories, selected by around half as many EAs as local groups. Notably, while SlateStarCodex is selected by significantly fewer highly engaged EAs than less engaged EAs, it is still selected by roughly as many EAs as the EA Forum itself, and both are selected more often than GiveWell, Giving What We Can or LessWrong.

Barriers to involvement in EA

Gender and Barriers to Involvement in EA

We found significant differences in the barriers selected by men and women, with greater proportions of women selecting EA being elitist, not diverse and not welcoming. Nevertheless, too few EA jobs was the category most commonly selected by women, and the second most commonly selected for men, while no EA friends was most commonly selected by men.

White/​non-white Respondents and Barriers to Involvement

We found no significant differences in this analysis.^[7]

Low/​High Engagement EAs and Barriers to Involvement

We found a number of significant differences in what low and high Engagement EAs considered barriers to further involvement EA. These differences may reflect differences in what count as barriers at different stages of involvement (and length of time involved) i.e. people tend to experience one set of barriers when initially engaged and then a different set of barriers when more engaged, rather than differences between the EAs in each group themselves.

Less engaged EAs were much more likely to indicate that having no EA friends and no group was a barrier to further involvement in EA. THey were also more likely to indicate that it being unclear how to get involved in movement building and unclear where to donate were barriers.

Highly engaged EAs were more likely to select that it being too hard to get an EA job was a barrier, as well as EA not being diverse. They were also much more likely to indicate that =that there was no barrier to them being more involved.

Predictors of NPS and Change in Interest in EA

We also examined the predictors of higher responses for the ‘Net Promoter Score’ question and the change in level of interest in EA question, using ordinal regressions. We did not have very strong prior hypotheses and so included the set of demographic and measures of engagement in EA we have used in other regressions in this series. We also added cause prioritization to the models based on the supposition that this might be relevant. To the extent that the movement prioritizes your preferred cause area you might be expected to be more likely to promote EA, and to the extent that it doesn’t you might have become less interested in EA (see, for example, the qualitative comments above, which cited dissatisfaction with EA lacking information on certain causes and being too dominated by others). In fact, the data we have suggests that higher rankings of any given cause area were either associated with higher Net Promoter Scores and increased interest in EA, or were unrelated to these outcomes. Therefore, we collapsed causes down into a mean cause rating for all causes. More details on the models can be found here.

Neither of our models for predicting NPS and change in level of interest in EA explained more than ~5% of the variance in outcomes, so their utility is probably limited.

Net Promoter Score

This model explained only ~5% of the variance in outcomes. For a one unit increase in mean cause rating, the odds of the high score of Net Promoter Score versus the lower Net Promoter Scores are 1.5 times greater, given that the other variables in the model are held constant. Those with higher self-reported engagement levels in EA were 1.5 times more likely to select a higher Net Promoter Score. Similarly, LessWrong and EA Facebook members were 1.3 times more likely to select higher Net Promoter Scores than non-members. Conversely, non-members of the EA Forum were 1.3 times more likely to select higher Net Promoter Scores than Forum members.

Male EAs were 1.3 times more likely to select higher Net Promoter Scores than female EAs . Being older and having been longer in EA is associated with lower Net Promoter Score responses. Veg*ns were 1.2 times more likely to select higher Net Promoter Scores. Those living outside of the UK are 1.5 times more likely to have a higher Net Promoter Score than those living there. Membership of GWWC and local EA groups, race, education, student status, and living in the USA or Europe were not significant.

We can also examine the descriptive differences in NPS responses across the groupings highlighted by this analysis. This may help give a sense of the practical significance of the effects identified as statistically significant in the model. For example, if a certain characteristic is associated with being 1.5 times more likely to select a higher response, we might also ask whether there is a substantial difference between the responses of respondents with or without the characteristic, and whether the responses of either of these groups is low or high in absolute terms. We think that this is likely to be particularly important to consider given the low amount of variability explained by our models and lack of strong theoretical explanations for some of our findings. Since the regressions attempt to control for the influence of different characteristics simultaneously, it is possible that we can find that a factor appears very influential in the model, but does not correspond to a difference in the descriptive results between groups. Likewise, there may be large differences in the results across groups, which do not correspond to this factor appearing to be influential in our model when controlling for other factors.

The finding that higher self-reported levels of engagement in EA was associated with higher responses to the NPS question appears to correspond to very large differences in responses across these groups. The mean response (on a 1-10 scale) rises from just over ⁵⁄₁₀ for those reporting the lowest levels of engagement with EA, to almost ⁸⁄₁₀ for those with the highest levels of engagement in EA.^[8] It seems intuitively plausible that those who are more engaged with EA would be more excited about introducing others to EA.

The effect of having been in EA longer also corresponded to quite large differences across different cohorts of EAs. Examining mean NPS scores, we can see that there is a somewhat linear trend, but those who first heard of EA in 2019 (the year of the survey) report markedly higher excitement than those from earlier cohorts. We think this may reflect a fairly intuitive tendency for those who have just (within the last year) heard about EA to be especially excited about it, accompanied by a slight tendency for excitement about sharing EA to decrease over subsequent years of involvement. Interestingly this tendency is the opposite of the effect we might expect to see based on differential attrition: we might expect that those who are still involved in EA after 10 years would be the most enthusiastic, whereas those who are less positive about EA might be be expected to have dropped out by this point, leading ‘veteran’ respondents to be generally more positive due to a survivorship bias.

In contrast, there was no statistically significant difference comparing Net Promoter Scores between male and female respondents^[9] with women on average giving a response of 7.05 and men a score of 7.17.^[10] When simply comparing the pattern of responses for EA Forum members and non-members, Forum members gave substantially and significantly higher responses^[11], despite the fact that, in our model, Forum membership was associated with negative scores, when controlling for other factors. Given this, the practical significance of the findings that being male is associated with higher NPS scores and Forum membership is associated with lower scores is unclear.

Change in interest in EA

This model explained only 5% of the variance in outcomes. Those who gave on average higher cause rankings were 1.8 times more likely to select a higher response category indicating their change in interest in EA. Those with higher self-reported engagement levels in EA were 1.1 times more likely to select a higher response category indicating their change in interest in EA.

Non-members of GWWC and non-white EAs were 1.3 times likely to select a higher response category indicating their change in interest in EA. Those who had been longer in EA were more likely to select lower response categories on the change in interest in EA measure. Gender, education, veg*nism, student status, membership in the EA Forum, EA Facebook, LessWrong and place of residence were not significant.

There were clear differences in the pattern of responses across different cohorts of EA, as in the case of NPS responses, with many more respondents who first heard about EA more recently indicating that their interest in EA had increased. However, in the oldest cohorts, it was typically the case that the majority of respondents reported that their level of interest had not changed in the last year.

Comparing the responses of white and non-white respondents to the change in level of interest in EA over the last year question, there were significant differences, with a greater proportion of non-white respondents selecting the higher categories indicating increased interest and more white respondents indicating no change or decreased interest. Nevertheless, the differences are fairly small.

The figures below plot the estimated odds ratios and their 95% confidence intervals of these factors for each model.^[12]

Conclusions

This post collated information across a variety of measures concerning respondents’ experience of the EA community. It is an important limitation of this data that individuals who have had particularly negative experiences of the EA community may be much less likely to take the EA Survey and express their views (although, in theory, it is also quite possible that individuals with criticisms of the EA community might be more likely to take the survey in order to express their complaints).

Nevertheless, the data can still offer insights into the views of the >2000 individuals who do continue to take the EA Survey and highlight influences on the experiences of these individuals. In addition, we can use the large number of qualitative responses offered to collate specific examples of the kinds of experiences individuals have and what kinds of things they find helpful or unhelpful for their involvement in EA.

One recurring theme was that lack of time or being too busy was a very significant issue for people, both as a cause of decreasing interest in EA and as a barrier to being more involved in EA. This seems a relatively hard problem to solve, though it might be worth considering whether there are any ways to make EA involvement less time-consuming for individuals (of course, this may be in tension with a focus on the importance of the most engaged individuals). It may also be worth considering how EA involving a high time or attention cost may serve as a barrier to inclusivity and reduce diversity.

Responses here also highlighted the importance of personal contacts (in line with our earlier post on getting involved in the EA community), which were selected as important for retaining people within EA by the largest number of respondents. The importance of local groups was also highlighted repeatedly, being the most commonly cited reason for increasing interest in EA, the second most commonly selected factor for retaining people in EA. Likewise, around 43% of respondents to the question about ways individuals wished to be more involved in EA selected each of participating in more EA events locally and hanging out with more EAs socially.

We found a number of significant differences in the responses of male vs female and low engagement vs high engagement respondents, though few significant differences based on race, which might suggest directions for further research.

Credits

The annual EA Survey is a project of Rethink Charity with analysis and commentary from researchers at http://​​rethinkpriorities.org/​​. This essay was written by David Moss, with contributions from Kim Cuddington and Neil Dullaghan. Thanks to Ben Todd, Peter Hurford, Daniela Waldhorn and Jason Schukraft for comments.

We would also like to express our appreciation to the Centre for Effective Altruism and the EA Meta Fund for supporting our work. Thanks also to everyone who took and shared the survey.

If you like our work, please consider subscribing to our newsletter. You can see all our work to date here.

Other articles in the EA Survey 2019 Series can be found here

Appendices

Appendix 1: Reasons for Change in Interest Categories

Appendix 2: Engagement levels of those respondents know who disengaged

We asked respondents to think of one individual who they think might have been a good fit for the EA community but who disengaged (if they knew any who disengaged), and what their level of engagement with EA was (using the engagement scale employed earlier in the survey).

This question was posed explicitly to those who answered “Yes” to the previous question about whether they knew anyone who disengaged but, as it happened, 57 respondents answered this and the subsequent follow-up question, who had answered “No” to the question about whether they knew anyone who had been an EA and subsequently disengaged. These individuals potentially missed the caveat “If yes” and simply thought of an individual who they thought might have been a good fit for the EA community anyway. Notably most of these respondents selected an individual with no engagement (27) or only mild engagement (such as reading a few articles) (20), so it is plausible that they thought these were not “EAs” who had subsequently disengaged, but they were people who might have been a good fit for the community anyway. Nevertheless, we dropped these responses from the analysis and only analysed the 421 responses of those who selected “Yes” to the previous question.

Among these respondents, very few cited a person with “No engagement” in EA (which makes sense, as these people would plausibly not be counted as “EAs”). Roughly equal numbers selected each of the other categories.

Since respondents were just asked to think of one individual who they think might have been a good fit, it is impossible to infer anything from this distribution about the distribution of engagement levels among those who disengaged from EA (who were known to respondents), since we don’t know the motivation for respondents to select one individual rather than another.

Notes

---

1. ↩︎

   We coded each response to a single thematic category we thought it fit best, though many alluded to multiple factors. As such the percentages below give, at best, a rough sense of the frequency of these factors being mentioned. In the following responses we split the reasons given by those who indicated that their level of interest decreased and increased separately. Full explanations of the categories are included in Appendix 1.

2. ↩︎

   85% of these self-reported their level of engagement with EA as 3 or higher and 46% as 4 or higher, on a 1-5 scale.

3. ↩︎

   This is not a perfect measure of the proportion of those with experience of each factor rating it as important, because individuals might experience a local group, find it unhelpful and cease being a member. This might lead to the category of current local group members containing fewer people who didn’t find them helpful than the ‘have attended an EA Global’ category, which will contain anyone who previously attended even if they then stopped attending. Another way in which this analysis may be misleading as a proxy for individuals’ experience of the two activities is that individuals can selectively decide to engage with each based on their estimations of whether they are likely to be useful. For example, if people judge that EA Global/​a local group wouldn’t be useful to them, they can choose not to engage at all, leading to lower numbers of people with experience of that category, but perhaps higher proportions of individuals who have engaged with that activity finding it useful. It is unclear how far such selection might play a role in these categories. However, from 2018 EA Survey data, we know that of those respondents who were not a member of a local group, (n=1560), only 17.3% said that they do not want to be engaged in a local group, while 49.6% said that they were unsure whether they want to be involved in a local group.

4. ↩︎

   The relationship between knowing someone who disengaged from EA and number of EAs one knows well enough to ask a favour appeared to be significant (x2 (6, n=1540) = 586, p=<0.001)) as did the relationship between knowing someone who had disengaged from EA and years in EA (x2 (10, n=1487) = 104, p=<0.001)).

5. ↩︎

   The 64 respondents who did not answer “Yes” to the first question and who did not specify an engagement level were dropped. Dropping these respondents not change the order of most often to least often selected options, or substantially change the totals. Nevertheless, these respondents did differ substantially in their responses compared to those who answered “Yes” to the first question and who specified an engagement level for a specific EA. These respondents were much more likely to indicate that they didn’t know why the person disengaged, or to say that the person was not exposed to enough information about effective altruism (https://​​i.ibb.co/​​yy5gjmm/​​image.png). This makes sense since, as noted in Appendix 2, these respondents may have been thinking of people who weren’t really engaged in EA in the first place, but who they thought might have been a “good fit” for EA, and these respondents were much more likely to indicate that the individuals who they were thinking of had “no engagement” with EA.

6. ↩︎

   Very few respondents specified an EA at level (1) ‘No Engagement’ who then disengaged, which makes sense, so we dropped these from the analysis below.

7. ↩︎

   We found similar patterns when comparing the groups ‘selected exclusively white’ and ‘selected any categories other than white’.

8. ↩︎

   We are aware that computing the mean score for an ordinal variable has the potential to be misleading, especially where, as in this case, responses are not normally distributed, however we are not aware of a clearly superior alternative summary statistic to highlight differences in such cases.

9. ↩︎

   As elsewhere, we dropped respondents who preferred not to specify their gender or who selected Other, as the very small number of such respondents would preclude meaningful statistical analysis of group differences.

10. ↩︎

    Welch’s t-test: women (m=7.05 , SD=2.25) and men (M=7.17, SD=2.16); t(867)=-1.03, p=0.303

11. ↩︎

    Welch’s t-test: non-member (m=6.86 , SD=2.28) and member (M=7.68, SD=1.85); t(1316)=-8.19, p=<0.001

12. ↩︎

    These are proportional odds ratios and don’t necessarily reflect the effect of each factor on each outcome level independently.

[Max_Daniel]

Thank you for your excellent work on the survey! Like the other posts in this series, I found this very interesting.

One quick suggestion: it’s unclear to me if respondents interpreted “EA job” as (a) “job at an EA organization” or (b) “high-impact job according to EA principles”. E.g. doing AI safety research in academia would be an EA job according to (b) but not (a) [except maybe for a few exceptions such as CHAI]. It could be good to clarify this in future surveys.

(This is assuming that “Too hard to get EA job” is all information respondents saw. If there was additional information indicating whether (a) or (b) was intended, it would be good to clarify this in the post.)

[David_Moss]

Thanks for your comment Max!

it’s unclear to me if respondents interpreted “EA job” as (a) “job at an EA organization” or (b) “high-impact job according to EA principles”

I agree. This was one of the questions which was requested externally, that I mentioned at the top of the post, which I included verbatim, so I don’t know which was the intended meaning. The precise wordings were “Too hard to get an EA job” and “Not enough job opportunities that seemed like a good fit for me” which I agree could be interpreted more narrowly or more broadly.

To perhaps gain a little insight, we can cross-reference this with our data on respondents’ career plans. Among those that included ‘Work at an EA non-profit’ in their plans (note that this was a multi-select question), 35.7% said that “Too hard to get an EA job” was a barrier to being more involved in EA. Conversely, among those that did not include working for an EA non-profit in their career plan, 20.1% selected this as a barrier. This is a significant difference (p<0.001), but notably it means that many participants who selected this as a barrier were not aiming to work in a specifically EA org. In fact, to put it another way, 49.3% of those who selected this as a barrier did not say they planned to work in an EA non-profit, whereas 50.7% did plan to work in an EA org (but note that many of these also included other routes, like academia, in their career plans, so it’s not clear that it being too hard to work in an EA org specifically was what they viewed as a barrier). Of course, it’s also possible that for some of these respondents it was because it was too hard to get a job in an EA org, which they viewed as a barrier, that they did not select EA org as part of their career plans.

[Linda Linsefors]

Thanks for doing this!

When comparing whites and non-whites, did you do anything to control for location.

I noticed non-whites ranked EAG as less important. Could this be becasue they are more likely to live far away from EAG events?

Or maybe there are so few EAs living in non-white majority countries, that they don’t skew the statistic? I.e. non-white EAs in majority white countries massively outnumber non-white EAs in non-white majority countries?

[David_Moss]

Hi Linda. Thanks for the question!

Could I ask to clarify which question you are looking at? I assume maybe the importance for retention question? There we observe that non-white respondents are more likely to select EAGx specifically (about twice as large a percentage of non-white respondents selected EAGx) and indeed I expect this is driven by geography for the reasons you say. There is no significant difference for EAG, though a slightly higher percentage of non-white respondents selected that as well.)

To answer your question about the analyses: the chi-square tests just look at whether there are differences in the proportions of white/​non-white, male/​non-male respondents selecting different categories don’t attempt to control for other characteristics. So you should just read these as identifying differences between these groups, rather than as necessarily showing that these differences are explained by the groupings themselves. Note that just knowing the proportions, even if they’re not causal may still be action-relevant, i.e. we might want to know what programs are actually helping a larger number of non-white EAs, even if this is ultimately explained by some third factor). In contrast, in the models at the end looking at predictors of NPS and change in level of interest in EA we do try to control for different influences simultaneously.