David_Moss

Karma: 7,038

I am the Principal Research Director at Rethink Priorities and currently lead our Surveys and Data Analysis department. Most of our projects involve private commissions for core EA movement and longtermist orgs, where we provide:

Private polling to assess public attitudes
Message testing / framing experiments, testing online ads
Expert surveys
Private data analyses and survey / analysis consultation
Impact assessments of orgs/programs

Formerly, I also managed our Wild Animal Welfare department and I’ve previously worked for Charity Science, and been a trustee at Charity Entrepreneurship and EA London.

My academic interests are in moral psychology and methodology at the intersection of psychology and philosophy.

David_Moss 24 Apr 2024 8:48 UTC
2 points
0 ∶ 0
in reply to: Linch’s comment on: On failing to get EA jobs: My experience and recommendations to EA orgs
I think this is one piece of information you would need to include to stop such a statement being misleading, but as I argue here, there are potentially lots of other pieces of information which would need to be included to make it non-misleading (i.e. information about any and all other confounders which explain the association).
Otherwise, applicants will not know that conditional on X, they are not less likely to be successful, if they do not have a PhD (even though disproportionately many people with X have a PhD).
Edit: TLDR, if you do not also condition on satisfying the role requirements, but only on applying, then this information will still be misleading (e.g. causing people who meet the requirements but lack the confounded proxy to underestimate their chances).

David_Moss 24 Apr 2024 8:43 UTC
3 points
1 ∶ 0
in reply to: ixex’s comment on: On failing to get EA jobs: My experience and recommendations to EA orgs
As I suggested in my first comment, you could do the same “by reporting other characteristics which play no role in selection, but which are heavily over-represented in successful applicants”: for example, you could report that >50% of successful applicants are male,^[1] white, live in certain countries, >90% have liberal political beliefs, and probably a very disproportionately large number have read Harry Potter fan fic.^[2] Presumably one could identify other traits which are associated with success via their association with these other traits e.g. if most successful applicants have PhDs and PhDs disproportionately tend to [drink red wine, ski etc.], then successful applicants may also disproportionately have these traits.
Of course, different people can disagree about whether or not each of these are causal. But even if they are predictive, I imagine that we would agree that at least one of these would likely mislead people. For example, having read Harry Potter fan fic is associated with being involved with communities interested in EA-related jobs for largely arbitrary historical reasons.^[3]
This concern is particularly acute when we take into account the pragmatics of employers highlighting some specific fact.^[4] People typically don’t offer irrelevant information for no reason. So if orgs go out of their way to say “>50% of successful applicants have PhDs”, even with the caveat about this not being causal, applicants will still reasonably wonder “Why are they telling me this?” and many will reasonably infer “What they want to convey is that this is a very competitive position and I should not apply.”
As I mentioned in the footnote of my comment above, there are jobs where this would be a reasonable inference. But I think most EA jobs are not like this.
If one wanted to provide applicants with full, non-misleading information, I think you would need to distinguish which of the cases applies, and provide a full account of the association which explains why successful applicants might often have PhDs, but that this is not the case when you control for x, y, z. That way (in theory), applicants would be able to know that conditional on them being a person who meets the requirements specified in the application (e.g. they can complete the coding test task), the fact that they don’t have a PhD does or does not imply anything about their chances of success. But I think that in practice, providing such an account for any given trait is either very difficult or impossible.^[5]
1. ^
  Though in EA Survey data, there is no significant gender difference in likelihood of having an EA job. In fact, a slightly larger proportion of women tend to have EA jobs.
2. ^
  None of these reflect real numbers from any actual hiring rounds, though they do reflect general disparities observed in the wider community.
3. ^
  Of course, you could describe a situation where having read Harry Potter fan fic actually serves as a useful indicator of some relevant trait like involvement in the EA community. But, again, I’m not referring to cases like this. Even in cases where involvement in the EA community is of no relevance to the role at all (e.g. all you need to do to be hired is to perform some technical, testable skill, like coding very well), applicants are likely to be disproportionately interested in EA, and successful applicants may be yet further disproportionately interested in EA, even if it has nothing to do with selection.
  This can happen if, for example, 50% of the applications are basically spam (e.g. applications from a large job site, who have barely read the job advert and don’t have any relevant skills but are applying for everything they can click on). In such cases, the subset of applications who are actually vaguely relevant, will be disproportionately people with an interest in EA, people with degrees etc.
4. ^
  In some countries there may be a norm of releasing information about certain characteristics, in which case this consideration doesn’t apply for those characteristics, but would for others.
5. ^
  And that is not taking into account the important question of whether all applicants would actually update on such information provided completely rationally, or would whether many would be irrationally inclined to be negative about their chances, and just conclude that they aren’t good enough to apply if they don’t have a PhD from a fancy institution.

David_Moss 23 Apr 2024 21:02 UTC
2 points
0 ∶ 0
in reply to: ixex’s comment on: On failing to get EA jobs: My experience and recommendations to EA orgs
I broadly agree that such a statement, taken completely literally, would not be misleading in itself. But it raises the questions:
- What useful information is being conveyed by such a statement?
  - If interpreted correctly, I think the applicant should take anything actionable away from the statement. But what I suspect many will do, will be to conclude that they probably shouldn’t apply if they don’t have a PhD, even if they meet all the requirements.
- What is pragmatically implied by such a statement?
  - People don’t typically go out of their way to state things which they don’t think are relevant (without some reason). So if employers go out of their way to state “>50% of successful applicants had a PhD...”, even with the caveat, people are reasonably going to wonder “Why are they telling me this?” and a natural interpretation is “They want to communicate that if I don’t have a PhD, I’m probably not suited to the role, even if I meet all the requirements”, which is exactly what employers don’t want to communicate (and is not true) in the cases I’m describing.^[1]
1. ^
  I think there are roles where unless you have a PhD, you are unlikely to meet the requirements of the role. In such cases, communicating that would be useful. But the cases I’m describing are not like that: in these cases, PhDs are really not relevant to the roles, but applicants will have very commonly undertaken PhDs. I imagine that part of the motivation for wanting to see the information are because people think that things are really like the former case, not the latter case.

David_Moss 23 Apr 2024 15:50 UTC
6 points
1 ∶ 0
in reply to: Jason’s comment on: On failing to get EA jobs: My experience and recommendations to EA orgs
Whether this information is good to include depends crucially on the causal relationships though.
In the simple test score case, academic ability causes both test scores and admission success, and test scores serve as a strong proxy for academic ability and we assume no other causal relationships complicating matters. Here test scores serve as a useful proxy for academic ability, and are relatively innocent as an indicator for likelihood of admission (i.e. they serve as a pretty good indicator of whether one is likely to succeed).
But telling people about something which was strongly associated with success, but not causally connected with the factors which determine success in the right way would be misleading.
In a more complex (but perhaps realistic) case, where completing a PhD is causally related to a bunch of other factors, then saying that most successful applicants have PhDs risks being misleading about one’s chances of success / suitability for the role and about the practical utility of getting a PhD for success.

David_Moss 23 Apr 2024 10:32 UTC
26 points
9 ∶ 4
on: On failing to get EA jobs: My experience and recommendations to EA orgs
Sometimes EA Orgs will say something like “we have no degree requirements”… but in reality will mostly hire people with PhDs. I appreciate your open-mindedness regarding degree and experience requirements, but saying something like “we expect most successful applicants to have X”… helps applicants assess whether the opportunity is worth spending time on.
I would advise against this specific policy, since I think it risks being misleading. My impression is that many EA employers genuinely assign ~0 weight to an applicant having a PhD, but nevertheless find themselves hiring disproportionately PhDs, simply because the pool of people who have the traits they are interested in (e.g. interested in autonomously completing very high-level research) disproportionately choose to complete PhDs.^[1] If an org simply reports “>50% of successful applicants have PhDs”, this will likely mislead applicants into thinking that having a PhD is important to success, even though it’s assigned no weight (one could imagine similar dynamics by reporting other characteristics which play no role in selection, but which are heavily over-represented in successful applicants).
To be clear, if employers actually do assign significant weight to a characteristic, but are open to considering exceptions, then I think it’s good to be transparent about both sides of that.
1. ^
  The EA community itself is very disproportionately skewed towards people with graduate degrees, which probably contributes to this. Last time we checked, in 2019, >45% of respondents had a graduate degree, and I would expect this number to have only increased since then, since many of the remainder were students, who will themselves likely go on to complete further degrees at quite high rates.

David_Moss 17 Apr 2024 18:38 UTC
13 points
0 ∶ 0
on: How good it is to donate and how hard it is to get a job
only 10% of non-student respondents worked at an Effective Altruism organization...
Donations are an amazing opportunity, and I think they are underemphasized
Our 2022 survey offers further illustration of this. Only 10% of respondents have earning to give as their career plan. And that masks a stark divide between highly engaged EAs, for whom less than 6% plan to pursue earning-to-give, compared to closer to 16% of less engaged EAs.

David_Moss 1 Apr 2024 11:28 UTC
112 points
16 ∶ 0
on: EA is now scandal-constrained
Our data suggests that the highest impact scandals are several times more impactful than other scandals (bear in mind that this data is probably not capturing the large number of smaller scandals).
If so, it seems plausible we should optimise for the very largest scandals, rather than simply producing a large volume of less impactful scandals.

David_Moss 29 Mar 2024 10:29 UTC
8 points
2 ∶ 0
in reply to: AnonymousEAForumAccount’s comment on: Updates on Community Health Survey Results
The relatively high frequency of people with high satisfaction temporarily stopping promoting EA (and the general flatness of this curve)

Agreed. I think that people temporarily stopping promoting EA is compatible with people who are still completely on board with EA, deciding that it’s strategically unwise to publicly promote it, at a time when there’s lots of negative discussion of it in the media. Likewise with still promoting EA, but stopping referring to it as “EA”, which also showed high levels across the board.
I think the prevalence of these behaviours points to the importance of more empirical research on the EA brand and how it compares to alternative brands or just referring to individual causes or projects (see our proposal here). I think it’s entirely possible that the term “EA” itself has been tarnished and that people do better to promote ideas and projects without explicitly branding them as EA. But there’s a real cost to just promoting things piecemeal or using alternative terms (e.g. “have you heard of “high impact careers” / “existential security”?”), rather than referring to a unified established brand. So it’s not clear a priori whether this is a positive move.
I was surprised that for the cohort that changed their behavior, “scandal” was just one of many reasons for dissatisfaction and didn’t really stand out. The data you provide looks quite consistent with Luke Freeman’s observation: “My impression is that that there was a confluence of things that peaking around the FTX-collapse...”
Agreed. One possible explanation, other than it just being a co-incidence of factors, is that the FTX crisis and subsequent revelations dented faith in EA leadership, and made people more receptive to other concerns. (I think historically, much of the community has been extremely deferential to core EA orgs and ~ assumed they know what they’re doing come what may).
Certainly it’s true that many of the other factors e.g. dissatisfaction with cause prioritisation, diversity, and elitism had been cited for a while. It’s also true that even before FTX (though it still holds for 2022), people who had been in the community longer tended to be less satisfied with the community, even though higher engagement was associated with higher satisfaction.^[1] While the implications of this for the average satisfaction level of the community depend on how many newer vs older EAs we have at a given time, this is compatible with a story where EAs generally become less satisfied with the community over time.
1. ^
  Note that this is the opposite direction to what you’d see if less satisfied people drop out, leaving more satisfied people remaining in earlier cohorts. That said the linked analyses (for individual years) can’t rule out the possibility that earlier cohorts have just always been distinctively less satisfied, which would require a comparison across years.

David_Moss 28 Mar 2024 16:17 UTC
8 points
2 ∶ 0
in reply to: AnonymousEAForumAccount’s comment on: Updates on Community Health Survey Results
This is a neat idea, but I think that’s probably putting more weight on the (absence) of small differences at particular levels of the response scale than the smaller sample size of the Extra EA Survey will support. If we look at the CIs for any individual response level, they are relatively wide for the EEAS, and the numbers selecting the lowest response levels were very low anyway.

David_Moss 28 Mar 2024 16:08 UTC
20 points
0 ∶ 0
in reply to: AnonymousEAForumAccount’s comment on: Updates on Community Health Survey Results
Many thanks!
I think it would be very valuable to closely examine the cohort of people who report having changed their behavior...
All behaviour changes were correlated with each other (except for stopping referring to EA, while still promoting it, which was associated with temporarily stopping promoting EA, but somewhat negatively associated with other changes).
All behaviour changes were associated with lower satisfaction, with most behavioural changes common only among people with satisfaction below the midpoint, and quite rare with satisfaction above the midpoint (again, with the exception of stopping referring to EA, while still promoting it, which was more common across levels).
People who reported a behavioural change were more likely, on the whole, to mention factors as reasons for dissatisfaction. (When interpreting these it’s important to account for the fact that people being more/less likely to mention a factor at a particular importance level might be explained by them being less/more likely to mention it at a different importance level, with less difference in terms of their overall propensity to mention it).
Similarly, there was no obvious pattern of particular factors being associated with lower satisfaction. In general, people who mentioned any given factor were less satisfied.
In principle, we could do more to assess whether any particular factors predict particular behavioural changes, controlling for relevant factors, but it might make more sense to wait for the next full iteration of the EA Survey, when we’ll have a larger sample size, and can ask people explicitly whether each of these things are factors (rather than relying on people spontaneously mentioning them.
For the other measures, differences are largely as expected, i.e. people who made a behaviour change are more likely to desire more community change, more likely to strongly agree there’s a leadership vacuum,^[1] and trust was higher among people who had not made a behaviour change.
Updating analyses of community growth seems like it should be a high priority… I’ve been a longstanding proponent of conducting regular analyses of community growth..
I still agree with this, unfortunately, we’ve been unsuccessful in securing any funding for more analysis of community growth metrics.
1. ^
  I personally don’t put too much weight on this question. I worry that it’s somewhat leading, and that people who are generally more dissatisfied are more likely to agree with it, but it’s unclear that leadership vacuum is really an active concern for people or that it’s what’s driving people’s dissatisfaction.

David_Moss 27 Mar 2024 17:48 UTC
4 points
0 ∶ 0
in reply to: Ben_West’s comment on: Updates on Community Health Survey Results
Thanks!
For satisfaction, we see the following patterns.
- Looking at actual satisfaction scores post-FTX, we see more engaged people were more highly satisfied than less engaged people. In comparison, for current satisfaction, this is no longer the case or is only minimally so (setting aside the least engaged who remain less satisfied than the moderately to highly engaged). Every group’s satisfaction has decreased, with moderate to highly engaged EAs’ satisfaction declining to similar levels (implying a larger decrease among the more highly engaged).
  - The pattern is roughly similar, but less clear, looking only at changes within matched subjects (smaller sample size).
- Looking at people’s recalled post-FTX satisfaction, there is no significant difference between the moderately to highly engaged (though they weakly lean in the opposite direction). So the recalled vs current comparison implies a slightly bigger positive gap for more highly engaged EAs (though we did not formally test this comparison).
For reasons for dissatisfaction, there are a few systematic differences across engagement levels:
- More highly engaged respondents are more likely to mention Leadership
- More highly engaged respondents were more likely to mention scandals
- More highly engaged respondents were more likely to mention JEID at the lower importance levels (Important or Slightly important vs Very important), but it’s a less clear pattern at the Very important level
- More highly engaged responents were more likely to mention Epistemics as Very important
- The most highly engaged respondents were much more likely to mention Funding (though still less than the top factors)
- Looking within the most highly engaged only, this that Leadership and Scandals are at the top, followed by Cause prioritization and JEID receiving similar levels of mentions.

David_Moss 20 Mar 2024 22:15 UTC
12 points
2 ∶ 0
in reply to: Jason’s comment on: Updates on Community Health Survey Results
Does “mainly a subset” mean that a significant majority of responses coded this way were also coded as cause prio?
That’s right, as we note here:
The Cause Prioritization and Focus on AI categories were largely, but not entirely, overlapping. The responses within the Cause Prioritization category which did not explicitly refer to too much focus on AI, were focused on insufficient attention being paid to other causes, primarily animals and GHD.
Specifically, of those who mention Cause Prioritization, around 68% were also coded as part of the AI/x-risk/longtermism category. That said, a large portion of the remainder mentioned “insufficient attention being paid to other causes, primarily animals and GHD” (which one may or may not think is just another side of the same coin). Conversely, around 8% of comments in the AI/x-risk/longtermism category were not also classified as Cause Prioritization (for example, just expressing annoyance about supporters of certain causes wouldn’t count as about Cause Prioritization per se).
So over 2/3rds of Cause Prioritization was explicitly about too much AI/x-risk/longtermism. A large part of the remainder is probably connected, as part of a ‘too much x-risk/too little not x-risk’ category. The overlap between categories is probably larger than implied by the raw numbers, but we had to rely on what people actually wrote in their comments, without making too many suppositions.

David_Moss 20 Mar 2024 22:02 UTC
16 points
3 ∶ 0
in reply to: titotal’s comment on: Updates on Community Health Survey Results
We did note this explicitly:
As we noted in our earlier report, individuals who are particularly dissatisfied with EA may be less likely to complete the survey (whether they have completely dropped out of the community or not), although the opposite effect (more dissatisfied respondents are more motivated to complete the survey to express their dissatisfaction) is also plausible.
I don’t think there’s any feasible way to address this within this, smaller, supplementary survey. Within the main EA Survey we do look for signs of differential attrition.

David_Moss 20 Mar 2024 18:27 UTC
22 points
6 ∶ 0
in reply to: Ulrik Horn’s comment on: Updates on Community Health Survey Results
Thanks Ulrik!
We can provide the percentages broken down by different groups. I would advise against thinking about this in terms of ‘what would the results be if weighted to match non-actual equal demographics’ though: (i) if the demographics were different (equal) then presumably concern about demographics would be different [fewer people would be worried about demographic diversity if we had perfect demographic diversity], and (ii) if the demographics were different (equal) then the composition of the different demographic groups within the community would likely be different [if we had a large increase in the proportion of women / decrease in the proportion of men, the people making up those groups would plausibly differ from the current groups].
That said, people identified as a woman or anything other than a man, were more likely to mention JEID as at last of somewhat importance, and they were also more likely to mention cause prioritization and excessive focus on AI/x-risk/longtermism as a concern. Conversely, men were more likely to refer to scandals, leadership and epistemics.
I would be even more cautious about interpreting the differences based on race due to the low sample size (the total number would be much larger in the full EA Survey), and the fact that the composition of non-white respondents as a group differs from what you would see in a ‘perfectly equal demographics’ scenario (i.e. more Asian, unequal representation across countries).

Updates on Community Health Survey Results

David_Moss20 Mar 2024 11:56 UTC

147 points

19 comments8 min readEA link

David_Moss 25 Jan 2024 9:01 UTC
7 points
0 ∶ 0
in reply to: andrew_richardson’s comment on: Most recent EA giving/prioritization survey?
Hopefully in the next couple/few weeks, though we’re prioritising the broader community health related questions from that followup survey linked above.

I can confirm that there’s not been so dramatic a shift since the 2020 cause results (below for reference), i.e. global poverty and AI are still very similarly ranked. The new allocation-of-resources data should hopefully give an even clearer sense of ‘how much of EA’ people want to be this or that.

David_Moss 24 Jan 2024 10:43 UTC
5 points
1 ∶ 0
on: Most recent EA giving/prioritization survey?
We did gather cause prioritization data in the most recent EA Survey, we just delayed publishing that report because we gathered additional cause prioritization data in this followup survey, which we ran in December. This was looking at what share of resources EAs would allocate to different causes, rather than just their rating of different causes, which I think adds an important new angle.
We stopped gathering information about donations to individual charities in 2020 as part of a drive to make the EA Survey shorter to increase participation. However, that does mean that you can access donation data more recent than 2019, in our 2020 report (and the accompanying bookdown), which reports a breakdown by charity cause area.

David_Moss 18 Dec 2023 14:45 UTC
4 points
1 ∶ 0
in reply to: Jason’s comment on: Announcing Surveys on Community Health, Causes, and Harassment
Thanks! I agree that allocating a percentage of “resources”, where this contains very different kinds of resources (money and labour) can be difficult. Still, we wanted this to largely match the question asked at the Meta Coordination Forum, which also combined this, so we matched their wording.

David_Moss 18 Dec 2023 14:41 UTC
7 points
0 ∶ 0
in reply to: James Herbert’s comment on: Are 1-in-5 Americans familiar with EA?
Thanks!

We’ll definitely be reporting on changes in awareness of and attitudes towards EA in our general reporting of EA Pulse in 2024. I’m not sure if/when we’d do a separate dedicated post towards changes in EA awareness/attitudes. We have a long list (this list is very non-exhausive) of research which is unpublished due to lack of capacity. A couple of items on that list also touch on attitudes/awareness of EA post-FTX, although we have run additional surveys since then.
Feel free to reach out privately if there are specific things it would be helpful to know for EA Netherlands.

David_Moss 18 Dec 2023 10:24 UTC
4 points
1 ∶ 0
in reply to: Luke Freeman’s comment on: Announcing Surveys on Community Health, Causes, and Harassment
Thanks Luke! Everyone who opted to receive additional surveys by email in the last EA Survey, will have received an email with this survey (December 11-12th). We find they often get sent to the spam folder though, so you might want to check.
If you follow the link in that email, it won’t automatically pre-fill your email or automatically track you, but you will be able to know which email address it was sent to, so that you can enter that one.