See the post introducing this sequence for context, caveats, credits, and links to prior discussion relevant to this sequence as a whole. This post doesn’t necessarily represent the views of my employers.
This post briefly highlights some things that I (and I think many others) have observed or believe, which I think collectively demonstrate that the current processes by which new EA-aligned research and researchers are “produced” are at least somewhat insufficient, inefficient, and prone to error. (Though I do think that what we have is far better than nothing, that it’s been getting better over time, and that it’s probably better than what most analogous communities/fields would have.)
I think that these observations/beliefs help (a) clarify the exact nature of the “problems” with the EA-aligned research pipeline and (b) hint at what interventions to improve the pipeline would need to do.
In brief:
There are many important open questions
Many orgs and funders want research(ers)
There are many aspiring/junior researchers
Some shouldn’t pursue EA-aligned research roles
Some should pursue and get EA-aligned research roles (but don’t)
Maybe more of them should try independent research?
Independent research attempts could be improved
Non-EA research efforts could be improved
Existing solutions are inefficient and insufficient
Senior people have capacity to help, if efficiently leveraged
That said, there are also many other ways one could break down, frame, and investigate problems in this area. For example, one could focus more on EA-aligned researchers insufficiently learning from or gaining credibility with actors outside the EA community (see also field building).[1] Or one could focus on more specific bottlenecks/pain points for specific groups of people, and conduct interviews and surveys with members of those groups to gather data on that.[2]
The following posts in this sequence will discuss what we could do about the problems highlighted here.
1. Many open questions
There’s a huge amount of high-priority research to be done.
It seems to me that this is strikingly clear, so, for brevity, I won’t provide much further justification here (but I’m happy to do so in the comments, if requested!).
As one piece of evidence, see A central directory for open research questions. Presumably many of those questions aren’t actually high-priority or are no longer “open”, but I’m pretty sure a substantial fraction are, and that’s a substantial fraction of avery large number of questions!
2. Orgs and funders want research(ers)
There are many orgs and funders who would be willing and able to hire or fund people to do such researchif there were people who the orgs/funders could trust would do it well (and without requiring too much training or vetting). But not if the orgs/funders think the people are choosing lower-priority questions, are inexperienced, or aren’t especially skilled, or if the orgs/funders simply would have a hard time assessing the people and/or their plans. (See also EA is vetting-constrained and Ben Todd discussing organizational capacity, infrastructure, and management bottlenecks.)[3]
3. Many aspiring/junior researchers
There seem to be many EAs[4] who don’t yet have substantial experience doing EA-relevant research and for whom one or more of the following is true:
They want to do EA-relevant research long-term
They want to test their fit for doing EA-relevant research
They want to do a bit of EA-relevant research in order to gain knowledge and skills that will improve their ability to do other high-impact things
E.g., by expanding their knowledge of global priorities in order to inform career or donation choices, or learning more about AI in order to inform their actions as a civil servant
They want to do a bit of EA-relevant research because that research could itself be valuable
This could either be because the person has to do a research project anyway (e.g., a thesis) or because they have some additional time on their hands
For convenience, I will sometimes lump all such people together under the label “aspiring/junior researchers”. But it’s important to note that that includes several quite different groups of people, who face different pain points and would benefit from different “solutions”.
4. Some shouldn’t pursue EA-aligned research roles
Many of those EAs would probably be better off focusing on activities/roles other than EA-aligned research (given their comparative advantage, the limited number of roles currently available, etc.). But it can be hard for those EAs to work out whether that’s the case for them, and thus they might (very understandably!) continue testing fit for such roles, trying to skill up, etc. This can lead to unnecessary time costs for these people, for hirers, for grantmakers, etc.
5. Some should pursue and get EA-aligned research roles (but don’t)
Meanwhile, many other EAs probably do have a comparative advantage for EA-aligned research, and yet self-select out of pursuing relevant jobs, funding, collaborations, or similar, or pursue such things but find it very hard to get them.[5] This may often be because orgs and funders have insufficient ability to vet, train, and manage these people (see also section 2 above).
6. Maybe more should try independent research?
It would probably be good for many EAs to simply try doing (semi-)independent EA-aligned research. (This could take the form of quite “cheap” efforts, like 10 hours trying to produce a blog post. This might help the EAs test, signal, or improve their fit for EA-aligned research, and might produce valuable outputs; see also.) But it seems that many of those EAs either never actually try this, or only get started after a delay that is much longer than necessary.
Some possible reasons for this include:
Some of these people are (understandably!) not willing or able to (a) try (semi-)independent research while working or (b) reduce their hours of paid work.
They find it hard to think of or find any research questions that seem (a) worth researching in general and (b) well-suited to their skills and interests.
They find it hard to determine which questions would be worth researching in general and are well-suited to their skills and interests.
They find it hard to pick between the questions that seem to do well on those criteria.
They find it hard to know exactly how to operationalise a research question, how to break it into sub-questions, where to start looking for relevant prior work, etc.
They find it hard to be motivated to start without some clearer signal that doing so would have a good payoff for them (e.g., an increased chance of a desired job), or that someone thinks they specifically should indeed try doing EA-aligned research.
They think or worry that they’ll do a bad or slow job at their current skill level without mentorship.
They don’t entirely feel they have “permission” to just get started, or they aren’t/don’t feel sufficiently “agenty”, or something like that.
7. Independent research attempts could be improved
When EAs do try doing EA-relevant research (semi-)independently, it seems that one or more of the following problems often occur:
The question they pick is substantially lower priority or less fitting for them than another question they could’ve picked.
They do a poor job of operationalising the question, breaking it into sub-questions, looking for relevant prior work, etc.
They had to spend a lot of time generating or sifting through question ideas, or figuring out how to operationalise questions, how to break questions into subquestions, where to start looking for relevant prior work, etc.
They find it hard to stay motivated and productive and do a good job without mentorship.
They don’t have a clear sense of who the target audiences for the research should be, what its path to impact / theory of change should be, etc.
As such, the aspiring/junior researcher doesn’t frame the research in the ideal way; doesn’t pursue the ideal constellation of sub-questions; and doesn’t take appropriate actions to ensure their research is critiqued, is built on, influences decisions, and/or is taken as a signal of the researcher’s fit for future projects.
(Additionally, sometimes the researcher does an impressive job, contributing to an org thinking the researcher is probably worth hiring, but the org still lacks sufficient funding or management capacity to hire the researcher.[6])
8. Non-EA research efforts could be improved
There are also a huge number of non-EAs who have to or want to do research projects and who could in theory be tackling higher-priority questions in a more useful/high-quality and efficient way than they currently are.
(Additionally, if these researchers were made aware of higher-priority questions and given tools that helped them tackle questions in a more useful/high-quality and efficient way, that might increase their awareness of, inclination towards, and/or engagement with EA ideas.)
9. Existing solutions are inefficient and insufficient
We have some fairly good ways of partially addressing those problems, such as a handful of research training programs, the post A central directory for open research questions, and 1-1 conversations between senior and aspiring/junior researchers (to help the aspiring/junior researchers learn about open questions, think about how high-priority and fitting various questions are, think about the paths to impact, get mentorship, etc.).
But:
Those partial solutions require more time from senior people in EA than seems ideal.
The above problems still partially persist. Reasons for that might include the following:
There is a limited availability of time from senior EAs
Collections of research questions often fail to provide clear ways to determine how high-priority each question is relative to other questions, what type of person is a good fit for addressing each question, what other questions each question connects to or could be broken down into, where to start looking for resources, what the paths to impact might be, etc.
In general, it’s hard to get people to do something on a volunteer basis and/or with little or no mentorship/management.
People may not be aware of clear examples of aspiring/junior researchers who independently tackled one or more open research questions and thereby provided direct value, gained status in the EA community, had an easier time finding a good job, or similar.
10. Senior people have capacity to help, if efficiently leveraged
I think that there are probably many “senior” EAs who would be happy to put insmallamounts of time to help aspiring/junior researchers.[7]
For example, I think many senior EAs might be willing to:
Have one or two 30 min calls with an aspiring/junior researcher
Add research topic ideas to a centralised resource
Add comments to a centralised resource about how research on a topic might inform their own future research or decisions
Providing some indication about how high-priority various questions seem, what type of person might be a good fit for them, etc.
You disagree with any of those observations or the inferences I draw from them
You want to suggest additional observations or inferences related to the EA-aligned research pipeline
You feel that you’ve personally “suffered” in some way from the EA-aligned research pipeline being somewhat insufficient and inefficient (e.g., you’ve found it very hard to get a relevant job or usefully try out independent research, or you’re spending lots of time giving 1-1 help to aspiring/junior researchers and feel that your input could be used more efficiently)
Indeed, I think that people (considering) spending a lot of resources trying to improve the EA-aligned research pipeline should consider doing things like conducting interviews and surveys with relevant groups, and one weakness of this sequence is that I haven’t done that myself.
Note that I’m not saying that additional funding no longer has value or that “EA no longer has any funding constraint”; we could still clearly do more with more funding.
I’m using the term “EAs” as shorthand for “People who identify or interact a lot with the EA community”; this would include some people who don’t self-identify as “an EA”.
To some extent, this is also true for jobs in general (in the big, wide, non-EA world). But it still seems worth thinking about whether and how the situation in relation to EA-aligned research could be improved. And it does seem like the situation might be unusually extreme in relation to EA-aligned research (see also After one year of applying for EA jobs: It is really, really hard to get hired by an EA organisation).
Reasons why I believe this include various conversations with senior EAs, various conversations with aspiring/junior researchers who got brief help from senior EAs, and the willingness of many senior EAs to act as mentors for various research training programs (e.g., SERI, Legal Priorities Project) and for Effective Thesis.
What’s wrong with the EA-aligned research pipeline?
See the post introducing this sequence for context, caveats, credits, and links to prior discussion relevant to this sequence as a whole. This post doesn’t necessarily represent the views of my employers.
This post briefly highlights some things that I (and I think many others) have observed or believe, which I think collectively demonstrate that the current processes by which new EA-aligned research and researchers are “produced” are at least somewhat insufficient, inefficient, and prone to error. (Though I do think that what we have is far better than nothing, that it’s been getting better over time, and that it’s probably better than what most analogous communities/fields would have.)
I think that these observations/beliefs help (a) clarify the exact nature of the “problems” with the EA-aligned research pipeline and (b) hint at what interventions to improve the pipeline would need to do.
In brief:
There are many important open questions
Many orgs and funders want research(ers)
There are many aspiring/junior researchers
Some shouldn’t pursue EA-aligned research roles
Some should pursue and get EA-aligned research roles (but don’t)
Maybe more of them should try independent research?
Independent research attempts could be improved
Non-EA research efforts could be improved
Existing solutions are inefficient and insufficient
Senior people have capacity to help, if efficiently leveraged
That said, there are also many other ways one could break down, frame, and investigate problems in this area. For example, one could focus more on EA-aligned researchers insufficiently learning from or gaining credibility with actors outside the EA community (see also field building).[1] Or one could focus on more specific bottlenecks/pain points for specific groups of people, and conduct interviews and surveys with members of those groups to gather data on that.[2]
The following posts in this sequence will discuss what we could do about the problems highlighted here.
1. Many open questions
There’s a huge amount of high-priority research to be done.
It seems to me that this is strikingly clear, so, for brevity, I won’t provide much further justification here (but I’m happy to do so in the comments, if requested!).
As one piece of evidence, see A central directory for open research questions. Presumably many of those questions aren’t actually high-priority or are no longer “open”, but I’m pretty sure a substantial fraction are, and that’s a substantial fraction of a very large number of questions!
2. Orgs and funders want research(ers)
There are many orgs and funders who would be willing and able to hire or fund people to do such research if there were people who the orgs/funders could trust would do it well (and without requiring too much training or vetting). But not if the orgs/funders think the people are choosing lower-priority questions, are inexperienced, or aren’t especially skilled, or if the orgs/funders simply would have a hard time assessing the people and/or their plans. (See also EA is vetting-constrained and Ben Todd discussing organizational capacity, infrastructure, and management bottlenecks.)[3]
3. Many aspiring/junior researchers
There seem to be many EAs[4] who don’t yet have substantial experience doing EA-relevant research and for whom one or more of the following is true:
They want to do EA-relevant research long-term
They want to test their fit for doing EA-relevant research
They want to do a bit of EA-relevant research in order to gain knowledge and skills that will improve their ability to do other high-impact things
E.g., by expanding their knowledge of global priorities in order to inform career or donation choices, or learning more about AI in order to inform their actions as a civil servant
They want to do a bit of EA-relevant research because that research could itself be valuable
This could either be because the person has to do a research project anyway (e.g., a thesis) or because they have some additional time on their hands
For convenience, I will sometimes lump all such people together under the label “aspiring/junior researchers”. But it’s important to note that that includes several quite different groups of people, who face different pain points and would benefit from different “solutions”.
4. Some shouldn’t pursue EA-aligned research roles
Many of those EAs would probably be better off focusing on activities/roles other than EA-aligned research (given their comparative advantage, the limited number of roles currently available, etc.). But it can be hard for those EAs to work out whether that’s the case for them, and thus they might (very understandably!) continue testing fit for such roles, trying to skill up, etc. This can lead to unnecessary time costs for these people, for hirers, for grantmakers, etc.
5. Some should pursue and get EA-aligned research roles (but don’t)
Meanwhile, many other EAs probably do have a comparative advantage for EA-aligned research, and yet self-select out of pursuing relevant jobs, funding, collaborations, or similar, or pursue such things but find it very hard to get them.[5] This may often be because orgs and funders have insufficient ability to vet, train, and manage these people (see also section 2 above).
6. Maybe more should try independent research?
It would probably be good for many EAs to simply try doing (semi-)independent EA-aligned research. (This could take the form of quite “cheap” efforts, like 10 hours trying to produce a blog post. This might help the EAs test, signal, or improve their fit for EA-aligned research, and might produce valuable outputs; see also.) But it seems that many of those EAs either never actually try this, or only get started after a delay that is much longer than necessary.
Some possible reasons for this include:
Some of these people are (understandably!) not willing or able to (a) try (semi-)independent research while working or (b) reduce their hours of paid work.
They find it hard to think of or find any research questions that seem (a) worth researching in general and (b) well-suited to their skills and interests.
They find it hard to determine which questions would be worth researching in general and are well-suited to their skills and interests.
They find it hard to pick between the questions that seem to do well on those criteria.
They find it hard to know exactly how to operationalise a research question, how to break it into sub-questions, where to start looking for relevant prior work, etc.
They find it hard to be motivated to start without some clearer signal that doing so would have a good payoff for them (e.g., an increased chance of a desired job), or that someone thinks they specifically should indeed try doing EA-aligned research.
They think or worry that they’ll do a bad or slow job at their current skill level without mentorship.
They don’t entirely feel they have “permission” to just get started, or they aren’t/don’t feel sufficiently “agenty”, or something like that.
7. Independent research attempts could be improved
When EAs do try doing EA-relevant research (semi-)independently, it seems that one or more of the following problems often occur:
The question they pick is substantially lower priority or less fitting for them than another question they could’ve picked.
They do a poor job of operationalising the question, breaking it into sub-questions, looking for relevant prior work, etc.
They had to spend a lot of time generating or sifting through question ideas, or figuring out how to operationalise questions, how to break questions into subquestions, where to start looking for relevant prior work, etc.
They find it hard to stay motivated and productive and do a good job without mentorship.
They don’t have a clear sense of who the target audiences for the research should be, what its path to impact / theory of change should be, etc.
As such, the aspiring/junior researcher doesn’t frame the research in the ideal way; doesn’t pursue the ideal constellation of sub-questions; and doesn’t take appropriate actions to ensure their research is critiqued, is built on, influences decisions, and/or is taken as a signal of the researcher’s fit for future projects.
(Additionally, sometimes the researcher does an impressive job, contributing to an org thinking the researcher is probably worth hiring, but the org still lacks sufficient funding or management capacity to hire the researcher.[6])
8. Non-EA research efforts could be improved
There are also a huge number of non-EAs who have to or want to do research projects and who could in theory be tackling higher-priority questions in a more useful/high-quality and efficient way than they currently are.
(Additionally, if these researchers were made aware of higher-priority questions and given tools that helped them tackle questions in a more useful/high-quality and efficient way, that might increase their awareness of, inclination towards, and/or engagement with EA ideas.)
9. Existing solutions are inefficient and insufficient
We have some fairly good ways of partially addressing those problems, such as a handful of research training programs, the post A central directory for open research questions, and 1-1 conversations between senior and aspiring/junior researchers (to help the aspiring/junior researchers learn about open questions, think about how high-priority and fitting various questions are, think about the paths to impact, get mentorship, etc.).
But:
Those partial solutions require more time from senior people in EA than seems ideal.
The above problems still partially persist. Reasons for that might include the following:
There is a limited availability of time from senior EAs
Collections of research questions often fail to provide clear ways to determine how high-priority each question is relative to other questions, what type of person is a good fit for addressing each question, what other questions each question connects to or could be broken down into, where to start looking for resources, what the paths to impact might be, etc.
In general, it’s hard to get people to do something on a volunteer basis and/or with little or no mentorship/management.
People may not be aware of clear examples of aspiring/junior researchers who independently tackled one or more open research questions and thereby provided direct value, gained status in the EA community, had an easier time finding a good job, or similar.
10. Senior people have capacity to help, if efficiently leveraged
I think that there are probably many “senior” EAs who would be happy to put in small amounts of time to help aspiring/junior researchers.[7]
For example, I think many senior EAs might be willing to:
Have one or two 30 min calls with an aspiring/junior researcher
Add research topic ideas to a centralised resource
Add comments to a centralised resource about how research on a topic might inform their own future research or decisions
Providing some indication about how high-priority various questions seem, what type of person might be a good fit for them, etc.
Call to action
Please comment below, send me a message, or fill in this anonymous form if:
You disagree with any of those observations or the inferences I draw from them
You want to suggest additional observations or inferences related to the EA-aligned research pipeline
You feel that you’ve personally “suffered” in some way from the EA-aligned research pipeline being somewhat insufficient and inefficient (e.g., you’ve found it very hard to get a relevant job or usefully try out independent research, or you’re spending lots of time giving 1-1 help to aspiring/junior researchers and feel that your input could be used more efficiently)
This could, for example, push in favour of more efforts to engage with and connect to various academic literatures and fields.
Indeed, I think that people (considering) spending a lot of resources trying to improve the EA-aligned research pipeline should consider doing things like conducting interviews and surveys with relevant groups, and one weakness of this sequence is that I haven’t done that myself.
Note that I’m not saying that additional funding no longer has value or that “EA no longer has any funding constraint”; we could still clearly do more with more funding.
I’m using the term “EAs” as shorthand for “People who identify or interact a lot with the EA community”; this would include some people who don’t self-identify as “an EA”.
To some extent, this is also true for jobs in general (in the big, wide, non-EA world). But it still seems worth thinking about whether and how the situation in relation to EA-aligned research could be improved. And it does seem like the situation might be unusually extreme in relation to EA-aligned research (see also After one year of applying for EA jobs: It is really, really hard to get hired by an EA organisation).
Of course, the org might just have bad judgement, not be worth funding, etc. But I think the problem is sometimes simply that it’s hard to get sufficient management capacity and/or that it’s hard for funders to vet organisations. (For further discussion of those points, see EA is vetting-constrained and Ben Todd discussing organizational capacity, infrastructure, and management bottlenecks.)
Reasons why I believe this include various conversations with senior EAs, various conversations with aspiring/junior researchers who got brief help from senior EAs, and the willingness of many senior EAs to act as mentors for various research training programs (e.g., SERI, Legal Priorities Project) and for Effective Thesis.