The value of graduate training for EA researchers: researchers seem to think it is worthwhile
Imagine the average “generalist” researcher employed by an effective altruist / longtermist nonprofit with a substantial research component (e.g. Open Philanthropy, Founders’ Pledge, Rethink Priorities, Center on Long-Term Risk). Let’s say that, if they start their research career with an undergraduate/bachelor’s degree in a relevant field but no graduate training, each year of full-time work, they produce one “unit” of impact.
In a short Google Form, posted on the Effective Altruism Researchers and EA Academia Facebook groups, I provided the above paragraph and then asked: “If, as well as an undergraduate/bachelor’s degree, they start their research career at EA nonprofits with a master’s degree in a relevant field, how many “units” of impact do you expect that they would produce each year for the first ~10 years of work?”* The average response, from the 8 respondents, was 1.7.
I also asked: “If, as well as an undergraduate/bachelor’s degree, they start their research career at EA nonprofits with a PhD in a relevant field, how many “units” of impact do you expect that they would produce each year for the first ~10 years of work?”* The average response was 3.9.
I also asked people whether they were a researcher at a nonprofit, in academia, or neither, and whether they had graduate training themselves or not.** Unsurprisingly, researchers in academia rated the value of graduate training more highly than researchers in nonprofits (2.0 and 4.3 for each year with a master’s and a PhD, respectively, compared to 1.2 and 1.7), as did respondents with graduate training themselves, relative to respondents without graduate training (2.0 and 5.2 compared to 1.2 and 1.7).
I asked a free-text response question: “Do you think that the value of graduate training would increase/compound, or decrease/discount, the got further into their career?” 4 respondents wrote that the value of graduate training would decrease/discount the got further into their career, but didn’t provide any explanations for this reasoning. This was also my expectation; my reasoning was that one or more years’ of graduate training, which would likely only be partly relevant to the nonprofit work that you would be doing, would become relatively less important later on, since your knowledge, skills, and connections would have increased through your work in nonprofits.
However, two respondents argued that the value of graduate training would increase/compound. One added: “People without PhDs are sadly often overlooked for good research positions and also under-respected relative to their skill. If they don’t have a PhD they will almost never end up in a senior research position.” The other noted that it would “increase/compound, particularly if they do things other than anonymous research, e.g. they build an impressive CV, get invited to conferences because of their track record. If one doesn’t have a PhD, the extent of this is limited, mostly unless one fits a high-credibility non-academic profile, e.g. founded an organization.”
I did some simple modelling / back of the envelope calculations to estimate the value of different pathways, accounting for 1) the multipliers on the value of your output as discussed in the questions on the form and 2) the time lost on graduate education.*** Tldr; with the multiplier values suggested by the form respondents, graduate education clearly looks worthwhile for early career researchers working in EA nonprofits, assuming they will work in an EA research nonprofit for the rest of their career. It gets a little more complex if you try to work it out in financial terms, e.g. accounting for tuition fees.
For my own situation (with a couple of years of experience in an EA research role, no graduate training), I had guessed multipliers of 1.08 and 1.12 on the value of my research in the ~10 years after completing graduate training, for a master’s and PhD, respectively. For the remaining years of a research career after that, I had estimated 1.01 and 1.02. Under these assumptions, the total output of a nonprofit research career with or without a master’s looks nearly identical for me; the output after completing a PhD looks somewhat worse. However, with the average values from the Google form then the output looks much better with a master’s than without and with a PhD than with just a master’s. Using the more pessimistic values from other EA nonprofit researchers, or respondents without graduate training, the order is still undergrad only < master’s < PhD, though the differences are smaller. In my case, tuition fees seem unlikely to affect these calculations much (see the notes on the rough models sheet).
Of course, which option is best for any individual also depends on numerous other career strategy considerations. For example, let’s think about “option value.” Which options are you likely to pursue if research in EA nonprofits doesn’t work out or you decide to try something else? Pursuing graduate training might enable you to test your fit with academia and pivot towards that path if it seems promising, but if your next best option is some role in a nonprofit that is unrelated to research (e.g. fundraising), then graduate education might not be as valuable.
I decided to post here partly in case others would benefit, and partly because I’m interested in feedback on/critiques of my reasoning, so please feel free to be critical in the comments!
*For both questions, I noted: “There are many complications and moderating factors for the questions below, but answering assuming the “average” for all other unspecified variables could still be helpful.)” and “1 = the same as if they just had a bachelor’s; numbers below 1 represent reduced impact, numbers above 1 represent increased impact.” **These questions were pretty simplified, not permitting people to select multiple options. *** Here, for simplicity, I assumed that: - You would produce no value while doing your graduate training, which seems likely to be false, especially during (the later years of) a PhD. - The value of 1 year after your graduate education was the same as 1 year before retirement, which seems likely to be false.
In a short Google Form, posted on the Effective Altruism Researchers and EA Academia Facebook groups, I provided the above paragraph and then asked: “If, as well as an undergraduate/bachelor’s degree, they start their research career at EA nonprofits with a master’s degree in a relevant field, how many “units” of impact do you expect that they would produce each year for the first ~10 years of work?”* The average response, from the 8 respondents, was 1.7.
The mechanism may not be causal. If you’re conditioning on type of person who can get accepted into graduate programs + get funding + manage to stick with a PhD program, you are implicitly drawing on a very different pool of people than if you don’t condition on this.
That’s a good point—my intention was that it would be the same individual in each instance, just with or without the training, but I didn’t word the survey question clearly to reflect that.
It’s an interesting analysis. Just a thought—since the value of 1 unit is up to the responder if I’ve understood correctly, it might be more meaningful to calculate ratios of the responses for each person and average these rather than average the responses to each part—for the latter, if any responder picked small “unit” sizes and correspondingly gave large numerical values, they would make an outsized contribution. Calculating ratios first cancels out whatever “unit” people have decided on. Though it should only matter much if people’s “units” differ considerably in size.
Hey Jamie, thanks for doing this, I find the results interesting. Just want to point out what I think are two small typos that made it harder to understand what you wrote:
I asked a free-text response question: “Do you think that the value of graduate training would increase/compound, or decrease/discount, the got further into their career?” 4 respondents wrote that the value of graduate training would decrease/discount the got further into their career
Could you correct what you put above?
Also, I’m curious on
1. What Master’s or Ph.D degrees are you considering to take?
2. What do you think would be a good Master’s or Ph.D degree to take for the average “generalist” researcher at an EA / longtermist non-profit (if this is different from what you personally would take)?
Psychology, sociology, (history), (political science). I imagine that that’s an unusually broad range to be considering, but I didn’t want to rule anything out prematurely. My undergraduate was in history but my research in nonprofits has been much more social science-y, and a bit more quantitative.
I imagine that there’s a very broad range that could be on the table. I haven’t thought about this question in general that much for “EA / longtermist” research orgs. For effective animal advocacy research organisations, my main guesses would be the same as the list above, plus economics. But there could be others that I haven’t thought about, related to those options, or an unusually good fit for some individuals etc.
The value of graduate training for EA researchers: researchers seem to think it is worthwhile
Imagine the average “generalist” researcher employed by an effective altruist / longtermist nonprofit with a substantial research component (e.g. Open Philanthropy, Founders’ Pledge, Rethink Priorities, Center on Long-Term Risk). Let’s say that, if they start their research career with an undergraduate/bachelor’s degree in a relevant field but no graduate training, each year of full-time work, they produce one “unit” of impact.
In a short Google Form, posted on the Effective Altruism Researchers and EA Academia Facebook groups, I provided the above paragraph and then asked: “If, as well as an undergraduate/bachelor’s degree, they start their research career at EA nonprofits with a master’s degree in a relevant field, how many “units” of impact do you expect that they would produce each year for the first ~10 years of work?”* The average response, from the 8 respondents, was 1.7.
I also asked: “If, as well as an undergraduate/bachelor’s degree, they start their research career at EA nonprofits with a PhD in a relevant field, how many “units” of impact do you expect that they would produce each year for the first ~10 years of work?”* The average response was 3.9.
I also asked people whether they were a researcher at a nonprofit, in academia, or neither, and whether they had graduate training themselves or not.** Unsurprisingly, researchers in academia rated the value of graduate training more highly than researchers in nonprofits (2.0 and 4.3 for each year with a master’s and a PhD, respectively, compared to 1.2 and 1.7), as did respondents with graduate training themselves, relative to respondents without graduate training (2.0 and 5.2 compared to 1.2 and 1.7).
I asked a free-text response question: “Do you think that the value of graduate training would increase/compound, or decrease/discount, the got further into their career?” 4 respondents wrote that the value of graduate training would decrease/discount the got further into their career, but didn’t provide any explanations for this reasoning. This was also my expectation; my reasoning was that one or more years’ of graduate training, which would likely only be partly relevant to the nonprofit work that you would be doing, would become relatively less important later on, since your knowledge, skills, and connections would have increased through your work in nonprofits.
However, two respondents argued that the value of graduate training would increase/compound. One added: “People without PhDs are sadly often overlooked for good research positions and also under-respected relative to their skill. If they don’t have a PhD they will almost never end up in a senior research position.” The other noted that it would “increase/compound, particularly if they do things other than anonymous research, e.g. they build an impressive CV, get invited to conferences because of their track record. If one doesn’t have a PhD, the extent of this is limited, mostly unless one fits a high-credibility non-academic profile, e.g. founded an organization.”
I did some simple modelling / back of the envelope calculations to estimate the value of different pathways, accounting for 1) the multipliers on the value of your output as discussed in the questions on the form and 2) the time lost on graduate education.*** Tldr; with the multiplier values suggested by the form respondents, graduate education clearly looks worthwhile for early career researchers working in EA nonprofits, assuming they will work in an EA research nonprofit for the rest of their career. It gets a little more complex if you try to work it out in financial terms, e.g. accounting for tuition fees.
For my own situation (with a couple of years of experience in an EA research role, no graduate training), I had guessed multipliers of 1.08 and 1.12 on the value of my research in the ~10 years after completing graduate training, for a master’s and PhD, respectively. For the remaining years of a research career after that, I had estimated 1.01 and 1.02. Under these assumptions, the total output of a nonprofit research career with or without a master’s looks nearly identical for me; the output after completing a PhD looks somewhat worse. However, with the average values from the Google form then the output looks much better with a master’s than without and with a PhD than with just a master’s. Using the more pessimistic values from other EA nonprofit researchers, or respondents without graduate training, the order is still undergrad only < master’s < PhD, though the differences are smaller. In my case, tuition fees seem unlikely to affect these calculations much (see the notes on the rough models sheet).
Of course, which option is best for any individual also depends on numerous other career strategy considerations. For example, let’s think about “option value.” Which options are you likely to pursue if research in EA nonprofits doesn’t work out or you decide to try something else? Pursuing graduate training might enable you to test your fit with academia and pivot towards that path if it seems promising, but if your next best option is some role in a nonprofit that is unrelated to research (e.g. fundraising), then graduate education might not be as valuable.
I decided to post here partly in case others would benefit, and partly because I’m interested in feedback on/critiques of my reasoning, so please feel free to be critical in the comments!
*For both questions, I noted: “There are many complications and moderating factors for the questions below, but answering assuming the “average” for all other unspecified variables could still be helpful.)” and “1 = the same as if they just had a bachelor’s; numbers below 1 represent reduced impact, numbers above 1 represent increased impact.”
**These questions were pretty simplified, not permitting people to select multiple options.
*** Here, for simplicity, I assumed that:
- You would produce no value while doing your graduate training, which seems likely to be false, especially during (the later years of) a PhD.
- The value of 1 year after your graduate education was the same as 1 year before retirement, which seems likely to be false.
The mechanism may not be causal. If you’re conditioning on type of person who can get accepted into graduate programs + get funding + manage to stick with a PhD program, you are implicitly drawing on a very different pool of people than if you don’t condition on this.
That’s a good point—my intention was that it would be the same individual in each instance, just with or without the training, but I didn’t word the survey question clearly to reflect that.
It’s an interesting analysis. Just a thought—since the value of 1 unit is up to the responder if I’ve understood correctly, it might be more meaningful to calculate ratios of the responses for each person and average these rather than average the responses to each part—for the latter, if any responder picked small “unit” sizes and correspondingly gave large numerical values, they would make an outsized contribution. Calculating ratios first cancels out whatever “unit” people have decided on. Though it should only matter much if people’s “units” differ considerably in size.
Hey Jamie, thanks for doing this, I find the results interesting. Just want to point out what I think are two small typos that made it harder to understand what you wrote:
Could you correct what you put above?
Also, I’m curious on
1. What Master’s or Ph.D degrees are you considering to take?
2. What do you think would be a good Master’s or Ph.D degree to take for the average “generalist” researcher at an EA / longtermist non-profit (if this is different from what you personally would take)?
Thanks!
Oops, I meant “the further they got”
Psychology, sociology, (history), (political science). I imagine that that’s an unusually broad range to be considering, but I didn’t want to rule anything out prematurely. My undergraduate was in history but my research in nonprofits has been much more social science-y, and a bit more quantitative.
I imagine that there’s a very broad range that could be on the table. I haven’t thought about this question in general that much for “EA / longtermist” research orgs. For effective animal advocacy research organisations, my main guesses would be the same as the list above, plus economics. But there could be others that I haven’t thought about, related to those options, or an unusually good fit for some individuals etc.