Error
Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?
Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?
Notably missing from this list, but related to 5,11, and 17 (and arguably 1 and 18) is increasing the number and EA alignment of currently non-EA or weakly EA-aligned senior researchers.
That is, increasing the number of senior EA aligned researchers not via the pipeline of
get interested in EA-> be a junior EA researcher → be a intermediate EA researcher → be a senior EA researcher,
but via
be a senior researcher → get interested in EA → be a senior EA researcher.
I don’t have very obvious examples in mind, but potential case studies so far include Phillip Tetlock, David Roodman, Rachel Glennester, Michael Kremer, Kevin Esvelt, and Stuart Russell.
Yeah, I think this is a quite important point that’s sort-of captured by the other paths you mention, but (in hindsight) not sufficiently highlighted/emphasised.
I think another possible example is Allan Dafoe—I don’t know his full “origin story”, and it’s possible he was already very EA-aligned as a junior researcher, but I think his actual topic selection and who he worked with switched quite a lot (and in an EA-aligned direction) after he was already fairly senior. And that seniority allowed him to play a key role in GovAI, which was (in my view) extremely valuable.
One place where I kind-of nod to the path you mention is:
I don’t think Alan’s really an example of this.
https://80000hours.org/podcast/episodes/allan-dafoe-politics-of-ai/
I think that quote makes it sound like Allan already had a similar worldview and cause prioritisation to EA, but wasn’t aware of or engaged with the EA community (though he doesn’t explicitly say that), and so he still seems like sort-of an example.
It also sounds like he wasn’t actively and individually reached out to by a person from the EA community, but rather just found relevant resources himself and then reached out (to Bostrom). But that still seems like it fits the sort of thing Linch is talking about—in this case, maybe the “intervention (for improving the EA-aligned research pipeline)” was something like Bostrom’s public writing and talks, which gave Allan a window into this community, which he then joined. And that seems like a good example of a field building intervention?
(But that’s just going from that quote and my vague knowledge of Allan.)
Fair enough. I guess just depends on exactly how broad/narrow of a category Linch was gesturing at.
I think the crux to me is to what extent Allan’s involvement in EAish AI governance is overdetermined. If, in a world with 75% less public writings on transformative AI of Bostrom’s calibre, Allan would still be involved in EAish AI governance, then this would point against the usefulness of this step in the pipeline (at least with the Allan anecdote).
I roughly agree, though would also note that the step could be useful by merely speeding up an overdetermined career move, e.g. if Allan would’ve ended up doing similar stuff anyway but only 5 years later.
Yes, I agree that speeding up career moves is useful.
Some quick notes on how my own career, project, and donation decisions have been influenced by thinking about the value of and methods for improving the EA-aligned research pipeline
(Note that most of these decisions were made before I drafted this sequence of posts, and thus weren’t based on my latest thinking. Also, I am likely missing some relevant things and will fail to explain some things well. Finally, as usual, this comment expresses my personal views only.)
Career decisions:
Thinking about the EA-aligned research pipeline was a key factor in me choosing to work for Rethink Priorities
I got other appealing job offers at the same time as the RP offer
A key selling point for RP for me was that, as far as I could tell before joining RP, RP had done well at scaling, being strategic, and assessing its impact, and seemed set to continue to do so
And it seemed like I could be a good fit for helping scale the longtermism team, e.g. through later taking on management responsibilities and helping develop RP’s longtermist research agendas/priorities
I am now more confident that those guesses were correct, and that it made accept the RP offer partly for these reasons
I’m currently focusing mostly on testing and improving my fit for research management roles/activities
I’ve also taken some steps to test my fit for grantmaking, and am likely to take more such steps soon
Project decisions:
I’ve spent a substantial amount of time helping with aspects of RP’s first research internship program
I’ve spent a substantial amount of time supporting other research training programs
E.g., sharing resources, creating a Slack workspace for people involved in these programs, reviewing and giving advice on strategic plans, acting as a mentor
I’ve spent a substantial amount of time having calls with EAs who are aspiring/junior researchers, to help with things like career planning, topic selection, connecting them to relevant people
I’ve written some relevant posts or docs, such as:
Notes on EA-related research, writing, testing fit, learning, and the Forum
A central directory for open research questions
Reasons for and against posting on the EA Forum
Suggestion: EAs should post more summaries and collections
Readings and notes on how to do high-impact research
Potential benefits and downsides of making and/or sharing a research agenda [this will be posted soon]
This sequence itself, of course!
I’ve made some relevant EA Forum Wiki tags, particularly scalably using labour and research training programs
I’m hoping to in some way help set up the sort of research questions database I’ll describe later in this sequence
Donation decisions:
A desire to improve the EA-aligned research pipeline was a notable factor in me donating to ALLFED and GCRI in 2020
Though not the single largest factor
I explained those donation decisions here
I’m considering donating this year to Effective Thesis and/or to someone who’s excited about working on the database idea I’ll describe in a later post
Just came here to comment something that’s been on my mind that I didn’t recall being suggested in the post, though it partly overlaps with your suggestions 1, 2, 4, 11, and 19.
Suggestion: Paid literature reviews with some (relatively low level) supervision.
Context: Since working at Sentience Institute, I’ve done quite a few literature reviews. (I’ve also done some more “rough and ready” ones at Animal Advocacy Careers.) I think that these have given me a much better understanding of how social sciences academia works, what sort of information is most helpful etc. A lot of the knowledge comes in handy in places that I wouldn’t necessarily have predicted, too. This makes me feel like the benefits might be comparable to the sorts of benefits that I expect lots of people get from PhDs—some methodological training / familiarity, and some useful knowledge. It wouldn’t give you some benefits of PhDs like signalling value, familiarity with the peer review process, or close mentorship relationships, but if you tried to get the literature reviews published in peer-reviewed journals, then that would add some of those benefits back in (and maybe help to improve the end product too).
Lit reviews can be quite time-consuming, but don’t necessarily require any very special skills—just willingness to spend time on it and look things up (e.g. methodological aspects) when you don’t know or understand them, rather than plowing on regardless. Obviously some methodological background in the topic would be helpful, but doesn’t always seem necessary; I’m a history grad and have done literature reviews on subjects from psychology to ethics to management.
It might be quite easy to explicitly offer (1) funding and (2) facilitation for independent researchers to be connected to potential reviewers of the end product. It could be up to the individual to suggest topics, or to some centralised body (as in your suggestion 7).
I’m not sure whose responsibility this should be. It could be EA Funds, Effective Thesis, or individual research orgs.
Caveats
I have found review + comments from colleagues helpful, so some supervision may be necessary, but these have tended to cluster at the start and end of projects with the vast majority of the work being independent.
To do rigorous systematic reviews, you generally want more than one person actually checking through the data, coding decisions etc, which would require more coordination. But this is not always necessary. Indeed, one of my lit reviews is currently going through the peer review process (and looks likely to be accepted) and didn’t use multiple author checks on these decisions. And less formal/systematic literature reviews can still be valuable, I think, both for the researcher and the readers.
Thanks! Yeah, this seems like a handy idea.
I was recently reminded of the “Take action” / “Get involved” page on effectivealtruism.org, and I now see that that actually includes a page on Write a literature review or meta-analysis. That Take action page seems useful, and should maybe be highlighted more often. In retrospect, I probably should’ve linked to various bits of it from this post.
True! I’d forgotten about that page. I think some sort of fairly minimal infrastructure might notably increase the number of people actually doing it though.
(Yeah, I didn’t mean that this meant your comment wasn’t useful or that it wouldn’t be a good idea to set up some sort of intervention to support this idea. I do hope someone sets up such an intervention, and I may try to help that happen sometime in future if I get more time or think of a particularly easy and high-leverage way to do so.)
In the EA Infrastructure Fund’s Ask Us Anything, I asked for their thoughts on the sorts of topics covered in this sequence, e.g. their thoughts on the intervention options mentioned in this post. I’ll quote Buck’s interesting reply in full. See here for precisely what I asked and for replies to Buck’s reply (including me agreeing or pushing back on some things).
---
“Re your 19 interventions, here are my quick takes on all of them
Yes I am in favor of this, and my day job is helping to run a new org that aspires to be a scalable EA-aligned research org.
I am in favor of this. I think one of the biggest bottlenecks here is finding people who are willing to mentor people in research. My current guess is that EAs who work as researchers should be more willing to mentor people in research, eg by mentoring people for an hour or two a week on projects that the mentor finds inside-view interesting (and therefore will be actually bought in to helping with). I think that in situations like this, it’s very helpful for the mentor to be judged as Andrew Grov suggests, by the output of their organization + the output of neighboring organizations under their influence. That is, they should think that one of their key goals with their research interns as having the research interns do things that they actually think are useful. I think that not having this goal makes it much more tempting for the mentors to kind of snooze on the job and not really try to make the experience useful.
Yeah this seems good if you can do it, but I don’t think this is that much of the bottleneck on research. It doesn’t take very much time to evaluate a grant for someone to do research compared to how much time it takes to mentor them.
My current unconfident position is that I am very enthusiastic about funding people to do research if they have someone who wants to mentor them and be held somewhat accountable for whether they do anything useful. And so I’d love to get more grant applications from people describing their research proposal and saying who their mentor is; I can make that grant in like two hours (30 mins to talk to the grantee, 30 mins to talk to the mentor, 60 mins overhead). If the grants are for 4 months, then I can spend five hours a week and do all the grantmaking for 40 people. This feels pretty leveraged to me and I am happy to spend that time, and therefore I don’t feel much need to scale this up more.
I think that grantmaking capacity is more of a bottleneck for things other than research output.
I don’t immediately feel excited by this for longtermist research; I wouldn’t be surprised if it’s good for animal welfare stuff but I’m not qualified to judge. I think that most research areas relevant to longtermism require high context in order to contribute to, and I don’t think that pushing people in the direction of good thesis topics is very likely to produce extremely useful research.
I’m not confident.
The post doesn’t seem to exist yet so idk
I think that it is quite hard to get non-EAs to do highly leveraged research of interest to EAs. I am not aware of many examples of it happening. (I actually can’t think of any offhand.) I think this is bottlenecked on EA having more problems that are well scoped and explained and can be handed off to less aligned people. I’m excited about work like The case for aligning narrowly superhuman models, because I think that this kind of work might make it easier to cause less aligned people to do useful stuff.
I feel pessimistic; I don’t think that this is the bottleneck. I think that people doing research projects without mentors is much worse, and if we had solved that problem, then we wouldn’t need this database as much. This database is mostly helpful in the very-little-supervision world, and so doesn’t seem like the key thing to work on.
I feel pessimistic, but idk maybe elicit is really amazing. (It seems at least pretty cool to me, but idk how useful it is.) Seems like if it’s amazing we should expect it to be extremely commercially successful; I think I’ll wait to see if I’m hearing people rave about it and then try it if so.
I think this is worth doing to some extent, obviously; I think that my guess is that EAs aren’t as into forecasting as they should be (including me unfortunately.) I’d need to know your specific proposal in order to have more specific thoughts.
I think that facilitating junior researchers to connect with each other is somewhat good but doesn’t seem as good as having them connect more with senior researchers somehow.
I’m into this. I designed a noticeable fraction of the Triplebyte interview at one point (and delivered it hundreds of times); I wonder whether I should try making up an EA interview.
Seems cool. I think a major bottleneck here is people who are extremely extroverted and have lots of background and are willing to spend a huge amount of time talking to a huge amount of people. I think that the job “spend many hours a day talking to EAs who aren’t as well connected as would be ideal for 30 minutes each, in the hope of answering their questions and connecting them to people and encouraging them” is not as good as what I’m currently doing with my time, but it feels like a tempting alternative.
I am excited for people trying to organize retreats where they invite a mix of highly-connected senior researchers and junior researchers to one place to talk about things. I would be excited to receive grant applications for things like this.
I’m not sure that this is better than providing funding to people, though it’s worth considering. I’m worried that it has some bad selection effects, where the most promising people are more likely to have money that they can spend living in closer proximity to EA hubs (and are more likely to have other sources of funding) and so the cheapo EA accommodations end up filtering for people who aren’t as promising.
Another way of putting this is that I think it’s kind of unhealthy to have a bunch of people floating around trying unsuccessfully to get into EA research; I’d rather they tried to get funding to try it really hard for a while, and if it doesn’t go well, they have a clean break from the attempt and then try to do one of the many other useful things they could do with their lives, rather than slowly giving up over the course of years and infecting everyone else with despair.
I’m not sure; seems worth people making some materials, but I’d think that we should mostly be relying on materials not produced by EAs
I am a total sucker for this stuff, and would love to make it happen; I don’t think it’s a very leveraged way of working on increasing the EA-aligned research pipeline though.
Yeah I’m into this; I think that strong web developers should consider reaching out to LessWrong and saying “hey do you want to hire me to make your site better”.
I think Ben Todd is wrong here. I think that the number of extremely promising junior researchers is totally a bottleneck and we totally have mentorship capacity for them. For example, I have twice run across undergrads at EA Global who I was immediately extremely impressed by and wanted to hire (they both did MIRI internships and have IMO very impactful roles (not at MIRI) now). I think that I would happily spend ten hours a week managing three more of these people, and the bottleneck here is just that I don’t know many new people who are that talented (and to a lesser extent, who want to grow in the ways that align with my interests).
I think that increasing the number of people who are eg top 25% of research ability among Stanford undergrads is less helpful, because more of the bottleneck for these people is mentorship capacity. Though I’d still love to have more of these people. I think that I want people who are between 25th and 90th percentile intellectual promisingness among top schools to try first to acquire some specific and useful skill (like programming really well, or doing machine learning, or doing biology literature reviews, or clearly synthesizing disparate and confusing arguments), because they can learn these skills without needing as much mentorship from senior researchers and then they have more of a value proposition to those senior researchers later.
This seems almost entirely useless; I don’t think this would help at all.
Seems like a good use of someone’s time.
---------------
This was a pretty good list of suggestions. I guess my takeaways from this are:
I care a lot about access to mentorship
I think that people who are willing to talk to lots of new people are a scarce and valuable resource
I think that most of the good that can be done in this space looks a lot more like “do a long schlep” than “implement this one relatively cheap thing, like making a website for a database of projects”.”
Thanks, I think this is a great topic and this seems like a useful list (although I do find reading through 19 different types of options without much structure a bit overwhelming!).
I’ll just ~repost a private comment I made before.
This feels like an especially promising area to me. I’d guess there are lots of cases where this would be very beneficial for the junior researcher and at least a bit beneficial for the experienced researcher. It just needs facilitation (or something else, e.g. a culture change where people try harder to make this happen themselves, some strong public encouragement to juniors to make this happen, …).
This isn’t based on really strong evidence, maybe mostly my own (limited) experience + assuming at least some experienced researchers are similar to me. And that there are lots of excellent junior researcher candidates out there (again from first hand impressions).
This also seems like a big deal and an area where maybe you could improve things significantly with a relatively small amount of effort. I don’t have great context here though.
Thanks for these thoughts!
Interesting. I received similar feedback on the previous post in the sequence, and re-organised it into “clusters” in response to that. And I’ve received similar feedback on a separate, upcoming draft of mine that also has a big list of things, and due to that feedback I plan to organise that list into clusters before publishing the post. Maybe this is a recurring issue with my writing that I should be on the lookout for. So thanks for that feedback :)
I guess this also relates to my caveat that “There are various other ways to carve up the space of options, various complementary framings that can be useful, etc.”, and to me trying to produce these posts relatively quickly and to be relatively thorough. I expect with more time, I could come up with better ways to organise the space of options—e.g. via creating diagrams representing various different pathways to getting more EA-aligned research or researchers, showing how each intervention could connect to one or more steps on those pathways, and then somehow using that to organise the interventions into broad types and then subtypes. (And if someone else did that, I’d be interested to read what they come up with!)
One (maybe?) low-effort thing that could be nice would be saying “these are my top 5” or “these are listed in order of how promising I think they are” or something (you may well have done that already and I missed it).
Ah, yes, this is probably useful and definitely low-effort (I’ve now done it in 1 minute, due to your comment).
The list was actually already in order of how promising I think they are, and I mentioned that in footnote 1. But I shouldn’t expect people to read footnotes, and your feedback plus that other feedback I got on other posts suggests that readers want that sort of thing enough / find it useful enough that that should be said in the main text. So I’ve now moved that info to the main text (in the summary, before I list the 19 interventions).
I think the main reason I originally put it in a footnote is that it’s hard to know what my ranking really means (since each intervention could be done in many different ways, which would vary in their value) or how much to trust it. But my ranking is still probably better than the ranking a reader would form, or than an absence of ranking, given that I’ve spent more time thinking about this. Going forward, I’ll be more inclined to just clearly tell readers things like my ranking, and less focused on avoiding “anchoring” them or things like that.
(So thanks again for the feedback!)
Additional intervention ideas
Here I’ll keep track of additional intervention ideas that have occurred to me since I finished drafting this post. Perhaps in future I’ll integrate some into the post itself.
Creating and/or improving EA-relevant journals
Could draw more people towards paying attention to important topics
Could make it easier for EAs doing graduate programs (especially PhDs) or pursuing academic careers to focus on high-priority topics and pursue them in the most impactful ways
That could in turn help with “Increasing and/or improving EAs’ use of non-EA options for research training, credentials, etc.”
Making high-quality data that’s relevant to high-priority topics more easily available
The idea here is that “a lot of researchers will follow good data wherever it comes from”
(This was suggested by a commenter on a draft of this post)
An idea from Linch:
(See also the comments on the shortform.)
An idea from Buck (see also the comments on the linked shortform itself):
Rough notes on another idea, following a call I just had:
Setting up something in between a research training program and a system for collaborations in high schools, universities, or local EA groups
Less vetting and probably lower average current knowledge, aptitude, etc. than research training program participants undergo/have
But this reduces the costs for vetting
And this opens this up to an additional pool of people (who may not yet be able to pass that vetting)
Plus, this could allow more people to test their fit for and get better at mentorship, by mentoring people in these “programs” or simply by collaborating with peers in these programs (since collaboration still has some mentorship-like elements)
E.g., in some cases, someone’s who just started a PhD student or just recently learned about the cause area they’re now focused on may not be able to usefully serve as a mentor for a participant in a research training program like SERI, but they may be able to usefully serve as a mentor for a high school student or some other undergrads
(I’m just saying there’d be some cases in that space in between—there’d also be some e.g. PhD students who can usefully serve as mentors for SERI fellows, and some who can’t usefully serve as mentors for high school students)
Complementary perspectives/framings that didn’t quite fit into this post
David Janku of Effective Thesis has written about interventions—other than Effective Thesis which also aim to influence which research is generated. I recommend reading that section, but here’s the list of interventions with the explanations and commentary removed:
influencing individuals by giving them information on what the potentially most impactful directions are and motivating them to pursue these directions
providing funding for research directions that seem promising
setting up research organisations producing research in a specific direction
organising research workshops
setting up prestigious prizes/awards
providing mentorship and space for exploration
David adds that an additional approach which doesn’t aim to influence which research is generated is “coordination—e.g. connecting students/researchers interested in the same topics”.
---
Meanwhile, Jonas Vollmer of EA Funds has written that, to achieve one possible vision for the EA Long-Term Future Fund:
I think that similar points could also be made for longtermist grantmaking by other actors (e.g., Open Philanthropy) and for grantmaking in some other areas (e.g., I’m guessing, wild animal welfare). And I think many of the interventions mentioned in this post might help address those needs.
Here are my thoughts on discovering, writing, and/or promoting positive case studies (moved to a comment since I tentatively think this intervention would be less valuable than the others):
I know of some cases (in addition to me) of people who are now doing impactful EA-aligned research and got to that point partly via something related to one of the interventions discussed elsewhere in this post or sequence
E.g., via doing independent research/writing published on the EA Forum, choosing a thesis and getting mentored via Effective Thesis, or doing a research training program
But I mostly know these cases because I’m now well-networked in EA, rather than because of easily findable public writeups. And I’d also guess that there are many more cases that I’m not aware of.
This could cause people to underestimate how achievable this is, underestimate the value of these “interventions” (e.g., writing on the Forum), or simply have a harder time motivating themselves to try (since success doesn’t feel like a real possibility)
So maybe it’d be valuable to simply:
Find and collect a larger set of positive case studies
Write many of them up (or record podcasts or videos or whatever)
Promote those writeups (or whatever) in such a way that they’ll be found by the people who’d benefit from them
E.g., so that the relevant people would stumble upon these case studies, or so that the people they’d reach out to (e.g., community-builders offering careers advice) would know to mention these case studies
This process could also provide useful data on which methods of entering and progressing through the EA-aligned research pipeline have been used, how successful the methods have been, how they could be supported, etc.^[Though I think the data collection that would be best for directly encouraging and guiding aspiring/junior researchers would differ from that which is best for guiding efforts to improve the pipeline.]
I haven’t thought much about how best to do this, who would be best placed to do it, how valuable it’d be, or what the most similar existing things are
Obviously there are already some things like case studies of successful-seeming EA-aligned careers, including research ones.
Maybe WANBAM have done something similar specifically for women, trans people of any gender, and non-binary people?
Obvious downside risk: Focusing solely on positive case studies could mislead people about how easy these pathways are and cause them to overly focus on pursuing research roles or roles at explicitly EA orgs
Readers of this post may also be interested in my rough collection of Readings and notes on how to do high-impact research.
You can update the EA CoLabs link (under Adding to and/or improving options...) with their website (Impact Colabs) which is a more functional update to this I think.
Thanks—done :)