My experience on a summer research programme
Earlier this year I successfully applied to a summer research programme. This program is a chance for young scholars to work on research projects related to long termism.
In this post, I will explain my experience so far with the program, from the application process to my daily life and the output I produced. My goal is to give an insider’s view of my experience and to encourage the reader to apply to similar programs.
Important caveat: from what I have been told, this year’s program has been quite different from last year’s, and it might be significantly different from next year’s depending on who is running it. So do not extrapolate too much on my experience to imagine how other programs will be like.
Key points
The application consists of writing a research proposal, and a short interview. You can get lots of information on your personal fit to research just by applying.
The program is quite self-directed, with lots of freedom to pursue the topics you want to research and learn about.
The research institution I visited is outstanding, with a very strong and deliberate research culture.
The main benefits I got out of this program are better understanding of my personal fit to research, entry-level experience as a researcher, a writing portfolio, a research network and an improved career plan.
As general advice, if you are early in your career or pivoting into a new job you should strongly consider spending a week searching and applying for internships, fellowships and workshops.
The application process
I applied to this program earlier this year as I was about to finish my Bachelor studies on Mathematics and Computer Science.
The application was fairly straightforward. They asked me for my CV and a research proposal. Writing the research proposal can be a little daunting if you do not have previous experience coming up with your own research ideas. If you are now in that position, I wrote a blogpost explaining my own process.
The research proposal is non-commital, and you may choose to work on something else if you are accepted in the program. If you are considering a career in research, going through the exercise of writing your own proposal can teach you a lot about your personal fit.
The applicants who passed the initial filter were invited to a 20-minute interview. My impression is that the goal was trying to get a sense of how aligned were the participants with EA intentions, how the applicants approached self-directed research and whether they would fit into the culture of the programme.
The rest of the fellows that were accepted came from many different backgrounds, including students of Biology, International Relations, Politics, Philosophy and Economics and generalist researchers working on biology and nuclear risks.
The program
The two main support structures provided by the program are individual mentors and the weekly check-in, where all fellows checkpoint what we are working on and we take turns to give short presentations on our research so far.
Furthermore, we have some activities scheduled and prompts for socializing in the fellows’ calendar. Plus we have access to internal seminars and workshops. We also are encouraged to come up and organize our own research and learning activities with and for other fellows. I took this chance to organize a research prioritization workshop.
As an addition to the above, I personally agreed with my mentor to have 10 minute daily check-ins. This is a great opportunity to ask him for some light advice and make a declaration of intentions for the day.
All in all, this program is quite self-directed. Most people spend their first weeks selecting a research project and do a deep dive on it. The choice of the research project had no major restrictions, although we could request feedback from our mentors.
The office is shared between multiple EA orgs and soon will be sharing with others, and is a thriving environment for intellectual exercise. The operations people work really hard to make the environment feel productive and welcoming, and you can find at any time other researchers to talk to with similar interests, eager to help the Summer Fellows with their research and provide career advice.
I took great advantage of the office’s Slack workspace, which I used to ask research questions and circulate drafts among other fellows and researchers for feedback.
I also benefited a lot from being able to book rooms full of whiteboards to set up deconfusing sessions with other fellows and researchers.
The program outcomes
Here are the main research outputs I have produced over the fellowship:
I published with Pablo Moreno the academic article Implications of Quantum Computing for AI Alignment Research in arXiv, and a summary of the paper on the EA Forum. I also drafted a preliminary report on research analysis of long term risks and benefits of Quantum Computing, which I will keep working on.
I wrote a preliminary exploration (unpublished) of a probabilistic graphical decision model for optimal timing of interventions. I have been awarded a grant by the Effective Altruism Foundation to continue this line of work.
I published Confounders Made Simple in Towards Data Science. This also led me to write a research proposal on causal discovery (unpublished), which I am looking to develop more in the future.
I published A summary of Nicholas Beckstead’s writing on Bayesian Ethics in the EA Forum. This was part of my exploratory research in Bayesian Ethics with Alex Hill.
I produced a list of ~80 research ideas (unpublished) that will set the basis of my research in the future, and an accompanying article explaining my research idea generation and selection process—How to generate research proposals—published in the EA Forum. Over the next year, I intend to work on shaping these ideas into a personal research agenda.
My approach is probably a bit unusual among the summer fellows, as I chose to focus on many research leads at the same time and producing very polished but also short articles, while the others made deep dives into particular topics. Not all fellows ended up producing written output, instead focusing on early exploration of novel research directions.
Some other activities I engaged in during the fellowship are:
I wrote a new version of my personal career plan, which then led me to apply to 7 different grant programs and jobs, which ultimately resulted in the aforementioned grant by EAF.
I organized a daily forecasting training exercise within the organisation using a forecasting platform Ozzie Gooen is developing.
I reviewed and gave feedback on Jah Ying’s work on the EA Development Framework and Vladimir Mikulik’s work on mesa optimizers.
I boggled with some ideas that I ultimately decided not to pursue, like an exploration of the math behind fat tail distributions, a report applying the EA Development Framework to Spain, a survey of research impact beyond technology development and an exploration of the simulation hypothesis using UDASSA.
I participated in several internal seminars, like Owen Cotton-Barratt’s Writing Workshop, presentations of the research from other fellows, a workshop on Independent Research by other summer fellow, etc.
I participated and organized some social activities for people in the office. The Nerf gunfight was definitely a highlight :)
In terms of intangibles, I believe I have gained the following:
A better understanding of my personal fit to research. These kinds of programs are a great way of finding out whether you actually enjoy the day to day of a researcher—long enough that you get to really immerse yourself, but short enough that the stakes are not that high if it does not work out for you.
A big level up to my researcher skills. I feel like I have shifted from the mindset of “being in love with the idea of being a researcher” to “being in love with research”. I feel like I have a much better sense of what research looks like in the day to day, I have set up lots of habits to streamline and improve my research and I have started developing my own research taste and style.
A network of researchers with aligned tastes that I will be able to reach out to for feedback and professional opportunities.
Some valuable lessons on how to interact with people in a focused research environment, and a better understanding of how these places work internally.
A much better sense of where I am in my career and where I am headed.
During the fellowship, I was also given space to pursue some side projects. I believe I would have done this even if I had not participated in the program, but I appreciate that I was able to make this compatible with my research during the fellowship, and I received some feedback and help on them from people within the office.
I mentored some EA Spain people on their projects and applications.
I have been directly collaborating with some early stage projects in EA Spain.
Overall, I am quite happy with how the fellowship went and what I was able to accomplish.
Similar programs you may want to look into
If all the above sounds appealing to you, I want to encourage you to apply to similar programs.
Some ideas of programs you may want to look into:
If you have further recommendations for others, please add them in a comment!
A word of warning: demand for these programs is relatively high, and it can be quite frustrating to get rejected over and over. I do still think applying is a good investment of your time.
This is because 1) in the process of applying you will learn a lot about your personal fit and 2) you might apply to a program and get redirected to a different one that matches your situation better. For example, I was recommended to apply to this program after I got rejected from a similar one.
Conclusions
It cannot be overemphasized how valuable these kinds of opportunities are.
If you are now early in your career (or if you are pivoting into a new job), I want to encourage you to apply to as many places as you can, even if you think you might be underqualified. It may take you many rejections, but persistence can pay off in the long run, and you will learn more about your fit and what you actually want to do in the process.
The program I was on, in particular, is an excellent environment for conducting research in various areas of strategic interest for long termist perspectives, from foundational decision making and Artificial Intelligence to nuclear risk and neuroscience.
This post was written by Jaime Sevilla. I’d like to thank Max Daniel, Habiba Islam, Rose Hadshar and Pablo Melchor for their feedback and help editing this article.
You have impressive outputs Jaime!
I would like to add that I believe Summer Research Fellowships/Internships at non-EA branded organisations may be more valuable than those at EA-branded ones. I believe there are some very high-quality programs out there, although I haven’t looked for them thoroughly. Reasons why I believe these could be better:
More dedicated training and supervision. EA-branded organizations are young and often run these programs without much prior experience.
Unique network. There are benefits to you personally and to the EA community (!) of building good professional networks outside of EA. These are especially valuable if you have academic ambitions, because EA research institutes cannot currently support PhD’s, nor would these be as well-regarded as one of the top institutes in a field.
These would be especially beneficial to people who have academic ambitions, people who are not in the top-20% of ‘self-directedness’, and to people who are relatively secure in their EA motivation (this limits the risk of value drift).
Drawbacks of researching at these non-EA institutes for a summer would be limited freedom and fewer EA-minded people around. (Although it’s probably a good opportunity to learn to work with non-EA’s while ‘remaining EA’ - a valuable and possibly rare skill!)
I agree with this. I think the setup of the CEA Summer Fellowship programme is a bit concerning.
Adding to the points you mentioned (little supervision, doesn’t provide good evidence for career paths outside EA organisations) it is also an unpaid programme that does not by default result in job offers being made to the best performers. *
I’m worried that students will think this programme will advance their future career, while I doubt this is true in most cases. Instead they might just pay high opportunity costs.
*At least this was true 1-2 years ago, I’m not entirely sure what the most recent iteration of the programme looks like.
[I co-directed the summer programme described in the OP, together with Rose Hadshar. Views are my own.]
Hi Denise and Siebe, thank you very much for your comments. I think they are valuable contributions to the conversation about the opportunities and risks of work at EA organizations.
I’m actually concerned (and have been so for a while) that many people in the community overestimate the value of work at EA organizations compared to alternatives. For example, if a friend told me that the only option they’ve considered for their summer was this summer programme, I’d strongly advise them to also look at opportunities outside the EA community. I think that widely promoted messages, e.g. about the value of ‘direct work’, have contributed to this (what I think is a) misperception (NB I don’t believe that the sources of these messages actually intended to promote this perception, more that the nuance of their intentions got inadvertently lost as messages were propagated through the community). I therefore particularly appreciate that you point out some specific downsides a summer programme such as the one discussed here, and give some actionable reasons to consider alternatives. (And that you do this in a public place, accessible to those with limited opportunity to have high-bandwidth conversations with people working at EA organizations or other relevant experience.)
That being said, my current view is that this summer programme can be a good choice for some people in some circumstances. Based on my experience this summer and the feedback from our summer fellows, I’d also guess that for several of them the summer fellowship was actually more valuable than what they’d have done otherwise. I’d like to gesture at my reasoning, in the hope that this is helpful for people considering applying to a similar programme. (Caveat: as the OP also points out, this programme might look quite different next year, if it’ll be run at all. I recommend checking with whoever will be directing it.)
Just to prevent misunderstandings, I’d like to mention that this year’s programme differed in major ways from previous versions (and was run by different people). In fact, I had been participating in the programme in 2018, and had a mixed experience; when planning this year’s programme, I’ve deliberately tried to learn from this experience (I had also reached out to my fellow fellows to ask them about their experience). E.g., I had appreciated the intellectual autonomy and the opportunity to informally talk to staff at CEA, FHI, and GPI (which shared an office at the time), but I had also often felt lonely and isolated and would have liked more supervision for one of my two projects. As a result, for this year we designed several activities aimed at fostering a collaborative and open culture among the summer fellows, giving them opportunities to introduce themselves and their projects to other staff at the office, and paired each summer fellow with a mentor.
Overall, I think my experience last year was very unlike the experience of this year’s fellows: For example, after a prompt to consider what they’d have done otherwise, this year 10 out of 11 people answered 9 or 10 on a 1-10 scale for “How much better or worse do you feel about the summer fellowship than about what you would have done otherwise?” (and the other one with an 8); on the same scale for “If the same program (same organizers, same structure etc.) happened next year, how strongly would you recommend a friend (with similar background to you before the fellowship) to apply?”, 9 answered with a 9 or 10 (and the other ones with 7 and 8). While I have concerns about the validity of such self-reported impressions for assessing the value of the programme (which I think will depend on things like future career outcomes), I’m confident that I’d have given significantly less positive responses about my experience last year.
Denise, you’re right that this year’s programme did “not by default result in job offers being made to the best performers,” and wasn’t intended to do so. I don’t believe this is necessarily a downside, however. In my view, one of the main benefit of this programme is to help people decide whether they’ll want to aim for similar work in the future. I think this will often be valuable for people who wouldn’t be available for full employment after the programme, e.g. because they’ll continue their university studies. In addition, I believe that several of the benefits of this year’s programme were enabled by us fostering a relatively non-evaluative atmosphere, which I worry would not have been possible in a programme designed to identify future hires. So overall I think that (a) programmes that are essentially extended “work trials” and (b) programmes that won’t usually lead to future employment have different upsides and downsides, and that there is room for both. (I do believe it’s important to be transparent about which of the two one is running. I’m not aware of any of this year’s summer fellows erroneously believing the programme would result in job offers, but I’d consider it a severe communications mistake on our part if this has happened.) As an aside, I think that a programme like this can to some extent further people’s careers. For example, for the summer fellows I mentored I’m now able to provide a credible reference for applications to jobs in EA, and in fact expect to do so over the next few weeks in 1-2 cases. I do, however, agree that these benefits are importantly different from the prospect of a relevant job offer, and I doubt that these benefits alone would make the summer programme worthwhile.
My impression is that our mentoring setup worked quite well, at least from the fellows’ perspective: 7 said they found mentoring “very valuable”, 3 “valuable”, and 1 “~neutral;” similarly, 10 out of 11 people said they’d like their mentor to mentor other people in the future, 9 of them maximally strongly. This seems consistent with responses on specific ways in which mentoring provided value and ways in which it could have been better. That being said, I do think there is room for improvement—for example, we weren’t in all cases able to find a mentor with relevant domain expertise. My loose impression from this year was that cases in which the mentor had domain expertise were a much “safer bet” in terms of mentoring being valuable. In other cases, I think we were able to overcome this challenge to some extent: for example, I worked with two of my mentees to help them reach out to various domain experts, as a result of which they had several conversations; I still don’t expect this provided as much value as direct and regular supervision from someone with domain expertise would have. (On the other hand, I think domain expertise is helpful all else equal, but far from sufficient for good mentoring. We did try to provide guidance to mentors, though I think with mixed success.) Anecdotally, based on my own experience in academia (and the distribution among my friends), my guess is that we’ve provided significantly more mentoring than the median academic internship, though except in 1-4 cases not as much quality-weighted mentorship as in the right tail (say, >95th percentile) of the academic mentorship distribution. (My wild guess is the situation is better in industry, though with other downsides, e.g. less autonomy.)
As mentioned earlier, I think the programme did a decent job at helping people to decide if they want to work in similar roles in the future—and does so in a way that is less costly to people than, say, applying for a full-time job at these organizations. So in this sense I do think it had career-related benefits, albeit by improving future decisions rather than by getting a job immediately. For instance, I think I would have benefitted significantly from the opportunity to work at CEA or FHI for a few weeks before deciding whether to accept the offer I got for FHI’s Research Scholars Programme, a 2-year role. (As I said, I participated in last year’s summer programme—however, I needed to decide to apply to RSP before the summer programme started, and I got the RSP offer quite early during the summer.) In fact, I believe that in at least one case the summer fellowship helped a fellow to decide that they do not want to accept an offer to work at an EA organization, and I think this was among the most tangible, valuable outcomes of this summer. Siebe, I agree with you that this has to weighed against the potential to get more information about other paths by doing internships elsewhere. However, I think that conversely these other internships would provide less information on how much one would like to work at, say, FHI. I think that it depends on the specifics of someone’s circumstances which of these things is more valuable to explore.
Summer fellows were paid. (With the exception of a very small number of people who couldn’t be paid for legal reasons.) That being said, I do believe there might be room for improvement here: for instance, on a scale from 1 to 5 on whether they felt the amount of financial support was adequate 8 out of 11 fellows responded with 4 or 5 - I think that ideally this would be 11 out of 11. I’m also concerned that 10 out of 11 fellows indicated that their savings stayed constant or (in 3 cases) decreased; while I don’t think this is catastrophic given the short duration of the programme, I think it suggests the programme is by far not as accessible to people with a less comfortable financial situation than I would like.
Siebe, I agree that “EA-branded organizations are young and often run these programs without much prior experience.” For example, my highest degree is a master’s in an unrelated field, and I’ve had maybe 2 years of work experience relevant to me running such a programme and mentoring researchers (though potentially somewhat less or much more depending on how you count things I did during my university years outside of a paid role). I agree this is a drawback, all else being equal. However, I think it might be relevant to note that (i) Rose Hadshar, the co-director of this year’s programme, has been the Project Manager for the Research Scholars Programme since last fall, which I think is very relevant work experience (more so than my previous one), (ii) we consulted with several people with significantly more relevant experience, including staff involved in running previous versions of the summer programme. I also think the “all else being equal” clause is important, and possibly not true in expectation: for example, I think it’s fair to say that Rose and I really cared about this programme and were highly motivated to make it go as well as we could. By contrast, I’ve heard many stories from my friends in which they did an internship at an established institution, working with people that had decades of work experience, but came away very frustrated because they felt the internship programme had been poorly thought out, consisted of boring work, or suffered from an uninspiring organizational culture. Clearly, many internship programmes outside of the EA community won’t suffer from these problems (and some at EA organizations will), but overall I believe it’s worth finding out as much as possible about the specifics of an internship programme one is considering rather than relying too much on proxies such as prior experience.
I think most of these points provide complementary information to the considerations you raised, rather than undermining them. I think they’re relevant if someone wanted to form an overall view on whether to apply to such a programme, but I’m not trying to dispute the value of your comments. I’m curious to what extent the points I raised change your overall view on the upsides and downsides of this summer programme, or if you have any further questions.
Awesome comment, Max—thanks for this very thoughtful response. (Minor: you may want to have shorter paragraphs—to partition the text more—to increase readability.)
Thank you, I much appreciate specific suggestions. This makes sense to me, and I’ll try to keep it in mind for future posts.
Thank you Max, strong upvoted. It sounds like you put a lot of thought into making the programme run better than in previous years and succeeded.