Effective Thesis project review
This article was jointly written by David Janku and Jan Kulveit from the Czech EA Association.
Within the Czech EA Association we have been running the Effective Thesis project for almost a year now, and would like to share some more information about the project, and our future plans.
For those interested in contributing, such as EA-aligned researchers, or even just anybody with a great research topic idea, please contact us at david.janku@efektivni-altruismus.cz
Introduction
Effective Thesis is a project that directs students’ research towards EA causes by offering them EA-related topics for their final dissertations and theses. The aim is to deliver three valuable outcomes:
Changing student trajectories at a particularly crucial juncture in their lives
Generating additional research in EA cause areas, with little cost to EA funding sources
Making current academics (students’ supervisors and committees) more familiar with EA perspective and EA topics
Our hope is this could be potentially high-leverage, even more so than usual career coaching, as we are focusing on a particularly crucial juncture.
Background
In most European universities, a dissertation is a compulsory part of any bachelor’s, master’s and doctoral degrees. People spend hundreds to thousands of hours working on it (dependent on study discipline and course requirements) and refer to it as the greatest challenge and achievement in their studies. Yet, most of them experience a huge struggle when trying to find the right topic. The quality of dissertations differs very much by university, course, supervisor availability and individual students’ factors. All students also have their own supervisor advising them throughout the research process and the academic quality of the dissertation is in the end checked by a committee who reads it and decides on the mark the student receives.
Main idea
Since dissertations are compulsory for many students, they invest much time and effort into them, they have academic support in increasing the quality of their research but often struggle to find a topic, so we had an idea to utilize this process by offering students EA-related topics.
To make it multiplicative, we have decided to ask EA orgs for these topics and agreed they will provide students with consultations to ensure that students’ work will be of best use to EA orgs. Consultations also serve as a motivator for students and should increase the quality of their work and, in a broader sense, should help to align academia and practice.
We have decided to create an online platform to make this project more scalable. We have created a website where people can read about why some problems are more important than others and see the topics filtered by their study discipline and interests. For each topic profile, we have put together descriptions, explaining why the topic is important and including sources to start with when the student is interested in the topic.
Reflections on the project development
Outcomes
We have put about 1500 hours into the project. So far, we have 12 students working on some of our topics and we estimate that each of them will invest about 200-300 hours into their theses, resulting in approx. 2400-3600 extra research hours. Up until now, the project budget was £5400, so each trajectory change cost roughly £450. From interviews with 2 students who have already finished their dissertations, it seems that our project helped them realize that they could do a dissertation on an EA topic and it also helped them to find sub-disciplines and specific problems to focus on within the EA problem areas. Both of them worked on AI safety problems and are now are looking for a job in this field, although the counterfactual influence of dissertation on their long-term career plans is difficult to evaluate since both of them also consulted with 80k and were in contact with other EA organizations and information sources.
Marketing
Since we operate online, one of our main marketing venues was an AdGrants Google account enabling us to spend $10.000 per month on AdWords. We got together with the marketing company Brainslab who offered help to EA projects and who took care of our account and increased its quality. However, after several months, it seems we still don’t have a good way to reach the right students, since it hasn’t brought us a single student who would like to work on some of our topics. We are now changing the strategy from using general keywords (like “thesis topic”) to using titles of well known and important papers and concepts specific to each study discipline. The most effective strategies so far seem to be posting in EA facebook groups and promotion by word of mouth. Interestingly, the strategy which may become several times more effective was to google our biggest competition in search results and ask them to promote our website. Since our main competitors in search results are blogs advising students on “how to choose a thesis topic” but not offering specific topics like we do, they can improve their content by linking us to their articles and we might get a great deal of relevant traffic from their websites. This idea came about from interviews with marketing professionals who we reached out to, and is a good example of non-zero-sum mindset.
Management
Since we offer topics via our website, success depends heavily on how the website is designed. We didn’t have very much experience with web development and design and therefore it took us longer to make changes, with results being sub-optimal. However, we managed to create and run the website much more cheaply than if we paid professionals and improved valuable skills (website development, project management, teamwork) along the way.
Steps that proved valuable included letting parts of the project which don’t require a deep understanding of EA be managed by professional organizations, who now take care of areas of it for free as a part of their CSR programme (as we did with AdWords). Another useful step was to consult with other individual professionals who we found via websites connecting nonprofits and skilled short-term volunteers (here is the list of such platforms we found).
Forthcoming plans
As we reflected on the project development, we found that current model of working via EA orgs is not optimal. Some of the top EA research organizations are heavily time-constrained, and unable to spend much effort on proposing thesis topics, so the current topics selection;
- Doesn’t represent the whole EA research landscape well (with more emphasis on wild animal suffering, for example, because the respective research organization was more cooperative in providing topics—a great help!)
Many topics seem to be highly interesting, but likely not tractable within the scope of masters or even doctoral thesis.
With many topics, it may be difficult to find an appropriate advisor.
In addition, the students, in general, do not have a comparative advantage in choosing topics, typically having neither previous experience with dissertation writing nor a very good overview of the research landscape.
Therefore, we have recently launched a different approach: individual Thesis Topic Coaching, a deliberative process in which we try to understand each student’s needs and opportunities, and suggest tailored topics or research directions. When students choose one of our topics, we will try to find them consultants focusing specifically on their topics and will share their work with relevant organizations. The change should result in an increased uptake of our services and the quality of final research pieces. This will also enable us to focus less on web development, which took a significant proportion of the time we devoted to the project.
Another thing we are considering doing is to reach out to current academics with topics relevant to their expertise and asking them to start offering these as dissertation topics for their students. This might potentially be a more effective way of addressing students than an online website since consulting with teachers is a default option for most students searching for dissertation topics. At the same time this would allow academics to explore and become more familiar with EA topics via their students.
We would like to encourage anyone interested in this path to contact us.
Request for help
As the project is ultimately trying to create a two-sided marketplace, it depends on the willingness of EA aligned researchers and research organizations to share promising research ideas and to offer the students consultations.
We want to ask EA aligned researchers for cooperation in this respect—if you have the capacity to help, please contact us. It does not have to be a particularly costly process and does not create a commitment on your side.
We also have a funding gap for 2018 sized circa 12000 USD, which we hope to cover mostly by grants, but individual donations are welcome and important—if you would like to donate, you can do it via Czech Effective Altruism Association in a tax-deductible way
Our takeaways
- We are still convinced that choosing a dissertation topic is an important branching moment in a researcher’s trajectory, which may not only generate hundreds of hours of additional research on an important topic, but may influence a researcher’s long-term career.
-
Even in case of students planning non-research careers, working on an EA thesis topic can lead to deeper engagement with the ideas and community, generating more knowledgeable EAs and making current academics more familiar with the EA perspective.
-
We are less sure that letting students choose for themselves from a list of topics proposed by EA research institutions is an effective way to utilize this branching moment.
Credits
David Janku manages the project and put in most of the work. The web development was mainly undertaken by Dan Hnyk (programming) and David Horák (copywriting), helped by many other volunteers. Jan Kulveit suggested the initial idea of steering research by promoting EA aligned theses topics and the direction change to thesis topic coaching.
The project was financially supported by Greg Colbourn with the help of the volunteers Pär Flodin, John Lidiard, Ronke Bankole, Nikita Hayward, Jiří Nádvorník and others, and uses an AdWords grant from Google managed by Brainlabs.
Come meet us!
We are holding an Effective Thesis meetup on 7.5. at 7 pm in MIRI/CFAR office in Berkeley, so if you´re interested and around, we will be happy to meet you!
- Effective Thesis seeks a project manager by (25 Nov 2020 15:47 UTC; 73 points)
- Effective Thesis: updates from 2019 and call for collaborators by (18 Jun 2020 12:39 UTC; 62 points)
- Effective Thesis project: second-year update by (9 May 2019 13:57 UTC; 54 points)
- Effective Thesis is hiring project manager and content manager by (13 May 2021 12:53 UTC; 30 points)
- Effective Thesis meetup by (LessWrong; 31 May 2018 19:49 UTC; 5 points)
Thanks for writing a summary of your progress and learnings so far, it’s so useful for the EA community to share its findings.
A few comments:
You might consider making the website more targeted. It seems best suited to undergraduate theses, so it would be useful to focus in on that. For example, it might be valuable to increase the focus on learning. During your degree, building career capital is likely to be the most impactful thing you can do. Although things like building connections can be valuable for career capital, learning useful skills and researching deeply into a topic are the expected goals a thesis and so what most university courses give you the best opportunity to do. Choosing a topic which gives you the best opportunity for learning could mean, for example, thinking about which people in your department you can learn the most from (whether because the best researchers, or because they are likely to be the most conscientious supervisors), and what topic is of interest to them so that they’ll be enthusiastic to work with you on it.
People in academia tend to be sticklers wrt writing style, so it could be worth getting someone to copy edit your main pages for typos.
Coming up with a topic to research is often a very personal process that happens when reading around an area. So it could be useful to have a page linking to recommended EA research / reading lists, to give people an idea of where they could start if they want to read around in areas where ideas are likely to be particularly useful. For example you might link to this list of syllabi and reading lists Pablo compiled.
Thanks for your comment!
I agree. This is one of the inefficiencies of the “list of predefined topics” concept we would like to improve by shifting to Thesis Topic Coaching. The plan is to count in the individual supervisor availability when offering topics. However, I guess that most students’ supervisors won’t be mainly focused on some EA topic so we will have to find a balance between advising to choose a topic which the supervisor is mainly focused on and a degree to which the topic relates to EA causes for each student individually.
You mean reading generally about EA? The value we wanted to add by this project is not to compile general reading lists in which students would have to search themselves, but helping them in the process of choosing a topic individually by directing them toward specific EA sub-topics relevant to their degree, experience and circumstances.
This is a great project! I agree unfunded theses are a huge untapped resource. We were trying to do something similar with our essay contest on global agricultural catastrophes, but it was not very successful. Joshua Pearce and I have dozens of ideas for effective theses, so we will reach out. Minor comment: it is good to spell out the month, because date conventions are different in the US than Europe.
Thanks for this summary. Just on comment: wouldn’t it be useful to have a kind of Effective Thesis prize? It might be convenient for advancing the idea among professors.
(This is the third − 1st in Open Thread #43, 2nd in a facebook comment—and last time I make this suggestion. Sorry if it’s getting boring)
In my opinion it would be definitely worth to try! What’s possibly not clear is that Effective Thesis is to some extent funding constrained, so in present it would need to get some additional sources of funding to run prizes
And prizes wouldn’t have to be super expensive. I mean, graduate students don’t need too much additional incentives to write a good thesis; the main one is to be acknowledged as “Effective Thesis of the year”
I’d contribute to that, especially if the theses were open for reading or download.
(Maybe we could for the right of voting on a thesis in a pre-selection phase. E.g., I’d be willing to pay U$50 to get access to them and vote on my favorite ones. But I haven’t really thought a lot about it)
Forethought just launched one a few hours ago!
This is a fantastic project! I encourage other EA university chapters to share the Effective Thesis website on their social media pages and internal groups 1-2x per year. When you share it on Facebook, make sure to mention the Effective Thesis Facebook page on your post.
This is awesome!
One thing which might be worth emphasizing is comparative advantage. For example, suppose I’m an econ student in a department that’s known for X, Y, and Z. My pool of potential advisors includes some of the world’s leading experts on X, Y, and Z, and I’m well-positioned to apply those ways of thinking to problems of EA interest. So a good strategy might be something like: Identify topics in your field of study (that you find personally interesting, that are currently hot research areas, that your advisor is interested in, etc.), and identify EA topics that you think are important/interesting, then create a 2d grid where you examine intersections of topics in your field/EA topics and see which are fertile.
To take this even further, thesis writers could ask questions like: What is the most beneficial/harmful research that has been done in my field so far from an EA perspective, and how can I do research that is likely to be beneficial and not harmful? If someone was to write a history of my field from an EA perspective, what might it look like? I would be interested to know what the “outside view” says about which research is likely to be beneficial in various fields.
You’ve probably already seen this thread, but if not you should take a look; many of the projects are research projects.
To make maximally good use of researcher time, it might be good to thoroughly familiarize yourself with publicly available info before having conversations. I added a comment to the other thread linking to some more obscure AI safety problem lists that could be useful (in addition to those in the main post).
This thread talks about the value of having a central clearinghouse for EA research, which is something you might aim to do in the long run.
Thanks for the comment!
That’s what we plan to do, with the fact that the grid work will be done by our coaches, who we believe have a comparative advantage in doing this since they have a better overview of EA research landscape and can assess better what students’ opportunities are.
I think this is worth re-considering. I see strong potential for applied projects on specific topics, tied to specific measures and datasets of interest. This might not be the best for people looking to pursue broad ‘theoretical/academic’ research careers. However, it could be very useful as a stepping stone to working at an EA research or impact-driven org.
I had a version of this (list of topics) when I was supervising the undergrad Econ dissertation modules at Essex and Exeter universities. Seemed to work well.
I especially see value in things like cost-effectiveness analysis, shallow reviews of potential high impact interventions/cause areas, fermi/Monte-carlo estimation, forecasting, meta-analysis/synthesis, and data science stuff, including re-analysis of existing data from trials and experiments in movement building, charitable giving, etc.
EA and EA research orgs often have more questions and more data than we have the capacity to handle. And these are often very interesting contexts from an academic and research PoV. Students can work on these, and get some feedback and recognition for their work.
There should be little risk, because if the students’ analysis ends up not being useful, the organization and decisionmakers do not have to use it.
E.g., a case in point (sketch):
High Impact Athletes is interested in modeling the estimated lifetime income over an athlete’s career, as a function of their sport, age, gender, ranking, etc. They would also like to model expected value of atheletes donations, and of their voice/impact on others. (The latter is harder still, but note there is some academic lit on, e.g., the value of celebrity Twitter posts
From experience, I know students are interested in this sort of topic and I suspect there is some research out there already. This could then be part of a larger model (Fermi Montecarlo guesstimate squiggle) of expected donations from athletes who pledge.
Why: this can help them consider which athletes to target more, and how much to emphasize their different theories of change (athlete donations, athletes getting their supporters to donate, etc.)