An update and personal reflections about AidGrade
(Loosely adapted from a post on my personal blog.)
As some of you know, back in 2012 I set up AidGrade, a small non-profit research institute, to collect the results of impact evaluations and synthesize them. It was actually while working on AidGrade that I learned about the Effective Altruism community, as someone who I was interacting with about AidGrade asked me if I’d heard of it.
Fast-forward 11 years. A global consortium of institutions, led by the World Bank, is going to be working on an open repository of impact evaluation results that could be used for meta-analysis and policy (the Impact Data and Evidence Aggregation Library, or IDEAL). This is really close to AidGrade’s mission, and we will be participating in the consortium, helping to design the protocols, contribute data, and perform cross-checks with the other institutions.
I am thrilled to see something like IDEAL develop. We made a case that this was a thing that should exist, and over time enough other people agreed that it will soon be a much larger thing (in which AidGrade will play the smallest of roles). All along, I was hoping that there could be a better institutional home for such a repository, and here we are. It’s the best possible outcome.
To anyone who supported AidGrade, through either time or money over the years, I hope you feel pleased with what you helped accomplish with AidGrade, and I hope you are as excited as I am about IDEAL.
With regards to institutional change more broadly, I also have some good news about another venture, the Social Science Prediction Platform. This platform enables researchers to gather forecasts of what their studies will find. The Journal of Development Economics has recently started encouraging authors of papers accepted through their pre-results review (“Registered Report”) track to collect forecasts on the SSPP, which should accelerate the use of forecasts in academia. We have been having discussions with other organizations about collecting forecasts and I hope to have more good news to share soon.
Both these projects were deeply rooted in academic work. I might be biased, but I think academic work is often underrated. It can be useful for many reasons, but part of it surely is that it can change the way people think about a topic and enable institutional change.
As an econ PhD student, it gives me so much motivation when economists set up organizations that do good, and whose success is based on their economics expertise! (e.g. AidGrade, GiveDirectly, Malengo)
I’d love to hear your perspective on what academic economists (including PhD students) can do to make their work more impactful, both in research and outside of it.
Thanks. A quick, non-exhaustive list:
Get feedback early on. Talking to people can save a lot of time
You should have a very clear idea of why it is needed. Good ideas sound obvious after the fact
That’s not to say people won’t disagree with you. If your idea takes off you will need to have a thick skin
A super-easy way to have more impact is to collaborate with others. This doesn’t help for job market papers, where people tend to want to have solo-authored work. But you can get a lot more done collaborating with others and the outputs will be higher-quality, too
Apart from collaborating with people on the actual project, do what you can to get buy-in from other people who have no relationship to the project. Other people can magnify the impact in big ways and small
It can take a while before early-career researchers find a good idea. Have more ideas than you would think
There’s another point I don’t quite know how to put but I’ll give it a go.
Despite the comments above about having many ideas and getting feedback early about one’s projects—which both point to having and abandoning ideas quickly—there’s another sense in which actually what one needs is an ability to stick to things. And the good taste to be able to evaluate when to try something else and when to keep going. (This is less about specific projects and more about larger shifts like whether to stay in academia/a certain line of work at all.)
I feel like sometimes people get too much advice to abandon things early. It’s advice that has intuitive appeal (if you can’t pick winners, at least cut your losses early), and it’s good advice in a lot of situations. But my impression is that while there are some people who would do better failing faster, there are also some people who would do better if they were more patient. At least for myself, I started having more success when sticking with things for longer. The longer you stick to a thing, the more expertise you have in it. That may not matter in some fields, but it matters in academia.
Now, obviously, you want to be very selective about what you stick to. That’s where having good taste comes in. But I’d start by looking honestly at yourself and looking at people near you that you see doing well for themselves in your chosen field, and asking which side of the impatient-patient spectrum you fall on compared to them. Some people are too patient. Some people are too impatient. I was too impatient and improved with more patience, and for some people it’s the opposite. Which advice applies the most to you depends on your starting point and field, and of course the outside options.
For econ PhDs, I think it’s worth having a lot of ideas and discarding them quickly especially in grad school because a lot of them are bad at first, but I also think there are people who jump ship from an academic career too early, like when they are on the market or in the first few years after. I suspect this might be generally true in academia where expertise really, really matters and you need to make a long-term investment, but I can’t speak for certain about other academic fields beyond economics. And I’ve definitely met many academics who played it too safe for maximizing impact, and many people who didn’t leave quickly enough. What I’m trying to emphasize is that it’s possible to make mistakes in both directions and you should put effort into figuring out which type of error you personally are more likely to make.