rossaokod
Thanks for all of the hard work on this, Howie (and presumably many others), over the last few months and (presumably) in the coming months
I think I would describe him as generally supportive of global health and wellbeing focused EA / effective giving. As others note above, he’s been aware of the community since roughly around the time it started and in many ways has been ‘practicing’ EA since long before then (e.g. founded IPA in 2002, wrote ‘more than good intentions’). He’s also engaged directly with the community in various ways—e.g. has been on the board of TLYCS for a long time (and advised them on charity selection for a while). He set up ImpactMatters, which evaluated and recommended a much broader range of charities than GiveWell. Overall, I think this is an extremely exciting appointment and I think he’ll do a huge amount of good in the role!
+1 as a vote for more summaries and thanks a lot for doing these! I’ll check in with Sven (who’s been organising our paper summaries) and we’ll get in touch soon
P.S. I’ve also just seen Joan’s write-up of the Focus University groups in the comments below, which suggests that there is already some decent self-evaluation, experimentation and feedback loops happening as part of these programmes’ designs. So it is very possible that there is a good amount of this going on that I (as a very casual observer) am just not aware of!
I completely agree that it is far easier to suggest an analysis than to execute one! I personally won’t have the capacity to do this in the next 12-18 months, but would be happy to give feedback on a proposal and/or the research as it develops if someone else is willing and able to take up the mantle.
I do think that this analysis is more likely to be done (and in a high quality way) if it was either done by, commissioned by, or executed with significant buy-in from CEA and other key stakeholders involved in community building and running local groups. This is partly a case of helping source data etc, but also gives important incentives for someone to do this research. If I had lots of free time over the next 6 months, I would only take this on if I was fairly confident that the people in charge of making decisions would value this research. One model would be for someone to write up a short proposal for the analysis and take it to the decision makers; another would be for the decision-makers to commission it (my guess is that this demand-driven approach is more likely to result in a well-funded, high quality study).
To be clear, I massively appreciate the work that many, many people (at CEA and many other orgs) do and have done on community building and professionalising the running of groups (sorry if the tone of my original comment was implicitly critical). I think such work is very likely very valuable. I also think the hits-based model is the correct one as we ramp up spending and that not all expenditure should be thoroughly evaluated. But in cases where it seems very likely that we’ll keep doing the same type of activity for many years and spend comparatively large resources on it (e.g. support for groups), it makes sense to bake self-evaluation into the design of programmes, to help improve their design in the future.
It’s bugged me for a while that EA has ~13 years of community building efforts but (AFAIK) not much by way of “strong” evidence of the impact of various types of community building / outreach, in particular local/student groups. I’d like to see more by way of baking self-evaluation into the design of community building efforts, and think we’d be in a much better epistemic place if this was at the forefront of efforts to professionalise community building efforts 5+ years ago.
By “strong” I mean a serious attempt at causal evaluation using experimental or quasi-experimental methods—i.e. not necessarily RCTs where these aren’t practical (though it would be great to see some of these where they are!), but some sort of “difference in difference” style analysis, or before-after comparisons. For example, how do groups’ key performance stats (e.g. EA’s ‘produced’, donors, money moved, people going on to EA jobs) compare in the year(s) before vs after getting a full/part time salaried group organiser? Possibly some of this already exists either privately or publicly and the relevant people know where to look (I haven’t looked hard, sorry!). E.g. I remember GWWC putting together a fundraising prospectus in 2015 which estimated various counterfactual scenarios. Have there been serious self-evaluations since ? (Sincere apologies if I’ve missed them or could find them easily—this is a genuine question!)
In terms of what I’d like to see more of with respect to self-evaluation, and tentatively think we could have done better on this over the last 5+ years:
When new initiatives are launched, serious consideration should be paid to how to get high quality evidence of the impact of those initiatives, which aspects of them work best.
E.g. with the recent scale-up of funding for EA groups and hiring or full time coordinators, it would be great if some sort of small-scale A/B test could be run and/or a phased-in introduction. E.g. you could take the top 30-40 universities/groups that we’d ideally have professional outreach at and randomly select half of them to start a (possibly phased-in) programme of professional group leading at the start of 2022-23, and another half at the start of 2023-24.
Possibly this is already happening and I don’t know—apologies if so! (I’ve had one very brief conversation with someone involved which suggested that it isn’t being approached like this)
One objection is that this would delay likely-valuable outreach and is hard to do well. This is true, but it builds knowledge for the future and I wish we’d done more of this 5+ years ago so we’d be more confident in the effectiveness of the increased expenditure today and ideally have a better idea what type of campus support is most effective!
I would love to see 1-4 people with strong quant / social science / impact evaluation skills work for ~6-12 months to do a retrospective evaluation of the evidence of the last ~13 years of movement-building efforts, especially support to local groups. They would need the support of people and organisations that led these efforts, to share data on expenditure and key outcomes. Even if lots of this relied on observational data, my guess is that distilling the information from various groups / efforts would be very valuable in understanding their effectiveness.
I just came across ‘Ordinary Pleasure’ by Toro y Moi, the lyrics of the chorus:
Maximize all the pleasure
Even with all this weather
Nothing can make it better
Maximize all the pleasure
Maximize all the pleasure
Even with all this weather
Nothing can make it better
Maximize all the pleasure
Yes please—that would be great! The easiest way is probably to just fill out the form and just clearly note somewhere that you’d be happy to be a mentor. Thanks!
Yep, there’s no requirement to have EA-related research ideas yet (and I’d expect that to be the case for a decent number of EA-interested and aligned people who participate in the mentoring programme)
Thanks a lot for sharing the syllabus, David, and for posting guidelines about using it. I think and hope this will serve as a really useful reference for people interested in pursuing (a career in) economics GPR. As you note, this is a bit broader than GPI’s current research focus (which is fairly narrowly focused on longtermism and associated questions for the time being), but I think there is valuable GPR to be done in these other areas too. As you also note, GPI is currently refreshing its research agenda to account for some of the exploration research we’ve done in economics over the last ~18 months—hopefully we’ll have a new and improved version out in the next 2-3 months.
I just wanted to explicitly add to this post that valuable GPR can, does, and should happen outside of an academic setting. I think this is implied in this post (e.g. the mention of OpenPhil and the link to the GPR roles on the 80k website), but is not quite explicit, so I just wanted to flag it. Researchers outside of academia face a different set of incentives to academics, and can sometimes have more freedom to work on questions that are more practically relevant but less ‘publishable’ in academic journals. The point is made quite nicely on the 80k GPR page here: https://80000hours.org/problem-profiles/global-priorities-research/#what-are-some-top-career-options-within-this-area
“That said, we expect that other centres will be established over the coming years, and you could also pursue this research in other academic positions.
One downside of academia, however, is that you need to work on topics that are publishable, and these are often not those that are most relevant to real decisions. This means it’s also important to have researchers working elsewhere on more practical questions.”Personally, I think/hope the field of GPR will develop in a similar way to ‘impact evaluation’ in development economics over the last ~20 years—i.e. significant progress has been made in academic research (including some of the more important methodological or foundational advances), but there has also been a lot of valuable non-academic impact evaluation research (including lots that is more directly relevant for decision-makers).
This is a great post—thanks a lot for writing it. I work at GPI, so want to add a bit of context on a couple of points, and add some of my own thoughts. Standard disclaimer that these are my personal views and not those of GPI though.
First, on GPI’s research agenda, and our progress in econ:“(One economics student told me that when reading the GPI research agenda, the economics parts read like it was written by philosophers. Maybe this contributes to the lack of headway on their economics research plans.)”
I think this is accurate and a reflection of how the research agenda was written and has evolved. For what it’s worth, we’re currently working on refreshing the research agenda to reflect some of the ‘exploration research’ we’ve done in economics in the past ~18 months—we should have an updated version in the next few months. More generally, we’ve had very little econ research capacity to date beyond pre-doctoral researchers (very junior in academic terms). This will improve very shortly—as Phil notes in a previous comment, we’ve hired two postdocs to start in the next month—but as others have noted, high quality academic work is hard and takes quite a lot of time, so this may not result in a step change in actionable econ research coming out of GPI in the short run, which leads on to my second comment…
Second, on theories of change—your point D1 is really important. We’ve actively discussed various ‘theories of change’ internally at GPI and how these should affect our strategy. A decent part of this discussion depends on what others are doing in EA and how we think GPI fits into the overall EA movement portfolio. Even within the (relatively narrow) scope of doing academic GP research in econ and philosophy, possible theories of change for GPI include (but are not limited to!) prioritising building up academic credibility for long-run influence, prioritising research that is more actionable for EAs/philanthropists and policymakers, prioritising influencing policymakers / the general public, or prioritising influencing the next generation through higher education. These are not mutually exclusive, but placing different emphasis on one or the other may imply different strategy. We are still very young, and so far we have mostly been focused on laying foundations for the first of these, and have so far made much more progress on this in philosophy than econ, though I expect things will evolve in the next few years. Personally, I don’t think we’ll be able to effectively target all of the possible theories of change, and I’d love to see more people and groups working on these.
Thanks, Aaron. Just to add to Steve’s response below:
1) We think that part of the reason for the large increase in people pledging ~1% of their income this academic year is due to (a) better training and messaging (largely because we now have a full time staff member doing this), and (b) our improved donation platform. (b) has allowed us to set different defaults for different chapters, and for donors to set their start date further in the future, so it coincides with graduation. Before that, our donation defaults were targeted at MBAs, so undergrads in particular would log on, see a massive default relative to their expected salary, and pledge something significantly lower. Juniors and sophomores would also be unable to pledge far enough in the future to start giving at graduation and so would often put in a ‘symbolic’ $10 pledge, which partly explains why the average donation (as percentage of average graduation income) was pretty low before this year.
2) Our donation data indicates that the power of defaults is pretty strong among our members (a large mass of people give the default amount on the sign up page). We plan on experimenting with changing these default options (e.g. having 2% and/or 5% as well as 1%) and testing whether this makes a difference. At first we will just do this on the online platform, but we’ll also consider randomly selecting some chapters to have messaging about higher percentages in future years, and again testing if this makes a difference.
One for the World: update after 6 months of our first staff member
Is this scheme still running? This page suggests the scheme is closed (https://eahub.org/actions/shopping/intro). Should we therefore all be using Amazon Smile instead?
Either way, it would be nice to see a short update, in particular how much this scheme moved to top charities (and how much effort it was to set up). Thanks!
This is useful feedback, and I’ve heard one or two similar sentiments before, though (in my experience) this type of “dismissive cynicism” has been quite rare.
We are quite careful in our messaging of the 1% figure, and try not to be self-congratulatory about giving this relatively small amount (but as you point out there is a tradeoff with trying to create a positive vibe vs being a bit more stoic about a small amount). For example, we often use the figure that Americans give 2.6% on average, to try to anchor people higher than 1% and show how normal that low level is. We also use this stat to have messaging along the lines of “you’ll give 2.6% on average, and will likely have a portfolio of charities where you give. Our ask is for at least 1% of that portfolio to go towards some of the most effective global poverty charities”. Increasingly, we do want to try to ‘upsell’ people more, but our efforts on this are fairly preliminary so far.
This is a really good and important point—thanks, Eli_Nathan. I don’t feel confident in having an ‘answer’ to this potential tradeoff (focusing on raising money vs deepening engagement), but a few thoughts:
1)
It seems that 1FTW attracts similar types of people that the GWWC pledge would, but at higher quantities due to the lower barrier. However, I’m skeptical that this lower barrier is necessarily a positive thing, because it would seem that, on average, these individuals are less likely to further engage with the EA community at large.
I think this is a reasonable view to take, and I agree that on average a OFTW members is less likely to engage deeply with EA than a GWWC member (or similar). I do think we have a number of cases, in particular some chapter leaders and ‘student ambassadors’, who have gone on to engage quite deeply with EA, and who may never have got involved without OFTW’s more broad-based approach. (I guess any evidence I have here is anecdotal though, and I don’t want to talk for others too much). So even if on average fewer people deeply engage with EA, I think it is very plausible that the total number is higher. I think the optimal setup at a university would be to have a thriving OFTW chapter (or something similar) that is engaging the broader student body with EA ideas, and a thriving general EA group that funnels those who are more interested in EA and other cause areas to get more involved. (See my other comments on the complementarity of these groups, and on the idea of ‘widening the funnel’ of engagement with EA, so more people just get involved, and more end up more deeply engaged).
2)
In my eyes, the comparative advantage for student groups is more about driving engagement and plan changes and less about raising funds.
I think this is also a very reasonable (and increasingly common) view to hold. Again, I think the ideal setup is for a general EA group to work with OFTW on this, but I think this is an area we would like our chapters to improve on. In discussions with GiveWell about the grant, they gave us feedback that they’d like to see more promotion of EA more generally by our chapters, and we also talked a bit about trying to offer 80,000-hours style material, to help generate engagement, plan changes and improve the ‘talent pool’ in EA. I think these are both areas that we want to improve on as we grow and increase our capacity.
3)
Of course, money still goes a long way, but I’m skeptical that group leaders should be spending their time focusing on (relatively) small donations over building communities of talented, engaged individuals.
On this, I think this is a fair short-term critique of the amount of money we are currently raising, but I think a lot of the (monetary) value in what we are doing is yet to be realised. I don’t know the stats off the top of my head, but X% of Wharton MBAs go on to become worth $Ym, and we want to try and engage these ‘future rich’ (and influential) people with EA at a relatively early stage (and while their preferences are still fairly malleable!). Keeping our members engaged over the years is going to be a key factor in our success going forward though!
Yeah, pretty much MBA students and Law students to a lesser extent. To be honest, when I was first approached by Josh and Kate (the founders) to join OFTW, I was fairly skeptical that it would catch on in the Wharton MBA class, but was impressed by the amount of market research and thought they’d put into the concept, the messaging and branding etc. One of the lessons I’ve learned over time is that the stereotype of a ‘typical’ MBA student caring more about money and a career than charities and doing good may not be (entirely!) fair—most people want to good, but haven’t thought deeply about it or know the best ways. We’re trying to make it easier and more convenient for them, and the idea has proved more popular than I had initially thought! I’m not sure to what extent this will extend to other communities, but one thing we want to start experimenting with is ‘corporate’ chapters, starting in companies where a lot of our members work.
I’m cross posting answers to some questions on the EA group organisers Facebook group below:
Have you found this to be significantly more successful than the 1% student version of GWWC?
In my personal experience, in terms of number of pledges, I would say yes. I’ve helped run Gwwc groups in Oxford (2011-2012) and Penn (2014-2017) and over those years I think we probably ‘caused’ 10-20 Gwwc pledges at Penn and <10 in my year at Oxford. The most established Oftw chapters seem capable of bringing in ~60-100+ each year. This compares favourably to the Gwwc groups I’ve been involved with, but probably not the most successful ones (eg Cambridge generated a huge number of Gwwc pledges through its pledge drive ~2.5(?) years ago). We also have some less established oftw chapters yet to hit those numbers.
In terms of deeper engagement with EA, I think a more general EA local group likely does a better job than a OFTW chapter on average. But overall I see the two approaches as complementary—Oftw to ‘broaden the funnel’ of engagement with EA and raise money, then a general EA group for those who become most engaged in EA, particularly in non-poverty cause areas. I think a OFTW pledge drive can help with engagement too, by giving concrete, tangible actions for members to work on.
Do you ever see people increase their pledge from 1%? Or do you see people feeling content with only 1%?
We haven’t seen many people increasing their donation past 1% so far, and have found this default pretty sticky. In the last year, we’ve emphasised the at least 1% messaging more, and changed defaults in our sign up page to include 2% options. We’ve also started experimenting with ‘upselling’ some of our more engaged members to higher amounts, but our first attempts didn’t yield much. Overall, this is an area (and donor engagement more generally) that we haven’t done much with so far, and are hoping to improve a lot on now we have more capacity.