Really interesting post, but I do want to flag a big concern I have in the comparative calculation. Broadly, estimated effects are almost always just going to be way more positive than well studied effects. For example if you estimated GD’s impact using standard income vs happiness adjustment measures (e.g. the value of double someone’s income on their happiness) you end up at a much higher number than the RCT results. I think this sort of thing happens pretty consistently and predictability. For example, it would be really easy to imagine Strong Minds treatments are different enough from the most studied ways of doing CBT for the treatment effects to only persist 1 year (which would reduce the cost-effectiveness to about equal), and it’s easy to imagine several such changes (almost all going in a more pessimistic direction).On the flip side, there has been extensive research, evaluation and huge numbers of charities founded in the global health space leading to a comparatively very small number of super strong charities, many of which are explicitly focused on cost-effectiveness/impact, etc. This same work (as far as I know) has not been done in the mental health area. In many ways, you are comparing a very strong global poverty charity to a much more average mental health charity. Thus personally I would not necessarily need to see a current mental health charity beating GiveWell’s best to be convinced the area as a whole could be very effective (if some strong research, evaluations and impact focused charities) were founded or identified in the area. Given my current work with Charity Entrepreneurship, the main case I am considering is if a new well researched and impact focused charity in mental health could be competitive with GiveWell top charities in effectiveness. I feel like the posts you have made over time have made this claim seem pretty plausible.
So I want to be pretty careful about going into details, but I can mix some stories together to make a plausible sounding story based on what I have heard. Please keep in mind this story is a fiction based off a composite of case studies I’ve witnessed, not a real example of any particular person.
Say Alice is an EA. She learns about it in his first year of college. She starts by attending an EA event or two and eventually ends up being a member of his university chapter and pretty heavily reading the EA forum. She takes the GWWC pledge and a year later she takes a summer internship at an EA organization. During this time she identifies strongly with the EA movement and considers it one of her top priorities. Sadly, as Alice is away at her internship her chapter suffers and when she gets back she hits a particularly rough year of school and due to long term concerns, she prioritizes school over setting the chapter back up, mainly thinking about her impact. The silver lining is at the end of this rough year she starts a relationship. The person is smart and well suited, but does not share her charitable interest. Over time she stops reading the EA content she used to and the chapter never gets started again. After her degree ends she takes a job in consulting that she says will give her career capital, but she has a sense her heart is not as into EA as she once was. She knows a big factor is her boyfriend’s family would approve of a more normal job than a charity focused one, plus she is confident she can donate and have some impact that way. Her first few paychecks she rationalizes as needing to move out and get established. The next few to build up a safe 6 month runway. The donations never happen. There’s always some reason or another to put it off, and EA seems so low on the priorities list now, just a thing she did in college, like playing a sport. Alice ends up donating a fairly small amount to effective charities (a little over 1%). Her involvement was at its peak when she was in college and she knows her college self would be disappointed. Each choice made sense at the time. Many of them even follow traditional EA advice, but the endline result is Alice does not really feel she is an EA anymore. She has many other stronger identities. In this story, with different recommendations from the EA movement and different choices from Alice, she could have ended up doing earning to give and donating a large percentage long term or working with an EA org long term, but instead she “value drifted”.
The following is a rough breakdown of the percentage of people who were not asked to move on to the next round in the Charity Science hiring process. These numbers assume one counterfactual hour of preparation for each interview and no preparation time outside of the given time limit for test tasks.~3* hour invested (50%) - Cover letter/resume~5 hours invested (20%) - Interview 1~10 hours invested (15%) - Test task 1~12 hours invested (5%) - Interview 2~17 hours invested (5%) - Test task 2~337 hours invested (2.5%) - paid 2-month work trialHired (2.5%)So, 95% of those not hired spend 17 hours or less, 85% spend 14 hours or less, and 70% spend 5 hours or less.
*changed from 1 hour to 3 hours based on comments
Just wanted to chip in on this. Although I do not think this addresses all the concerns I have with representativeness, I do think CEA has been making a more concerted and genuine effort at considering how to deal with these issues (not just this blog post, but also in some of the more recent conversations they have been having with a wider range of people in the EA movement). I think it’s a tricky issue to get right (how to build a cause neutral EA movement when you think some causes are higher impact than others) and there is still a lot of thought to be done on the issue, but I am glad steps are happening in the right direction.