Evaluating Small, Effective Charities Vs. Large, Less Effective Charities
Hi EA’ers,
I’ve got a question for you all about the best way to invest limited benefit-cost evaluation resources. I wonder if folks have any thoughts about the relative value of evaluating small group of highly effective—and typically small—charities, versus spending additional resources to evaluate large charities which may not be as cost-effective..
Ideally, everyone would give to the most effective charities, but the fact is that larger charities—from United Way on down—receive the bulk of giving dollars and have a disproportionate impact on how giving happens and how it’s evaluated.
EA tends to focus most of its evaluation resources on a few, targeted charities. While this is obviously important, it also seems to me that we are missing some “low-hanging fruit” by not spending more evaluation resources on a wider range of charities. Directing resources to more fully evaluate and improve the cost-effectiveness of large charities could have a larger “meta” benefit-cost ratio, in the sense of the value of an evaluation dollar in increasing the benefit-cost ratio of other dollars spent.
For example, the Salvation Army received $2 billion in donations last year. Assuming that the money donated to it is roughly fixed, and not (unfortunately) very sensitive to rigorous measures of impact, then increasing its cost-effectiveness by only a small amount could have a greater net effect than further in-depth evaluations of smaller charities.
I’ve longed dreamed of a mega-database that would include detailed cost-effectiveness measures for lots of charities, as I think this would import the ideas of EA into a much greater pool of giving money and help it gain political strength. Such a project would need some pretty intensive resources, however.
I’m interested in your thoughts!
This sounds like a more rigorous and detailed version of the Back of the Envelope Guide to Philanthropy.
Thanks, I hadn’t heard of that site yet.
This is an excellent question.
Givewell has written about this:
http://www.givewell.org/search/google/Mega-charities?query=Mega-charities&cx=007119284953973998335%3Arwakucpx-to&cof=FORID%3A11&sitesearch=&safe=medium
My sense is that UNICEF is the most effective big (health) charity—they do lots of highly effective health interventions:
they deliver 20-30 million bednets every year—for comparison AMF has delivered 6-7 since its inception
they also spend lots of money on vaccinations, which are likely to be more effective than bednets (though better funded due to GAVI)
big on HIV spending
But their effectiveness might be diluted because they spend money on non-priority interventions and countries that are not among the least developed countries such as the DRC, where AMF is active.
One thing that has changed since the Givewell posts have been written is that UN agencies (among them UNICEF) have become more transparent, but it’s still hard to see what exactly they spend their money on: ati.publishwhatyoufund.org/index-2014/results/
You can see UNICEF’s spending on their 7 programme areas here:
http://www.unicef.org/strategicplan/files/2013-ABL4-Integrated_budget-ODS-English.pdf
and then look at the thematic reports here:
http://www.unicef.org/publicpartnerships/66662_66837.html
where they give a breakdown of what they spend their money on in the programme areas (it’s still quite crude, I couldn’t find out what they spend on which vaccine, but the data might be out there- I just couldn’t find it).
You could do a back of the envelope calculation and see whether they’re on average better than AMF, and I would not be super terribly surprised, but that doesn’t mean that your marginal dollar that you donate to UNICEF will have a bigger impact. It’s very difficult to properly ring fence money and restrict funding to priority areas when giving to UNICEF as a small donor, because they might just shift money around (using their considerable unrestricted funding to do that). I’m also not sure whether when you go on the UNICEF website and click ‘I want to provide a family with a bednet’, whether that then is really going into the restricted funding pot.
If the EA movement were to grow considerably, we might be able to ringfence money properly, by looking at UNICEFs projected spending and the immunization expenditure line and then say ‘we want you to spend more on immunizations’ or use Social Impact Bonds: ‘if everyone in country x is immunized against measles, the EA community will pay X million $’.
Givewell has recently had a post on ‘Charities we’d like to see’ and they wish for an immunization charity, but I’m not sure whether there ever will be one.
The mechanism of action is not entirely clear to me: When, hypothetically, you’re able to run a cost-effectiveness evaluation on the Salvation Army and publish the results, then the only people who will see it will be EAs who wouldn’t donate to it anyway. All the while the Salvation Army would have no more incentive to improve its efficiency than it has now. However, such an evaluation would require substantial cooperation from the charity being evaluated, and that’s already where the endeavor would be stopped short because the Salvation Army will probably see no reason to cooperate on that. Does that clarify your question?
One way to raise the efficiency waterline, as it were, is probably to reach out to people running inefficient charities, explain considerations of cause-neutrality to them, and hope that some will update and improve their charities. This may succeed in some cases, but it requires overcoming powerful biases, because these people have already invested many years of their lives inefficiently and will be the lone dissenting voice in their organization if they update. Plus, they may have to accept raising less money if they wish to invest it more efficiently (of course more than offsetting the absolute decrease). If they had been doing fundraising for fundraising’s sake, they’ll have to abandon their bottom line of money raised in favor of their new favorite impact proxy, also a momentous change.
Another approach is what prioritization organizations (like GiveWell and ACE) and most organizations working on movement building try to accomplish. What they do instead is to awaken donors to the importance of impact, thus shifting donations from less efficient to more efficient charities, and thus creating an incentive (ideally a forcible one) for less efficient charities to become more efficient in order to retain or gain donors. They try to establish a more efficient market for altruistic investments, so that it is then the market that forces charities to improve.
Basically, money raised – as opposed to impact – is bottom line that no charity can afford to ignore lest it perish. So they’re trying to turn money raised into a less disastrous proxy for impact. ^^