One key takeaway I took from it was to have more confidence in GiveWell’s analysis of it’s current top recommended charities. The suggested changes here mostly move numbers by 10%-30% which is significant but not huge. I do CEAs for my job and this seems pretty good to me. After reading this I feel like GiveWell’s cost effectiveness analysis are OK. Not perfect but as a rough decision heuristic they probably work fine. CEAs are ultimately rough approximations at best and this is how CEAs should be / are used by GiveWell.
My suggestion to GiveWell on how they can improve, after reading this post would be: Maybe it is more valuable for GiveWell to spend their limited person hours doing rough assessments of more varied and different interventions, than perfecting their analysis of these top charities. I would be more excited to see GiveWell committing to develop very rough speculative back of the envelop style analyses of a broader range of interventions (mental health, health policy, economic growth, system change, etc) than keep improving their current analyses to perfection. (Although maybe more of both is the best plan if it is achievable.)
I think this is a sentiment that the MichaelPlant (one of the post authors) has expressed in the past, see his comment here. I would be curious to hear if the post authors have thoughts, after doing this analysis, on the value of GiveWell investing more in exploration verses more in accuracy.
I’d prefer more investment in exploration and accuracy[1]. I wouldn’t be surprised if GiveWell pursues but doesn’t publish more speculative research of the type you’re mentioning.
I found most of these issues in a couple of hours. I also didn’t have time to publish critiques on all of the issues I found. So I am not quite as reassured as you are, particularly because relatively small changes in these figures can still lead to changes in the allocation of millions of dollars.
I like this post.
One key takeaway I took from it was to have more confidence in GiveWell’s analysis of it’s current top recommended charities. The suggested changes here mostly move numbers by 10%-30% which is significant but not huge. I do CEAs for my job and this seems pretty good to me. After reading this I feel like GiveWell’s cost effectiveness analysis are OK. Not perfect but as a rough decision heuristic they probably work fine. CEAs are ultimately rough approximations at best and this is how CEAs should be / are used by GiveWell.
My suggestion to GiveWell on how they can improve, after reading this post would be: Maybe it is more valuable for GiveWell to spend their limited person hours doing rough assessments of more varied and different interventions, than perfecting their analysis of these top charities. I would be more excited to see GiveWell committing to develop very rough speculative back of the envelop style analyses of a broader range of interventions (mental health, health policy, economic growth, system change, etc) than keep improving their current analyses to perfection. (Although maybe more of both is the best plan if it is achievable.)
I think this is a sentiment that the MichaelPlant (one of the post authors) has expressed in the past, see his comment here. I would be curious to hear if the post authors have thoughts, after doing this analysis, on the value of GiveWell investing more in exploration verses more in accuracy.
I’d prefer more investment in exploration and accuracy[1]. I wouldn’t be surprised if GiveWell pursues but doesn’t publish more speculative research of the type you’re mentioning.
I found most of these issues in a couple of hours. I also didn’t have time to publish critiques on all of the issues I found. So I am not quite as reassured as you are, particularly because relatively small changes in these figures can still lead to changes in the allocation of millions of dollars.
See here for a general argument supporting the cost-effectiveness of funding more research.
I see that they often publish summaries of that kind of research in various places, two random examples:
Malnutrition: https://docs.google.com/spreadsheets/d/15Q3ww3vmZINDsJ2lGNlwdw3amRiiR_y2mkSkbUf1i-g/edit#gid=1814038446 (from https://blog.givewell.org/2021/11/19/malnutrition-treatment/ )
Family planning radio campaigns: https://docs.google.com/document/d/1_k7SBtgCOwMITTyseaz9rzWv3skb7XWts89m8oL6Wb8/edit and https://docs.google.com/spreadsheets/d/1KtjwUbh5za5LKkH2WDe6g8UQ3cOcaRs-dxZgGcvYnDE/edit#gid=18893171 (from their September newsletter)
Thank you Joel. Makes sense. Well done on finding these issues!