GiveWell recently announced the latest update of their top charities list. This post is a collection of my notes on the changes, mostly focusing on the cost-effectiveness estimates.A year ago I put a lot of hours into understanding the cost-effectiveness calculations for deworming and cash transfers, the results of which I wrote up in this post. I had planned to make this a fairly short post, merely indicating the important changes from last year, but I ended up going into a bit of detail (it’s still a lot shorter than last year’s).Before I get into all that, a few preliminaries:
The Against Malaria Foundation distributes insecticide-treated bednets; the primary benefit is preventing child deaths from malaria. I think of its effectiveness in units of dollars per (statistical) life saved.
The Schistosomiasis Control Initiative (SCI) funds and helps to implement mass deworming programmes. The Deworm the World Initiative (DtWI) encourages governments to implement mass deworming programmes. The primary benefits from deworming school-aged children are that they grow into healthier, stronger adults, and are more productive in adulthood. (A secondary benefit comes from improving the health of worm-infected children in the short-term.) I think of the benefits in units of “proportional increase in (discounted) adult consumption relative to income, per donated dollar”.
GiveDirectly transfer cash to poor households in Kenya and Uganda. Some of this money is consumed, some is invested; the benefit is in units of “proportional increase in (discounted) consumption relative to income, per donated dollar”.
I use the following approximate equivalences (your mileage may vary!):
1 DALY averted is equivalent to consumption of 3 times annual income;
1 statistical life saved by bednets is equivalent to 34 DALY’s averted.
It is a little silly to be so precise as to say “34” DALY’s and in my head I usually round to 30. I only wrote 34 above so that it’s consistent with a discount rate of 1.7% – I use that discount rate because that’s roughly around where I feel it should be.
Against Malaria Foundation
GiveWell (perhaps only temporarily) removed AMF from its recommended charities because they don’t think that AMF has room for more funding. I can’t comment particularly usefully on this topic; those who want the details can read this post on the GiveWell blog. The important question is whether or not AMF will be able to secure agreements for large net distributions with one or more governments. If they can’t, then they won’t be able to spend the money directed to them via GiveWell as fast as it comes in. If they can’t, it won’t be so bad – the money donated will be spent well, it just won’t be spent for a while.
My subjective guess is that the probability of AMF securing an agreement for a large net distribution is high enough to warrant continuing donating to them.
It is also important note some changes to GiveWell’s assessment of bednet distributions. The rather surprising headline figure is a cost per life saved of $3400 (using average costs) or $3200 (using marginal costs). This is up almost $1000 from last year. The changes come from an increase in average net costs ($5.54 to $6.13) and the introduction of a fudge factor to take into account mosquito resistance to the insecticides used to treat the nets. GiveWell subjectively estimate this factor at 80% (so that cost per life saved is multiplied by 1⁄0.8 = 1.25).
I don’t have much of a feel for what this number should be. On the one hand, the resistance seems to develop over a number of years, it’s been sporadically monitored for a while, resistance on the map here is defined at at least 20% mosquito survival, and there are plenty of dots on the map which don’t show such levels of resistance. On the other hand, I was reading a report a few weeks ago (I forget which one) on this topic and it was dispiriting. On balance, I feel that 80% is being a little too pessimistic for the time being, and I’ll round down GiveWell’s $3400 figure to $3000, while hoping that a properly effective vaccine will eventually render this problem moot.
Deworming
This calculation remains horrendously convoluted.
The same host of subjective adjustment factors are present in this year’s calculation – the results rely on a study that was run in an area with particularly high levels of worm infection, and there’s some disagreement in the literature over how effective mass deworming programmes are (as opposed to more targeted deworming treatment). For the most part I am happy with the range of subjective guesses I used last year. An important not-so-subjective change is SCI’s cost per treatment, up from $0.51 to somewhere between $0.72 and $1.13. The GiveWell staff use $0.99 as their best guess, but their SCI review describes this value as “conservative”, and I’d be happier with a figure closer to $0.72. I’ve used $0.80 as my input into the spreadsheet.
GiveWell have tidied up the calculation in one place – last year it was assumed (at least implicitly) that all benefits from deworming, both developmental and short-term, came from giving the treatment to children. This year the short-term health benefits to adults are included. This gives a non-trivial boost to the overall cost-effectiveness, almost cancelling out the effect of the increase in estimated cost per treatment.
GiveWell’s spreadsheet has a “Prevalence/intensity adjustment—SCI” input, set to 66.67%. This is new; last year I argued that this should be incorporated into the external validity adjustment, so I’m happy to see it there (though I set it to 100% and use the same external validity adjustment as I did last year).
Putting it all together, my preferred inputs say that a donated dollar to SCI increases discounted consumption for one person by 1.4% of annual income, down from my best guess of 1.7% last year. By the equivalences I listed at the start, that equates to a cost per life-saved-equivalent of $7000.
DtWI’s advocacy efforts are so hard to quantify that I will ignore the cost-effectiveness calculation for it.
Cash transfers
The cost-effectiveness of cash transfers depends on the percentage of the transfers that get invested, and the return on those investments. Last year I followed the GiveWell staff and estimated that 75% of GiveDirectly’s cash transfers would be invested, and assumed an ROI of 25%.
This year, the GiveWell staff mostly estimate 50% rate of investment and a 15% ROI. The rate of investment is lower than what is reported in GiveDirectly surveys (around 70%), and comes from somewhere in a jumble of econometric tables that I couldn’t be bothered deciphering.
The ROI of 15% is estimated from the ROI from buying an iron roof, the most commonly-mentioned large investment made by GiveDirectly recipients. Reported ROI’s for the iron rooves range from 7% to 23%, so the figure of 15% seems reasonable.
The result is that a donated dollar increases discounted consumption by 0.57%, down from my best guess of 0.96% last year. The 0.57% figure equates to a cost per life-saved equivalent of $18,000. (As was the case last year, I end up with cash transfers being more cost-effective than the GiveWell staff, because I use a lower discount rate than they do.)
That is a disappointingly high number, and it is worth reflecting on it for a bit. There have now been quite a few studies on various types of cash transfers, most of them finding much higher ROI’s than 15%. GiveDirectly themselves apparently designed their plans around 25%. It is possible that as GD expands their programmes elsewhere, ROI’s will improve; it’s also possible (and this is what I suspect is more likely) that the fully hands-off approach by GD leads to lower returns than cash transfer programmes that put some conditions on the grants. (Of course the latter would require more overhead, but the differences in ROI are big.)
While I remain very positive about cash transfers, and so will continue to support GD as the leading cash transfer charity (I’m also a huge fan of their randomised rollouts, which lets us have this sort of discussion at all; they also experiment with the design of their transfers, also worth supporting), I am a little less positive about GD than I used to be.
GiveWell?
GiveWell still has a funding gap. I’ve seen people want to have discussions about the merits of donating to GiveWell directly, given that Good Ventures can fund them (the question is whether GiveWell should have a large, independent revenue source separate from GV), but I found it all very unsatisfying. It gets into the sort of governance issues that I really don’t have a clue about.
I donated US$5000 to GiveWell today. I didn’t overthink it, just made a decision.
Conclusions
GiveWell’s cost-effectiveness spreadsheet with my inputs for deworming and cash transfers (in the ‘DB’ sheet) is here (XLSX).
Last year I had deworming as 1.7 times as cost-effective as cash transfers. This year I have it 2.5 times as cost-effective as GiveDirectly’s cash transfers. All of these numbers are rubbery, of course, but it’s enough for me to weight my giving more towards deworming this coming year.
If we ignore the room-for-more-funding issue, AMF is an even more clear stand-out number 1 charity.
There are strong arguments for delaying donations until the AMF situation becomes clearer. I suspect I will continue to support AMF at a reduced level this coming year. I’d be uncomfortable if the effective altruist community just stopped donating for a while, so I think I will continue my regular donations, with SCI getting the greatest share.
Current plan for 2013 donation splits: SCI 40, AMF 30, GD 20, DtWI 10. That’ll probably change.
Notes on GiveWell’s recommended charities
GiveWell recently announced the latest update of their top charities list. This post is a collection of my notes on the changes, mostly focusing on the cost-effectiveness estimates.A year ago I put a lot of hours into understanding the cost-effectiveness calculations for deworming and cash transfers, the results of which I wrote up in this post. I had planned to make this a fairly short post, merely indicating the important changes from last year, but I ended up going into a bit of detail (it’s still a lot shorter than last year’s).Before I get into all that, a few preliminaries:
I use the following approximate equivalences (your mileage may vary!):The Against Malaria Foundation distributes insecticide-treated bednets; the primary benefit is preventing child deaths from malaria. I think of its effectiveness in units of dollars per (statistical) life saved.
The Schistosomiasis Control Initiative (SCI) funds and helps to implement mass deworming programmes. The Deworm the World Initiative (DtWI) encourages governments to implement mass deworming programmes. The primary benefits from deworming school-aged children are that they grow into healthier, stronger adults, and are more productive in adulthood. (A secondary benefit comes from improving the health of worm-infected children in the short-term.) I think of the benefits in units of “proportional increase in (discounted) adult consumption relative to income, per donated dollar”.
GiveDirectly transfer cash to poor households in Kenya and Uganda. Some of this money is consumed, some is invested; the benefit is in units of “proportional increase in (discounted) consumption relative to income, per donated dollar”.
It is a little silly to be so precise as to say “34” DALY’s and in my head I usually round to 30. I only wrote 34 above so that it’s consistent with a discount rate of 1.7% – I use that discount rate because that’s roughly around where I feel it should be.1 DALY averted is equivalent to consumption of 3 times annual income;
1 statistical life saved by bednets is equivalent to 34 DALY’s averted.
Against Malaria Foundation
GiveWell (perhaps only temporarily) removed AMF from its recommended charities because they don’t think that AMF has room for more funding. I can’t comment particularly usefully on this topic; those who want the details can read this post on the GiveWell blog. The important question is whether or not AMF will be able to secure agreements for large net distributions with one or more governments. If they can’t, then they won’t be able to spend the money directed to them via GiveWell as fast as it comes in. If they can’t, it won’t be so bad – the money donated will be spent well, it just won’t be spent for a while.My subjective guess is that the probability of AMF securing an agreement for a large net distribution is high enough to warrant continuing donating to them.
It is also important note some changes to GiveWell’s assessment of bednet distributions. The rather surprising headline figure is a cost per life saved of $3400 (using average costs) or $3200 (using marginal costs). This is up almost $1000 from last year. The changes come from an increase in average net costs ($5.54 to $6.13) and the introduction of a fudge factor to take into account mosquito resistance to the insecticides used to treat the nets. GiveWell subjectively estimate this factor at 80% (so that cost per life saved is multiplied by 1⁄0.8 = 1.25).
I don’t have much of a feel for what this number should be. On the one hand, the resistance seems to develop over a number of years, it’s been sporadically monitored for a while, resistance on the map here is defined at at least 20% mosquito survival, and there are plenty of dots on the map which don’t show such levels of resistance. On the other hand, I was reading a report a few weeks ago (I forget which one) on this topic and it was dispiriting. On balance, I feel that 80% is being a little too pessimistic for the time being, and I’ll round down GiveWell’s $3400 figure to $3000, while hoping that a properly effective vaccine will eventually render this problem moot.
Deworming
This calculation remains horrendously convoluted.The same host of subjective adjustment factors are present in this year’s calculation – the results rely on a study that was run in an area with particularly high levels of worm infection, and there’s some disagreement in the literature over how effective mass deworming programmes are (as opposed to more targeted deworming treatment). For the most part I am happy with the range of subjective guesses I used last year. An important not-so-subjective change is SCI’s cost per treatment, up from $0.51 to somewhere between $0.72 and $1.13. The GiveWell staff use $0.99 as their best guess, but their SCI review describes this value as “conservative”, and I’d be happier with a figure closer to $0.72. I’ve used $0.80 as my input into the spreadsheet.
GiveWell have tidied up the calculation in one place – last year it was assumed (at least implicitly) that all benefits from deworming, both developmental and short-term, came from giving the treatment to children. This year the short-term health benefits to adults are included. This gives a non-trivial boost to the overall cost-effectiveness, almost cancelling out the effect of the increase in estimated cost per treatment.
GiveWell’s spreadsheet has a “Prevalence/intensity adjustment—SCI” input, set to 66.67%. This is new; last year I argued that this should be incorporated into the external validity adjustment, so I’m happy to see it there (though I set it to 100% and use the same external validity adjustment as I did last year).
Putting it all together, my preferred inputs say that a donated dollar to SCI increases discounted consumption for one person by 1.4% of annual income, down from my best guess of 1.7% last year. By the equivalences I listed at the start, that equates to a cost per life-saved-equivalent of $7000.
DtWI’s advocacy efforts are so hard to quantify that I will ignore the cost-effectiveness calculation for it.
Cash transfers
The cost-effectiveness of cash transfers depends on the percentage of the transfers that get invested, and the return on those investments. Last year I followed the GiveWell staff and estimated that 75% of GiveDirectly’s cash transfers would be invested, and assumed an ROI of 25%.This year, the GiveWell staff mostly estimate 50% rate of investment and a 15% ROI. The rate of investment is lower than what is reported in GiveDirectly surveys (around 70%), and comes from somewhere in a jumble of econometric tables that I couldn’t be bothered deciphering.
The ROI of 15% is estimated from the ROI from buying an iron roof, the most commonly-mentioned large investment made by GiveDirectly recipients. Reported ROI’s for the iron rooves range from 7% to 23%, so the figure of 15% seems reasonable.
The result is that a donated dollar increases discounted consumption by 0.57%, down from my best guess of 0.96% last year. The 0.57% figure equates to a cost per life-saved equivalent of $18,000. (As was the case last year, I end up with cash transfers being more cost-effective than the GiveWell staff, because I use a lower discount rate than they do.)
That is a disappointingly high number, and it is worth reflecting on it for a bit. There have now been quite a few studies on various types of cash transfers, most of them finding much higher ROI’s than 15%. GiveDirectly themselves apparently designed their plans around 25%. It is possible that as GD expands their programmes elsewhere, ROI’s will improve; it’s also possible (and this is what I suspect is more likely) that the fully hands-off approach by GD leads to lower returns than cash transfer programmes that put some conditions on the grants. (Of course the latter would require more overhead, but the differences in ROI are big.)
While I remain very positive about cash transfers, and so will continue to support GD as the leading cash transfer charity (I’m also a huge fan of their randomised rollouts, which lets us have this sort of discussion at all; they also experiment with the design of their transfers, also worth supporting), I am a little less positive about GD than I used to be.
GiveWell?
GiveWell still has a funding gap. I’ve seen people want to have discussions about the merits of donating to GiveWell directly, given that Good Ventures can fund them (the question is whether GiveWell should have a large, independent revenue source separate from GV), but I found it all very unsatisfying. It gets into the sort of governance issues that I really don’t have a clue about.I donated US$5000 to GiveWell today. I didn’t overthink it, just made a decision.
Conclusions
Crossposted from David Barry’s blogGiveWell’s cost-effectiveness spreadsheet with my inputs for deworming and cash transfers (in the ‘DB’ sheet) is here (XLSX).
Last year I had deworming as 1.7 times as cost-effective as cash transfers. This year I have it 2.5 times as cost-effective as GiveDirectly’s cash transfers. All of these numbers are rubbery, of course, but it’s enough for me to weight my giving more towards deworming this coming year.
If we ignore the room-for-more-funding issue, AMF is an even more clear stand-out number 1 charity.
There are strong arguments for delaying donations until the AMF situation becomes clearer. I suspect I will continue to support AMF at a reduced level this coming year. I’d be uncomfortable if the effective altruist community just stopped donating for a while, so I think I will continue my regular donations, with SCI getting the greatest share.
Current plan for 2013 donation splits: SCI 40, AMF 30, GD 20, DtWI 10. That’ll probably change.