I picked one claim from this more or less at random to check—the fourfold assumption about government spending—and I don’t think it’s accurate.
The sentence: “they also argue that the resources governments contribute would have been used less efficiently elsewhere, often assuming a fourfold difference in effectiveness (GiveWell, 2018b).”
I don’t think the 2018 post cited contains that assumption. The only 4:1 ratio I can see comes from a hypothetical: a charity buys $5,000 of “magic pills”, the government spends $5,000 distributing them, 1,000 lives are saved, and without the charity the government’s money would have saved 250.
The parameter GiveWell actually states in that post is that the counterfactual value of government funds is ~75% as cost-effective as GiveDirectly.
That’s also quite an old blog post, so I checked the current figure used in GiveWell’s SMC analysis. There, each dollar of in-kind government resources is estimated at around 1⁄20 of the value of SMC spending, not 1⁄4.
They also explain where the 1⁄20 comes from. Interestingly, it doesn’t imply a particularly low opinion of government effectiveness.
GiveWell guesses the displaced resources split 80/10/10 across health, education and social security, then estimates the value of each separately.
Health comes out at 0.0056 units of value per dollar. The composition is from Uganda’s National Health Expenditure Accounts 2013-14 — roughly 31% HIV/AIDS, 22% malaria, then maternal conditions, respiratory infections, perinatal conditions, injuries, nutritional deficiencies, diarrhoeal disease. Each of the eight largest categories gets a guessed cost per life saved ($30,000 for HIV/AIDS, $10,000 for malaria, $20,000 for most of the rest) and is classified as saving either adult or under-five lives. Categories without a guess, about a quarter of spending, are dropped and the rest rescaled. That produces a weighted $12,254 per child life and $30,000 per adult life, split 45⁄55, which GiveWell converts using its moral weights of 116 for a child death and 73 for an adult death.
Education (0.0028) and social security (0.0026) are done similarly, benchmarked against GiveDirectly rather than against lives saved. Weighted 80/10/10, that gives an overall estimate of 0.005 units of value per dollar.
Cash transfers sit at 0.0034 in the same model. So government spending is being valued at roughly 1.5x unconditional cash transfers — above GiveWell’s unit of account, though well below the 10x bar it requires to fund something.
The gap is large because SMC is modelled as exceptional, not because domestic institutions are rated poorly.
I’ve only checked this one claim, so I can’t say anything about the rest. I know Dirk-Jan a bit—he spoke at EAGxAmsterdam in December—and Martijn is on EAN’s board (for full disclosure). I’ll look through this in more detail and get in touch with Dirk-Jan. I haven’t sent this comment to Martijn to check; as he mentions in his comment, most of the writing was done by Dirk-Jan.
Edit: To be clear, none of this necessarily undermines the broader argument—you could read it as strengthening it. But it does look like a factual error, and I’d argue “GiveWell assumes a fourfold difference” is a mischaracterisation. It isn’t fourfold, and it isn’t an assumption: it’s an estimate, built up from stated components, and in the SMC case it comes out around 1⁄20.
The version I’d defend is closer to: GiveWell estimates a large difference, around twentyfold in one case.
It would be cool to hear from Dirk-Jan how he’d improve on that estimate!
I picked one claim from this more or less at random to check—the fourfold assumption about government spending—and I don’t think it’s accurate.
The sentence: “they also argue that the resources governments contribute would have been used less efficiently elsewhere, often assuming a fourfold difference in effectiveness (GiveWell, 2018b).”
I don’t think the 2018 post cited contains that assumption. The only 4:1 ratio I can see comes from a hypothetical: a charity buys $5,000 of “magic pills”, the government spends $5,000 distributing them, 1,000 lives are saved, and without the charity the government’s money would have saved 250.
The parameter GiveWell actually states in that post is that the counterfactual value of government funds is ~75% as cost-effective as GiveDirectly.
That’s also quite an old blog post, so I checked the current figure used in GiveWell’s SMC analysis. There, each dollar of in-kind government resources is estimated at around 1⁄20 of the value of SMC spending, not 1⁄4.
They also explain where the 1⁄20 comes from. Interestingly, it doesn’t imply a particularly low opinion of government effectiveness.
GiveWell guesses the displaced resources split 80/10/10 across health, education and social security, then estimates the value of each separately.
Health comes out at 0.0056 units of value per dollar. The composition is from Uganda’s National Health Expenditure Accounts 2013-14 — roughly 31% HIV/AIDS, 22% malaria, then maternal conditions, respiratory infections, perinatal conditions, injuries, nutritional deficiencies, diarrhoeal disease. Each of the eight largest categories gets a guessed cost per life saved ($30,000 for HIV/AIDS, $10,000 for malaria, $20,000 for most of the rest) and is classified as saving either adult or under-five lives. Categories without a guess, about a quarter of spending, are dropped and the rest rescaled. That produces a weighted $12,254 per child life and $30,000 per adult life, split 45⁄55, which GiveWell converts using its moral weights of 116 for a child death and 73 for an adult death.
Education (0.0028) and social security (0.0026) are done similarly, benchmarked against GiveDirectly rather than against lives saved. Weighted 80/10/10, that gives an overall estimate of 0.005 units of value per dollar.
Cash transfers sit at 0.0034 in the same model. So government spending is being valued at roughly 1.5x unconditional cash transfers — above GiveWell’s unit of account, though well below the 10x bar it requires to fund something.
The gap is large because SMC is modelled as exceptional, not because domestic institutions are rated poorly.
I’ve only checked this one claim, so I can’t say anything about the rest. I know Dirk-Jan a bit—he spoke at EAGxAmsterdam in December—and Martijn is on EAN’s board (for full disclosure). I’ll look through this in more detail and get in touch with Dirk-Jan. I haven’t sent this comment to Martijn to check; as he mentions in his comment, most of the writing was done by Dirk-Jan.
Edit: To be clear, none of this necessarily undermines the broader argument—you could read it as strengthening it. But it does look like a factual error, and I’d argue “GiveWell assumes a fourfold difference” is a mischaracterisation. It isn’t fourfold, and it isn’t an assumption: it’s an estimate, built up from stated components, and in the SMC case it comes out around 1⁄20.
The version I’d defend is closer to: GiveWell estimates a large difference, around twentyfold in one case.
It would be cool to hear from Dirk-Jan how he’d improve on that estimate!