On the robustness of cost-effectiveness estimates

A good post from Jonah Sinick on the Ro­bust­ness of cost-effec­tive­ness es­ti­mates here. In brief: GiveWell have con­sis­tently found that the cost-effec­tive­ness es­ti­mates of the best in­ter­ven­tions are too op­ti­mistic. This is a to-be-ex­pected ex­am­ple of re­gres­sion to the mean . But Jonah has found that this effect has oc­curred much more than he would have thought be­fore he worked for GiveWell. So he urges you to go with more ro­bust ev­i­dence of good but lower cost-effec­tive­ness over less ro­bust but higher cost-effec­tive­ness, and sug­gests that this might be a mis­take that EAs make more gen­er­ally.

I’m pretty sym­pa­thetic to the thrust of the post, but I had a cou­ple of im­me­di­ate thoughts.

First, Jonah talks about “lives saved per dol­lar”—but what EAs are ul­ti­mately con­cerned about is “good done per dol­lar”. But, for me, the amount of good I can do per dol­lar is far greater than I ini­tially would have thought. This is be­cause, in my ini­tial anal­y­sis—and in what I’d pre­sume are most peo­ple’s ini­tial analy­ses—benefits to the long-term fu­ture of civil­i­sa­tion weren’t taken into ac­count, or weren’t thought to be morally rele­vant. (Ex­am­ple: sav­ing a life doesn’t just benefit that per­son, but also, be­cause the per­son is eco­nom­i­cally pro­duc­tive, benefits all of so­ciety. More­over, these eco­nomic benefits con­tinue through the gen­er­a­tions and in fact com­pound at a rate roughly equal to gen­eral eco­nomic growth). But those (ex­pected) benefits strike me, and strike most peo­ple I’ve spo­ken with who agree with the moral rele­vance of them, to be far greater than the short-term benefits to the per­son whose life is saved. So, in terms of my ex­pec­ta­tions about how much good I can do in the world, I’m able to ex­ceed those by a far greater amount than I’d pre­vi­ously thought likely. And that holds true whether it costs $2000 or $20000 to save a life. So the les­son to take from past up­dates on ev­i­dence can look quite differ­ent de­pend­ing on whether you’re talk­ing about “good done per dol­lar” or “lives saved per dol­lar”, and the former is what we ul­ti­mately care about.

Se­cond, some­thing Jonah doesn’t men­tion is that, when you find out that your ev­i­dence is poorer than you’d thought, two gen­eral les­sons are to pur­sue ac­tivi­ties with high op­tion value and to pay to gain new ev­i­dence (though this les­son do only fol­low if you think you can get a de­cent amount of new ev­i­dence in the fu­ture). Build­ing a move­ment of peo­ple who are aiming to do the most good with their marginal re­sources, and who are try­ing to work out how best to do that, strikes me as a good way to achieve both of these things, and it’s been a ma­jor fac­tor in my de­ci­sion to fo­cus on build­ing the effec­tive al­tru­ism move­ment in gen­eral, rather than fo­cus­ing on any one spe­cific ac­tivity.

No comments.