Fact checking comparison between trachoma surgeries and guide dogs

In a 2013 TED talk Peter Singer claims

“It costs about 40,000 dol­lars to train a guide dog and train the re­cip­i­ent so that the guide dog can be an effec­tive help to a blind per­son. It costs some­where be­tween 20 and 50 dol­lars to cure a blind per­son in a de­vel­op­ing coun­try if they have tra­choma.”

Un­for­tu­nately, this claim is not ac­cu­rate. To be­gin with, blind­ness from tra­choma is ir­re­versible so it’s only pos­si­ble to pre­vent blind­ness from tra­choma, not to cure it. Ac­cord­ing to a GiveWell blog post, it does cost ~$20-60 to perform one tra­choma surgery but “there can be a small im­prove­ment in vi­sion fol­low­ing surgery”. Ac­cord­ing to their back-of-en­velope calcu­la­tion with some as­sump­tions, 1 case of full-blown blind­ness is averted for ev­ery 6-20 suc­cess­ful surg­eries. In any case, my point is that peo­ple who use this ex­am­ple to ad­ver­tise GiveWell don’t read what GiveWell has to say about it.


EDIT (2017-05-16): Even though GiveWell haven’t made such claim and may have a differ­ent opinion, one doc­tor (who has a much deeper un­der­stand­ing of these is­sues than me) com­mented that she “would be com­fortable with say­ing that for about $100 we can pre­vent tra­choma-in­duced blind­ness” and that Singer’s claim was not as nearly as in­ac­cu­rate as I made it seem.


As of 2017-05-10, Giv­ing What We Can also gives a similar ex­am­ple:

“In the de­vel­op­ing world there are more than a mil­lion peo­ple suffer­ing from tra­choma-in­duced blind­ness and poor vi­sion which could be helped by a safe eye op­er­a­tion, cost­ing only about $100 and pre­vent­ing 1-30 years of blind­ness and an­other 1-30 years of low vi­sion, ac­cord­ing to GiveWell.org

They also do some­thing EAs (in­clud­ing me) don’t do of­ten enough — provide a source. The source is a GiveWell page which is pub­lished in 2009 and has a dis­claimer

“The con­tent on this page has not been re­cently up­dated. This con­tent is likely to be no longer fully ac­cu­rate, both with re­spect to the re­search it pre­sents and with re­spect to what it im­plies about our views and po­si­tions.”

The page has the fol­low­ing text:

“We have not done thor­ough cost-effec­tive­ness anal­y­sis of this pro­gram. Be­cause such anal­y­sis is highly time-con­sum­ing—and be­cause the re­sults can vary sig­nifi­cantly de­pend­ing on de­tails of the con­text—we gen­er­ally do not provide cost-effec­tive­ness anal­y­sis for an in­ter­ven­tion un­less we find what we con­sider to be a strong as­so­ci­ated giv­ing op­por­tu­nity.

We provide some pre­limi­nary figures based on the Disease Con­trol Pri­ori­ties in Devel­op­ing Coun­tries re­port, which we pre­vi­ously used for cost-effec­tive­ness es­ti­mates un­til we vet­ted its work in 2011, find­ing ma­jor er­rors that raised gen­eral con­cerns.

We have rel­a­tively lit­tle in­for­ma­tion about the likely im­pact of this pro­gram, so it’s difficult to es­ti­mate the cost-effec­tive­ness.”


“Us­ing a sim­ple con­ver­sion calcu­la­tion, we es­ti­mate that $100 pre­vents 1-30 years of blind­ness and an ad­di­tional 1-30 years of low vi­sion when spent on surg­eries (though in­signifi­cant benefits, in these terms, when spent on an­tibiotics). The source of the Disease Con­trol Pri­ori­ties in Devel­op­ing Coun­tries re­port’s es­ti­mate is un­clear and these figures should be taken with ex­treme cau­tion.

It seems un­fair to just provide the num­bers and skip all these dis­claimers. De­spite know­ing about this un­cer­tainty, some­times I feel temp­ta­tion to also omit dis­claimers and just pre­sent the num­bers to be more con­vinc­ing. After all, the goal is very ad­mirable—to help more peo­ple liv­ing in ex­treme poverty. But I be­lieve that in the long run EA will achieve more if we are be­ing to­tally hon­est and up­front about un­cer­tain­ties and never take any short­cuts. Other­wise we might not be trusted the next time we have some­thing to say. Fur­ther­more, to in­fluence the world we need our com­mu­nity to have a cor­rect model of the world.

On the other hand, tra­choma is a hor­rible dis­ease. Just watch this ex­cerpt:

tl;dw: eye­lids turn in­wards and eye­lashes scrape the eye­ball, caus­ing in­tense pain on ev­ery blink. That scrap­ing even­tu­ally causes blind­ness. Peo­ple treat them­selves by pul­ling out their eye­lashes with tweez­ers. One woman said she does it ev­ery 2 weeks. Hor­rible.

If you worry about be­ing con­vinc­ing, you can talk about that and then hon­estly talk about un­cer­tainty re­gard­ing num­bers. Most peo­ple are scope in­sen­si­tive any­way. Or you can talk about cataract surgery in­stead of tra­choma be­cause dis­claimers in this page seem slightly less se­vere. Or just talk about your fa­vorite char­ity and then add “imag­ine that suffer­ing could be pre­vented so cheaply in our coun­try, ac­tion would be taken ur­gently”. But the main points of this post are

  • many of us were over­stat­ing the point that money goes fur­ther in poor countries

  • many of us don’t do enough fact check­ing, es­pe­cially be­fore mak­ing pub­lic claims

  • many of us should com­mu­ni­cate un­cer­tainty better


EDIT (2017-05-15):

Many peo­ple in the com­ments gave other rea­sons not to use the com­par­i­son but if you de­cide to use it any­way and want to quote GiveWell, you could also use figures from this Peter Singer’s com­ment. Alter­na­tively, you can use one of the other com­par­i­sons pro­posed by Ben Todd.