If you (mostly) believe in worms, what should you think about WASH?

[alex lawsen (previously alexrjl)]

Thanks go to Aaron Gertler, Louis_Dixon, and Adam Bricknell for their extremely helpful feedback on earlier drafts of this post. This question, and the thinking behind the post, started from research Sanjay and I were performing on deworming for SoGive. SoGive is an organisation which provides services to donors to help them to achieve high impact donations.

Epistemic Status: Highly uncertain, though much of the uncertainty is around deworming rather than WASH. I think the most likely way in which this post might not matter is if deworming just isn’t actually doing very much.

Below, I lay out reasons that donors and organisations who currently value deworming interventions may wish to consider funding water, sanitation, and hygiene (WASH) interventions or research. In broad brushstrokes, the reasoning is as follows:

• According to GiveWell’s research, the overwhelming majority of the value of mass deworming interventions (DW) comes from expected long-term economic effects rather than short-term effects on health. The mechanism by which these long-term effects occur is unclear, especially as the health effects are so small.

• The best WASH interventions (in particular, Dispensers for Safe Water, but possibly also Development Media International) have larger health effects than DW.

• If the long-term effects of deworming are related to the health effects of worms, it is likely that the long-term economic effects of WASH interventions are at least as good. If the effects are somehow specific to worms, given that a significant part of the benefit of WASH is in preventing parasitic worm infestation, there should still be significant long-term effects of WASH interventions.

The third bullet point is by far the most likely to be wrong, and I’ve attempted to explain which parts of it I am most uncertain about below. The post assumes some knowledge of historical debate around the history of DW in the EA movement. A good starting point for those wishing to know more would be these two blog posts by GiveWell, and this overview, which contains links to several of the original papers. A brief overview of the evidence is available here, but as it was written in 2014 it misses a couple of recent metastudies, neither of which have been very positive on the health effects of DW. I have deliberately not questioned the cost-effectiveness analyses performed by GiveWell, because I don’t think deviating from them is required in order to draw the conclusions I have from the arguments below.

GiveWell notes in their cost-effectiveness analysis for Dispensers for Safe Water that they “are not aware of any literature examining the potential of reduced childhood diarrheal illness to lead to long-term increases in income”. It’s not clear exactly how much time was spent looking for such literature, and one thing I am hoping may come out of this post is that more research (including secondary research) happens in this area. My own brief review did not turn up any long-term controlled studies, though research looking at the association between sanitation and economic outcomes does appear to exist and look positive. I have included a brief collection of links as a starting point in Appendix 1.

GiveWell’s cost-effectiveness analysis gives no weight to the immediate health impacts of mass deworming.

In GiveWell’s cost-effectiveness analysis, DW benefits are modeled as being entirely due to long-term economic effects like those seen in the follow-up studies to Miguel and Kremer’s 2004 study. Despite being heavily discounted due to concerns about replicability, in expectation these economic effects are expected to dominate the short term health effects of the interventions. The concerns about replicability are comprehensively discussed in the links above, but come in part from the lack of strong evidence for positive short term health effects of DW, and uncertainty about the mechanism by which the longer term economic effects are mediated, if the short term health effects are indeed small or nonexistent. Current hypotheses about possible mechanisms are discussed in Appendix 2.

The best wash interventions have larger health effects than mass deworming programmes.

GiveWell’s cost-effectiveness analysis for Dispensers for Safe Water estimates that the health impacts of the program are about 35% as good as AMF. This analysis only counts contributions from reducing deaths of children under five; the disease burden of non-fatal diarrhea is not considered, and nor is any potential long-term effect.

The short-term health effects of DW do not appear to be large enough to count: the evidence for any positive effect on weight gain is extremely thin, studies appear equivocal on whether there is any effect on haemoglobin levels, and in the case of other measures like height, the evidence looks even less positive.

If deworming has long term economic effects, there are good reasons to think WASH does too.

As mentioned above, I’m pretty uncertain about this. Most of this uncertainty is related to the fact that even people who believe in the long-term benefits of deworming just don’t have a very good idea of where those benefits come from. In terms of what it would mean for WASH interventions, there are two categories of potential mechanism by which DW could have a long-term effect:

• The long-term negative effects of worm infestation could be due to the health problems worms cause, either directly (for example diarrhea leading to stunting), or indirectly (via school attendance dropping as people feel unwell). In this case, given that WASH interventions seem better at preventing such health problems than mass deworming is, we should take seriously the possibility that they consequently have larger long-term effects. As discussed in Appendix 2, GiveWell appears to think that this mechanism, specifically the effect that worms have on nutrition, is most likely. If nutrition is the mechanism, WASH looks extremely promising, given its effects on diarrhea.

• The long term negative effects of worm infestation could be due to the worms themselves, or due to health effects that are unique to worms. In this case, WASH interventions would have long term effects only to the extent that they are effective in preventing parasitic worm infestations. They do appear to be effective at doing so ([1], [2]). However, I haven’t at this stage attempted to directly compare the cost-effectiveness of WASH and DW specifically on worms.

What would I like to see?

A long-term follow-up study of a WASH-focused RCT would be nice. Ideally, several long-term studies, both of WASH and DW. I think it’s somewhat noteworthy here that a WASH study finding no economic effects would make me much more pessimistic about worms. In the short term, I would be very excited to see a more comprehensive review of the literature around the economic effects of WASH interventions than I have yet been able to perform.

Appendix 1: What does the literature currently say about the economic effects of sanitation interventions?

My investigation of the literature so far does not come close to a full review, but may provide some useful starting points for someone planning to perform one. Below is a collection of links to papers I found which discussed the economic effects of sanitation improvement. I did not find any RCTs looking at closely related questions.

Bartram, J., & Cairncross, S. (2010). Hygiene, sanitation, and water: forgotten foundations of health. PLoS medicine, 7(11), e1000367. https://​​doi.org/​​10.1371/​​journal.pmed.1000367

Haller, L., Hutton, G., Bartram, J. (2007). Estimating the costs and health benefits of water and sanitation improvements at global level. J Water Health 5 (4): 467–480. https://​​doi.org/​​10.2166/​​wh.2007.008

Haller, L., Hutton, G., Bartram, J. (2007). Global cost-benefit analysis of water supply and sanitation interventions. J Water Health 5 (4): 481–502. https://​​doi.org/​​10.2166/​​wh.2007.009

Hanjra, M. A. and Gichuki, F. (2008). Investments in agricultural water management for poverty reduction in Africa: Case studies of Limpopo, Nile, and Volta river basins. Natural Resources Forum, 32: 185-202. https://​​doi.org/​​10.1111/​​j.1477-8947.2008.00191.x

Hunter, P. R., MacDonald, A. M., Carter, R. C. (2010). Water Supply and Health. PLOS Medicine, 7(11), e1000361. https://​​doi.org/​​10.1371/​​journal.pmed.1000361

Jeuland, M. A., Fuente, D. E., Ozdemir, S., Allaire, M. C., & Whittington, D. (2013). The long-term dynamics of mortality benefits from improved water and sanitation in less developed countries. PloS one, 8(10), e74804. https://​​doi.org/​​10.1371/​​journal.pone.0074804

Sanctuary, M., Tropp, H., Haller, L. (2005). Making water a part of economic development : the economic benefits of improved water management and services. Stockholm International Water Institute (SIWI) https://​​www.ircwash.org/​​resources/​​making-water-part-economic-development-economic-benefits-improved-water-management-and

Appendix 2: Mechanisms by which DW might cause long-term economic effects

Much of the uncertainty around DW comes from the very limited evidence of any short term effects. Without short term effects, it’s difficult to understand how longer-term effects could occur, so most explanations I’ve come across assume some short term effects.

David Roodman’s blog post which I linked to in the introduction seems to conclude that the most likely cause of the long-term effects is through weight gain in treated children.

In this light, the simplest theory that is compatible with the evidence arrayed so far is that deworming does improve nutrition in infected children while leaving uninfected children unaffected; and that available studies of mass deworming tend to lack the statistical power to detect the diluted benefits of mass deworming, even when combined in a meta-analysis.

Duflo & Banerjee briefly discuss deworming in Chapter 2 of Poor Economics, and imply that they believe the benefits of deworming are related to nutrition. I should note that Poor Economics is now almost a decade old.

For example, the study of the long-term effect of deworming children in Kenya, mentioned above, concluded that being dewormed for two years instead of one (and hence being better nourished for two years instead of one) would lead to a lifetime income gain of $3,269 USD PPP

This summary by J-PAL, on the other hand, seems to imply that school attendance is a key factor.

At a cost of less than US$0.60 per child per year, school-based deworming reduced serious worm infections by 61 percent and reduced school absenteeism by 25 percent… …A long-term follow-up found that deworming also improved school performance and future earnings.

[cole_haus]

A few of the summary points in Safe Drinking Water for Low-Income Regions are interesting and may provide reason for a bit of pessimism:

1. Safe drinking water from “source to sip” consists of a series of interactions between technologies, their delivery models, their scales and costs of production, and consumer uptake and consistent use. Safe drinking water is a system, not a product or an intervention.

1. It seems unlikely that household treatment and safe storage systems—with the possible exception of boiling—can be transformative at scale under current prices, delivery models, and preferences, but they are effective and protective in specific contexts.

1. Cost analyses for “low-cost” systems are usually reported on a partial basis, with installation costs and some operational costs included. The enabling costs of social marketing, mobilization, education, reminders, and community- or household-based unpaid labor are mentioned but not explicitly accounted for.

1. Delivery models and business models significantly affect costs and uptake, at all three scales of service. Yet they are rarely made explicit.

1. Safe water systems can be highly effective, but consumers undervalue drinking water quality and have low willingness and/​or ability to pay for safety. This is a particular challenge for arsenic mitigation or avoidance, as arsenicosis is only evident after several years of exposure.

Without an in-depth of understanding of either deworming or WASH, it seems like the implementation issues for WASH may be more severe?

[cole_haus]

This doesn’t directly engage with the point of this post, but Safe Drinking Water for Low-Income Regions is a pretty good introduction to WASH IMO:

Well into the 21st century, safe and affordable drinking water remains an unmet human need. At least 1.8 billion people are potentially exposed to microbial contamination, and close to 140 million people are potentially exposed to unsafe levels of arsenic. Many new technologies, water quality assessments, health impact assessments, cost studies, and user preference studies have emerged in the past 20 years to further the laudable goal of safe drinking water for all. This article reviews (a) the current literature on safe water approaches with respect to their effectiveness in improving water quality and protectiveness in improving human health, (b) new work on the uptake and use of safe water systems among low-income consumers, (c) new research on the cash and labor costs of safe water systems, and (d) research on user preferences and valuations for safe water. Our main recommendation is that safe water from “source to sip” should be seen as a system; this entire system, rather than a discrete intervention, should be the object of analysis for technical, economic, and health assessments.

(Providing Safe Water: Evidence from Randomized Evaluations too.)

[alex lawsen (previously alexrjl)]

I wrote most of the post before anyone was talking about coronavirus, and therefore used Dispensers for Safe Water as the example comparison, which doesn’t at first glance look like it would help whatsoever with Covid-19.

DMI were mentioned in the post however, and their mass media communications designed to effect behaviour changes around, among other things, hand-washing, look even better in the light of the first case of Covid-19 being recorded in sub-saharan Africa. If there’s interest, I’ll try to write up a more detailed look at DMI soon.