Hi, Joel,
This is a big question, and one we’d very much like to have a better explanation for! It’s important to note that our estimates are uncertain, so the gap between the mortality benefit of water quality interventions and mortality attributed to diarrhea is not necessarily as large as it appears. But we do find it plausible that a larger-than-expected mortality reduction would exist, for a few reasons:
Global Burden of Disease (GBD) estimates of diarrhea deaths in children under five assume that each death has just one cause. However, deaths are often the result of several contributing causes, and in some situations removing any single one of those causes would mean averting a death. So we think it’s likely that just extrapolating diarrhea mortality from diarrhea morbidity underestimates the total mortality benefit of avoiding diarrhea.
On a related note, there is observational evidence that repeated bouts of diarrhea may be a significant cause of malnutrition, and evidence that malnutrition increases risk of dying from infectious diseases, such as malaria and measles. Frequent diarrhea generally seems to lead to undernourishment and worse health, leaving people, especially children, vulnerable to other illnesses such as respiratory infections.
There is evidence from studies of historical water quality improvement projects that improving a population’s drinking water leads to mortality benefits beyond what would be expected from a reduction in waterborne diseases alone, such as reductions in death from respiratory infections; this is known as the Mills-Reincke phenomenon. The effect on child mortality observed in these studies is consistent with the estimate we arrived at in our own meta-analysis. More recent observational evidence points to a link between diarrheal infection and increased risk of respiratory illness.
For more details and citations for the above, see this section of our water quality intervention report (particularly beginning with “We are uncertain about our central estimate of the mortality impact of chlorination…”).
In the course of our investigation, we also spoke to several researchers who found it plausible that water quality improvements would have a larger-than-expected mortality benefit. These researchers offered their own insight into why water treatment might also reduce non–diarrheal illness, or why the reduction in diarrhea mortality might be larger than what one would calculate indirectly from diarrhea morbidity. See our notes from these conversations here, here, and here.
In sum, we still have uncertainty about what explains the gap between mortality reduction estimates, but the impact of averting waterborne diseases on death from non-waterborne diseases probably plays a role. We think that further empirical evidence of the size of the mortality effect, or a better understanding of the mechanisms that link water quality to mortality, could help to substantially reduce our uncertainty.
Thanks for the flag, Joel.
Brian, our team is working on our own reports on how we view interpersonal group therapy interventions and subjective well-being measures more generally. We expect to publish our reports within the next 3-6 months.
We have spoken to HLI about their work, and HLI has given us feedback on our reports. It’s been really helpful to discuss this topic with Michael, Joel, and the team at HLI. Their work has provided some updates to how we view this topic, even if we do not ultimately end up reaching the same conclusions.
We’re still looking into this area and some of the important questions HLI has raised. While we plan to provide a more detailed view once our reports are published, a few areas where we differ from HLI are below:
The most concrete methodological difference between our approach and HLI’s is that we have a stronger view that there are larger intra-household spillovers for cash transfers than for therapy. HLI assumes any spillover effects to other household members are proportional across interventions—i.e., if a cash transfer benefits other household members’ subjective well-being x% as much as it benefits the recipient, the same is true for therapy. We have a strong prior that, since income is shared within households, other household members would benefit much more from a cash transfer than from therapy delivered to one of its members. If there are 4 to 5 members per household (roughly what we estimate for participants in GiveDirectly’s program) and there were no household multipliers from therapy, psychotherapy would be 2x-3x as cost-effective as cash transfers, taking HLI’s other assumptions as given. HLI has flagged this as an uncertainty and something they may look further into, and we would be interested to see what they find. It seems possible there are significant spillovers from therapy, but our current best guess is these would be much smaller than for cash.
Another concrete difference is that we’re not sure comparing standard deviation effects across interventions is appropriate. HLI compares the effect on well-being in standard deviations (SDs) across cash transfer and therapy interventions. Our impression is that therapy interventions like StrongMinds target individuals with depression, who might be concentrated at the lower end of subjective well-being scales, to a greater extent than cash transfer interventions do. If this is the case, then the measure of well-being per SD may be lower for therapy than for cash transfer interventions. For example, a SD for psychotherapy may be a 0.5 on a 10-point scale of well-being, while the SD for cash transfers may be 1. HLI notes this as an uncertainty, and we’d be interested in more evidence on this, too.
We put less weight on subjective well-being as a measure to compare different interventions. As HLI notes, subjective well-being measures could provide a common currency for comparing interventions like cash and therapy. We agree this is a useful perspective. However, we think there are some limitations to these measures, and give weight to other factors. I expect that GiveWell would find interventions that have benefits beyond subjective well-being more cost-effective than HLI would. As a result, even if a subjective well-being approach showed a program was more cost-effective than programs we’d expect to recommend marginal funding to, we wouldn’t necessarily make a recommendation based on that alone (though we would give it some weight).
When we’ve looked at group therapy from perspectives other than comparing different interventions’ effects on subjective well-being measures, we find much lower cost-effectiveness than HLI is finding. We’ve considered a few different angles. First, we’ve looked at the effect of therapy under our current moral weights, which we use to trade off outcomes like increasing consumption, averting deaths, and averting morbidity. Under this approach and using the effect of depression on DALYs, we find similar cost-effectiveness between therapy and cash transfers. Second, we’ve taken a very shallow look at a recent trial that compares cash transfers with therapy head-to-head. This trial finds a much smaller effect of therapy vs. cash. Third, it seems intuitively surprising that a $1,000 cash transfer for a household in a low-income country would have a substantially smaller effect than 3-4 months of therapy (0.5 SD for cash transfers vs. 1.6 SD for therapy in HLI’s study). If the average GiveDirectly household has consumption of $2,000-$3,000, a $1,000 transfer amounts to a 30%-50% increase in baseline consumption for an entire household. We would guess that if we, for example, gave households the choice between 3-4 months of therapy and a $1,000 transfer, many more would choose the transfer. While the above approaches definitely have drawbacks and we don’t think we should put 100% weight on them either, the implications seem substantially different enough from HLI’s estimates that they give us some additional skepticism.
We still have a lot of uncertainty about how to compare different interventions like cash transfers and therapy, and making these comparisons is crucial to our decisions on what funding opportunities to recommend to our donors. As a result, we hope to continue to discuss this topic with individuals who have a differing view than us on our moral weights so that we can continue to refine our approach.
We look forward to engaging once we publish a fully vettable report. Until then, I hope this answers the immediate questions you have about where the views of GiveWell and HLI differ.