Finally, about the other points you raised, we recommend you (and everyone interested in the theme) check the new version of the report published by Ambitious Impact/Charity Entrepreneurship when recommending this intervention for CE’s next incubation round [1].
AIM/CE researchers dedicated a few hours to polishing the remaining loose ends in our previous version. It includes a model for calculating the mortality reductions based on diarrhea case-fatality rates. This is a slightly enhanced version of our model and represents an interesting proposition on how to extrapolate the effect of the intervention from cases to mortality—given more time, it could even be expanded to consider pathogen-specific case-fatality rates, which should be even better.
- ^
We, the authors of the originial report, had only very limited participation in the changes made between the two versions.
Hi again, Nick! Sorry it took so long to answer your remaining points!
So, first about this one:
Taking a more careful look into the footnotes and supplemental materials for GiveWell’s discussion of the Mills-Reincke effect, it appears that they have gathered mixed evidence on its existence for the over-5 population:
This supplementary spreadsheet lists six econometric studies including overall all-cause mortality effects from water quality improvements. These are mostly natural experiments regarding the implementation of municipal water quality improvements in now-developed countries (Germany, USA, Sweden, Japan), during the 20th and early 21st centuries (a summary of all papers is available here; we have only briefly reviewed the relevant ones). Both the mean and the median effect sizes on the population-wide all-cause mortality were −19%, ranging from +0.9 to −58%. Surprisingly, this is an even larger effect size than the one observed when considering under-5 mortality alone (median: −11%; mean: −10 to −13%). This can be taken at least as some rough evidence that the MR phenomenon can be widespread across age groups. [1]
On the other hand, a long note to the “Adjustment for smaller Mills-Reincke effect in over-5s” cell of their CEA, authors of the GW report on water quality interventions mention one study (Newman et al. 2020) that looked into one of the plausible mechanisms for explaining the MR phenomenon and found no evidence of such effects on adults. However, the report’s authors argue that it may still be true for older children and adolescents:
In any case, we added a sensitivity analysis to our Effectiveness Supplement where we consider no Mills-Reincke effect for age groups over 5 y.o. And it does make a great difference. On average, it reduces the estimated (cost-)effectiveness of filtration interventions by 33%, and by 21% for chlorination interventions across all countries (simple average). For our top 5 prioritized countries, the difference is even bigger: a 43% reduction for filtration, and 36% for chlorination [2]. For Nigeria, for instance, the effectiveness lowers from ~24k DALYs/100,000 people served, to ~13k DALYs/100,000 people served.
Given the magnitude of the difference, we suggest further research would greatly benefit from trying to better understand the evidence base for or against the existence of the Mills-Reincke effect among older children, adolescents, and adults.
Note that GiveWell itself mentioned reasons to take these estimates with a grain of salt. Quoting them (end of the “Studies of historical water quality improvements” section of their report):
Differences across countries and intervention types are due to the different YLL and YLD proportions coming from each pathogen and each age group in each country (according to data from GBD 2019).