Thanks for this report, it makes a lot of sense and reads well! I agree that ceramic water filters can be a great solution. We use ceramic candles at home to filter the rainwater we drink, that we catch off our outdoor toilet roof. This actually reminds me to replace our candles, we haven’t in far too long...
My intuition likes this intervention especially in areas which have...
Very high disease burden plus one of the 2 issues below
Sparse population, where other cleaning methods might be even more expensive
Conflict, as one off deliveries could reach households in areas where ongoing chlorine projects might be hard to maintain
My first thought is have you reached out directly to Evidence Action and Givewell to ask about why they went so hard on chlorination and not ceramic filters? Given that in many scenarios (especially in remote hard to reach areas) ceramic filters seem to clean the water more cheaply than chlorine (either with direct or inline) I would imagine they have good reasons why they preferred chlorination to filters, given that they have gone all in on them spending tens of millions of dollars as their preferred cleaning method.
I’m most interested in your decisions around your primary outcome measure (I think the crux of your DALY burden reduction), which was summarised in your full article like this
”Impact through mortality and morbidity was obtained by multiplying the total DALYs lost per year due to those diarrheal diseases by 1 minus the (adjusted—see next section) risk ratios for ceramic filters (without education) found in our subgroup. Each GiveWell moral weight unit equals the doubling in consumption for one person during a year. A disability-adjusted life year averted equals 2.3 moral units. This number was then multiplied by GiveWell’s estimates of the Mills-Reincke effect (3.7x for under 5s and 2.35x for over 5s) to get the effects on all-cause mortality and morbidity.”
The 2 assumptions here that I think might be shaky here are...
Your assumption here that there is a direct connection between cases of diarrhoea and mortality. We have very little clue why clean water leads to a mortality reduction, so I’m not sure its reasonable to make the leap, first that there is a causal relationship at all between diarrhoea reduction and the huge mortality reduction, and second that that connection moves linearly like is assumed here (For every % of diarrhoea reduction, mills-reincke effect moves the same).
I can’t immediately think of a great solution here, but the current method doesn’t seem right.
(Potentially shaky assumption shared by GiveWell). The 2.35 Mills Reincke multiplier for over 5s. I could be wrong but I’m not sure there’s any good data at all on mortality reductions for clean water in over 5s. The Kremer study Is the only one I know which looks directly at mortality at all, and then only in under 5s. Where does this number come from? My intuition would that the multiplier should be far smaller than this (very very uncertain).
A few other more minor points
I think 2 years is ambitious number for lifespan. Between families neglecting to use it (perhaps most important) kids breaking, families selling it (not the end of the world), dirty water clogging it etc. I would have perhaps gone for 1 or 1.5 years use as realistic, but this isn’t so important. I struggled in a 5 minute search to find decent research on ceramic filter lifespan in the real world, do you have one?”
From my experience it is borderline implausible that 5.8% of diarrhoeal episodes lead to hospitalisation. I’m not sure if you referenced where this number came from? I’ve worked in Northern Uganda for 10 years and diarrhoea is a far less common than you would expect reason for presentation to primary care (Under 3% of our OneDay Health Patients present with diarrhoea) and at best an uncommon reason for hospitalisation. I even removed treating diarrhoea from our OneDay Health cost-effectiveness analysis because it appeared insignificant (both because the effect of treating one patient is so small, and because we treat so few patients for it)
Taking a less anecdata approach, doing a ’sanity check” (Thanks @NunoSempere) We know fairly well that sub-saharan African children have something like 2-5 episodes of diarrhoea a year (data varies) If 6% of these episodes led to hospitalisation, then that would mean every 5 year old would have on average about one hospital admission for diarrhoea before the age of 5 (6% of 15 episodes), which is an absurdly high number of admissions
Kenya for example has about 7 million kids under 5, which would mean about 1.3 million hospital admissions for diarrhoea yearly which can’t eb right. Assuming each kid was admitted for 2 nights, this would mean 7100 hospital beds permanently filled with kids with diarrhoea in Kenya—most certainly not the case.
(Note this calculation was done quickly so could be plain wrong)
Also quick correction I think you mean the $8 country is Nigeria here.
“We found the cost-effectiveness of our proposed ceramic filter intervention to be $67 dollars per DALY in Kenya, $54 dollars per DALY in Uganda, and $8 dollars per DALY in Kenya.”
Anyway great job and an important contribution to the clean water discussion!
If you could be bothered as well, you could definitely get a shorter version of this published in a health economics journal—if you are interested in this maybe message me I might be able to help
Hi Nick! Thanks so much for your thoughtful response. Here are some of our thoughts on your questions:
**Have you reached out directly to Evidence Action and GiveWell to ask about why they went so hard on chlorination and not ceramic filters? **
Since this comment, I have reached out to Givewell and Evidence Action! Thanks for the suggestion—we had not previously reached out due to time constraints.
Your assumption here that there is a direct connection between cases of diarrhoea and mortality. We have very little clue why clean water leads to a mortality reduction, so I’m not sure its reasonable to make the leap, first that there is a causal relationship at all between diarrhoea reduction and the huge mortality reduction, and second that that connection moves linearly like is assumed here (For every % of diarrhoea reduction, mills-reincke effect moves the same).
You’re right, the leap in assuming a causal relationship between diarrhea reduction and mortality reduction is not ideal and we aren’t assuming this is exactly correct. However, using diarrhea cases as a proxy for morbidity and mortality was the best we could come up with. We welcome any additional suggestions.
In addition, from what we saw from the individual studies considered in the systematic reviews we took into account, none seem to be powered to detect effects on mortality, and most do not report it as an outcome. To make sure reductions in diarrhea cases translate into mortality reductions, one would probably need to do what Kremer and his colleagues did for their 2015 systematic review—individually reach the authors of each study and ask whether they collected any mortality data that went undisclosed due to low statistical power. This could be further work for researchers to perform in the future.
**I could be wrong but I’m not sure there’s any good data at all on mortality reductions for clean water in over 5s. **The Kremer studyIs the only one I know which looks directly at mortality at all, and then only in under 5s. Where does this number come from? My intuition would that the multiplier should be far smaller than this (very very uncertain).
This is a really good point. When applying the 2.35 Mills Reincke multiplier in our cost-effectiveness analysis, we were more concerned with comparing different water quality interventions than getting the exact number for a CEA (dollars per DALY). To compare ceramic filters to chlorination, we applied the same methodology as Givewell to create as close of a comparison between the two.
However, we can perform an analysis removing the Mills-Reinke effect for over 5s. This would affect our estimates both for chlorination and ceramic filters in similar ways but would lower the overall cost-effectiveness. We can likely do this next week!
I think 2 years is ambitious number for lifespan. Between families neglecting to use it (perhaps most important) kids breaking, families selling it (not the end of the world), dirty water clogging it etc. I would have perhaps gone for 1 or 1.5 years use as realistic, but this isn’t so important. I struggled in a 5 minute search to find decent research on ceramic filter lifespan in the real world, do you have one?”
You hit on an important topic, as the lifespan of the ceramic filter actually can significantly impact the overall cost-effectiveness analysis.
Our assumption of a two-year lifespan for ceramic filters in our analysis is based on a Unicef report (Unicef, 2020), which suggests a minimum two-year effectiveness. However, field insights from organizations like “Potters Without Borders” and experts at Pure Home Water indicate that, with proper care, these filters can endure beyond this conservative timeframe.
Michael Anyekase, with over 10 years of experience working with manufacturing and distributing ceramic filters in Uganda at Purehome Water (similar to our theory of change), proposes a potential lifespan of 5 to 6 years under clear water conditions.
Yvette Neh suggests an upper limit of 4 years.. Yvette Neh (who also works at purehome water), proposes an upper limit of 4 years for somewhat good water conditions. In addition, none of the experts we spoke with were overly concerned with filter breakage, and agree that this 2 year lifespan is a conservative timeline?
**We ultimately chose a 2-year filter lifespan as we believe this might be a conservative estimate, and that, in practice, these filters often last much longer. **
From my experience it is borderline implausible that 5.8% of diarrhoeal episodes lead to hospitalisation
This intuitively makes a lot of sense, and we will look into this more deeply in the upcoming days and weeks.
Also quick correction I think you mean the $8 country is Nigeria here.
Thank you! We updated this in the forum and the report!
Again, thank you so much for your thoughtful response.
Hi again, Nick! Sorry it took so long to answer your remaining points!
So, first about this one:
(Potentially shaky assumption shared by GiveWell). The 2.35 Mills Reincke multiplier for over 5s. I could be wrong but I’m not sure there’s any good data at all on mortality reductions for clean water in over 5s. The Kremer study Is the only one I know which looks directly at mortality at all, and then only in under 5s. Where does this number come from? My intuition would that the multiplier should be far smaller than this (very very uncertain).
Taking a more careful look into the footnotes and supplemental materials for GiveWell’s discussion of the Mills-Reincke effect, it appears that they have gathered mixed evidence on its existence for the over-5 population:
This supplementary spreadsheet lists six econometric studies including overall all-cause mortality effects from water quality improvements. These are mostly natural experiments regarding the implementation of municipal water quality improvements in now-developed countries (Germany, USA, Sweden, Japan), during the 20th and early 21st centuries (a summary of all papers is available here; we have only briefly reviewed the relevant ones). Both the mean and the median effect sizes on the population-wide all-cause mortality were −19%, ranging from +0.9 to −58%. Surprisingly, this is an even larger effect size than the one observed when considering under-5 mortality alone (median: −11%; mean: −10 to −13%). This can be taken at least as some rough evidence that the MR phenomenon can be widespread across age groups. [1]
On the other hand, a long note to the “Adjustment for smaller Mills-Reincke effect in over-5s” cell of their CEA, authors of the GW report on water quality interventions mention one study (Newman et al. 2020) that looked into one of the plausible mechanisms for explaining the MR phenomenon and found no evidence of such effects on adults. However, the report’s authors argue that it may still be true for older children and adolescents:
[...] However, we do not have much evidence on whether it operates in people over 5. Based on conversations with experts[...], we believe a likely mechanism by which the Mills-Reincke phenomenon operates is that enteric diseases impair nutritional status and body energy reserves, increasing risk from subsequent infectious diseases. This mechanism is expected to affect young children the most since they have smaller body reserves of energy, protein, and other nutrients. We identified one study that addresses this hypothesis empirically in under-5s and over-5s side-by-side. Newman et al. 2020 measured the risk of respiratory infection following a bout of diarrhea in infants and mothers. Infants had a substantially elevated risk of respiratory infection following diarrhea, while mothers did not. This provides some evidence that the Mills-Reincke phenomenon may not operate in adults. It is worth noting that the “over 5” category we use in this CEA is not composed solely of adults, and children over 5 may still be susceptible to the Mills-Reincke phenomenon. To derive this adjustment, we assume that Mills-Reincke has an impact half as large in over-5s as in under-5s. This is a very uncertain guess. We also assume that water quality interventions avert 2.7 non-enteric deaths for each enteric death averted (see water quality report for calculation, link below), and that the 50% downward adjustment only applies to non-enteric deaths.
In any case, we added a sensitivity analysis to our Effectiveness Supplement where we consider no Mills-Reincke effect for age groups over 5 y.o. And it does make a great difference. On average, it reduces the estimated (cost-)effectiveness of filtration interventions by 33%, and by 21% for chlorination interventions across all countries(simple average). For our top 5 prioritized countries, the difference is even bigger: a 43% reduction for filtration, and 36% for chlorination [2]. For Nigeria, for instance, the effectiveness lowers from ~24k DALYs/100,000 people served, to ~13k DALYs/100,000 people served.
Given the magnitude of the difference, we suggest further research would greatly benefit from trying to better understand the evidence base for or against the existence of the Mills-Reincke effect among older children, adolescents, and adults.
Note that GiveWell itself mentioned reasons to take these estimates with a grain of salt. Quoting them (end of the “Studies of historical water quality improvements” section of their report):
Several factors lead us to be uncertain about the relevance of these studies to modern interventions that target waterborne disease in low-income settings:
- Most were conducted in contexts that differ substantially from the contexts in which water quality interventions would be applied today. This includes differences in location, infectious disease profile, baseline mortality rates, and other factors.
- The water quality interventions represented in these studies vary, and none is identical to the interventions to which we might direct funding.
- These studies are observational and use different strategies to attempt to isolate the causal impact of water quality on mortality. We believe some of these strategies are more convincing than others, but all of them may be susceptible to some degree of confounding from other variables that changed alongside water quality.
- We do not know whether unsupportive studies went unpublished, potentially creating publication bias. This would make the overall literature appear more supportive than it would otherwise be.
- None of these studies were preregistered, increasing the risk of bias.
For these reasons, we view these studies as providing rough triangulation for our mortality reduction estimate and a plausible mechanism to explain its unexpectedly large effect size. We do not see them as providing precise effect size estimates for the impact of water quality interventions on mortality.
Differences across countries and intervention types are due to the different YLL and YLD proportions coming from each pathogen and each age group in each country (according to data from GBD 2019).
From my experience it is borderline implausible that 5.8% of diarrhoeal episodes lead to hospitalisation. I’m not sure if you referenced where this number came from? I’ve worked in Northern Uganda for 10 years and diarrhoea is a far less common than you would expect reason for presentation to primary care (Under 3% of our OneDay Health Patients present with diarrhoea) and at best an uncommon reason for hospitalisation. I even removed treating diarrhoea from our OneDay Health cost-effectiveness analysis because it appeared insignificant (both because the effect of treating one patient is so small, and because we treat so few patients for it)
It took a while, but we finally found some good empirical data regarding hospitalization rates due to diarrhea in LMICs, beyond the four smaller studies GiveWell references for justifying their 5.8% estimate [1].
So, the Global Enteric Multicenter Study of Diarrheal Disease in Infants and Young Children in Developing Countries (GEMS) was a 3-year, multi-site case-control study focusing on diarrhea in children under 5 years old living in seven low-income countries in sub-Saharan Africa and South Asia (source).
Even though the main GEMS study had a focus on moderate and severe cases, it was preceded by a preparatory survey called “Health Care Utilization and Attitudes Survey (HUAS)”. According to their “Establishing a Sampling Frame for the Case/Control Study and Selecting Health Centers for Case Recruitment” section:
In preparation for the case/control study, we performed a Health Care Utilization and Attitudes Survey (HUAS). An age-stratified sample of approximately 1000 children aged 0–59 months per site randomly selected from each updated DSS [demographic surveillance system] dataset was visited at home, and parents/primary caretakers were asked whether their child had experienced diarrhea during the previous 14 days. If so, the presence of findings suggestive of MSD [moderate-to-severe diarrhea] was solicited (sunken eyes, wrinkled skin, hospitalization, receipt of intravenous hydration, or dysentery), and source(s) of healthcare were recorded. These data were used to adjust the size of the DSS population at each site as necessary to contribute the requisite number of cases of MSD to each age stratum, and to select 1 or more “sentinel” health centers (SHCs) serving the DSS population at each site (Table 1) as venues for the case/control study based on their potential to capture MSD cases from the DSS.
It turns out that the resulting data from the HUAS study is easily accessible for anyone to consult. And it recorded the number of children under 5 who reportedly had diarrhea during the 14 days previous to the interview, and whether they were admitted to a hospital.
According to this data, out of 5,171 who reportedly suffered from diarrhea in the previous 14 days, and for whom data for this variable was available, 270 children were admitted to hospitals. This gives a 5.22% hospitalization rate (95% CI: 4.65-5.86%), which is roughly what GiveWell estimated from a set of smaller studies.
Therefore, we think this part of the CEA is also roughly aligned with evidence.
Finally, about the other points you raised, we recommend you (and everyone interested in the theme) check the new version of the report published by Ambitious Impact/Charity Entrepreneurship when recommending this intervention for CE’s next incubation round[1].
AIM/CE researchers dedicated a few hours to polishing the remaining loose ends in our previous version. It includes a model for calculating the mortality reductions based on diarrhea case-fatality rates. This is a slightly enhanced version of our model and represents an interesting proposition on how to extrapolate the effect of the intervention from cases to mortality—given more time, it could even be expanded to consider pathogen-specific case-fatality rates, which should be even better.
Thanks for this report, it makes a lot of sense and reads well! I agree that ceramic water filters can be a great solution. We use ceramic candles at home to filter the rainwater we drink, that we catch off our outdoor toilet roof. This actually reminds me to replace our candles, we haven’t in far too long...
My intuition likes this intervention especially in areas which have...
Very high disease burden plus one of the 2 issues below
Sparse population, where other cleaning methods might be even more expensive
Conflict, as one off deliveries could reach households in areas where ongoing chlorine projects might be hard to maintain
My first thought is have you reached out directly to Evidence Action and Givewell to ask about why they went so hard on chlorination and not ceramic filters? Given that in many scenarios (especially in remote hard to reach areas) ceramic filters seem to clean the water more cheaply than chlorine (either with direct or inline) I would imagine they have good reasons why they preferred chlorination to filters, given that they have gone all in on them spending tens of millions of dollars as their preferred cleaning method.
I’m most interested in your decisions around your primary outcome measure (I think the crux of your DALY burden reduction), which was summarised in your full article like this
”Impact through mortality and morbidity was obtained by multiplying the total DALYs
lost per year due to those diarrheal diseases by 1 minus the (adjusted—see next
section) risk ratios for ceramic filters (without education) found in our subgroup. Each GiveWell moral weight unit equals the doubling in consumption for one person during a
year. A disability-adjusted life year averted equals 2.3 moral units. This number was then multiplied by GiveWell’s estimates of the Mills-Reincke effect (3.7x for under 5s and 2.35x for over 5s) to get
the effects on all-cause mortality and morbidity.”
The 2 assumptions here that I think might be shaky here are...
Your assumption here that there is a direct connection between cases of diarrhoea and mortality. We have very little clue why clean water leads to a mortality reduction, so I’m not sure its reasonable to make the leap, first that there is a causal relationship at all between diarrhoea reduction and the huge mortality reduction, and second that that connection moves linearly like is assumed here (For every % of diarrhoea reduction, mills-reincke effect moves the same).
I can’t immediately think of a great solution here, but the current method doesn’t seem right.
(Potentially shaky assumption shared by GiveWell). The 2.35 Mills Reincke multiplier for over 5s. I could be wrong but I’m not sure there’s any good data at all on mortality reductions for clean water in over 5s. The Kremer study Is the only one I know which looks directly at mortality at all, and then only in under 5s. Where does this number come from? My intuition would that the multiplier should be far smaller than this (very very uncertain).
A few other more minor points
I think 2 years is ambitious number for lifespan. Between families neglecting to use it (perhaps most important) kids breaking, families selling it (not the end of the world), dirty water clogging it etc. I would have perhaps gone for 1 or 1.5 years use as realistic, but this isn’t so important. I struggled in a 5 minute search to find decent research on ceramic filter lifespan in the real world, do you have one?”
From my experience it is borderline implausible that 5.8% of diarrhoeal episodes lead to hospitalisation. I’m not sure if you referenced where this number came from? I’ve worked in Northern Uganda for 10 years and diarrhoea is a far less common than you would expect reason for presentation to primary care (Under 3% of our OneDay Health Patients present with diarrhoea) and at best an uncommon reason for hospitalisation. I even removed treating diarrhoea from our OneDay Health cost-effectiveness analysis because it appeared insignificant (both because the effect of treating one patient is so small, and because we treat so few patients for it)
Taking a less anecdata approach, doing a ’sanity check” (Thanks @NunoSempere) We know fairly well that sub-saharan African children have something like 2-5 episodes of diarrhoea a year (data varies) If 6% of these episodes led to hospitalisation, then that would mean every 5 year old would have on average about one hospital admission for diarrhoea before the age of 5 (6% of 15 episodes), which is an absurdly high number of admissions
Kenya for example has about 7 million kids under 5, which would mean about 1.3 million hospital admissions for diarrhoea yearly which can’t eb right. Assuming each kid was admitted for 2 nights, this would mean 7100 hospital beds permanently filled with kids with diarrhoea in Kenya—most certainly not the case.
(Note this calculation was done quickly so could be plain wrong)
Also quick correction I think you mean the $8 country is Nigeria here.
“We found the cost-effectiveness of our proposed ceramic filter intervention to be $67 dollars per DALY in Kenya, $54 dollars per DALY in Uganda, and $8 dollars per DALY in Kenya.”
Anyway great job and an important contribution to the clean water discussion!
If you could be bothered as well, you could definitely get a shorter version of this published in a health economics journal—if you are interested in this maybe message me I might be able to help
Nice one!
Hi Nick! Thanks so much for your thoughtful response. Here are some of our thoughts on your questions:
Since this comment, I have reached out to Givewell and Evidence Action! Thanks for the suggestion—we had not previously reached out due to time constraints.
You’re right, the leap in assuming a causal relationship between diarrhea reduction and mortality reduction is not ideal and we aren’t assuming this is exactly correct. However, using diarrhea cases as a proxy for morbidity and mortality was the best we could come up with. We welcome any additional suggestions.
In addition, from what we saw from the individual studies considered in the systematic reviews we took into account, none seem to be powered to detect effects on mortality, and most do not report it as an outcome. To make sure reductions in diarrhea cases translate into mortality reductions, one would probably need to do what Kremer and his colleagues did for their 2015 systematic review—individually reach the authors of each study and ask whether they collected any mortality data that went undisclosed due to low statistical power. This could be further work for researchers to perform in the future.
This is a really good point. When applying the 2.35 Mills Reincke multiplier in our cost-effectiveness analysis, we were more concerned with comparing different water quality interventions than getting the exact number for a CEA (dollars per DALY). To compare ceramic filters to chlorination, we applied the same methodology as Givewell to create as close of a comparison between the two.
However, we can perform an analysis removing the Mills-Reinke effect for over 5s. This would affect our estimates both for chlorination and ceramic filters in similar ways but would lower the overall cost-effectiveness. We can likely do this next week!
You hit on an important topic, as the lifespan of the ceramic filter actually can significantly impact the overall cost-effectiveness analysis.
Our assumption of a two-year lifespan for ceramic filters in our analysis is based on a Unicef report (Unicef, 2020), which suggests a minimum two-year effectiveness. However, field insights from organizations like “Potters Without Borders” and experts at Pure Home Water indicate that, with proper care, these filters can endure beyond this conservative timeframe.
Michael Anyekase, with over 10 years of experience working with manufacturing and distributing ceramic filters in Uganda at Purehome Water (similar to our theory of change), proposes a potential lifespan of 5 to 6 years under clear water conditions.
Yvette Neh suggests an upper limit of 4 years.. Yvette Neh (who also works at purehome water), proposes an upper limit of 4 years for somewhat good water conditions. In addition, none of the experts we spoke with were overly concerned with filter breakage, and agree that this 2 year lifespan is a conservative timeline?
**We ultimately chose a 2-year filter lifespan as we believe this might be a conservative estimate, and that, in practice, these filters often last much longer. **
This intuitively makes a lot of sense, and we will look into this more deeply in the upcoming days and weeks.
Thank you! We updated this in the forum and the report!
Again, thank you so much for your thoughtful response.
Hi again, Nick! Sorry it took so long to answer your remaining points!
So, first about this one:
Taking a more careful look into the footnotes and supplemental materials for GiveWell’s discussion of the Mills-Reincke effect, it appears that they have gathered mixed evidence on its existence for the over-5 population:
This supplementary spreadsheet lists six econometric studies including overall all-cause mortality effects from water quality improvements. These are mostly natural experiments regarding the implementation of municipal water quality improvements in now-developed countries (Germany, USA, Sweden, Japan), during the 20th and early 21st centuries (a summary of all papers is available here; we have only briefly reviewed the relevant ones). Both the mean and the median effect sizes on the population-wide all-cause mortality were −19%, ranging from +0.9 to −58%. Surprisingly, this is an even larger effect size than the one observed when considering under-5 mortality alone (median: −11%; mean: −10 to −13%). This can be taken at least as some rough evidence that the MR phenomenon can be widespread across age groups. [1]
On the other hand, a long note to the “Adjustment for smaller Mills-Reincke effect in over-5s” cell of their CEA, authors of the GW report on water quality interventions mention one study (Newman et al. 2020) that looked into one of the plausible mechanisms for explaining the MR phenomenon and found no evidence of such effects on adults. However, the report’s authors argue that it may still be true for older children and adolescents:
In any case, we added a sensitivity analysis to our Effectiveness Supplement where we consider no Mills-Reincke effect for age groups over 5 y.o. And it does make a great difference. On average, it reduces the estimated (cost-)effectiveness of filtration interventions by 33%, and by 21% for chlorination interventions across all countries (simple average). For our top 5 prioritized countries, the difference is even bigger: a 43% reduction for filtration, and 36% for chlorination [2]. For Nigeria, for instance, the effectiveness lowers from ~24k DALYs/100,000 people served, to ~13k DALYs/100,000 people served.
Given the magnitude of the difference, we suggest further research would greatly benefit from trying to better understand the evidence base for or against the existence of the Mills-Reincke effect among older children, adolescents, and adults.
Note that GiveWell itself mentioned reasons to take these estimates with a grain of salt. Quoting them (end of the “Studies of historical water quality improvements” section of their report):
Differences across countries and intervention types are due to the different YLL and YLD proportions coming from each pathogen and each age group in each country (according to data from GBD 2019).
About this other point:
It took a while, but we finally found some good empirical data regarding hospitalization rates due to diarrhea in LMICs, beyond the four smaller studies GiveWell references for justifying their 5.8% estimate [1].
So, the Global Enteric Multicenter Study of Diarrheal Disease in Infants and Young Children in Developing Countries (GEMS) was a 3-year, multi-site case-control study focusing on diarrhea in children under 5 years old living in seven low-income countries in sub-Saharan Africa and South Asia (source).
Even though the main GEMS study had a focus on moderate and severe cases, it was preceded by a preparatory survey called “Health Care Utilization and Attitudes Survey (HUAS)”. According to their “Establishing a Sampling Frame for the Case/Control Study and Selecting Health Centers for Case Recruitment” section:
It turns out that the resulting data from the HUAS study is easily accessible for anyone to consult. And it recorded the number of children under 5 who reportedly had diarrhea during the 14 days previous to the interview, and whether they were admitted to a hospital.
According to this data, out of 5,171 who reportedly suffered from diarrhea in the previous 14 days, and for whom data for this variable was available, 270 children were admitted to hospitals. This gives a 5.22% hospitalization rate (95% CI: 4.65-5.86%), which is roughly what GiveWell estimated from a set of smaller studies.
Therefore, we think this part of the CEA is also roughly aligned with evidence.
The four studies are Burton et al. 2011, Page et al. 2011, Breiman et al. 2011, and Omore et al. 2013. See footnote #92 of GW’s report.
Finally, about the other points you raised, we recommend you (and everyone interested in the theme) check the new version of the report published by Ambitious Impact/Charity Entrepreneurship when recommending this intervention for CE’s next incubation round [1].
AIM/CE researchers dedicated a few hours to polishing the remaining loose ends in our previous version. It includes a model for calculating the mortality reductions based on diarrhea case-fatality rates. This is a slightly enhanced version of our model and represents an interesting proposition on how to extrapolate the effect of the intervention from cases to mortality—given more time, it could even be expanded to consider pathogen-specific case-fatality rates, which should be even better.
We, the authors of the originial report, had only very limited participation in the changes made between the two versions.