Hi Nick! Thanks so much for your thoughtful response. Here are some of our thoughts on your questions:
**Have you reached out directly to Evidence Action and GiveWell to ask about why they went so hard on chlorination and not ceramic filters? **
Since this comment, I have reached out to Givewell and Evidence Action! Thanks for the suggestion—we had not previously reached out due to time constraints.
Your assumption here that there is a direct connection between cases of diarrhoea and mortality. We have very little clue why clean water leads to a mortality reduction, so I’m not sure its reasonable to make the leap, first that there is a causal relationship at all between diarrhoea reduction and the huge mortality reduction, and second that that connection moves linearly like is assumed here (For every % of diarrhoea reduction, mills-reincke effect moves the same).
You’re right, the leap in assuming a causal relationship between diarrhea reduction and mortality reduction is not ideal and we aren’t assuming this is exactly correct. However, using diarrhea cases as a proxy for morbidity and mortality was the best we could come up with. We welcome any additional suggestions.
In addition, from what we saw from the individual studies considered in the systematic reviews we took into account, none seem to be powered to detect effects on mortality, and most do not report it as an outcome. To make sure reductions in diarrhea cases translate into mortality reductions, one would probably need to do what Kremer and his colleagues did for their 2015 systematic review—individually reach the authors of each study and ask whether they collected any mortality data that went undisclosed due to low statistical power. This could be further work for researchers to perform in the future.
**I could be wrong but I’m not sure there’s any good data at all on mortality reductions for clean water in over 5s. **The Kremer studyIs the only one I know which looks directly at mortality at all, and then only in under 5s. Where does this number come from? My intuition would that the multiplier should be far smaller than this (very very uncertain).
This is a really good point. When applying the 2.35 Mills Reincke multiplier in our cost-effectiveness analysis, we were more concerned with comparing different water quality interventions than getting the exact number for a CEA (dollars per DALY). To compare ceramic filters to chlorination, we applied the same methodology as Givewell to create as close of a comparison between the two.
However, we can perform an analysis removing the Mills-Reinke effect for over 5s. This would affect our estimates both for chlorination and ceramic filters in similar ways but would lower the overall cost-effectiveness. We can likely do this next week!
I think 2 years is ambitious number for lifespan. Between families neglecting to use it (perhaps most important) kids breaking, families selling it (not the end of the world), dirty water clogging it etc. I would have perhaps gone for 1 or 1.5 years use as realistic, but this isn’t so important. I struggled in a 5 minute search to find decent research on ceramic filter lifespan in the real world, do you have one?”
You hit on an important topic, as the lifespan of the ceramic filter actually can significantly impact the overall cost-effectiveness analysis.
Our assumption of a two-year lifespan for ceramic filters in our analysis is based on a Unicef report (Unicef, 2020), which suggests a minimum two-year effectiveness. However, field insights from organizations like “Potters Without Borders” and experts at Pure Home Water indicate that, with proper care, these filters can endure beyond this conservative timeframe.
Michael Anyekase, with over 10 years of experience working with manufacturing and distributing ceramic filters in Uganda at Purehome Water (similar to our theory of change), proposes a potential lifespan of 5 to 6 years under clear water conditions.
Yvette Neh suggests an upper limit of 4 years.. Yvette Neh (who also works at purehome water), proposes an upper limit of 4 years for somewhat good water conditions. In addition, none of the experts we spoke with were overly concerned with filter breakage, and agree that this 2 year lifespan is a conservative timeline?
**We ultimately chose a 2-year filter lifespan as we believe this might be a conservative estimate, and that, in practice, these filters often last much longer. **
From my experience it is borderline implausible that 5.8% of diarrhoeal episodes lead to hospitalisation
This intuitively makes a lot of sense, and we will look into this more deeply in the upcoming days and weeks.
Also quick correction I think you mean the $8 country is Nigeria here.
Thank you! We updated this in the forum and the report!
Again, thank you so much for your thoughtful response.
Hi Nick! Thanks so much for your thoughtful response. Here are some of our thoughts on your questions:
Since this comment, I have reached out to Givewell and Evidence Action! Thanks for the suggestion—we had not previously reached out due to time constraints.
You’re right, the leap in assuming a causal relationship between diarrhea reduction and mortality reduction is not ideal and we aren’t assuming this is exactly correct. However, using diarrhea cases as a proxy for morbidity and mortality was the best we could come up with. We welcome any additional suggestions.
In addition, from what we saw from the individual studies considered in the systematic reviews we took into account, none seem to be powered to detect effects on mortality, and most do not report it as an outcome. To make sure reductions in diarrhea cases translate into mortality reductions, one would probably need to do what Kremer and his colleagues did for their 2015 systematic review—individually reach the authors of each study and ask whether they collected any mortality data that went undisclosed due to low statistical power. This could be further work for researchers to perform in the future.
This is a really good point. When applying the 2.35 Mills Reincke multiplier in our cost-effectiveness analysis, we were more concerned with comparing different water quality interventions than getting the exact number for a CEA (dollars per DALY). To compare ceramic filters to chlorination, we applied the same methodology as Givewell to create as close of a comparison between the two.
However, we can perform an analysis removing the Mills-Reinke effect for over 5s. This would affect our estimates both for chlorination and ceramic filters in similar ways but would lower the overall cost-effectiveness. We can likely do this next week!
You hit on an important topic, as the lifespan of the ceramic filter actually can significantly impact the overall cost-effectiveness analysis.
Our assumption of a two-year lifespan for ceramic filters in our analysis is based on a Unicef report (Unicef, 2020), which suggests a minimum two-year effectiveness. However, field insights from organizations like “Potters Without Borders” and experts at Pure Home Water indicate that, with proper care, these filters can endure beyond this conservative timeframe.
Michael Anyekase, with over 10 years of experience working with manufacturing and distributing ceramic filters in Uganda at Purehome Water (similar to our theory of change), proposes a potential lifespan of 5 to 6 years under clear water conditions.
Yvette Neh suggests an upper limit of 4 years.. Yvette Neh (who also works at purehome water), proposes an upper limit of 4 years for somewhat good water conditions. In addition, none of the experts we spoke with were overly concerned with filter breakage, and agree that this 2 year lifespan is a conservative timeline?
**We ultimately chose a 2-year filter lifespan as we believe this might be a conservative estimate, and that, in practice, these filters often last much longer. **
This intuitively makes a lot of sense, and we will look into this more deeply in the upcoming days and weeks.
Thank you! We updated this in the forum and the report!
Again, thank you so much for your thoughtful response.