It looks to me like the Kremer paper and the Cochrane review authors have both different methodology and ask different questions—the Cochrane review analysis RCTs as they stand and asks if clean water reduces diarrhoea (which it did), while Kremer mines extra mortality data from previous RCTs then meta-analysis it to look for mortality reduction.
I completely agree the Kremer paper is far more ambitious, and has potential for p-hacking. One of my points in the article though is that Kremer’s mortality reduction finding is eerily similar to what Mills and Reinke found 100 years ago which adds a little more credence I think. Also I like Givewell’s approach of agreeing that there is likely to be a significant mortality benefit, but being more conservative in their approach than the results of Kremer’s study.
What different assessments did you think Kremer made from the Cochrane review authors?
It looks to me like the Kremer paper and the Cochrane review authors have both different methodology and ask different questions—the Cochrane review analysis RCTs as they stand and asks if clean water reduces diarrhoea (which it did), while Kremer mines extra mortality data from previous RCTs then meta-analysis it to look for mortality reduction.
I completely agree the Kremer paper is far more ambitious, and has potential for p-hacking. One of my points in the article though is that Kremer’s mortality reduction finding is eerily similar to what Mills and Reinke found 100 years ago which adds a little more credence I think. Also I like Givewell’s approach of agreeing that there is likely to be a significant mortality benefit, but being more conservative in their approach than the results of Kremer’s study.
What different assessments did you think Kremer made from the Cochrane review authors?
Nice one.