Hi there, really enjoying this piece (just discovered it). My grad school advisor often asks: “what evidence would convince a determined skeptic?” and I think that’s broadly in the same vein.
Incidentally, my entry to GiveWell’s Change Our Mind contest does for SMC what you did LLINs, though I came away much less convinced. I think the core difference between us is that I am, by default, skeptical of pre-replication-crisis research. I think that if you find papers from 20 years ago where the authors themselves say that their designs were underpowered to detect an effect, then the odds of successful replication (contingent on a new team getting all the implementation details right) are disquietingly low.
My beliefs on this were shaped by writing a pretty critical meta-analysis of the ‘contact hypothesis’. Lots of experts said that the salubrious effects of contact on prejudice had been proven beyond reasonable doubt, but when we zoomed in on the very strongest research, we just didn’t see it. Right around then, some political scientist ran some very nice intergroup contact experiments in post-conflict areas, and they found much less encouraging results (one, two).
Basically, I’ve come to believe that most published research findings are false, and I don’t give pre-replication-crisis studies the benefit of the doubt. But, as you say, if no IRB would give LLIN replication its approval, we’re kind of at a dead end.
Hi there, really enjoying this piece (just discovered it). My grad school advisor often asks: “what evidence would convince a determined skeptic?” and I think that’s broadly in the same vein.
Incidentally, my entry to GiveWell’s Change Our Mind contest does for SMC what you did LLINs, though I came away much less convinced. I think the core difference between us is that I am, by default, skeptical of pre-replication-crisis research. I think that if you find papers from 20 years ago where the authors themselves say that their designs were underpowered to detect an effect, then the odds of successful replication (contingent on a new team getting all the implementation details right) are disquietingly low.
My beliefs on this were shaped by writing a pretty critical meta-analysis of the ‘contact hypothesis’. Lots of experts said that the salubrious effects of contact on prejudice had been proven beyond reasonable doubt, but when we zoomed in on the very strongest research, we just didn’t see it. Right around then, some political scientist ran some very nice intergroup contact experiments in post-conflict areas, and they found much less encouraging results (one, two).
Basically, I’ve come to believe that most published research findings are false, and I don’t give pre-replication-crisis studies the benefit of the doubt. But, as you say, if no IRB would give LLIN replication its approval, we’re kind of at a dead end.