CEO of Fortify Health, Mulago and Jacobs Fellow, Ex-IDinsight and Management Consulting.
I lead Fortify Health, a GiveWell, Coefficient Giving and Founder’s Pledge supported non-profit dedicated to reducing and preventing iron-deficiency anaemia. I love thinking about how to scale impactful, evidence-based, cost-effective interventions to alleviate poverty.
Hi NOVAH team,
I have a lot of admiration for your transparency and public commitment. In general, I think this kind of public commitment and pre-analysis planning for key evaluation results is laudable and pushes the sector towards evidence-based decision making.
That said, I do have some concerns about setting a pre-commitment on an effect size from an RCT, particularly at such an early stage of an intervention’s maturity. How are you thinking about mitigating the risk of a null result? I understand that in that scenario you’d pause for a few months and conduct exploratory research. But would there not be an opportunity, at earlier stages of your theory of change, to better test the inputs-to-outputs and outputs-to-outcomes links through things like:
simple observational studies
mixed methods
process evaluations, run in parallel with or prior to the RCT?
I’ll put my biases on the table. I think RCTs are incredibly useful tools, but they’re often not the most helpful tools for understanding the complex theory of change behind an intervention. They give you a single output, usually an effect size, that can be hard to disentangle and act on operationally, especially if that effect size turns out to be null.