Thanks so so much for this fantastic response, really appreciate it and this helps clarify a few things :). I don’t have anything significant to add, but a few comments.
- On the measurement issue, yeah I 100% agree that downstream effects are the ones that need be measured. If its a procurement reform, then I would agree the primary outcme would be to reduce stockouts. This might be one of the easier RCTs to do (depending on the system). I would ave thought in many cases randomising by district and following up over 1-2 years should answer the question pretty clearly? I agree some reforms it might be hard to find a clear outcome measure, but on procurement/supply chains it mighe be doable in many cases.
- Yes I agree with community health workers in terms of producing great flow-on effects, although like you said wit those exemplar countries like Bangladesh its difficult to know how much to attribute those changes to CHWs. Also the E”xemplars” case study pointed to a bunch of health interventions, not just CHWs which probably contributed to the amazing improvements there, including Access to health centers, maternal health vouchers, procurement changes etc. I really like the “Exemplars in Global Health” studies.
- Small critique I think saying “In Bangladesh, for example, CHWs played a transformative role not only in delivering services but also in shifting norms around family planning, increasing female education, and building trust in the formal healthcare system, all of which helped increase demand for institutional care over time.” could be true but we can’t say it with much confidence at all. Especially saying they played a “transformative role” seems like overreach. How can we seperate the effect of the CHWs from all the other amazing health system inputs Bangladesh was imputting?
- You’re right that sometimes countries want to move fast and don’t care much about research. That’s great if they are funding things themselves. I think if its being externally funded, its is on the funder to make sure research happens in cases where it is important/possible. If we’re funding it, we’re doing an RCT to test it—take it or leave it. Its an under-appreciated point that governments everywhere and especially in low income countries are often not interested in evidence at all. I’ve been blown away by the extent of it here in Uganda.
- On the vaccine point, I’m not sure $33 per extra person vaccinated (from the Meriggi study) would ever worth it for covid (maybr for some other disease). As a side note I think it was a huge waste of resources to try (and fail) vaccinate most Africans for covid. I think it should have just been older people and those with comorbidities vaccinated,( wrote a bit about Covid in Uganda here)
-I think with Health Systems Strengthening, especially when it comes to areas like Governance and supply chains, the question of whether there are “gaps” isn’t always the most important. Yes there are gaps—huge gaps that need addressing. The bigger question is whether we have a consistently effective method to address those gaps.
The gaps are enormous throughout Health Systems, and I think even more than with other interventions cost-effective tractability is the big question.
On a completely unrelated note, this graph from the Bangladesh Exemplars study is one of the coolest I’ve ever seen :D :D :D.
Thanks so so much for this fantastic response, really appreciate it and this helps clarify a few things :). I don’t have anything significant to add, but a few comments.
- On the measurement issue, yeah I 100% agree that downstream effects are the ones that need be measured. If its a procurement reform, then I would agree the primary outcme would be to reduce stockouts. This might be one of the easier RCTs to do (depending on the system). I would ave thought in many cases randomising by district and following up over 1-2 years should answer the question pretty clearly? I agree some reforms it might be hard to find a clear outcome measure, but on procurement/supply chains it mighe be doable in many cases.
- Yes I agree with community health workers in terms of producing great flow-on effects, although like you said wit those exemplar countries like Bangladesh its difficult to know how much to attribute those changes to CHWs. Also the E”xemplars” case study pointed to a bunch of health interventions, not just CHWs which probably contributed to the amazing improvements there, including Access to health centers, maternal health vouchers, procurement changes etc. I really like the “Exemplars in Global Health” studies.
- Small critique I think saying “In Bangladesh, for example, CHWs played a transformative role not only in delivering services but also in shifting norms around family planning, increasing female education, and building trust in the formal healthcare system, all of which helped increase demand for institutional care over time.” could be true but we can’t say it with much confidence at all. Especially saying they played a “transformative role” seems like overreach. How can we seperate the effect of the CHWs from all the other amazing health system inputs Bangladesh was imputting?
- You’re right that sometimes countries want to move fast and don’t care much about research. That’s great if they are funding things themselves. I think if its being externally funded, its is on the funder to make sure research happens in cases where it is important/possible. If we’re funding it, we’re doing an RCT to test it—take it or leave it. Its an under-appreciated point that governments everywhere and especially in low income countries are often not interested in evidence at all. I’ve been blown away by the extent of it here in Uganda.
- On the vaccine point, I’m not sure $33 per extra person vaccinated (from the Meriggi study) would ever worth it for covid (maybr for some other disease). As a side note I think it was a huge waste of resources to try (and fail) vaccinate most Africans for covid. I think it should have just been older people and those with comorbidities vaccinated,( wrote a bit about Covid in Uganda here)
-I think with Health Systems Strengthening, especially when it comes to areas like Governance and supply chains, the question of whether there are “gaps” isn’t always the most important. Yes there are gaps—huge gaps that need addressing. The bigger question is whether we have a consistently effective method to address those gaps.
The gaps are enormous throughout Health Systems, and I think even more than with other interventions cost-effective tractability is the big question.
On a completely unrelated note, this graph from the Bangladesh Exemplars study is one of the coolest I’ve ever seen :D :D :D.