Governance, financing, and supply chain interventions can be randomised at state or district level
While I agree this is true in theory, is it practical? I imagine the size needed to power such a study is prohibitive except for the largest organisations, the answer still wouldn’t be definitive (eg due to generalisation concerns), and there would be lots of measurement issues (eg residents in one district crossing to another district with better funded healthcare).
If you tell me I’m wrong, I’d definitely bow to your experience and knowledge in this field but this isn’t obviously true.
I don’t have great experience and knowledge here AT ALL as a caveat. Never “bow” to anything I say, my takes are often more on the “loose” than “rock solid” end of things :D.
I think if we can randomise things like socrecard studies and IMCI across hundreds of health facilities (done a number of times), then I don’t see why we can’t do the same with a supply chain interventions or governance interventions. The community Health worker movement has done some impressive large scale RCTs like this one. Perhaps 1-3 million dollars could make these studies happen without too much trouble. Give 10 randomised districts the governance/ intervention and 10 not, then just see if healthcare outputs improve. I actually think its easier than many other types of studies because
1. I think its good enough to measure outcomes in terms of facility level outputsd, so we don’t necessarily need community level morbidity/mortality data 2. Outcome measures (no. of patients treated, correct diagnosis) would be super easy and not expensive to measure compared with other studies. In many cases routinely collected DHIS data should be enough to answer the primary outcome question so we don’t even necessarily need much expense on data collection (a big study cost)
I would say from an RCT perspective if people crossed to another district because healthcare was getting that much better, that would be a strong sign that the intervention is working insanely well. If it was a financing type intervention, then making it close to cost-neutral between the intervention and control group. People are NOT very mobile in places like Uganda at least. Where I work in rural places Transport is often (if not usually) the biggest healthcare cost people incur.
I think the biggest reasons these studies haven’t happened more (there are some) are less practicality and more...
1. Most governance, financing and supply chain interventions are funded by bilateral aid not philanthropy, so they don’t usually think about rigorous testing. Instead they often spend quite large amounts of money on “baseline and endline surveys” which I think are often like pouring money down a sink. Or to be slightly more generous perhaps a direct cash transfer to the richest 1% of people in low income countries ;)
2. The kind of people who are into HSS interventions are often pretty religious about HSS being the “only” way to go and are also often RCT averse, so are less likely to commit time and resources to an RCT as part of their intervention.
While I agree this is true in theory, is it practical? I imagine the size needed to power such a study is prohibitive except for the largest organisations, the answer still wouldn’t be definitive (eg due to generalisation concerns), and there would be lots of measurement issues (eg residents in one district crossing to another district with better funded healthcare).
If you tell me I’m wrong, I’d definitely bow to your experience and knowledge in this field but this isn’t obviously true.
I don’t have great experience and knowledge here AT ALL as a caveat. Never “bow” to anything I say, my takes are often more on the “loose” than “rock solid” end of things :D.
I think if we can randomise things like socrecard studies and IMCI across hundreds of health facilities (done a number of times), then I don’t see why we can’t do the same with a supply chain interventions or governance interventions. The community Health worker movement has done some impressive large scale RCTs like this one. Perhaps 1-3 million dollars could make these studies happen without too much trouble. Give 10 randomised districts the governance/ intervention and 10 not, then just see if healthcare outputs improve. I actually think its easier than many other types of studies because
1. I think its good enough to measure outcomes in terms of facility level outputsd, so we don’t necessarily need community level morbidity/mortality data
2. Outcome measures (no. of patients treated, correct diagnosis) would be super easy and not expensive to measure compared with other studies. In many cases routinely collected DHIS data should be enough to answer the primary outcome question so we don’t even necessarily need much expense on data collection (a big study cost)
I would say from an RCT perspective if people crossed to another district because healthcare was getting that much better, that would be a strong sign that the intervention is working insanely well. If it was a financing type intervention, then making it close to cost-neutral between the intervention and control group. People are NOT very mobile in places like Uganda at least. Where I work in rural places Transport is often (if not usually) the biggest healthcare cost people incur.
I think the biggest reasons these studies haven’t happened more (there are some) are less practicality and more...
1. Most governance, financing and supply chain interventions are funded by bilateral aid not philanthropy, so they don’t usually think about rigorous testing. Instead they often spend quite large amounts of money on “baseline and endline surveys” which I think are often like pouring money down a sink. Or to be slightly more generous perhaps a direct cash transfer to the richest 1% of people in low income countries ;)
2. The kind of people who are into HSS interventions are often pretty religious about HSS being the “only” way to go and are also often RCT averse, so are less likely to commit time and resources to an RCT as part of their intervention.