Thanks @Madeleine Ballard. As you know I’m a big fan of the Pro-CHW movement and all the work you do. I agree facility care can’t reach everyone. I agree we need pro CHWs across Africa and like you say CHWs are often the main people doing the work facilitating EA funded vertical programs like mosquito nets and deworming. Sometimes this platform isn’t recognised, and the cost borne by governments and donors who support these CHWs isn’t appreciated properly in calculations—which I think I’ve pointed out to GiveWell before.
I also agree that EA doesn’t pay nearly enough attention to gradual government adoption of the most cost-effective interventions, and I hope we’re going to see more discussion about this in the near future.
When looking at RP’s report, this was what I said in context.
”I agree that CHWs are an essential, scalable part of many LMIC health systems. CHWs however are touted to be very cost-effective when they often aren’t. This statement made my eyes pop a little. “Many studies have found CHW programs to be cost-effective by various metrics. In Mozambique, the annual cost per beneficiary was just $47.12.
That number $47.12 I quoted from the RP report is quite different from $10 you’ve quoted as the median cost per capita here and doesn’t scream cost-effectiveness. The report which your $10 number came from wasn’t out at the time I wrote that response. $10 per beneficiary still seems high to me but it’s hard to compare to other interventions (see below)
One thing I’m confused about, is thatI don’t think a cost as low as $0.59 per capita is possible for a salaried CHW. How did the researchers get to this number?Trying to be as conservative as possible—here’s my sense check. The ceiling for the no. of people a CHW can cover on a door-to-door model is perhaps 800. A lower end CHW salary might be $40 a month. Then we add commodity and supervision costs (excluding digital/connectivity costs). I’ve estimated this at around 20 people tested for malaria, 10 given treatment for malaria and 5 each treated for pneumonia and diarrhoea. Often CHWs would treat more people than this, which would mean higher costs.
- Salary (12x40) = $480 - Malaria tests 20 a month (20 x $0.5 x 12 months ) = $120 - Malaria treatment 10 a month (10 x $0.3 x 12) = $36 - Pneumonia treatment 5 a month (5 x $0.5 x 12) = $30 - Diarrhoea ORS + zinc 5 a month (5 x $0.5 x 12) = $30 - Supervision ? $60 a year
= $756 / 800 people = $0.95 per capita per year under pretty conservative assumptions. I might be missing something here though!
Also “Per capita cost” and “Cost per beneficiary” are useful metrics for governments to decide if CHWs are affordable/cheap enough, but not useful metrics for cost-effectiveness comparisons. What can these metrics be compared to? We can’t compare this metric to any other intervention. I’ve considered we could use a metric such as “cost of universal primary care per-capita” which would combine the cost of CHWs + other services, which could then be compared to facility only based models + extra transport costs. This would be tricky though…
For CHWs a more straightforward “Cost per DALYs averted” would be more useful to compare with other EA funded interventions, or a more thorough GiveWell style CEA.
Also on this point “Any cost-effectiveness analysis that attributes the full fixed cost of that worker to one of those service lines is going to produce a misleading number.” I don’t think Givewell attributes the full fixed cost of CHWs in CEAs. They might apportion some of the cost, or perhaps none. I often have the opposite criticism that they don’t always appreciate the underlying cost of government health workers used in mass-distribution campaigns. I could be wrong here though I haven’t looked closely recently.
Again I’m a huge fan of CHWs and your work and I believe in your mission. I’m just challenging whether your numbers make sense, and I think we need more than per-capita/per-beneficiary numbers to compare CHWs to other cost-effective interventions under EA frameworks.
Thanks so much, Nick! We published a massive multi-paper cost-effective review only weeks after you shared your original post! You can dig into all the papers/numbers/methodology here: https://joinchic.org/resources/cost-effectiveness/
TL;DR: The numbers are medians across 380 scenarios in 130 studies. As I said in the footnote, data is heterogeneous (mostly due to platform design differences, including salaries). Ranges are big and depend on context, but would say a) your $0.95 BOTE is a fair floor for a professional program at modest catchment, but b) that your earlier take (CHWs are “often not cost-effective,” they treat “few patients per month,” they can only treat “a handful of conditions in young children”) is not reflected in these data.
Re: DALYs: you’ll see in the papers that vertical evidence is already in EA terms (e.g. there are DALY numbers for iCCM) but none of the 42 horizontal integrated scenarios in the BMJ GH piece report cost per DALY. The research I’d most want EA to fund is a prospective platform-economics evaluation: does bundling 6–8 service lines on one salaried CHW compound DALYs and lower cost-per-DALY the way fixed-cost-sharing predicts? (or to your point, maybe not?)
My bigger point though is that maybe this is already a good enough bet based on the vertical DALY evidence we both already accept (i.e. why not apply hits-based giving on the handover: catalytic financing to professionalize a salaried, skilled, supervised, supplied workforce via national, government). Esp, as this has what a hit almost never has, which is a decade of cost-effectiveness evidence already behind it. If the handover lands, the donor pays once and the government runs the whole portfolio forever (and the vertical $/DALY figures are the floor). Have added 2 footnotes re:the above.
I think It wouldn’t cost much at all to make forward a pretty robust cost-effectiveness model for a CHW which rolls out a wide range of interventions. (I think Living Goods +- others might well have decent models already here?). I think you could even build this yourself? Some of the package would be easy to do because data is there (malaria, diarrhoea, pneumonia treatment, family planning antenatal care), while screenings and referrals are much harder to quantify and might have to be left out of the analysis pending better data.
I agree the bet argument is pretty good if the goal is government adoption. Regardless of the nuance of cost-effectiveness, CHWs will always be more cost-effective than most health things govt. could do and it would likely displace less cost-effective things. Unfortunately I don’t think EA funders have seriously considered scale through govt. something worth chasing as a bet, but I really like the idea.
Thanks @Madeleine Ballard. As you know I’m a big fan of the Pro-CHW movement and all the work you do. I agree facility care can’t reach everyone. I agree we need pro CHWs across Africa and like you say CHWs are often the main people doing the work facilitating EA funded vertical programs like mosquito nets and deworming. Sometimes this platform isn’t recognised, and the cost borne by governments and donors who support these CHWs isn’t appreciated properly in calculations—which I think I’ve pointed out to GiveWell before.
I also agree that EA doesn’t pay nearly enough attention to gradual government adoption of the most cost-effective interventions, and I hope we’re going to see more discussion about this in the near future.
When looking at RP’s report, this was what I said in context.
”I agree that CHWs are an essential, scalable part of many LMIC health systems. CHWs however are touted to be very cost-effective when they often aren’t. This statement made my eyes pop a little. “Many studies have found CHW programs to be cost-effective by various metrics. In Mozambique, the annual cost per beneficiary was just $47.12.
That number $47.12 I quoted from the RP report is quite different from $10 you’ve quoted as the median cost per capita here and doesn’t scream cost-effectiveness. The report which your $10 number came from wasn’t out at the time I wrote that response. $10 per beneficiary still seems high to me but it’s hard to compare to other interventions (see below)
One thing I’m confused about, is that I don’t think a cost as low as $0.59 per capita is possible for a salaried CHW. How did the researchers get to this number? Trying to be as conservative as possible—here’s my sense check. The ceiling for the no. of people a CHW can cover on a door-to-door model is perhaps 800. A lower end CHW salary might be $40 a month. Then we add commodity and supervision costs (excluding digital/connectivity costs). I’ve estimated this at around 20 people tested for malaria, 10 given treatment for malaria and 5 each treated for pneumonia and diarrhoea. Often CHWs would treat more people than this, which would mean higher costs.
- Salary (12x40) = $480
- Malaria tests 20 a month (20 x $0.5 x 12 months ) = $120
- Malaria treatment 10 a month (10 x $0.3 x 12) = $36
- Pneumonia treatment 5 a month (5 x $0.5 x 12) = $30
- Diarrhoea ORS + zinc 5 a month (5 x $0.5 x 12) = $30
- Supervision ? $60 a year
= $756 / 800 people = $0.95 per capita per year under pretty conservative assumptions. I might be missing something here though!
Also “Per capita cost” and “Cost per beneficiary” are useful metrics for governments to decide if CHWs are affordable/cheap enough, but not useful metrics for cost-effectiveness comparisons. What can these metrics be compared to? We can’t compare this metric to any other intervention. I’ve considered we could use a metric such as “cost of universal primary care per-capita” which would combine the cost of CHWs + other services, which could then be compared to facility only based models + extra transport costs. This would be tricky though…
For CHWs a more straightforward “Cost per DALYs averted” would be more useful to compare with other EA funded interventions, or a more thorough GiveWell style CEA.
Also on this point “Any cost-effectiveness analysis that attributes the full fixed cost of that worker to one of those service lines is going to produce a misleading number.” I don’t think Givewell attributes the full fixed cost of CHWs in CEAs. They might apportion some of the cost, or perhaps none. I often have the opposite criticism that they don’t always appreciate the underlying cost of government health workers used in mass-distribution campaigns. I could be wrong here though I haven’t looked closely recently.
Again I’m a huge fan of CHWs and your work and I believe in your mission. I’m just challenging whether your numbers make sense, and I think we need more than per-capita/per-beneficiary numbers to compare CHWs to other cost-effective interventions under EA frameworks.
Thanks so much, Nick! We published a massive multi-paper cost-effective review only weeks after you shared your original post! You can dig into all the papers/numbers/methodology here: https://joinchic.org/resources/cost-effectiveness/
TL;DR: The numbers are medians across 380 scenarios in 130 studies. As I said in the footnote, data is heterogeneous (mostly due to platform design differences, including salaries). Ranges are big and depend on context, but would say a) your $0.95 BOTE is a fair floor for a professional program at modest catchment, but b) that your earlier take (CHWs are “often not cost-effective,” they treat “few patients per month,” they can only treat “a handful of conditions in young children”) is not reflected in these data.
Re: DALYs: you’ll see in the papers that vertical evidence is already in EA terms (e.g. there are DALY numbers for iCCM) but none of the 42 horizontal integrated scenarios in the BMJ GH piece report cost per DALY. The research I’d most want EA to fund is a prospective platform-economics evaluation: does bundling 6–8 service lines on one salaried CHW compound DALYs and lower cost-per-DALY the way fixed-cost-sharing predicts? (or to your point, maybe not?)
My bigger point though is that maybe this is already a good enough bet based on the vertical DALY evidence we both already accept (i.e. why not apply hits-based giving on the handover: catalytic financing to professionalize a salaried, skilled, supervised, supplied workforce via national, government). Esp, as this has what a hit almost never has, which is a decade of cost-effectiveness evidence already behind it. If the handover lands, the donor pays once and the government runs the whole portfolio forever (and the vertical $/DALY figures are the floor).
Have added 2 footnotes re:the above.
I think It wouldn’t cost much at all to make forward a pretty robust cost-effectiveness model for a CHW which rolls out a wide range of interventions. (I think Living Goods +- others might well have decent models already here?). I think you could even build this yourself? Some of the package would be easy to do because data is there (malaria, diarrhoea, pneumonia treatment, family planning antenatal care), while screenings and referrals are much harder to quantify and might have to be left out of the analysis pending better data.
I agree the bet argument is pretty good if the goal is government adoption. Regardless of the nuance of cost-effectiveness, CHWs will always be more cost-effective than most health things govt. could do and it would likely displace less cost-effective things. Unfortunately I don’t think EA funders have seriously considered scale through govt. something worth chasing as a bet, but I really like the idea.