Your paraphrasing is amazing (probably better than my original post). I just fear you know my brain a bit better than I do. Are you the first GAI? I also don’t feel like your analysis is that abstracted at all—your points seem quite concrete actually.
One small correction I might make is that most Ugandans who get malaria would get treatment, not just a few. We target the 5-10% of places which are really remote and getting treatment is difficult—that’s what where here for.
It’s an interesting idea to do “Badness of untreated malaria” x “Chance the treatment was counterfactual”. This is a cleaner method than what I did that’s for sure. The first issue with this is that I’m not sure we have a clear data point for badness of untreated malaria (although I can look into this more). Obviously impossible to study now and we need to rely on older data.
The chance of treatment (yes or no) is counterfactual would be more realistic to find, but is very black and white when really there’s a lot more too it than getting treatment or not. Quality of care is important—but perhaps even more important like @Ray_Kennedypointed out is how quickly people get the treatment. Malaria is an exponentially replicating parasite, and hours can make a differece.
On your counterfactual adjustments (love it)
The most severe cases would be more likely to get treated yes (interesting point never thought of ths) - but as a counterpoint as already discussed early treatment is really important. Often (not always) severe malaria is a direct result of inadequate, late or non-treatment.
Yes there’s definitely the counterfactual of the government doing less because NGOs are doing the work. A given thing which should probably warrant a dscount with basically any NGO program! As a caveat here, the Ugandan government made the active decision to spend their resources upgrading current facilities rather than building more in remote places. In the last 10 years the government has opened 0 (or maybe 1) new health center in Northern Uganda. Also of course like many other NGOs we have the dream that perhaps someday a government could see the value in our model and potentially take it over or roll out a version of their own. So this could be a tiny point in our favour (see my bias kicking in again...)
(Side note) Givewell seems quite rough on AMF on this front, positing a 50% chance that other funders would come in if they weren’t there in some countries. This feels overconservative to me—what might well be happening is that Global Fund (or others) instead fund more nets in other places instead of working where AMF are working
The only comment I didn’t really understand was the difference between where the vale comes from.
“My biggest uncertainty would be to what extent the value of ODH comes from [counterfactually averting deaths/saving lots of DALYs], vs [providing better quality of care, or more convenient treatment].”
I might well be missing something, but better quality of care and more “convenient” treatment (meaning people get earlier treatment) both avert deaths and save DALYs, just like getting treatment vs. not getting at all does. So doesn’t it all play into the same value proposition?
I might well be missing something, but better quality of care and more “convenient” treatment (meaning people get earlier treatment) both avert deaths and save DALYs, just like getting treatment vs. not getting at all does. So doesn’t it all play into the same value proposition?
See, me missing context matters here. I was imagining that the most pessimistic scenario would be that:
ODH provides treatment for malaria which is faster, nearer & more convenient
but patients would have otherwise gotten the same treatment, just later, further away and paying more costs to get it
So the value of ODH wouldn’t be the value of the treatment, it would be the value of making it more convenient
But as you point out (“Quality of care is important—but perhaps even more important like @Ray_Kennedypointed out is how quickly people get the treatment. Malaria is an exponentially replicating parasite, and hours can make a differece.”) you can’t just neatly separate getting faster care from getting better care. There are some fun things you could do with distributions, i.e., explicitly model the benefit as a function of how fast you get treatment, and then estimate the counterfactual value as
∫∫ (Value of getting treatment in h hours—Chance of having otherwise gotten treatment in (h + x) hours instead × Value of getting treatment in (h + x) hours) dx dh
(where the double integral just means that you are explicitly estimating the value of each possible pair of x and h and then weighing them according to how likely they are)
But I think this would be overkill, and only worth coming back to do explicitly if/when ODH is spending a few million a year. Still they might add some clarity if we don’t do the calculations. Anyways, best of luck.
On the counterfactual of the government potentially doing less, I speculate that it would be politically difficult for the government to copy ODH’s business model under which 2⁄3 of total costs are covered by patient fees. Specifically, my understanding is that user fees for public healthcare were dropped in the early 2000s, although as a practical matter the public system isn’t always free. Reinstating official fees only in certain areas probably wouldn’t fly well politically. So the government would likely have to spend several times what ODH does to set up the same health centers, and that is probably relevant to assessing the odds that it might counterfactually do so.
Your paraphrasing is amazing (probably better than my original post). I just fear you know my brain a bit better than I do. Are you the first GAI? I also don’t feel like your analysis is that abstracted at all—your points seem quite concrete actually.
One small correction I might make is that most Ugandans who get malaria would get treatment, not just a few. We target the 5-10% of places which are really remote and getting treatment is difficult—that’s what where here for.
It’s an interesting idea to do “Badness of untreated malaria” x “Chance the treatment was counterfactual”. This is a cleaner method than what I did that’s for sure. The first issue with this is that I’m not sure we have a clear data point for badness of untreated malaria (although I can look into this more). Obviously impossible to study now and we need to rely on older data.
The chance of treatment (yes or no) is counterfactual would be more realistic to find, but is very black and white when really there’s a lot more too it than getting treatment or not. Quality of care is important—but perhaps even more important like @Ray_Kennedy pointed out is how quickly people get the treatment. Malaria is an exponentially replicating parasite, and hours can make a differece.
On your counterfactual adjustments (love it)
The most severe cases would be more likely to get treated yes (interesting point never thought of ths) - but as a counterpoint as already discussed early treatment is really important. Often (not always) severe malaria is a direct result of inadequate, late or non-treatment.
Yes there’s definitely the counterfactual of the government doing less because NGOs are doing the work. A given thing which should probably warrant a dscount with basically any NGO program! As a caveat here, the Ugandan government made the active decision to spend their resources upgrading current facilities rather than building more in remote places. In the last 10 years the government has opened 0 (or maybe 1) new health center in Northern Uganda. Also of course like many other NGOs we have the dream that perhaps someday a government could see the value in our model and potentially take it over or roll out a version of their own. So this could be a tiny point in our favour (see my bias kicking in again...)
(Side note) Givewell seems quite rough on AMF on this front, positing a 50% chance that other funders would come in if they weren’t there in some countries. This feels overconservative to me—what might well be happening is that Global Fund (or others) instead fund more nets in other places instead of working where AMF are working
The only comment I didn’t really understand was the difference between where the vale comes from.
“My biggest uncertainty would be to what extent the value of ODH comes from [counterfactually averting deaths/saving lots of DALYs], vs [providing better quality of care, or more convenient treatment].”
I might well be missing something, but better quality of care and more “convenient” treatment (meaning people get earlier treatment) both avert deaths and save DALYs, just like getting treatment vs. not getting at all does. So doesn’t it all play into the same value proposition?
See, me missing context matters here. I was imagining that the most pessimistic scenario would be that:
ODH provides treatment for malaria which is faster, nearer & more convenient
but patients would have otherwise gotten the same treatment, just later, further away and paying more costs to get it
So the value of ODH wouldn’t be the value of the treatment, it would be the value of making it more convenient
But as you point out (“Quality of care is important—but perhaps even more important like @Ray_Kennedy pointed out is how quickly people get the treatment. Malaria is an exponentially replicating parasite, and hours can make a differece.”) you can’t just neatly separate getting faster care from getting better care. There are some fun things you could do with distributions, i.e., explicitly model the benefit as a function of how fast you get treatment, and then estimate the counterfactual value as
∫∫ (Value of getting treatment in h hours—Chance of having otherwise gotten treatment in (h + x) hours instead × Value of getting treatment in (h + x) hours) dx dh
(where the double integral just means that you are explicitly estimating the value of each possible pair of x and h and then weighing them according to how likely they are)
But I think this would be overkill, and only worth coming back to do explicitly if/when ODH is spending a few million a year. Still they might add some clarity if we don’t do the calculations. Anyways, best of luck.
On the counterfactual of the government potentially doing less, I speculate that it would be politically difficult for the government to copy ODH’s business model under which 2⁄3 of total costs are covered by patient fees. Specifically, my understanding is that user fees for public healthcare were dropped in the early 2000s, although as a practical matter the public system isn’t always free. Reinstating official fees only in certain areas probably wouldn’t fly well politically. So the government would likely have to spend several times what ODH does to set up the same health centers, and that is probably relevant to assessing the odds that it might counterfactually do so.