Emily Oster declared that “treating HIV doesn’t pay.” “It is humane to pay for AIDS drugs in Africa,” she wrote, “but it isn’t economical. The same dollars spent on prevention would save more lives.”
Twenty years later, with $100 billion dollars appropriated[26] under both Democratic and Republican administrations, and millions of lives saved, it’s hard to argue a different foreign aid program would’ve garnered more support, scaled so effectively, and done more good. It’s not that trade-offs don’t exist, we just got the counterfactual wrong.
It’s not clear to me that the core point of the essay goes through. For instance, the same amount of money as applied to malaria would also have helped many people, driven down prices, encouraged innovation—maybe the equivalent would have been a malaria vaccine, a gene drive, or mass fumigations.
i.e., it seems plausible that both of these could be true:
PEPFAR was worth doing
There are other large health megaprojects that would have been better
At the risk of over-simplification: I think you should at least discount the expected value of all those alternative health mega-projects by (a) the probability that congress would’ve funded them, and (b) the probability they would’ve worked as intended at scale. If the first probability was, say, 10%, and the latter maybe 25%, numbers get pretty small pretty fast. (Of course, PEPFAR only scores 100% and 100% in retrospect. But it seems it would’ve still scored quite high circa late 2002.)
Longer version...
Fully agree that $100b investment in malaria eradication might have changed a whole bunch of parameters too. This is a good point. But it also reinforces the point that static, comparative cost-effectiveness analysis is a bad framework for making these choices. What frustrates me is that even after 20 years, we lack a good analytical framework for thinking about why PEPFAR worked so well.
I didn’t harp on this point in the original post, but some of the “better” alternative uses of PEFAR money that were put forward both then and now—e.g., HIV prevention rather than treatment -- lack the same kind of proven effectiveness at scale that we’ve seen for ARVs. Development is littered with ‘proven’ interventions that fall apart at scale. Did the White House get lucky in picking something with sufficiently simple logistics that it could work across dozens of countries and millions of patients? Or did Paul Farmer, Mark Dybul, etc know something we don’t? Not sure.
It’s always difficult to know whether to build politics into the model when doing economic calculations. I’m sure there are instances where saying “I know you (policymakers) don’t want to do it, but policy A would be way better than B.” PEPFAR just illustrates how much money we potentially leave on the table by respecting that partition between politics and technical advice. If the doctors summoned to the White House had lectured the policy folks on how a big push on HIV treatment was irresponsible, would they realistically have gotten something better? Who knows, but there’s little in the historical record of PEPFAR’s genesis to suggest that was a real possibility.
The first point seems to be saying that we should factor in the chance that a program works into cost-effectiveness analysis. Isn’t this already a part of all such analyses? If it isn’t, I’m very surprised by that and think it would be a much more important topic for an essay than anything about PEPFAR in particular.
The second point, that people should consider whether a project is politically feasible, is well taken. It sounds like the lesson here is “if you find yourself in a situation where you have to recommend either project A or B, and both are good, but A is better than B, but if you do activism for A it still won’t happen, but if you do activism for B your input will push it over the edge into happening, do activism for B.” I agree with this as far as it goes, but there seem like some important caveats:
You shouldn’t lie, for the normal reasons against lying. It sounds like some of these economists were just publishing reports or articles saying that antimalarials were better than PEPFAR, and I think if that was what they found, then publishing that in reports is correct regardless of the politics. Then when they put on their activist hats they can support whatever is most effective to support.
It’s a really hard problem to know whether to pursue less ambitious vs. more ambitious goals. If you’re a socialist, should you spend your time fighting for a $1 minimum wage increase, for Medicare For All, or for completely dismantling capitalism and replacing it with workers’ communes? I don’t think there’s an obvious answer, even though the first is clearly more likely to succeed than the second two. In retrospect it’s clear that PEPFAR worked politically, and if you say that more cost-effective options wouldn’t have worked politically at that time then I believe you, but I don’t want to conclude that therefore less ambitious political projects are always better than more ambitious ones, and I don’t know how else to apply the lesson from this story more generally.
Good points. Agree that “always go for a big push instead of incrementalism” is waaayyyy too simple and sweeping a lesson to draw from PEPFAR. Also, three cheers for not lying. I think the World Bank was right not to suppress its data on the low cost-effectiveness of ARV drugs circa the mid-2000s. But in retrospect, I think people drew bad policy conclusions from that data.
My piece above is largely a plea for a little bit of intellectual humility and introspection on the part of the cost-effectiveness crowd (of which I’m often an active participant). If we find our analytical framework leads us to oppose the core of PEPFAR (i.e., the distribution of free ARVs in poor countries), then we ought—in the face of its historic achievements—to pause and reflect a bit, rather than shrugging and insisting “yeah, but I could’ve done better.”
Your first paragraph gets at one reason why I think that kind of confidence isn’t credible. Successful trials don’t automatically translate into successful national or multi-national, multi-billion dollar programs. So we should be more skeptical that something that scored higher on cost-effectiveness terms ex ante would’ve converted into a massive global policy win. I genuinely don’t think cost-effectiveness analysis, at least as practiced in global development/global health, really takes those questions of ‘scalability’ and implementation feasibility seriously enough. As a result, we confuse stuff that really could be done a million times over if money was available with stuff that looks cheap but faces much deeper political and organizational hurdles.
When we find something that works and really can scale with an influx of cash, we should run with it. With the benefit of hindsight, PEPFAR seems to fit that description. And, coincidentally, setting aside broader philosophical debates to stick with this concrete case, PEPFAR happens to be up for reauthorization in congress this year...
I noticed that much of the political tractability discussion has focussed on counterfactuals involving other diseases (e.g. malaria) but I’m more confused why the government prioritised treatment over prevention.
Oster’s article argued that prevention would be a better bet than treatment and that was written in 2005 when Congress had approved the budget and the prices of the drugs had come down. You also highlight the importance of, “the evidence from those who had worked in the field that antiretroviral drugs and preventive measures could be deployed effectively and at reasonable cost, even in very poor settings”. (emphasis added)
So both options had evidence of feasibility and efficacy, the money was already there, and it seems like, at that time, they didn’t know which option would scale better or get cheaper quicker.
I’m struggling to see what Oster got wrong. Was the decision to prioritise treatment over prevention mostly driven by the emotional appeal of the ‘Lazarus effect’?
It’s not clear to me that the core point of the essay goes through. For instance, the same amount of money as applied to malaria would also have helped many people, driven down prices, encouraged innovation—maybe the equivalent would have been a malaria vaccine, a gene drive, or mass fumigations.
i.e., it seems plausible that both of these could be true:
PEPFAR was worth doing
There are other large health megaprojects that would have been better
At the risk of over-simplification: I think you should at least discount the expected value of all those alternative health mega-projects by (a) the probability that congress would’ve funded them, and (b) the probability they would’ve worked as intended at scale. If the first probability was, say, 10%, and the latter maybe 25%, numbers get pretty small pretty fast. (Of course, PEPFAR only scores 100% and 100% in retrospect. But it seems it would’ve still scored quite high circa late 2002.)
Longer version...
Fully agree that $100b investment in malaria eradication might have changed a whole bunch of parameters too. This is a good point. But it also reinforces the point that static, comparative cost-effectiveness analysis is a bad framework for making these choices. What frustrates me is that even after 20 years, we lack a good analytical framework for thinking about why PEPFAR worked so well.
I didn’t harp on this point in the original post, but some of the “better” alternative uses of PEFAR money that were put forward both then and now—e.g., HIV prevention rather than treatment -- lack the same kind of proven effectiveness at scale that we’ve seen for ARVs. Development is littered with ‘proven’ interventions that fall apart at scale. Did the White House get lucky in picking something with sufficiently simple logistics that it could work across dozens of countries and millions of patients? Or did Paul Farmer, Mark Dybul, etc know something we don’t? Not sure.
It’s always difficult to know whether to build politics into the model when doing economic calculations. I’m sure there are instances where saying “I know you (policymakers) don’t want to do it, but policy A would be way better than B.” PEPFAR just illustrates how much money we potentially leave on the table by respecting that partition between politics and technical advice. If the doctors summoned to the White House had lectured the policy folks on how a big push on HIV treatment was irresponsible, would they realistically have gotten something better? Who knows, but there’s little in the historical record of PEPFAR’s genesis to suggest that was a real possibility.
The first point seems to be saying that we should factor in the chance that a program works into cost-effectiveness analysis. Isn’t this already a part of all such analyses? If it isn’t, I’m very surprised by that and think it would be a much more important topic for an essay than anything about PEPFAR in particular.
The second point, that people should consider whether a project is politically feasible, is well taken. It sounds like the lesson here is “if you find yourself in a situation where you have to recommend either project A or B, and both are good, but A is better than B, but if you do activism for A it still won’t happen, but if you do activism for B your input will push it over the edge into happening, do activism for B.” I agree with this as far as it goes, but there seem like some important caveats:
You shouldn’t lie, for the normal reasons against lying. It sounds like some of these economists were just publishing reports or articles saying that antimalarials were better than PEPFAR, and I think if that was what they found, then publishing that in reports is correct regardless of the politics. Then when they put on their activist hats they can support whatever is most effective to support.
It’s a really hard problem to know whether to pursue less ambitious vs. more ambitious goals. If you’re a socialist, should you spend your time fighting for a $1 minimum wage increase, for Medicare For All, or for completely dismantling capitalism and replacing it with workers’ communes? I don’t think there’s an obvious answer, even though the first is clearly more likely to succeed than the second two. In retrospect it’s clear that PEPFAR worked politically, and if you say that more cost-effective options wouldn’t have worked politically at that time then I believe you, but I don’t want to conclude that therefore less ambitious political projects are always better than more ambitious ones, and I don’t know how else to apply the lesson from this story more generally.
Good points. Agree that “always go for a big push instead of incrementalism” is waaayyyy too simple and sweeping a lesson to draw from PEPFAR. Also, three cheers for not lying. I think the World Bank was right not to suppress its data on the low cost-effectiveness of ARV drugs circa the mid-2000s. But in retrospect, I think people drew bad policy conclusions from that data.
My piece above is largely a plea for a little bit of intellectual humility and introspection on the part of the cost-effectiveness crowd (of which I’m often an active participant). If we find our analytical framework leads us to oppose the core of PEPFAR (i.e., the distribution of free ARVs in poor countries), then we ought—in the face of its historic achievements—to pause and reflect a bit, rather than shrugging and insisting “yeah, but I could’ve done better.”
Your first paragraph gets at one reason why I think that kind of confidence isn’t credible. Successful trials don’t automatically translate into successful national or multi-national, multi-billion dollar programs. So we should be more skeptical that something that scored higher on cost-effectiveness terms ex ante would’ve converted into a massive global policy win. I genuinely don’t think cost-effectiveness analysis, at least as practiced in global development/global health, really takes those questions of ‘scalability’ and implementation feasibility seriously enough. As a result, we confuse stuff that really could be done a million times over if money was available with stuff that looks cheap but faces much deeper political and organizational hurdles.
When we find something that works and really can scale with an influx of cash, we should run with it. With the benefit of hindsight, PEPFAR seems to fit that description. And, coincidentally, setting aside broader philosophical debates to stick with this concrete case, PEPFAR happens to be up for reauthorization in congress this year...
I noticed that much of the political tractability discussion has focussed on counterfactuals involving other diseases (e.g. malaria) but I’m more confused why the government prioritised treatment over prevention.
Oster’s article argued that prevention would be a better bet than treatment and that was written in 2005 when Congress had approved the budget and the prices of the drugs had come down. You also highlight the importance of, “the evidence from those who had worked in the field that antiretroviral drugs and preventive measures could be deployed effectively and at reasonable cost, even in very poor settings”. (emphasis added)
So both options had evidence of feasibility and efficacy, the money was already there, and it seems like, at that time, they didn’t know which option would scale better or get cheaper quicker.
I’m struggling to see what Oster got wrong. Was the decision to prioritise treatment over prevention mostly driven by the emotional appeal of the ‘Lazarus effect’?