As someone predisposed to like modeling, the key takeaway I got from Justin Sandefur’s Asterisk essay PEPFAR and the Costs of Cost-Benefit Analysis was this corrective reminder – emphasis mine, focusing on what changed my mind:
Second, economists were stuck in an austerity mindset, in which global health funding priorities were zero-sum: $300 for a course of HIV drugs means fewer bed nets to fight malaria. But these trade-offs rarely materialized. The total budget envelope for global public health in the 2000s was not fixed. PEPFAR raised new money. That money was probably not fungible across policy alternatives. Instead, the Bush White House was able to sell a dramatic increase in America’s foreign aid budget by demonstrating that several billion dollars could, realistically, halt an epidemic that was killing more people than any other disease in the world.
...
A broader lesson here, perhaps, is about getting counterfactuals right. In comparative cost-effectiveness analysis, the counterfactual to AIDS treatment is the best possible alternative use of that money to save lives. In practice, the actual alternative might simply be the status quo, no PEPFAR, and a 0.1% reduction in the fiscal year 2004 federal budget. Economists are often pessimistic about the prospects of big additional spending, not out of any deep knowledge of the budgeting process, but because holding that variable fixed makes analyzing the problem more tractable. In reality, there are lots of free variables.
More detail:
Economists’ standard optimization framework is to start with a fixed budget and allocate money across competing alternatives. At a high-level, this is also how the global development community (specifically OECD donors) tends to operate: foreign aid commitments are made as a proportion of national income, entirely divorced from specific policy goals. PEPFAR started with the goal instead: Set it, persuade key players it can be done, and ask for the money to do it.
Bush didn’t think like an economist. He was apparently allergic to measuring foreign aid in terms of dollars spent. Instead, the White House would start with health targets and solve for a budget, not vice versa. … Economists are trained to look for trade-offs. This is good intellectual discipline. Pursuing “Investment A” means forgoing “Investment B.” But in many real-world cases, it’s not at all obvious that the realistic alternative to big new spending proposals is similar levels of big new spending on some better program. The realistic counterfactual might be nothing at all.
In retrospect, it seems clear that economists were far too quick to accept the total foreign aid budget envelope as a fixed constraint. The size of that budget, as PEPFAR would demonstrate, was very much up for debate.
When Bush pitched $15 billion over five years in his State of the Union, he noted that $10 billion would be funded by money that had not yet been promised. And indeed, 2003 marked a clear breaking point in the history of American foreign aid. In real-dollar terms, aid spending had been essentially flat for half a century at around $20 billion a year. By the end of Bush’s presidency, between PEPFAR and massive contracts for Iraq reconstruction, that number hovered around $35 billion. And it has stayed there since.
Compared to normal development spending, $15 billion may have sounded like a lot, but exactly one sentence after announcing that number in his State of the Union address, Bush pivoted to the case for invading Iraq, a war that would eventually cost America something in the region of $3 trillion — not to mention thousands of American and hundreds of thousands of Iraqi lives. Money was not a real constraint.
Tangentially, I suspect this sort of attitude (Iraq invasion notwithstanding) would naturally arise out of a definite optimism mindset (that essay by Dan Wang is incidentally a great read; his follow-up is more comprehensive and clearly argued, but I prefer the original for inspiration). It seems to me that Justin has this mindset as well, cf. his analogy to climate change in comparing economists’ carbon taxes and cap-and-trade schemes vs progressive activists pushing for green tech investment to bend the cost curve. He concludes:
You don’t have to give up on cost-effectiveness or utilitarianism altogether to recognize that these frameworks led economists astray on PEPFAR — and probably some other topics too. Economists got PEPFAR wrong analytically, not emotionally, and continue to make the same analytical mistakes in numerous domains. Contrary to the tenets of the simple, static, comparative cost-effectiveness analysis, cost curves can sometimes be bent, some interventions scale more easily than others, and real-world evidence of feasibility and efficacy can sometimes render budget constraints extremely malleable. Over 20 years later, with $100 billion dollars appropriated under both Democratic and Republican administrations, and millions of lives saved, it’s hard to argue a different foreign aid program would’ve garnered more support, scaled so effectively, and done more good. It’s not that trade-offs don’t exist. We just got the counterfactual wrong.
Aside from his climate change example above, I’d be curious to know what other domains economists are making analytical mistakes in w.r.t. cost-benefit modeling, since I’m probably predisposed to making the same kinds of mistakes.
From the PEPFAR report, the same thing Justin Sandefur mentioned above:
When PEPFAR was announced, many economists thought it would not be cost-effective to treat AIDS. They were wrong. Their initial concern was understandable: when PEPFAR began in 2004, first-line antiretroviral therapy cost about $1,000 per year, so treating everyone with HIV would cost tens of billions of dollars. Many experts worried that PEPFAR would lead to funding cuts for other highly cost-effective aid efforts. But the pessimistic forecasts didn’t come true. First, the Bush Administration decided to fund PEPFAR on top of existing aid efforts. Second, the massive increase in funding for antiretroviral drugs created demand that helped drive competition and innovation, and supercharged an existing race to develop cheaper, more effective generic drugs.3 Today, a year of first-line antiretroviral medication costs about $60.
As a result, PEPFAR has saved between 7.5 and 30 million lives, at a cost between $1,500 and $10,000 per life saved, and has also prevented at least 5.5 million babies from being born with HIV. It has also gotten more cost-effective each year as medication costs decline, doing more and more on a budget that has been declining in real dollars since 2009.
Some other quotes that stood out to me:
PEPFAR was conceived of and launched during a time of widespread skepticism about foreign aid. It had become clear that lots of foreign aid was corrupt, wasteful, or unhelpful. But the thesis behind PEPFAR was that health interventions might succeed where larger development interventions often had not. Accountability for health interventions is easier—we can tell whether or not a drug has reached a patient. Results are more measurable. And while development interventions are often premised on ideological claims that may come and go, or on theories of investment that may not hold up, health interventions have only the simple thesis that health is better than sickness and life is better than death.
In the present day, PEPFAR has been successfully handing off responsibilities in partner nations, targeting 70% of funding through local organizations including partner country governments. Some countries, including Botswana and South Africa, have successfully transitioned to funding a majority of their own HIV efforts, with PEPFAR now playing a smaller supporting role.
PEPFAR requires very good accounting controls, with every expenditure documented and demonstrated to be in line with program requirements. For this report, we read three recent Office of the Inspector General audits of PEPFAR program recipients. The three audits we reviewed found undocumented expense rates ranging from 0% to 2% of program expenses, and they demanded repayment of every dollar unaccounted for.
Quoting Goldsmith, Horiuchi, and Wood 2014, “PEPFAR has, indeed, positively affected how publics in recipient countries regard the US … what types of aid, under what conditions, might be effective in influencing foreign public opinion about the donor. Specifically, our theory is that foreign aid that is targeted, sustained, effective, and visible is more likely to affect mass opinion. … as Goldsmith and Horiuchi(2012) show, changes in public opinion within Country B about Country A can influence Country B’s foreign policy behavior toward Country A.” (emphasis ours).
As someone predisposed to like modeling, the key takeaway I got from Justin Sandefur’s Asterisk essay PEPFAR and the Costs of Cost-Benefit Analysis was this corrective reminder – emphasis mine, focusing on what changed my mind:
More detail:
Tangentially, I suspect this sort of attitude (Iraq invasion notwithstanding) would naturally arise out of a definite optimism mindset (that essay by Dan Wang is incidentally a great read; his follow-up is more comprehensive and clearly argued, but I prefer the original for inspiration). It seems to me that Justin has this mindset as well, cf. his analogy to climate change in comparing economists’ carbon taxes and cap-and-trade schemes vs progressive activists pushing for green tech investment to bend the cost curve. He concludes:
Aside from his climate change example above, I’d be curious to know what other domains economists are making analytical mistakes in w.r.t. cost-benefit modeling, since I’m probably predisposed to making the same kinds of mistakes.
From the PEPFAR report, the same thing Justin Sandefur mentioned above:
As a result, PEPFAR has saved between 7.5 and 30 million lives, at a cost between $1,500 and $10,000 per life saved, and has also prevented at least 5.5 million babies from being born with HIV. It has also gotten more cost-effective each year as medication costs decline, doing more and more on a budget that has been declining in real dollars since 2009.
Some other quotes that stood out to me: