We Need Major, But Not Radical, FDA Reform

Link post

Fellow progress blogger Alex Telford and I have had a friendly back-and-forth going over FDA reform. Alex suggests incremental reforms to the FDA, which I strongly support, but these don’t go far enough. The FDA’s failures merit a complete overhaul: Remove efficacy requirements and keep only basic safety testing and ingredient verification. Any drug that doesn’t go through efficacy trials gets a big red warning label, but is otherwise legal.

Before getting into Alex’s points let me quickly make the positive case for my position.

The FDA is punished for errors of commission: drugs they approve which turn out not to work or to be harmful. They don’t take responsibility for errors of omission: drugs they could have approved earlier but delayed, or drugs that would have been developed but were abandoned due to the cost of approval. This asymmetry predictably leads to overcaution.

Every week the Covid-19 vaccines were delayed, for example, cost at least four thousand lives. Pfizer sent their final Phase 3 data to the FDA on November 20th but was not approved until 3 weeks later on December 11th. There were successful Phase I/II human trials and successful primate-challenge trials 5 months earlier in July. Billions of doses of the vaccine were ordered by September. Every week, thousands of people died while the FDA waited for more information even after we were confident that the vaccine would not hurt anybody and was likely to prevent death. The extra information that the FDA waited months to get was not worth the tens of thousands of lives it cost. Scaling back the FDA’s mandatory authority to safety and ingredient testing would correct for this deadly bias.

This isn’t as radical as it may sound. The FDA didn’t have efficacy requirements until 1962. Today, off-label prescriptions already operate without efficacy requirements. Doctors can prescribe a drug even if it has not gone through FDA-approved efficacy trials for the malady they are trying to cure. These off-label prescriptions are effective, and already make up ~20% of all prescriptions written in the US. Removing mandatory efficacy trials for all drugs is equivalent to expanding this already common practice.

Now, let’s get to Alex’s objections. Most of his post was focused on my analogy between pharmaceuticals and surgery. There are compelling data and arguments on both sides and his post shifted my confidence in the validity and conclusions of the analogy downwards, but in the interest of not overinvesting in one particular analogy I’ll leave that debate where it stands and focus more on Alex’s general arguments in favor of the FDA.

Patent medicines and snake oil

Alex notes that we can look to the past, before the FDA was created, to get an idea of what the pharmaceutical market might look like with less FDA oversight.

Maxwell argues that in the absence of government oversight, market forces would prevent companies from pushing ineffective or harmful drugs simply to make a profit. Except that there are precedents for exactly this scenario occurring. Until they were stamped out by regulators in the early 20th century, patent medicine hucksters sold ineffective, and sometimes literally poisonous, nostrums to desperate patients. We still use “snake oil” today as shorthand from a scam product.

There is no denying that medicine has improved massively over the past 150 years alongside expanding regulatory oversight, but this relationship is not causal. The vast majority of gains in the quality of medical care are due to innovations like antibiotics, genome sequencing, and robotic surgery. A tough and discerning FDA in the 1870s which allows only the best available treatments to be marketed would not have improved medical quality much because the best available treatments at the time were still bad. And even the most lackadaisical and laissez-faire FDA today would not make us forget vaccines or antibiotics or to wash our hands before surgery. The strength of regulatory oversight explains little of the differences in medical quality over the past century.

The main problem with 19th century medicine is not that hucksters sold fake treatments, its that real treatments did not exist and no one knew how to make them. Learning how to make new treatments is still the largest source of gains today. If we can cure cancer, double lifespan, or develop these 8 vaccines, this innovation will far outstrip any gains from sorting through the medicines we already have.

If we further consider that the FDA’s filtration is paid for by a tax on innovation, the tradeoff looks worse. The ever-growing cost of regulatory approval drains hundreds of billions from the research that we rely on for progress.

Chesterton’s fence

In a related point, Alex points out that the FDA has been built incrementally over time, with each new piece a reaction to fill a gap revealed by a scandal. This means that the institutional design of the FDA encodes a lot of information which we ignore at our own risk.

The FDA did not spring into existence fully formed, its powers were granted and expanded over time in response to specific safety scandals and societal pressure (I cover these events in some detail in act 3 of my long post on the history of the pharmaceutical industry, thalidomide being the most notable). Anyone proposing we eliminate the FDA should keep in mind that we’ve already tried an unregulated pharmaceutical industry, and we decided that we didn’t much like it, thank you very much.

I agree with Alex’s description of how the FDA came to be, but this description is exactly why I am confident that the FDA is far too cautious. The FDA is built out of reactions to safety scandals and societal pressure, but the scandal per-life-lost is much higher for errors of commission than errors of omission. The FDA ought to be optimizing to save lives, but instead it’s shaped in reaction to scandal and pressure so it is willing to send millions to the invisible graveyard in exchange for avoiding a few dozen deaths of malpractice.

The FDA shut down Strep A vaccine research for thirty years, for example, based on an uncontrolled study with 21 participants that showed a scandalous risk, ignoring previous successful trials with tens of thousands of participants and a human challenge trial with no serious adverse safety events. Strep A kills five hundred thousand people a year globally.

We know why Chesterton’s fence is there and we know why it’s too high.

Does the FDA actually help innovation?

At the end of the essay, Alex steelmans the case for how drug regulators may help progress and innovation in pharmaceuticals.

The existence of the FDA is a forcing function which incentivises drug companies to generate data that shows that their drugs have a positive risk-benefit profile. Information on what works and doesn’t work is fed back into the development process for future drugs, improving the quality of future drug development decisions
(as a corollary to #1) Because drug companies make money if their products are demonstrated to work, they are incentivised to pursue scientific research in a truth-seeking manner. Over time, market forces conspire to eliminate companies that do bad science and reward those who make discoveries that benefit humanity. By creating a mechanism for the market to reward good science, academic science (as a source of talent and ideas for biotech) is also indirectly incentivised to produce higher quality work
By creating a regulatory pathway which encodes some value judgements on what type of technologies are worth developing and in what manner, regulators can help funnel talent, money, research, and economic activity (incl. entrepreneurship) down channels that are ultimately beneficial for society
A defined regulatory pathway with established value generating milestones (IND, phase I, phase II, phase III…) makes investing in biotech more predictable and attractive

The first three points are closely related. Drug regulation creates incentives for drug companies to produce useful data and research. This data and research has big positive externalities, so its good to provide incentives for them. However, the FDA provides this incentive by making all drug development more expensive, but making failed drug development even more expensive. Because the FDA provides these incentives with a stick instead of a carrot, the net effect on the output of the industry is negative.

The FDA’s growing power does correlate with higher levels of medical quality over time, but the rate of growth is negatively correlated with FDA oversight. There are many possible reasons for this but it is suggestive that the period before FDA efficacy requirements 1900-1962 saw much faster medical progress than the subsequent 60 years.

Imagine we applied the FDA’s incentive strategy to some other public good like carbon removal. Before you can start your carbon removal machine, you have to go through a multi-year, multi-million dollar testing gauntlet to verify exactly how much carbon the machine removes and whether it is more or less effective than planting trees. This process would produce some useful data, and it would make inefficient machines less rewarding relative to machines which pass the tests, but it won’t incentivize entry into the field on net. It would be much better to subsidize the public good industry for producing something we want, or have a government agency directly run the tests rather than taxing all public goods producers, but the good ones a little less.

New medical treatments have massive positive externalities so we want to sort through and differentially reward them without decreasing the overall output of the industry. Instead of raising the cost of entry for all drugs with extra punishments for ineffective ones, we should lower barriers and provide extra rewards to the most effective and safe drugs. One easy way to accomplish this is to remove efficacy requirements, lowering development costs for all drugs, and retrospectively reward the drugs whose early results hold up best in voluntary efficacy trials, post-market surveillance, or government funded trials.

Many parts of clinical research are public goods. Instead of taxing the private rewards that pharmaceutical companies get from this research and returning only a portion of the tax to the public-good-producing industry, we could spend much more on producing research directly. Doubling the NIH budget and running a bunch of clinical trials is a far cheaper and more effective way to incentivize the research, data, and talent that Alex and I both want than the FDA’s current strategy.

Alex’s fourth point about a defined regulatory pathway seems outside of the clash of our debate. The regulatory structure I am proposing is certainly not less defined or predictable than the current one, so it doesn’t seem like a relevant metric by which to compare them.

Alex and I agree on more about the FDA than either of us and the median person you’d find on the street would. Still, we have our disagreements. Fundamentally, I think our disagreements stem from differing weights on the importance of innovation vs selection in medical progress and on the importance of errors of omission vs commission in the FDA’s decision making. I would much rather live in a world with lots of medical innovation and poor selection than the other way around, and I view commission bias as fundamental and pervasive in all of the FDA’s decisions.

To correct for this bias I propose a major, but not a radical, reform to the FDA. Return to the FDA of 1960. Expand the standards we already apply to a large proportion of prescriptions: Remove efficacy requirements for approval. If there are still market distortions which favor ineffective drugs, fix them with subsidies rather than regulatory hurdles so that we don’t siphon resources from the innovative engines which are responsible for the vast majority of medical progress.

Addendum

Supplements

Alex also highlights the modern day supplement industry as a cautionary tale for pharmaceutical deregulation. An investigation from Scott Alexander, which Alex cites, finds that even niche nootropic supplements by-and-large contain what they claim to. Still, most supplements aren’t really good treatments for anything.

I put this section in the addendum because I think supplements are a separate issue from what most discussion of the FDA centers on. Usually, the FDA is framed as a correction to misalignments between the interests of consumers and pharmaceutical companies. People want effective medicine but pharmaceutical companies can give them sugar and sawdust for cheap and it is difficult for patients to find out.

The main problem with supplements, though, is that people don’t want effective treatments. It’s not that consumers ask for one thing and get another, it’s that they get what they want but we think what they want is bad for them. More like gambling or alcohol than snake oil. Whether and when this kind of paternalism is justified is a complicated philosophical debate which is worth further investigations but I thought was orthogonal to our other points about the FDA.