Malaria vaccines: how confident are we?
Alternative title: should SoGive red-team malaria vaccines?
We’ve been seeing a lot of excitement about malaria vaccines – e.g. the first thing mentioned by the EA wins 2023 post was the R21 vaccine.
We at SoGive looked into malaria vaccines about a year ago, and came away with a slightly more cautious impression. Bear in mind though, (a) we were trying to answer a different question[1]; (b) a lot has changed in a year.
The purpose of this post is to outline these (currently tentative) doubts, and explore whether there’s appetite for us to research this more carefully.
The main things we’re still unsure of
At first glance, malaria vaccines appear less cost-effective than existing malaria interventions (nets/SMC[2]). Are they, in fact, less cost-effective?
In light of this, does it make sense to advocate for their rollout?
We thank 1Day Sooner for their helpful comments and constructive collaboration – we sent them a draft of this shortly before publishing. We also thank our contacts at Malaria Consortium and AMF; when we spoke to them in 2022 for our earlier review of malaria vaccines, their comments were very helpful. Some earlier work done by current/former members of the SoGive team has also provided useful groundwork for the thinking here, so thank you to Isobel Phillips, Ishaan Guptasarma, Scott Smith.
Be aware that any indications of cost-effectiveness in this post are extremely rough, and may change materially if we were to conduct this research.
Malaria vaccines may be materially (10x??) less effective than nets/SMC
Based on the research we did a year ago, it seems that malaria vaccines significantly underperform bednets and SMC. Several items in this table are caveated; it’s worth reviewing the version in the appendix which sets out the details.
| Several items in this table are caveated; it’s worth reviewing the version in the appendix which sets out the details. | |||
| RTS,S vaccine | R21 vaccine | Bednets* | |
| Cost-related considerations | |||
| Cost per person treated | $56.40 (estimated) | >$8, based on WHO info; ~$25, based on info from 1Day Sooner | $2.18 |
| Number of doses needed per person | 4 (i.e. 3 + a booster) | 4 (i.e. 3 + a booster) | 0.49 bednets per person protected |
| Logistics: cold chain? | Yes | Yes, but less demanding than RTS,S | No |
| Efficacy-related considerations | |||
| Reduction in clinical malaria** | 55.8% | 77% | 45% |
| Reduction in severe malaria** | 32.2% | Unknown, estimated at 44.4% | 45% |
* SMC is only excluded from this table for brevity, not because of any preference for bednets over SMC.
** Malaria reduction figures are estimates under study conditions
Vaccine costs look high…
When we created this table c.1 year ago, the key message from this table is that costs for vaccines are materially higher than for bednets or SMC, which is significantly driven by logistical difficulties, such as the need for multiple doses and a cold supply chain (i.e. the vaccines have to be kept at a low temperature while they are transported). At the time, we focused on RTS,S because there was more information available.
At that stage, we guessed that R21 would likely have similar costs to RTS,S. Somewhat to our surprise, it does seem that R21 costs may be lower than RTS,S costs. We weren’t clear on the costs of R21, however when we shared a draft of this with 1Day Sooner, they helpfully pointed us to their Dec 2023 Vaccination Status Report. It seems they believe that each dose costs $3.90 on its own, and the all-in cost of delivering the first dose to a person is $25 per full course.
… and there doesn’t * seem * to be an offsetting efficacy benefit.
Although the efficacy numbers look similar, there are several complicating factors not captured in this table. For example, a consideration about the ages of beneficiaries may somewhat make bednets look better for anyone doing a back-of-the-envelope calculation from the above table. This is expanded on in the research plan below.
Numbers are caveated – we haven’t done this research yet.
And just to stress, the cost-effectiveness comparisons (10x less effective) are very rough, back-of-the-envelope figures. 10x is calculated solely by observing that the cost per person appears to be about an order of magnitude higher for R21; there are several other considerations which could move the needle (possibly significantly).
To indicate how materially the 10x estimate could move after doing this work properly, a gut-feel-based 75% confidence interval is 2x-30x, and a gut-feel-based 90% confidence interval is 0.3x-100x.
Is worse cost-effectiveness reason not to advocate for something?
Should we use advocacy efforts on getting pre-qualification of R21 to happen sooner, or should we use our advocacy efforts on something else, e.g. to get more aid money spent more effectively, which presumably includes more malaria nets and SMC?
It’s not obvious what the answer to that question is, but it seems reasonable to believe that the cost-effectiveness of vaccines may at least be a relevant part of the story.
The research plan (see below) sets out a number of ways advocating for malaria vaccines might make sense, even if they are not the most cost-effective.
What happens next?
At this stage, the purpose of writing this is to see whether someone else has already resolved these questions.
If not, we would consider looking into this more carefully. Here’s our plan for what we would do:
Research plan
Review and update our understanding of the R21 vaccine as at today.
Is our understanding of the costs correct, or could we be misunderstanding? (e.g. maybe the $2-$4 mentioned by the WHO only captures part of the costs) Ideally, it would also be useful to understand why the R21 costs are materially lower than the RTS,S costs.
Update: after we shared a draft of this with 1Day Sooner, they pointed us to their Dec 2023 Vaccination Status Report. It seems they believe that each dose costs $3.90 on its own, and the all-in cost of delivering the first dose to a person is $25 per full course. The research should review/verify this.
What’s an up-to-date picture of the efficacy? The numbers in the table are from older papers, and more up to date figures may change the figures.
How strong is the evidence base? Normally the evidence from, say, just one RCT would not be considered strong evidence, and might cause us to apply a replicability adjustment.
Does the efficacy percentage appropriately reflect the complications of multiple doses (e.g. not all those who got the first dose will get the next dose, etc; is that appropriately captured)
Plus any other considerations which would help us determine whether the cost-effectiveness of R21 really is less good than the cost-effectiveness of bednets or SMC, and if so, by how much.
This would include any of the other factors in the GiveWell CEA that have been glossed over in this short write-up, such as
Adjustments for differences between trial conditions and actual usage (e.g. net usage may be worse in real life)
Adjustment for net coverage years lost due to residual nets from previous distributions
Insecticide resistance (note that we have previously critiqued GiveWell’s approach to insecticide resistance adjustments, arguing that they have not incorporated relevant RCT evidence; we might not incorporate these concerns in this analysis)
This list is not exhaustive
This would likely be implemented by simply taking a copy of the GiveWell model and adjusting it.
Critiquing/red-teaming GiveWell’s analysis is out of scope for this piece of research.
The cost-effectiveness analysis should consider ages – vaccines are geared towards under-5s, who have a higher baseline malaria mortality than the general population. However the c.$2 cost per person for bednets is not focused on under-5s. This means that a back-of-the-envelope cost-effectiveness estimate from the figures in the table above would flatter the relative cost-effectiveness figures for bednets. Thank you to 1Day Sooner for highlighting this point.
Understand the Imperial College cost-effectiveness study on R21 (Schmit et al 2023)
When we did our earlier review, the Imperial College cost-effectiveness study on RTS,S (Topazian, Schmit et al 2022) seemed to confirm our impression that RTS,S is not the most cost-effective intervention (at least most of the time).
The pre-print for the equivalent paper for R21 states “Under an assumed dose price of US$3 we estimated a median incremental cost-effectiveness ratio compared with current interventions of <...> $36 [range $126, $34] and $33 [range $158, $27] per DALY averted in perennial and seasonal settings, respectively”.
We understand this report was co-authored by the R21 study team.
If the findings are correct, this appears to be in the same ballpark as bednets and SMC, which seems surprising given the high costs.
Note that the point around ages above may be an offsetting factor which could conceivably bridge this gap (at least partially)
Thank you to the team at 1Day Sooner for flagging this.
Does the information about malaria vaccines change our views on the cost-effectiveness of bednets and SMC?
In principle there could be interaction effects (i.e. the extent to which bednets or SMC reduce malaria might be smaller if the beneficiaries are also receiving vaccines). In practice, this might not be such a big deal, since the overlap probably won’t be large.
The new information about the fact that malaria vaccines seem to have a smaller effect on more severe malaria than on malaria as a whole might cause us to think twice about the assumption in GiveWell’s models which says that this effect size is the same regardless of the severity of the malaria. (This is essentially questioning the “Ratio of the reduction in malaria mortality to the reduction in malaria incidence” item in GiveWell’s CEA). Note that we largely want to avoid red-teaming GiveWell’s analysis for this research, so this point may be de-scoped.
Arguably the policy decision should be based not on today’s cost-effectiveness, but on future cost-effectiveness.
A deeper understanding of the cost structure – is there much scope for it to come down further?
How might vaccine efficacy evolve over time?
With any of these considerations about future cost effectiveness, it’s useful to distinguish between future changes which will happen anyway, and those which will be accelerated/supported by rolling out the vaccines sooner.
Might some kind of forecasting help with this?
It might be appropriate to consider this comparatively – i.e. will the cost-effectiveness of bednets/SMC decline over time (e.g. because of diminishing marginal returns of some sort)? If so, might vaccine cost-effectiveness overtake it? If so, when would we expect vaccine cost-effectiveness to overtake bednet/SMC cost-effectiveness? How much sooner than that overtaking point should we start rolling out vaccines?
Are there reasons to want to support the roll-out of the vaccine, even if R21’s cost-effectiveness is somewhat behind that of the most cost-effective malaria interventions?
For example, those bodies whom you are lobbying to fund the vaccine roll-out might otherwise have been expected to use funds on activities which are less cost-effective than the R21 vaccine. E.g. because they have different cost-effectiveness thresholds.
Is there a reason why including a mixture of cost-effective and somewhat less cost-effective interventions together could be a high-impact outcome? E.g. maybe there are reasons why full malaria elimination is particularly valuable, or, related to the previous point, perhaps funders who are being lobbied are keen to ensure that full malaria elimination is achieved.
If anyone thinks this research plan is incomplete, we would be interested to hear further thoughts.
Similarly, if anyone has immediate answers to the questions, we would be happy to hear their thoughts.
We would be very happy if this analysis weren’t needed, and the purpose of this post is to explore whether it’s useful to conduct this work. If it seems like it’s useful, and there’s enough people who would value it, we may conduct it, especially since our past work on this topic leaves us well-positioned. Also, this is not intended in a mercenary way, but if donors were willing to provide SoGive with financial contributions towards this work, that would make us significantly more likely to prioritise this work.
Appendix: table with more details
This is a repeat of the table which appeared earlier in this document, but with more details/sources/references.
| RTS,S vaccine | R21 vaccine | Bednets1 | |
| Cost-related considerations | |||
| Cost per person treated | $56.40 (estimated) | >$8, based on WHO info; ~$25, based on info from 1Day Sooner2 | $2.183 |
| Number of doses needed per person | 4 (i.e. 3 + a booster) | 4 (i.e. 3 + a booster) | 0.49 bednets per person protected4 |
| Logistics: cold chain needed? | Yes | Yes, but less demanding than RTS,S5 | No |
| Efficacy-related considerations | |||
| Estimated reduction in clinical malaria (under study conditions) | 55.8%6 | 77%7 | 45%8 |
| Estimated reduction in severe malaria (under study conditions) | 32.2%9 | Unknown, estimated at 44.4%10 | 45%11 |
Note 1:
For brevity, this table references malaria nets but not SMC. This is not intended to indicate some kind of preference for nets over SMC. Indeed, if the table did include a column for SMC, the overall picture would be very similar to what we see for bednets.
Note 2:
When we first produced this table, the cost per person for R21 was populated as
Unknown, but likely similar to the costs for RTS,S.
The rationale we gave was:
We haven’t done a careful review of the costs of the R21 vaccine, and we haven’t seen an explicit estimate of the costs, but we know that there are structural issues with both vaccines which drive costs upwards for both. Furthermore, the R21 vaccine uses an “adjuvant” (or additive) called Matrix M, and we know that this adjuvant has at least one feature which will lead to it being less cost-effective than the AS01 adjuvant used for the RTS,S vaccine. Source: “No such TLR4 ligand is present in the MM adjuvant which could lead to an improved safety profile and lower costs of manufacture” Venkatraman et al 2019 preprint https://www.medrxiv.org/content/10.1101/19009282v1.full.pdf or https://www.medrxiv.org/content/10.1101/19009282v1 Note that we haven’t done a careful review of all the components, and there may be other elements which point in the other direction.
When we updated it for publication recently, we populated this cell with
$8-$16, see caveats
We expanded on this as follows:
We didn’t carefully review this number, so this is caveated, but the WHO said $2-$4 per dose, so we just multiplied that by 4 (4 doses).
Later in the document, we clarified that we had not checked if this was the “all-in” cost, i.e. whether it included logistical and other costs.
After we shared a draft of this with 1Day Sooner, they pointed us to their Dec 2023 Vaccination Status Report. It seems they believe that each dose costs $3.90 on its own, and the all-in cost of delivering the first dose to a person is $25 per full course.
Thank you to 1Day Sooner for this helpful contribution.
Note 3:
The cost per person protected by a bednet is calculated as a straight (i.e. unweighted) average of the cost per malaria net (i.e. this row in GiveWell’s CEA), which comes to $4.48. This is multiplied by the number of bednets needed per person (0.49 nets, see elsewhere in the table) to get to $2.18.
Note 4:
It’s common for more than one person to sleep under one net because of bed-sharing. Source of the estimate of 2.06 people sleeping under one net is GiveWell: Number of people covered per net [Oct 2022] (public). 0.49 bednets per person (i.e. about half a bednet per person) is calculated as (1/2.06).
Note 5:
RTS,S must be kept between 2-8°C. R21 also requires 2-8°C, but exhibits “thermostability for 2 weeks at 25°C and 40°C”, which would make distribution in Sub-Saharan Africa easier. (See the table on page 3 of this UNICEF document: https://www.unicef.org/supply/media/19456/file/Malaria%20-%20Vaccine%20-%20QA%20-%20October%202023%20-%20English%20.pdf)
Thank you to 1Day Sooner for flagging this.
Note 6:
RTS,S Clinical Trials Partnership, 2015, The Lancet: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5626001/ “The coprimary endpoints of efficacy to clinical malaria over the first 12 months after dose 3 were 55·8% (97·5% CI 50·6–60·4) in children aged 5–17 months and 31·3% (23·6–38·3) in infants aged 6–12 weeks.” Note that this is at 12 months (another row of this table looks at more severe cases of malaria, which is assessed at 18 months)
Note 7:
Datoo et al, 2021, The Lancet: Efficacy of a low-dose candidate malaria vaccine, R21 in adjuvant Matrix-M, with seasonal administration to children in Burkina Faso: a randomised controlled trial—The Lancet. “At 1 year, vaccine efficacy remained high, at 77% (67–84)”
Note 8:
This 45% figure is the reduction in the incidence of uncomplicated episodes of Plasmodium falciparum malaria according to Pryce et al 2018, a Cochrane review of bednets. Study conditions are largely similar to real life, but this figure doesn’t include any adjustments for insecticide resistance, or for the possibility that the 36 months between distributions might be longer than the time period at which malaria incidence was assessed in the studies (which varied between the 5 studies in the Pryce metastudy).
Note 9:
RTS,S Clinical Trials Partnership, 2015, The Lancet: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5626001/ “compared with 171 children who experienced at least one episode of severe malaria in the C3C [control] group, 116 children experienced at least one episode of severe malaria in the R3R [treatment] group (32·2%, 13·7 to 46·9)” This is at 18 months.
Note 10:
The estimate of 44.4% was calculated by simple rescaling, i.e. 32.2% * 77%/55.8%. We have not checked whether the linearity assumptions implicit in this calculation are reasonable.
Note 11:
GiveWell assumes that the same 45% factor applies to both general malaria reductions and to the more severe reductions associated with fatal malaria. This isn’t obviously correct, and they provide some discussion of it in their models (see the comment in this cell), or in their intervention report on SMC.
- ^
The question we were answering was not “should we advocate for more malaria vaccines?”, but rather “should the existence of malaria vaccines change our minds about funding existing malaria interventions (bednets and SMC)?”
- ^
SMC = seasonal malaria chemoprevention. GiveWell recommends Malaria Consortium’s SMC work.
I feel like as president of 1Day Sooner I should probably chime in—first, I wanted to say this type of work—critiquing advocacy campaigns and analyses from EA or EA-aligned groups—is very valuable and should be encouraged. I’m appreciative of SoGIve for publishing this and think they should be commended for spending the time to conduct this analysis. I think creating a healthy ecosystem for disagreement and the right incentives to encourage criticism and full-throated debate is important.
On the object-level question, I’m obviously biased but I think most of the difference in cost-effectiveness in the SoGive analysis goes away if you adjust for the fact that vaccines are only given to children under 5 but only ~15-20% of bednets cover children under 5. Because 75% of malaria mortality is in children under 5, bednets are cheaper per person protected but the vaccines are much more targeted to people whose protection is most valuable. (The development benefit effects of reducing morbidity in children are also age-skewed in vaccines’ favor though that’s less dramatic).
Insecticide resistance (probably reduces bednets’ effectiveness to about 80% of what they’d otherwise be) and durability (GW estimates each bednet purchased provides about 1.7 year of coverage) are probably also relevant. The AMF tab of the GiveWell spreadsheet is a useful resource in thinking through these questions.
For more of my thinking, here’s my side of the email correspondence with Sanjay at SoGive. (I didn’t include text from other people on the thread because I haven’t asked their permission to share). An interesting meta-question is what should be the norm about making these sort of red-teaming or adversarial post-review correspondences public. My guess is it’s probably a good thing to default to because it incentivizes people to be on their best behavior (and the benefits to confidentiality of being able to speak frankly don’t seem that strong in these cases). But I don’t think it’s obvious either way and would be curious what other people think.
Overall, I’m eager to see more analysis done digging into the Imperial/Oxford modeling of cost benefit of the R21 vaccine (which comes to about 630 lives saved per 100K vaccinated, see Table 2) and what’s publicly available about the WHO estimate of 13% all-cause mortality reduction from RTS,S. (Here’s an older preprint that finds a smaller benefit—more recent data that was publicly reported is apparently higher). So I think generally the follow-on research plan Sanjay discusses makes sense from my perspective, and I’d be personally supportive of anyone who wants to contribute to that work.
Thank you Josh. I’ve found 1Day Sooner’s collaborative spirit to be exemplary here—both being welcoming of the challenge and adding useful thoughts.
It seems intuitive to me that the following package of considerations may lead to vaccines and nets/SMC having roughly the same cost-effectiveness:
vaccines are 10x (ish) more expensive (bad for vaccines)
vaccines are more targeted at the most vulnerable ages (good for vaccines)
misc other considerations, like insecticide resistance (this is a bit hand-wavey at the moment, but I guess probably nets out to being good for vaccines)
A hint at the possibility that this might end up with similar cost-effectiveness is the Imperial/Oxford study that Josh mentioned.
Other considerations include:
Replicability: when we create these sorts of models, we don’t normally give 100% credit to the efficacy figures for vaccines. Rather we recognise there isn’t yet a large evidence base, and typically the efficacy is more likely to go down than up with more evidence; estimating how much it would go down by as we got more and more evidence is sometimes called an internal validity adjustment, or replicability adjustment.
For bednets this is negligible (adjust by 5%), and with good reason—nets are a very well-established intervention with plenty of evidence
I expect that such an adjustment for vaccines might be significantly less favourable, but I’d need to look at the evidence properly to say anything more precise than that
My best guess, having not researched it properly, is that the cost-effectiveness of the R21 vaccine probably will still be clearly behind that of existing interventions (bednets/SMC), but it may well be above the cost-effectiveness thresholds of the bodies you’re advocating to (e.g. because they are large aid agencies with large budgets), meaning that the advocacy still makes sense. I wouldn’t be surprised if either of these guesses were incorrect
Thanks SoGive for the post! We wanted to share some of GiveWell’s current thinking around malaria vaccines in case it’s helpful. We also wrote a report on RTS,S in 2022 here and have recommended a couple grants for vaccine rollout and research.
On a cost-per-person-reached basis, we agree ITNs and SMC are superior to either of the two WHO-approved malaria vaccines. However, we think there’s less of a differential in cost-effectiveness than this post implies, for a number of reasons:
The difference in all-in delivery costs is probably less substantial: We think it roughly costs $6 to deliver an ITN to a household[1], and roughly $7 to provide a child with a full course of SMC. (See also impact metrics here). We estimate the total costs for fully immunizing a child range from $23-$43, depending on the choice of vaccine.[2]
We would approach the comparison for the reduction in malaria incidence differently: Using the effect size from the Pryce et al meta-analysis for ITNs (which consists mostly of trials that lasted one year) and comparing it with the incidence reduction observed after 12 months in RTS,S/R21 trials is not as straightforward as is seems:
The reduction found in Pryce et al has to be considerably updated in light of recent developments, notably new types of nets (PBO, dual active ingredient) and increased insecticide resistance.
The RTS,S Clinical Partnership trial includes results from a follow-up four years after the start of the intervention and finds a reduction in clinical malaria from 28% (3 dose group) to 36% (4-dose group) at endpoint. Our malaria team recommends these results rather than the earlier snapshots, as they are less noisy.
Our best guess is that R21 and RTS,S are similarly effective at preventing malaria. Available evidence suggests that short-term effectiveness is broadly similar between RTS,S and R21. No data has been published for the impact of R21 on malaria incidence over the long run (>20 months after the first dose). Because the short-term outcomes are broadly similar, and both vaccines employ the same mechanism to induce an immune response in the vaccinated person[3], our best guess is that four doses of R21 probably offer similar levels of protection as four doses of RTS,S over the long run.
We believe the apparent difference cited in the post is most likely due to the different setup between trials (Datoo 2022, unlike the RTS,S trial, was carried out in a seasonal setting). Note also that results from a phase III trial on R21 are now available (Datoo 2023 (preprint)).
The duration of protection differs: Due to factors such as attrition and physical decay, we currently estimate an ITN to provide between 1.2 − 2.0 years of effective protection. As indicated above, we estimate that malaria vaccines offer around 30% protection over four years.
There are, of course, additional factors that need to be taken into account to get a full picture (for example, what coverage levels are achievable for each intervention?). However, our current best guess is that even with those included, nets (and SMC) will be more cost-effective than malaria vaccines—just not by an order of magnitude.
Costs per child reached are much higher, roughly $15-$26.
The price per dose of RTS,S was $9.30 in 2022, and estimates for R21 indicate a price per dose of $3.90. We expect that, on average, 70% of children who received three doses will also get a booster shot, which implies vaccine costs per child between $14-$37. The best costing estimates for the delivery of the doses suggest around $9 per child.
“The leading malaria vaccine in development is the circumsporozoite protein (CSP)-based particle vaccine, RTS,S, which targets the pre-erythrocytic stage of Plasmodium falciparum infection. It induces modest levels of protective efficacy, thought to be mediated primarily by CSP-specific antibodies. We aimed to enhance vaccine efficacy by generating a more immunogenic CSP-based particle vaccine and therefore developed a next-generation RTS,S-like vaccine, called R21. The major improvement is that in contrast to RTS,S, R21 particles are formed from a single CSP-hepatitis B surface antigen (HBsAg) fusion protein, and this leads to a vaccine composed of a much higher proportion of CSP than in RTS,S.” Collins et al. 2017, “Abstract”
Thanks very much for this, much appreciated. Your best guess of vaccines being less cost-effective than bednets and SMC, but not by an order of magnitude, sounds sensible.
Thank you for posting this, really interesting! I am pretty excited for malaria vaccine roll-out, but think that posts which go against current thinking are valuable, and I appreciate this being posted.
Three things that I’d be super curious about (and i think might have the effect of being more optimistic);
(1) IIRC the WHO found a 13% reduction in all-cause mortality (!) with a vaccination coverage rate of ~65%. This was in Ghana/ Kenya and Malawi, I think. I assume these areas already had some availability of SMC/ bednets, and this makes me think that there’s likely quite a substantial impact beyond nets/ SMC?
(2) Beyond immediate impacts of the vaccine- will the malaria vaccine likely bring eradication forward beyond the impact of nets/ SMC? (my current understanding is yes, and I think that bringing eradication forward is super important.)
(3) As you already point out, where would money have otherwise been spent. My impression here is that the money which funds malaria vax roll-out would likely come from a mix of sources, rather than nescessarily pulling from SMC/ bednets (but I still have some concern/ uncertainty here).
I said this in another comment, but in case it gets missed, I just want to highlight that 1Day Sooner has shown an excellent attitude. When we reached out to them, they were consistently welcoming of the criticism and had constructive useful comments. I’ve found these virtues to be more common in the EA community than elsewhere, but I still like to call them out when I see it.
I always enjoy some good malaria cost-effectiveness chat, however I think we need to move beyond prioritisation of the roll-out of health interventions at the global/donor level. While there is a place for generalised cost-effectiveness analysis, on the whole it’s better to think of cost-effectiveness not as a generalisable property of a technology but as a product of the technology and the context. There is no doubt malaria vaccination (R21 or RTS,S) will be highly cost-effective in some populations and less so in others. The same goes for bed nets and SMC due to mosquito type, seasonality and many other factors. Plus, as others have said, the original post too often treats malaria interventions as alternatives when, in many cases, more important malaria planing will be about where to focus interventions rather than which technology to chose.
There are two broad ways to approach linking cost-effectiveness with context: i) large-scale multi-country, multi-intervention, cost-effectiveness analyses at national or sub-national level. There is a place for this and many organisations do great work in this space, including Givewell. ii) Countries undertake their own analyses and set priorities compared to other healthcare investment options, not so much other malaria interventions both other healthcare. For highly endemic regions, it could well be that nets and vaccination are both very cost-effective and that an MoH should prioritise this over, say, certain oncology treatments. If the MoH still accepts donated vaccine or earmarked vaccine financing (and why wouldn’t it), then the donors aren’t really buying more people vaccinated, they’re buying more people treated for cancer—which might still be a positive contribution, but not what they think they are buying and not the best option.
From where I’m sitting, it seems that global health is rightly moving away from campaigns to back single technologies or disease areas and looking at how to provide less fragmented and more foundational support. That’s not as easy a story to tell to philanthropic donors but there’s increasing recognition of the harm that hundreds of separate earmarked offers of support for this technology or that disease area does. Even in most low-income countries, the government is the main funder of health services and those looking to help should identify appropriate auxiliary programmes to support, not displace or distract.
(Note all the above refers to *roll-out* of vaccines. For R&D a best-guess overall cost-effectiveness estimate makes sense and so does intervention-focused advocacy/engagement.)
Tl;dr it is usually better to consider cost-effectiveness of technologies as context-dependent, rather than an property of the technology, and we should consider systems support over financing technologies that would/should have been bought with domestic financing anyway. We need to empower local planners to consider the full range of health investment options, not try to decide for them.
--
A few bonus comments on some questions in the post:
“Is there a reason why including a mixture of cost-effective and somewhat less cost-effective interventions together could be a high-impact outcome?”
- Yes, for example if the combination of interventions brings local R below 1. But it might also be less than expected too. Effectiveness for infectious diseases is non-linear so it’s better to simulate transmission effects. This has been done by several groups.
“Will the cost-effectiveness of bednets/SMC decline over time (e.g. because of diminishing marginal returns of some sort)? If so, might vaccine cost-effectiveness overtake it? “
- Yes, insecticide or drug resistance can result in declining effectiveness of bed nets or prophylaxis.
And on the age-factor of bed net vs vx cost-effectiveness—I would certainly expect this to be taken into account in all decent modelling so would be wary of over correcting in botecs.
The burden of malaria as it exists right now, exists even amidst the decades of use of all the available interventions (including ITNs and chemotherapies). Is there any progression of To add to that, compliance in the use of the ITNs is also challenge (several reports exists on how ITNs in Africa end up being used for fishing, or how some resell them. The ITNs reduce all cause mortality by 17% and the RTS’S vaccine reduces mortality by 13% and the projection for R21 is expected to be higher when the data becomes available.
I am not 100% certain that I follow all the math. However, when it comes to the malaria burden and which intervention is most cost effective, the general rule I’d apply is the permanence of the intervention. An intervention such as the use of ITNs may appear cost effective if you view it in a period of maximum 3 years. But because such an intervention is not permanent, the need to replenish ITN supplies to households repeatedly means that nets are a recurring cost. This implies that the actual cost of ITNs can’t fully be quantified in the sense of how long malaria as a disease will continue to exist. Vaccines on the other hand are more permanent and have a predictive cost even when measured against waning immunity.
Vaccines will bring us closer towards herd immunity and potentially the elimination of malaria and vaccines must be used complimentary to other interventions in the malaria toolkit.
Thanks Zacharia, I think you make some interesting points but I disagree with much of the substance here. My overall point would be, that when we talk about cost-effectiveness a lot of it is a question of math, so I don’t think its necessarily helpful to try and apply overarching “rules” or “principles” to cost-effectiveness calculations. Better I think to just take all the steps as dispassionately as possible needed to make an accurate estimate, leaving pre-determined principles out of it.
But to respond to your specific points...
The current vaccine efficacy seems to start waning after 18 months, so I’m not sure why you are talking about vaccines being a more “permanent” solution. The current vaccines need to be “replenished” similar to mosquito nets. Interestingly though 3ish years of protection either from nets or a vaccine might be enough to get a child through the most malaria-deadly period of their life.
Even if the vaccines lasted a lot longterm I don’t think “permanence” is a good general rule to apply to see how cost-effective an intervention might be. What matters more is how many lives will be saved (or DALYs averted), and how much the intervention will cost. Obviously the longer an intervention lasts (the more permanent it is) the more likely the intervention to be cost-effective.
I don’t think the current vaccines with their mediocre efficacy bring us anywhere near even the most liberal application of “herd immunity” (and I don’t think anyone is claiming they do). Sure with new close to 100% effective vaccines we could start thinking about herd immunity and the insanely high potential cost-effectiveness of elimination but there’s nothing even on the horizon which is yet even close to this good.
As much as I am for malaria vaccine, I feel it does not solve the issues caused by mosquitoes. Where does the vaccines solution leave the pain caused by mosquito sting? The bed nets protect us from both malaria and physical pain caused by mosquito bite.
How long do the effects of the vaccine last / how was this incorporated into your estimates?
My naive guess is that this should make vaccines more cost effective on the time scale of years (since you won’t need to re-vaccinate people), but maybe it’s already incorporated into the efficacy estimate?
With the current vaccines where protection starts waning after 18 months, I’m not sure there is a huge benefit in terms of “longevity” vs. nets. And yes I would be fairly sure they have incorporated this into the efficacy estimate although I haven’t checked!
Thanks Nick that’s helpful!
My two cents, leaning (weakly) towards there being a danger of comparing apples to oranges, and ‘red teaming’ being more valuable when more data is available.
Much of the cost of the malaria vaccine will be borne by governments or non-EA international aid (current $25 cost-estimate of a programme may also include committed govt health spend on vaccination infrastructure)
afaik the marginal cost per impact of a hypothetical EA vaccine rollout funding program is currently unknown and likely lower than current total cost per vaccination estimate
other organizations involved already fund many interventions with weaker cost/benefit and ring fence causes, so them diverting funding to malaria vaccination may be significantly net positive even if better interventions exist
Cost effectiveness has already been a consideration for those involved in the programmes.(most notably the Gates Foundation dropped out of funding RTS’S primarily based on cost-effectiveness related concerns. I am not sure of their position on R21 or if it is still evolving). So it hasn’t been neglected and public-facing critical analysis may already be available.
It’s unlikely ‘red teaming’ medical trials will uncover the sort of methodological flaws present in less rigorous RCTs and analysis of more speculative causal relationships.
Vaccination is intended to be a complement rather than a substitute for nets/SMC and an idealised antimalarial intervention should include both.
BOTEC calculation based on above figures $27 for an 88% reduction; likely to compare very favourably with most global health interventions
I suspect most of the funding for the vaccination program won’t be diverted from bednets or other extremely cost effective interventions, and organisations which backed vaccine development continue to spend on bed nets
The efficacy criteria are not equivalent as the 55% of [severe] malarial cases not prevented by nets represent cases where the nets have no effect (e.g. daytime infections) whereas the vaccination may still have had a positive immune effect on the the 23% of clinical malarial cases still diagnosed post-vaccination. Long term economic and health impacts of a vaccine attenuating [still severe] malaria infection even where symptoms exist are outside the scope of evidence collected, but plausibly significant.
Malaria vaccination is a new programme involving many non-EA stakeholders and facing numerous political and logistical challenges to roll out. Caution is needed to avoid “red team” material being misinterpreted as arguing R21 is ineffective or poor value relative to the median global healthcare intervention rather than relative to the most cost-effective interventions studied; perfect can be the enemy of good. Current evidence suggests the interventions are cost effective relative to many global health programs with public support, and on a cost-neutral basis R21 appears significantly more effective than other standalone antimalarial interventions
Downside risk is greater if the preliminary data overstates marginal costs or underestimates long-term benefit (plausibly both)
Likelihood and salience of misinterpretation lower (and benefit of analysis of more up-to-date data/estimates higher) after govt commitments made, with research more likely to be appropriately used by EAs exploring a specific donation option
I would update the other way if there are major institutional donors ready but unwilling to commit in the absence of thorough independent study. Ideally, these are the sort of donors that ought to be willing to fund you too!
(Disclaimer: not a sector expert. I may help 1Day Sooner with their request for ad hoc research task support in the near future but opinions and any errors are entirely mine)
Executive summary: The cost-effectiveness of malaria vaccines like R21 is uncertain and potentially much worse than bednets or seasonal malaria chemoprevention. More research is warranted before advocating for rollout.
Key points:
Malaria vaccines may cost over $25 per person protected versus under $3 for bednets, with similar efficacy. This 10x difference merits investigation.
Key uncertainties include future costs as production scales, efficacy duration, and impact on severe malaria. Interaction effects with nets are also unclear.
An updated cost-effectiveness analysis is proposed, focusing on R21 versus nets, incorporating new data, but avoiding full red-teaming.
Even if less cost-effective, advocacy might be reasonable if it unlocks additional funding or enables elimination.
More evidence is needed before confidently advocating for rollout over alternative interventions.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.