Hi Nick,
Thank you for providing this feedback! My name is Vicky, and I am a Research Associate at GiveWell, on the vaccines team. We really appreciate these kinds of rough sense checks on our work and thought this was a great approach.
Our lookback includes children enrolled across multiple years of programming (roughly covering 2020 to 2026) whereas the enrollment figures in your estimate only include a single year of program operations.
Our lookback aims to roughly capture the impact of our grants which cover New Incentives’ operations from 2020 through 2026. The estimated 1.5 million children enrolled that you cite is based on a single operating year, 2023.[1] We think New Incentives will enroll closer to 6.3 million children between 2020 and 2026.[2]
Using the same assumption that roughly 81% of children enrolled would be counterfactually vaccinated absent the program,[3] we estimate that roughly 1.2 million children would be counterfactually vaccinated by the program from 2020 through 2026.[4]
This implies an upper bound of roughly 60,000 deaths averted with your approach (assuming 5% of deaths in unvaccinated children might be preventable as an upper bound).[5]
We think this difference–the assumed number of children enrolled with GiveWell funding—is the main reason the upper bound you estimated for the number of deaths averted appears significantly lower than the estimates in our lookback, although we’re still exploring other potential discrepancies between the numbers in your approach and our estimates.[6]
Thanks again for your engagement!
- ^
In 2023, New Incentives reported enrolling 1,518,904 children across 9 states. See New Incentives, 2023 Annual Report, p.8-9
- ^
We estimated this by taking the total amount of funding (roughly $120 million) divided by the cost per child enrolled (roughly $19 per child enrolled) between 2020 and 2024. This assumes that the cost per child enrolled between 2025 and 2026 will remain similar to the historical weighted average.
We have not published our most recent estimates yet, but you can see our approach to estimating cost per child enrolled based on data from 2020 through 2023 here.
New Incentives publishes enrollment figures here. When calculating the cost per child, we adjust their estimates to account for potential repeat enrollments.
- ^
The 81% in our public report is based on a single state, Bauchi, and the exact percentage differs across states depending on baseline coverage and New Incentives’ expected impact in that state. In addition, we’ve made some internal updates to the model since the last version of our intervention report was published.
- ^
6.3 million * (1 − 81%) = roughly 1.2 million children counterfactually vaccinated by the program.
- ^
1.2 million children counterfactually vaccinated * 5% risk of dying from causes that might be preventable through vaccination = 60,000 deaths potentially averted as an upper bound.
The 5% risk of dying parameter is derived from two assumptions in your post:
12.5% of unvaccinated kids might die from non-birth/neonatal and non-malarial causes.
40% of non-birth/neonatal and non-nalarial deaths could potentially be vaccine-preventable.
12.5% * 40% = 5%
- ^
Across states where the New Incentives program operates, we estimate that unvaccinated children experience roughly a 3% to 8% chance of dying from vaccine-preventable diseases and that vaccination reduces their risk of dying by roughly 50%, which appears more in-line with your estimates. For more on how we estimate these, see our public report here.
Hi Nick,
Thanks for noting that section of the post could have been clearer! We’ve edited the article to clarify that New Incentives went from serving 70,000 to 1.5 million children per year.
We agree that the extra lives saved (“indirect deaths” in our analysis) is an interesting question. Both the magnitude of the adjustment and the exact mechanisms (i.e., which other causes those deaths are coming from in the GBD bucket) are major sources of uncertainty in our model, and we don’t currently specify what other deaths are being averted through vaccination in our analysis. We may follow up with a post to share more about our work on indirect deaths in the future.
Thanks again for the feedback!