Is GiveWell underestimating the health value of lead eradication?
[EDIT: after responses from GiveWell in the comments, I believe that the answer to the question in the title is “NO”. I have left the post up for future reference, but now believe that GiveWell’s rough estimate for the health value of lead eradication is a very good guess.]
In 2021, GiveWell decided to support Pure Earth, an organization working on reducing lead exposure in low- and middle income countries. In support of this grant, GiveWell modeled the impacts of the grant on the future incomes of the beneficiaries in detail. They also made a “rough guess” about potential health impacts of the grant, adjusting the total benefits up by 20% (arguing that “we thought it was unlikely they would change our decision to recommend this grant”). In this post I argue that health benefits of lead poisoning are likely to make up a ~2x larger share of the total than the 16.7% implied by GiveWell’s rough guess (derived by taking 20%/(100%+20%)), and that lead abatement therefore can be somewhat more cost effective.
Why work to reduce lead exposure?
For a quick introduction to how lead is harmful, consider reading explainers by Our World in Data or Vox, or a more detailed report by Rethink Priorities or Pure Earth. In short, lead harms cognitive development in children, reducing their future expected earnings, and it increases rates of cardiovascular disease, leading to increased health burdens and premature deaths.
According to Attina (2013), the economic costs in lower- and middle income countries (LMICs) corresponded to 1.2% of global GDP in 2011 (with most concentrated impacts in Africa at >4% of regional GDP). According to the IHME Global Burden of Disease database (accessed through Our World in Data), the disease burden from lead exposure was ~22 Mn DALYs in 2022 (of which 94% is in LMICs), which compares to 2.5 Bn DALYs lost in total (so around 0.9% of the global burden).
There is uncertainty around these numbers, and the Rethink Priorities Lead Exposure Report has argued that the health burden could be 30-100% larger while the economic burden is only 30-50% of the estimated magnitude. However, for this article, I’ll assume that the numbers above are correct. Furthermore, I’ll assume that the split between economic impacts and health impacts for the most effective interventions corresponds to the global aggregate split; though I briefly discuss in the summary that the most effective interventions may actually have a skew towards more economic benefits.
Estimating the relative impacts on health and incomes
Let’s start by estimating the value, within the GiveWell framework, of avoiding the health burden linked to lead exposure. GiveWell measures the value of improved health in units of doubling consumption, which makes it a bit hard to translate directly from a DALY number to a $ number. I will therefore use $100k/DALY value that the Open Philanthropy GHW team uses, assuming that the two organizations are closely value-aligned. This leads to a potential value of $2.2 Tn from avoiding the health impacts of lead poisoning, of which ~$2.0 Tn are linked to LMICs. There is some delay between when GiveWell would fund an intervention, and when the beneficiaries would experience better health, so this value should be discounted using GiveWell’s 4% discount rate. However, “about 99% of the amount of lead taken into the body of an adult will leave in the waste within a couple of weeks”, according to Barry (1975), so this is unlikely to change the value by a lot.
By comparison, the economic value of eradicating lead poisoning in LMICs could be around 1.20% of GDP, or around $1.1 Tn using the World Bank estimate of $94 Tn global GDP in 2021 in nominal terms. Since the 1.20% number is from 2011, this is likely slightly too low—the health burden has increased by 3% since 2011, and the economic burden has likely increased faster, since LMICs are growing faster than the global average. One further nuance is which currency you choose to use—with ~9% inflation in the US this year, it matters which year you’re using as a reference for your dollar estimates. For now, I will generously assume that these factors bring the economic value up to $1.5 Tn/year.
As mentioned, GiveWell focuses on logarithmic increases in income, which means that each dollar is worth more if it is used to increase incomes of the globally poor. How poor are people affected by lead poisoning? Based on World Bank data on GDP per capita and the Global Disease Burden data on lead poisoning health burden, I estimate that the typical person affected by lead poisoning has an income of $3.8-6.7k per year. The higher number represents the weighted average GDP per capita, using the total disease burden in each country as weights, and assuming that lead exposure affects a representative sample of people within each country. However, poor people may be more effected by lead exposure, and this estimate does not capture the variance in incomes among the affected people. As a rough proxy, I therefore also calculated the weighted average logarithmic income of the affected people, resulting in the $3.8k number. These numbers are roughly one order of magnitude less than the $50k number used as a benchmark by Open Philanthropy.
You may now think that this would drive the economic value of eradicating lead poisoning to ~$15 Tn or so (using the order-of-magnitude scaling factor to the $1.5 Tn number; if instead comparing $50k to the exact values of $3.8-6.7k yields a range of $11.2-19.7 Tn), but economic gains from eradicating lead poisoning are lagged by many years. GiveWell assume that it takes 15 years for a child exposed to lead to join the workforce, and that the impacts are distributed over 40 years. Using a 4% discount rate, this means that the average value of a dollar 15-55 years into the future is just $0.27. Therefore, I estimate that the economic value of eradicating lead is around $4 Tn (0.27* $15 Tn).
Summary
Based on the calculations above, eradication of lead exposure in LMICs would be worth roughly $4 Tn/year in economic outcomes and $2 Tn/year in improved health outcomes. Health outcomes are therefore ~33% of the overall value linked to lead exposure eradication, or a ~50% uplift compared to only the economic value.[1]
If the higher multiplier of 50% had been used instead, the estimated cost effectiveness of the Pure Earth grant would have increased from 18x to ~24x that of cash transfers. By comparison, the best AMF program is estimated at a cost-effectiveness of ~15x cash transfers. Lead exposure can therefore be cost competitive with top GiveWell interventions regardless of which multiplier is used for the value of health outcomes, but the difference to AMF is ~3x larger if the higher health value is used.
- ^
Note that this corresponds to what GiveWell estimates in a separate model, where they argue that the total burden of exposure to lead leads to 94 Mn of their units of value, of which 31 Mn (which corresponds exactly to the former number of 33% of the value, or a 50% uplift compared to the economic value) are linked to health outcomes.
I’m not sure why the two separate GiveWell models use different relative shares for health benefits and economic benefits; it’s possible that different lead abatement interventions may have slightly different profiles (e.g., lead pollution in communities with many kids may have higher economic returns per DALY averted, since the future incomes of a larger number of kids is affected; furthermore, these interventions are likely to be among the most cost-effective ones), but I have not seen GiveWell or others argue this.
This is Adam Salisbury, a senior research associate at GiveWell, responding from GiveWell’s EA Forum account. Thank you for taking the time to engage with our work. Changes to how much we value the health benefits of lead reduction could change how much we could recommend funding to charities working in this area in the future, and the conceptual approach you lay out is both sensible and easy to follow.
I wanted to clarify how we came to the previous adjustment figure, in case it provides helpful context, and then raise a question about your methodology.
Our approach to estimating the health benefits of lead reduction
Our 120% adjustment is a best guess based on i) baseline mortality from lead exposure and ii) the hypothetical value derived from the Pure Earth intervention averting 5%, 15%, or 30% of these deaths. We thought we could get a more accurate estimate if we spent more time looking at the lead mortality/morbidity literature, but we ultimately decided not to do this, as we did not think this parameter would make a difference to this particular grant decision. This was because the economic benefits alone put Pure Earth above our ‘bar’ of cost-effectiveness.
The decision not to dive deeper was broadly reflective of GiveWell’s research approach: in general, the amount of time we spend researching a parameter is roughly proportional to how critical we think that parameter is in influencing our bottom-line estimate/overall decision. We think this helps us to prioritize our time effectively, by keeping us focused on the areas most relevant to our funding decisions.
Question on the approach you recommended
With this said, if we were to consider more lead grants in the future, it is possible that the economic/qualitative arguments would change, which might make us want to dig deeper on the health impacts of lead exposure. For this reason, feedback such as this is really useful to us. On this note, I have a question about your methodology:
I have tried to replicate your back-of the-envelope calculation here, and have a query related to your approach to discounting. In sum: I don’t think it’s necessary to discount the economic benefits calculated on line 17, because discount rates are already ‘baked-in’ to the 1.2% of GDP estimate this figure derives from.
As I understand it, the 1.2% estimate comes from Attina & Trasande (2013), who estimate the economic burden of lead exposure by: i) estimating baseline blood lead levels (BLLs) in LMICs; ii) estimating the relationship between BLLs and IQ; iii) estimating the relationship between IQ and lifetime economic productivity (LEP); iv) applying this to LEP estimates across LMICs, to ‘back out’ lost LEP due to lead exposure. Their LEP estimates are benchmarked against estimates from the US, which “assume annual growth in productivity of 1% and a 3% discount rate” (page 3).
In other words, I think the 1.2% figure comes from the following equation:
This is admittedly not easy to infer from the text, but I think is clearer in the Supplementary Appendix, where they report lost LEP per cohort for each LMIC (see Table S2). In any case, I think that because discount rates enter the numerator of this equation, we don’t want to further discount the 1.2% figure, since then we’d be ‘double-discounting’ future benefits.
If I don’t discount the economic benefits in line 17, then I get an implied adjustment for health benefits of 115% (line 24). This is broadly similar to the 120% adjustment we had before. Again, this could easily change if we decided to dip deeper into this parameter in the future.
I could easily be missing something, so please let me know if you disagree with my interpretation of Attina and Trasande (2013). And thanks again for your post – we really appreciate the feedback.
Hi Adam! Thanks for the detailed reply. From a brief look at your model, it seems you’ve understood my reasoning in this post correctly. I had indeed overlooked that their numbers were already discounted.
However, since they use a 3% discount rate and you use a 4% discount rate, you would still need to adjust for the difference. If we still assume that the economic impacts hit throughout your entire career, from 15 to 60 years into the future (note: 15 years into the future is not the average, but the initial year of impacts!), then you get to around $0.7 of NPV for each $1 today—much better than the $0.28 in my analysis, but still less than the $1 without discounting. Using this number, the result would be very close to GiveWell’s 20% estimate. How curious!
Best,
Jakob
Thanks for the flag, Jakob!
This is Miranda Kaplan, a communications associate with GiveWell—apologies for the slight delay in responding. Thanks for your engagement with our work! We’re looking into this and may post a detailed response here later depending on our capacity.
Thank you, Miranda!
I’m curious, why apply a 4% discount rate?
You can see their rationale in their public model: https://docs.google.com/spreadsheets/d/1tytvmV_32H8XGGRJlUzRDTKTHrdevPIYmb_uc6aLeas/edit#gid=1362437801
It’s the sum of 1.7% “improving circumstances over time”, 0.9% “compounding non-monetary benefits” and 1.4% “temporal uncertainty”. They have 0.0% “pure time preference”