Preliminary Analysis of Intervention to Reduce Lead Exposure from Adulterated Turmeric in Bangladesh Shows Cost Benefit of About US$1 per DALY

[Kate Porterfield]

Pure Earth is a GiveWell Grantee dedicated to reducing lead exposure in low- and middle-income countries. In collaboration with Stanford University and the International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b), a preliminary cost-effectiveness analysis (CEA) was performed to assess the effectiveness of an intervention in Bangladesh. The CEA presents an encouraging outlook, with a cost per disability-adjusted life year (DALY)-equivalent averted estimated at just under US$1. As this assessment is preliminary, it may contain methodological inconsistencies with GiveWell’s. As such, we welcome any comments and corrections.

---

In 2019, after investigations concluded that turmeric was the primary source of lead exposure among residents of rural Bangladesh, Stanford University and Bangladeshi non-profit icddr,b embarked on a mission to eliminate lead poisoning from turmeric. Stanford and icddr,b’s investigations had revealed that lead chromate (an industrial pigment sometimes called “School Bus Yellow”) was being added to turmeric roots to make them more attractive for sale. Armed with this evidence, the team coordinated with the Bangladeshi Food Safety Authority to conduct a crack-down of the adulteration by enforcing policies at the markets and raising awareness among businesspeople and the public nationwide. These efforts successfully halted the practice of adding lead chromate to turmeric: the prevalence of lead in turmeric dropped from 47% in 2019 to 0% in 2021. In collaboration with Pure Earth, icddr,b continues to monitor turmeric and other spices and coordinate with government agencies to maintain the safety of these and other food products.

To gauge the effectiveness of this program in advancing the mission of reducing lead exposures globally, it is important to assess both impact and cost-effectiveness. To approach this task, Pure Earth and Stanford have completed a back-of-the-envelope cost-effectiveness analysis (CEA), incorporating preliminary data from blood lead level assessments and various assumptions. This model is built off of previous models created by LEEP and Rethink Priorities.

The preliminary findings are that this program can avert an equivalent DALY for just under $1. This result is extraordinary, albeit deserving of further scrutiny. It indicates that certain interventions in the lead space could be enormously cost-effective. The body of work to reduce lead exposures in LMICs is nascent, and not all interventions are likely to be as cost-effective as spices. But clearly, more work on spices is called for, and Pure Earth, Stanford, icddr,b, and others are pursuing funding to expand these programs into other countries.

Program Implementation Costs

To establish a framework for cost-effectiveness assessment, it is essential to define the terms “cost” and “effectiveness” within the context of the Stanford-led mission. The concept of “cost” encompasses the resources utilized by the project team and those expended by the Bangladesh government as a direct result of the project’s activities. Specifically, we consider monetary expenses incurred by the program, which we estimate to be upfront costs of $360,000. These expenses include both the costs to identify the sources of lead exposure and implement the program, as well as continuing costs of $100,000 to monitor and continue the program after the initial implementation. Additionally, the Government of Bangladesh is expected to spend $100,000 over the course of the intervention.

To facilitate comparisons with other global health interventions, we define the project’s”effectiveness” as the prevention of negative outcomes resulting from its efforts. Lead contamination adversely affects children in two primary ways: first, it harms their physical health, leading to premature death or physical disabilities; and second, it permanently impairs their cognitive abilities, resulting in decreased productivity and lower earnings over their lifetimes.

To quantify the first effect, the World Health Organization employs a metric called “disability-adjusted life years″ (DALYs), where each unit represents the loss of one full year of healthy life. The second effect is evaluated by measuring the income loss, which we convert into DALY-equivalents. In line with GiveWell’s approach, we equate one DALY to 2.8 years of income. Therefore, our effectiveness measurement relies on the total number of DALY-equivalents averted through the project’s interventions.

DALY-Equivalents

Through our program, we anticipate a significant reduction in the consumption of lead through turmeric, consequently lowering the number of children in Bangladesh who are exposed to lead each year. This, in turn, contributes to a decrease in the total number of DALY-equivalents attributed to lead poisoning. However, even in the hypothetical scenario where the project had not intervened, it is theoretically possible (though highly unlikely) that Bangladesh would have eventually enforced the existing regulations on leaded additives to spices and achieved similar benefits over time.

Therefore, the project’s impact lies not in identifying and enforcing food safety regulations, but rather in expediting its implementation by several years. In the model we use a very conservative estimate of this acceleration to be 8 years. The practice of adding lead chromate had been going on for over four decades and no signs were seen of it stopping without this intervention. By “averting” DALY-equivalents, we mean reducing the number of DALY-equivalents that would have otherwise been incurred if our efforts had not intervened, thereby minimizing the overall burden of lead-related health and cognitive impairments.

Cost Effectiveness Analysis

To determine the number of DALY-equivalents that will be prevented by the program, we rely on data and modeling derived from two prominent research studies. The Global Burden of Disease Study 2019, provides insights into the impact of lead poisoning on physical health, while a widely cited 2013 study assesses the economic consequences in low- and middle-income countries (LMICs).

The project measured blood lead levels among 1,398 women and children prior to the program implementation, and again after the turmeric lead levels reached zero. Fortunately, the preliminary results indicate a sizable reduction of 30% in blood lead levels (unpublished data).

Converting these reductions in lead exposures, our calculations indicate we can expect to avert approximately 1,000,000 DALY-equivalents in Bangladesh. Among these, around 20% (200,000 DALYs) can be attributed to improved health outcomes, while the remaining 80% (800,000 DALY-equivalents) stem from increased income. It is important to note that these figures are time-discounted, meaning that the value of future benefits is slightly diminished.

Simultaneously, we estimate that expenditures for the same period will amount to approximately $560,000. We determine the roughly estimated cost-effectiveness of the project’s Bangladesh program to be just under $1 per DALY-equivalent averted.

Lastly, we can consider a cost-effectiveness of just under US$1 per DALY-equivalent in the broader context of programs within global health and development. Take, for example, GiveDirectly, one of GiveWell’s top-rated charities. Like icddr,b and partners, it raises the incomes of people in low-income countries, but does so by giving unconditional cash transfers. Applying a unit conversion to a CEA by GiveWell, we find that GiveDirectly has a cost-effectiveness of approximately $836 per DALY-equivalent averted. (It’s important to bear in mind that this comparison is not exactly like for like, since GiveDirectly’s program has been studied extensively and is operating at a much larger scale).

Where could our analysis go wrong?

Exploring the uncertainties in our model enables decision-makers to assess the reliability of cost-effectiveness estimates and determine whether further information is needed to address existing knowledge gaps. Here, we outline three significant uncertainties that warrant attention.

Firstly, the model relies on determining the proportion of lead exposure reduction that is attributable to this program. Uniquely for this program we do have direct measures on lead reduction in various areas of Bangladesh, but this is pre-post data, and at the end of the day we have had to make an estimate for what we believe is the amount of lead exposure reduction attributable to this program. Similarly we have had to make estimates for how much lead reduction is occurring in areas where we do not have direct measures of lead reduction. For now, we have taken the mean reduction of 1.64 μg/​dl reduction in blood lead levels measured in the study area populations and have estimated that this effect happens across 50 percent of the population of Bangladesh. We do know, however, that there are no other active lead mitigation programs in place in Bangladesh at the moment (although some are in planning). We believe this to be a conservative estimate, but recognize this is not a very precise figure in our model.

Secondly, there is uncertainty regarding the number of years by which this project halted the contamination in advance of other future efforts. We chose 8 years as an estimate since, to our knowledge, no other entities were addressing the issue and the problem had existed for over 40 years before we intervened. We also believe this is a conservative estimate.

Furthermore, like any cost-effectiveness analysis that models changes over time, the choice of time discount rate significantly impacts the results. We apply a 4% per year time discount for future costs and benefits, following the GiveWell approach. Given that some effects of lead exposure prevention, such as income improvements, occur in the distant future, the chosen time discount rate can considerably alter the outcomes.

Conclusion

Despite being a preliminary assessment, this cost-effectiveness analysis (CEA) of this intervention in Bangladesh presents an exceptionally encouraging outlook, with a cost per DALY-equivalent averted estimated at just under US$1. It is crucial not to overlook the profound significance of this outcome: US$1 represents a small investment for the equivalent of an additional year of life in optimal health.

Early results from Pure Earth’s Rapid Market Assessment project find that between 6 and 12 countries may have similar problems with contaminated spices. Large parts of northern India (also highly populated) are similarly affected. Other lead salts are also highly colored, in reds and oranges, and found in other products. Programs to halt intentional contamination of spices and other foodstuffs are enormously impactful, and ought to be a first response in the fight against lead poisoning globally.

Finally, other significant sources of lead exposure (including leaded pottery and aluminum cookware, paint, medicines etc) require a similar regulatory response, and are likely to show cost benefit ratios that are also very strong.

Acknowledgements

A special thanks to Erik Hausen (especially), Jenna Forsyth, Steve Luby, James Snowden, and Rich Fuller for their contributions to this post.

[Tom Barnes]

This looks very exciting, thanks for posting!

I’ll quickly mention a couple of things that stuck out to me that might make the CEA potentially overoptimistic:

1. IQ points lost per μg/​dl of lead—this is likely a logarithmic relationship (as suggested by Bernard and Schukraft). For a BLL of 2.4 − 10 μg/​dl, IQ loss from 1 μg/​dl increase may be close to 0.5, but above 10, it’s closer to 0.2 per 1 μg/​dl increase, and above 20, closer to 0.1. Average BLL in Bangladesh seem to be around 6.8 μg/​dl, though amongst residents living near turmeric sources of lead, it could plausibly be (much) higher, and thus a lower IQ gain will be had from a 1 μg/​dl reduction in lead.

2. Income loss per IQ points loss—The CEA assumes that 1 IQ point loss leads to a 2.1% reduction in income. However, some work by GiveWell (here) suggests this might be closer to 0.67% (and there might be some reasons to discount this further, e.g. due to replicability concerns)

3. Replicability of intervention—as noted in the text, it’s hard to estimate how much the Bangladesh program reduced lead exposure by. If Bangladesh’s average BLL level is around 6.8 μg/​dl, then a 1.64 reduction from the intervention implies this intervention cut BLL by 25% for half of children in Bangladesh. This is great, but I can see several reasons why this may not be informative of future programs’ cost-effectiveness

   1. Maybe Turmeric is much more prevalent in rural Bangladesh than other regions

   2. Maybe it was unusually easy to get regulators to agree to introduce /​ enforce standards

   3. Maybe it was unusually easy to get producers to switch away from lead chromate

Each of these reasons on their own is fairly weak, but the likelihood of at least one being true gives us reason to discount future cost-effectiveness analyses. More generally, we might expect some regression to the mean w.r.t reducing exposure from tulmeric—maybe everything went right for this particular program, but this is unlikely to be true in future programs. To be clear, there are likely also reasons that this analysis is too pessimistic, and thus on net it may be the case that cost-effectiveness remains at $1/​ DALY (or even better). Nonetheless, I think it’s good to be cautious, since $1 /​ DALY implies this program was >800x better than cash transfers and >80x better than GiveWell’s top charities—a strong claim to make (though still possible!)

[Kate Porterfield]

Thanks so much for your response! This is excellent feedback and we’re grateful for the interest.

1. IQ loss: We did go to the experts to estimate based on actual numbers, so we’re reasonably confident about this. And even if it were out by 50 percent, the results are still strong.

2. Income loss by IQ has been a moving target in the literature. It could be at that lower end—we used a mid-point of studies. Again, even if we are out by half, the results are positive.

3. Replicability is certainly an issue. We do know that turmeric is used throughout the country anecdotally, and that other sources of lead are important in Bangladesh (aluminum pots, contaminated sites, perhaps fish). Additionally, we know that no other interventions have been undertaken—expect for a small site clean up that will not impact nationally. We’re happy to report that the government is very engaged and committed, and have asked WB for their support for these other sources. These conditions might be less available in other countries, including Central Asia, India (northern states), Balkans (some) Middle East (some), and North Africa. Not all countries have the issue, but many do. So, yes, viability of replication of this solution will be mixed. On the plus side, a similar intervention in Georgia had the same rapid result, and without a large expense. With that, we hope to continue to repeat the success.

All, please do send more thoughts and comments! We’re new to this kind of assessment, and need your feedback!

Best,

Pure Earth Team

[Lizka]

I really appreciate cost-effectiveness analyses like this and am curating this post (despite the fact that it was shared a while back). (See also this comment & discussion.)

I’m also wildly impressed by the project itself (I highlighted the project in the recent EA Newsletter). See the story of the research project and other work behind this (let me know if you know of a better link!). Pure Earth’s work here seems incredible, and I’d be really interested in learning more about it.

[Joel Tan🔸]

Great curation. I would have missed this otherwise.

[Joel Tan🔸]

Thanks for the analysis! I’m enthusiastic about the lead reduction work Pure Earth and LEEP have been doing.

Some points for consideration:

(1) It may be that GBD underestimates the health burden—specifically, the loss in cognitive ability—insofar as the GBD examines the increased risk of intellectual disability and the consequence burden; but there is also a case to be made that loss of cognitive ability by the overall population that nonetheless does not put one below the threshold of intellectual disability would still be a significant loss that a fuller analysis would want to take into consideration.

(2) I would agree that 8 years is conservative. Generally, to discount for counterfactual introduction, I’m a fan of a applying an annual discount when calculating the aggregate multi-year benefits, rather than coming up with an estimate of “years brought forward”—partly because it’s easier to calculate (by using data on past introductions of a policy etc, and comparing that against the total number of country-years in which introduction could have potentially happened), but also partly because I worry that the “bringing forward” model may overestimate the benefit if your other temporal discounts are significant enough. Given the lack of past policies on adulterated tumeric elimination, I would probably use the rate of introduction of anti-lead paint regulations as a baseline (and adjusting downwards given dissimilarities). In any case, I doubt the discount will be >=1%, since even soda taxes (a far more widespread policy) is barely being introduced at that rate.

(3) A quick analysis (regressing DALY burden per capita of lead exposure in Bangladesh over time) suggests (surprisingly, to me) potential growth over time; this may be worth looking more into, and taking into consideration on top of the population growth effect.

(4) I think my biggest concern is just the intervention effect size given (a) reliance on pre-post data with significant potential for endogeneity and confounding. (b) I also suspect the intervention is disproportionately effective here given lead exposure in Bangladesh is extraordinarily high (9th higher per capita).

(5) Final thing that jumped out at me is the issue of how lead poisoning isn’t reversible (at least from quickly scanning the old GBD Comparative Quantification of Health Risks report, so I don’t think modelling the intervention as having an impact after 30 years makes sense—rather, the benefit trickles in on a year-by-year basis, as new cohorts are (counterfactually) not poisoned, and have better health/​income outcomes. I can’t say for sure without having done a deeper analysis, but I suspect this would improve your results, because frontloading benefits tend to increase the overall cost-effectiveness when discounts are high (and you have that 4% discount from GiveWell).

Hope that’s helpful, and do let me know if you want additional inputs!

[Karthik Tadepalli]

I would not at all be surprised if lead elimination interventions (and lead in spices in particular) are better than GiveWell top charities. But the #1 concern I have is with tractability. Developing countries have lots of regulations on the books that are unenforced because of state capacity constraints. The most likely failure mode for a lead elimination intervention is that it fails to make governments actually enforce laws.

This looks like an exceptionally effective advocacy campaign where government regulators were convinced to spend a lot of time and money to enforce laws. I think the best thing Pure Earth could do is open up the black box of how exactly that happened.

• How did the Bangladesh government react when you approached them?

• How was the crackdown implemented? How much personnel did it require?

• How organized was this practice? Organized vendors adulterating spices could fight against regulators more effectively than disorganized vendors.

• What implementation support did you provide?

[SummaryBot]

Executive summary: An intervention to reduce lead exposure from adulterated turmeric in Bangladesh had an estimated cost per DALY-equivalent averted of just under US$1.

Key points:

1. Turmeric was found to be the primary source of lead exposure in rural Bangladesh due to intentional adulteration with lead chromate.

2. An intervention coordinated with the Bangladeshi government successfully reduced lead in turmeric to 0% from 47% previously.

3. A preliminary cost-effectiveness analysis estimates approximately 1 million DALY-equivalents will be averted at a cost of $560,000.

4. The estimated cost per DALY-equivalent averted is under $1, which is highly cost-effective compared to other global health interventions.

5. There are uncertainties around attributable lead reductions and timeframe, but the analysis provides an encouraging outlook on reducing lead exposures globally.

6. Further interventions on contaminated spices and other sources of lead exposure in low- and middle-income countries are likely to have similarly favorable cost-benefit ratios.

This comment was auto-generated by the EA Forum Team. Contact us if you have feedback.

[Benevolent_Rain]

I might be able to help with work in this field. In addition for it to be easier for me to travel to Bangladesh (family and No Visa Required status), some relevant networks including the government and some familiarity with the customs (and even rudimentary language) I also think I have some proof points that entrepreneurship is what I can offer to EA. Please reach out if thinking about doing something here.

[RayTaylor]

That’s brilliant—is the work being extended to other countries?

[RayTaylor]

That’s brilliant—is the work being extended to other countries?