I’ll quickly mention a couple of things that stuck out to me that might make the CEA potentially overoptimistic:
IQ points lost per μg/​dl of lead—this is likely a logarithmic relationship (as suggested by Bernard and Schukraft). For a BLL of 2.4 − 10 μg/​dl, IQ loss from 1 μg/​dl increase may be close to 0.5, but above 10, it’s closer to 0.2 per 1 μg/​dl increase, and above 20, closer to 0.1. Average BLL in Bangladesh seem to be around 6.8 μg/​dl, though amongst residents living near turmeric sources of lead, it could plausibly be (much) higher, and thus a lower IQ gain will be had from a 1 μg/​dl reduction in lead.
Income loss per IQ points loss—The CEA assumes that 1 IQ point loss leads to a 2.1% reduction in income. However, some work by GiveWell (here) suggests this might be closer to 0.67% (and there might be some reasons to discount this further, e.g. due to replicability concerns)
Replicability of intervention—as noted in the text, it’s hard to estimate how much the Bangladesh program reduced lead exposure by. If Bangladesh’s average BLL level is around 6.8 μg/​dl, then a 1.64 reduction from the intervention implies this intervention cut BLL by 25% for half of children in Bangladesh. This is great, but I can see several reasons why this may not be informative of future programs’ cost-effectiveness
Maybe Turmeric is much more prevalent in rural Bangladesh than other regions
Maybe it was unusually easy to get regulators to agree to introduce /​ enforce standards
Maybe it was unusually easy to get producers to switch away from lead chromate
Each of these reasons on their own is fairly weak, but the likelihood of at least one being true gives us reason to discount future cost-effectiveness analyses. More generally, we might expect some regression to the mean w.r.t reducing exposure from tulmeric—maybe everything went right for this particular program, but this is unlikely to be true in future programs. To be clear, there are likely also reasons that this analysis is too pessimistic, and thus on net it may be the case that cost-effectiveness remains at $1/​ DALY (or even better). Nonetheless, I think it’s good to be cautious, since $1 /​ DALY implies this program was >800x better than cash transfers and >80x better than GiveWell’s top charities—a strong claim to make (though still possible!)
Thanks so much for your response! This is excellent feedback and we’re grateful for the interest.
1. IQ loss: We did go to the experts to estimate based on actual numbers, so we’re reasonably confident about this. And even if it were out by 50 percent, the results are still strong.
2. Income loss by IQ has been a moving target in the literature. It could be at that lower end—we used a mid-point of studies. Again, even if we are out by half, the results are positive.
3. Replicability is certainly an issue. We do know that turmeric is used throughout the country anecdotally, and that other sources of lead are important in Bangladesh (aluminum pots, contaminated sites, perhaps fish). Additionally, we know that no other interventions have been undertaken—expect for a small site clean up that will not impact nationally. We’re happy to report that the government is very engaged and committed, and have asked WB for their support for these other sources. These conditions might be less available in other countries, including Central Asia, India (northern states), Balkans (some) Middle East (some), and North Africa. Not all countries have the issue, but many do. So, yes, viability of replication of this solution will be mixed. On the plus side, a similar intervention in Georgia had the same rapid result, and without a large expense. With that, we hope to continue to repeat the success.
All, please do send more thoughts and comments! We’re new to this kind of assessment, and need your feedback!
This looks very exciting, thanks for posting!
I’ll quickly mention a couple of things that stuck out to me that might make the CEA potentially overoptimistic:
IQ points lost per μg/​dl of lead—this is likely a logarithmic relationship (as suggested by Bernard and Schukraft). For a BLL of 2.4 − 10 μg/​dl, IQ loss from 1 μg/​dl increase may be close to 0.5, but above 10, it’s closer to 0.2 per 1 μg/​dl increase, and above 20, closer to 0.1. Average BLL in Bangladesh seem to be around 6.8 μg/​dl, though amongst residents living near turmeric sources of lead, it could plausibly be (much) higher, and thus a lower IQ gain will be had from a 1 μg/​dl reduction in lead.
Income loss per IQ points loss—The CEA assumes that 1 IQ point loss leads to a 2.1% reduction in income. However, some work by GiveWell (here) suggests this might be closer to 0.67% (and there might be some reasons to discount this further, e.g. due to replicability concerns)
Replicability of intervention—as noted in the text, it’s hard to estimate how much the Bangladesh program reduced lead exposure by. If Bangladesh’s average BLL level is around 6.8 μg/​dl, then a 1.64 reduction from the intervention implies this intervention cut BLL by 25% for half of children in Bangladesh. This is great, but I can see several reasons why this may not be informative of future programs’ cost-effectiveness
Maybe Turmeric is much more prevalent in rural Bangladesh than other regions
Maybe it was unusually easy to get regulators to agree to introduce /​ enforce standards
Maybe it was unusually easy to get producers to switch away from lead chromate
Each of these reasons on their own is fairly weak, but the likelihood of at least one being true gives us reason to discount future cost-effectiveness analyses. More generally, we might expect some regression to the mean w.r.t reducing exposure from tulmeric—maybe everything went right for this particular program, but this is unlikely to be true in future programs. To be clear, there are likely also reasons that this analysis is too pessimistic, and thus on net it may be the case that cost-effectiveness remains at $1/​ DALY (or even better). Nonetheless, I think it’s good to be cautious, since $1 /​ DALY implies this program was >800x better than cash transfers and >80x better than GiveWell’s top charities—a strong claim to make (though still possible!)
Thanks so much for your response! This is excellent feedback and we’re grateful for the interest.
1. IQ loss: We did go to the experts to estimate based on actual numbers, so we’re reasonably confident about this. And even if it were out by 50 percent, the results are still strong.
2. Income loss by IQ has been a moving target in the literature. It could be at that lower end—we used a mid-point of studies. Again, even if we are out by half, the results are positive.
3. Replicability is certainly an issue. We do know that turmeric is used throughout the country anecdotally, and that other sources of lead are important in Bangladesh (aluminum pots, contaminated sites, perhaps fish). Additionally, we know that no other interventions have been undertaken—expect for a small site clean up that will not impact nationally. We’re happy to report that the government is very engaged and committed, and have asked WB for their support for these other sources. These conditions might be less available in other countries, including Central Asia, India (northern states), Balkans (some) Middle East (some), and North Africa. Not all countries have the issue, but many do. So, yes, viability of replication of this solution will be mixed. On the plus side, a similar intervention in Georgia had the same rapid result, and without a large expense. With that, we hope to continue to repeat the success.
All, please do send more thoughts and comments! We’re new to this kind of assessment, and need your feedback!
Best,
Pure Earth Team