Thanks for the analysis! I’m enthusiastic about the lead reduction work Pure Earth and LEEP have been doing.
Some points for consideration:
(1) It may be that GBD underestimates the health burden—specifically, the loss in cognitive ability—insofar as the GBD examines the increased risk of intellectual disability and the consequence burden; but there is also a case to be made that loss of cognitive ability by the overall population that nonetheless does not put one below the threshold of intellectual disability would still be a significant loss that a fuller analysis would want to take into consideration.
(2) I would agree that 8 years is conservative. Generally, to discount for counterfactual introduction, I’m a fan of a applying an annual discount when calculating the aggregate multi-year benefits, rather than coming up with an estimate of “years brought forward”—partly because it’s easier to calculate (by using data on past introductions of a policy etc, and comparing that against the total number of country-years in which introduction could have potentially happened), but also partly because I worry that the “bringing forward” model may overestimate the benefit if your other temporal discounts are significant enough. Given the lack of past policies on adulterated tumeric elimination, I would probably use the rate of introduction of anti-lead paint regulations as a baseline (and adjusting downwards given dissimilarities). In any case, I doubt the discount will be >=1%, since even soda taxes (a far more widespread policy) is barely being introduced at that rate.
(3) A quick analysis (regressing DALY burden per capita of lead exposure in Bangladesh over time) suggests (surprisingly, to me) potential growth over time; this may be worth looking more into, and taking into consideration on top of the population growth effect.
(4) I think my biggest concern is just the intervention effect size given (a) reliance on pre-post data with significant potential for endogeneity and confounding. (b) I also suspect the intervention is disproportionately effective here given lead exposure in Bangladesh is extraordinarily high (9th higher per capita).
(5) Final thing that jumped out at me is the issue of how lead poisoning isn’t reversible (at least from quickly scanning the old GBD Comparative Quantification of Health Risks report, so I don’t think modelling the intervention as having an impact after 30 years makes sense—rather, the benefit trickles in on a year-by-year basis, as new cohorts are (counterfactually) not poisoned, and have better health/income outcomes. I can’t say for sure without having done a deeper analysis, but I suspect this would improve your results, because frontloading benefits tend to increase the overall cost-effectiveness when discounts are high (and you have that 4% discount from GiveWell).
Hope that’s helpful, and do let me know if you want additional inputs!
Thanks for the analysis! I’m enthusiastic about the lead reduction work Pure Earth and LEEP have been doing.
Some points for consideration:
(1) It may be that GBD underestimates the health burden—specifically, the loss in cognitive ability—insofar as the GBD examines the increased risk of intellectual disability and the consequence burden; but there is also a case to be made that loss of cognitive ability by the overall population that nonetheless does not put one below the threshold of intellectual disability would still be a significant loss that a fuller analysis would want to take into consideration.
(2) I would agree that 8 years is conservative. Generally, to discount for counterfactual introduction, I’m a fan of a applying an annual discount when calculating the aggregate multi-year benefits, rather than coming up with an estimate of “years brought forward”—partly because it’s easier to calculate (by using data on past introductions of a policy etc, and comparing that against the total number of country-years in which introduction could have potentially happened), but also partly because I worry that the “bringing forward” model may overestimate the benefit if your other temporal discounts are significant enough. Given the lack of past policies on adulterated tumeric elimination, I would probably use the rate of introduction of anti-lead paint regulations as a baseline (and adjusting downwards given dissimilarities). In any case, I doubt the discount will be >=1%, since even soda taxes (a far more widespread policy) is barely being introduced at that rate.
(3) A quick analysis (regressing DALY burden per capita of lead exposure in Bangladesh over time) suggests (surprisingly, to me) potential growth over time; this may be worth looking more into, and taking into consideration on top of the population growth effect.
(4) I think my biggest concern is just the intervention effect size given (a) reliance on pre-post data with significant potential for endogeneity and confounding. (b) I also suspect the intervention is disproportionately effective here given lead exposure in Bangladesh is extraordinarily high (9th higher per capita).
(5) Final thing that jumped out at me is the issue of how lead poisoning isn’t reversible (at least from quickly scanning the old GBD Comparative Quantification of Health Risks report, so I don’t think modelling the intervention as having an impact after 30 years makes sense—rather, the benefit trickles in on a year-by-year basis, as new cohorts are (counterfactually) not poisoned, and have better health/income outcomes. I can’t say for sure without having done a deeper analysis, but I suspect this would improve your results, because frontloading benefits tend to increase the overall cost-effectiveness when discounts are high (and you have that 4% discount from GiveWell).
Hope that’s helpful, and do let me know if you want additional inputs!