Alex here, responding to your comment. Thank you for taking the time to give us this feedback!
In response to some of your specific points:
You’re right that we should have characterized the results from Lång and Nystedt (2018) as mixed rather than positive. Thanks for pointing out that mistake. We will update the spreadsheet so that study is correctly color-coded, and update the relevant part of the post. With this adjustment, among the studies we looked at, 3 suggest decreasing effects over time, 2 suggest increasing effects over time, and 5 show mixed effects. This still doesn’t seem like it adds up to strong evidence for either increasing or decreasing effects, so my prior of a flat effect over time remains the same.
We excluded Duflo et al. 2021 because it didn’t appear to include much about life cycle impacts on income from the intervention. It does report some increases in income for women in the treatment group between 2019 and 2020. However, I’d be reluctant to interpret that as evidence for increases over adulthood, because it represents only one year and because it compares pre-COVID results with results during COVID, which means other factors are probably at play.
That said, I agree that a more in-depth analysis might lead to a different prior for how we should expect early-life health interventions to affect income over the life cycle. We didn’t prioritize an in-depth analysis for this adjustment, but we would be open to more work to create a better-informed prior of deworming’s income effects over time. This would require deeper engagement with the studies we looked at to better understand their methodologies, relevance to deworming, and other factors. At the moment, it’s not a high-priority project for GiveWell staff, but we’re considering an external partnership to explore this further. We imagine that having a better grasp on how income effects change over time could inform our analysis not just of deworming but also of other programs we support, including vitamin A supplementation and seasonal malaria chemoprevention.
We’ll continue to share here if more work on this leads us to further updates.
Hi, Joel,
Alex here, responding to your comment. Thank you for taking the time to give us this feedback!
In response to some of your specific points:
You’re right that we should have characterized the results from Lång and Nystedt (2018) as mixed rather than positive. Thanks for pointing out that mistake. We will update the spreadsheet so that study is correctly color-coded, and update the relevant part of the post. With this adjustment, among the studies we looked at, 3 suggest decreasing effects over time, 2 suggest increasing effects over time, and 5 show mixed effects. This still doesn’t seem like it adds up to strong evidence for either increasing or decreasing effects, so my prior of a flat effect over time remains the same.
We excluded Duflo et al. 2021 because it didn’t appear to include much about life cycle impacts on income from the intervention. It does report some increases in income for women in the treatment group between 2019 and 2020. However, I’d be reluctant to interpret that as evidence for increases over adulthood, because it represents only one year and because it compares pre-COVID results with results during COVID, which means other factors are probably at play.
That said, I agree that a more in-depth analysis might lead to a different prior for how we should expect early-life health interventions to affect income over the life cycle. We didn’t prioritize an in-depth analysis for this adjustment, but we would be open to more work to create a better-informed prior of deworming’s income effects over time. This would require deeper engagement with the studies we looked at to better understand their methodologies, relevance to deworming, and other factors. At the moment, it’s not a high-priority project for GiveWell staff, but we’re considering an external partnership to explore this further. We imagine that having a better grasp on how income effects change over time could inform our analysis not just of deworming but also of other programs we support, including vitamin A supplementation and seasonal malaria chemoprevention.
We’ll continue to share here if more work on this leads us to further updates.
Best,
Alex