This is a really comprehensive effort, and I strongly support EAs getting more in the weeds with expert reports like the DCP. However, I downvoted this post because I worry it goes about this effort in the wrong way. This is a truly dizzying number of proposed interventions, and the extent of argumentation for them seems to be citing a $/DALY number from DCP3. I think it would be a lot more transparent if you elaborated more on what exactly DCP3 was evaluating, where that evidence is from, and how credible the evidence base is (one study? ten?) Here’s what could go wrong without any of these:
If OpenPhil were to begin funding one of these interventions but in a different form than DCP3 evaluated, that could end up being much less cost effective. For example, a proposal like “smoking prevention programs” is not a helpful level of specificity, since there can obviously be more effective and less effective smoking prevention programs.
Without knowing where the evidence is from, OpenPhil could end up recommending it in places where the contextual factors that made the intervention work are not present. That would make the intervention much less cost effective than advertised.
Making massive funding allocations based off a single study, or worse a single small scale study, is obviously not a good idea. The intervention might not scale appropriately—e.g. it worked in a lab study or with a small set of highly trained doctors, but not when you roll it out to thousands of hospitals with less well trained doctors. We aren’t always able to have gold standard evidence, but we should at least know what we are working with so we can decide how much to revise down our credences.
Providing the necessary context for these $/DALY numbers is the heart of cost-effectiveness analysis, in my opinion. $/DALY numbers aren’t handed down to us from heaven, and we need to know a lot more about interventions than a single number in order to evaluate whether we should fund them.
That said, these interventions do look really promising and I’m sure the numbers have some basis. If you elaborated in detail on a few of them I think this could be a really excellent contribution to our understanding of what interventions to prioritize.
what exactly DCP3 was evaluating, where that evidence is from, and how credible the evidence base is (one study? ten?)
DCP3 was evaluating the DALY cost-effectiveness of interventions. You may be familiar but just to restate, DALY is the sum of years of life lost and equivalent of years of life lost due to living with disability (Vos et al., 2020, p. 1431). The Institute of Health Metrics and Evaluation took the first term of the sum from national vital registration records and disability weights from surveys that focused on one’s ability to perform tasks similar to those that others do (p. 473). A limited sample of people was asked and the weights assume death as the worst state. So, DALY can be understood as a weighted sum of medical conditions where survival and ability to perform tasks are the objectives.
The evidence for various recommendation varies, some are based on a single study and some on meta-analyses and multiple studies. When a $/DALY range is specified, then it seems to me that a meta-analysis or several high-quality studies were used. For point estimates, such as $12.88/DALY for hernia repair with a bednet (vol. 1, 161), only one study can be cited. I assume that not only RCTs are considered but are strongly preferred, for example vol. 8, p. 156 shows a table with evidence for outcomes from 1, 2, or 3 RCTs.
funding one of these interventions but in a different form than DCP3 evaluated, that could end up being much less cost effective … not a helpful level of specificity
I agree. For example, info on returns to education can provide 43 additional years of schooling for $100 spent (p. 33) or 0.23. Where the relationship between outputs and DALYs averted is understood, depending on several factors, then top cost-effective programs can be found by the cost and the factors. Programs where the output-DALY relationship is not known, such as the target health product ads, would need to be evaluated on a small scale before larger investment is made.
I could broadly say that experts who studied the output-DALY values should estimate the cost-effectiveness of the former program types and experts on geography, method, and covered health topics should estimate the cost-effectiveness of the latter type.
If you elaborated in detail on a few of them I think this could be a really excellent contribution to our understanding of what interventions to prioritize.
That makes sense. Also, especially for the programs that OPP is considering. If the GAVI program that is used as an example is considered, then going through the references to estimate the cost-effectiveness of individual vaccines in different settings, and trying to understand the likely development of pharmaceutical company drug provision commitments, also depending on philanthropic funding of distribution, research, and manufacturing, can be valuable. I further assume that several OPP employees read the collection and use the cited studies for grant evaluation in funded cause areas (such as alcohol taxation). However, I do not believe that the volumes have been read in a way to come up with innovative solutions that cover multiple interventions, such as the four which I am suggesting (that includes, for example surrogate alcohol regulation in addition to taxation of recorded alcohol), which have not been extensively evaluated. So, I think that my greatest marginal contribution can be to suggest these programs.
This is a really comprehensive effort, and I strongly support EAs getting more in the weeds with expert reports like the DCP. However, I downvoted this post because I worry it goes about this effort in the wrong way. This is a truly dizzying number of proposed interventions, and the extent of argumentation for them seems to be citing a $/DALY number from DCP3. I think it would be a lot more transparent if you elaborated more on what exactly DCP3 was evaluating, where that evidence is from, and how credible the evidence base is (one study? ten?) Here’s what could go wrong without any of these:
If OpenPhil were to begin funding one of these interventions but in a different form than DCP3 evaluated, that could end up being much less cost effective. For example, a proposal like “smoking prevention programs” is not a helpful level of specificity, since there can obviously be more effective and less effective smoking prevention programs.
Without knowing where the evidence is from, OpenPhil could end up recommending it in places where the contextual factors that made the intervention work are not present. That would make the intervention much less cost effective than advertised.
Making massive funding allocations based off a single study, or worse a single small scale study, is obviously not a good idea. The intervention might not scale appropriately—e.g. it worked in a lab study or with a small set of highly trained doctors, but not when you roll it out to thousands of hospitals with less well trained doctors. We aren’t always able to have gold standard evidence, but we should at least know what we are working with so we can decide how much to revise down our credences.
Providing the necessary context for these $/DALY numbers is the heart of cost-effectiveness analysis, in my opinion. $/DALY numbers aren’t handed down to us from heaven, and we need to know a lot more about interventions than a single number in order to evaluate whether we should fund them.
That said, these interventions do look really promising and I’m sure the numbers have some basis. If you elaborated in detail on a few of them I think this could be a really excellent contribution to our understanding of what interventions to prioritize.
Thank you for your suggestions.
DCP3 was evaluating the DALY cost-effectiveness of interventions. You may be familiar but just to restate, DALY is the sum of years of life lost and equivalent of years of life lost due to living with disability (Vos et al., 2020, p. 1431). The Institute of Health Metrics and Evaluation took the first term of the sum from national vital registration records and disability weights from surveys that focused on one’s ability to perform tasks similar to those that others do (p. 473). A limited sample of people was asked and the weights assume death as the worst state. So, DALY can be understood as a weighted sum of medical conditions where survival and ability to perform tasks are the objectives.
The evidence for various recommendation varies, some are based on a single study and some on meta-analyses and multiple studies. When a $/DALY range is specified, then it seems to me that a meta-analysis or several high-quality studies were used. For point estimates, such as $12.88/DALY for hernia repair with a bednet (vol. 1, 161), only one study can be cited. I assume that not only RCTs are considered but are strongly preferred, for example vol. 8, p. 156 shows a table with evidence for outcomes from 1, 2, or 3 RCTs.
I agree. For example, info on returns to education can provide 43 additional years of schooling for $100 spent (p. 33) or 0.23. Where the relationship between outputs and DALYs averted is understood, depending on several factors, then top cost-effective programs can be found by the cost and the factors. Programs where the output-DALY relationship is not known, such as the target health product ads, would need to be evaluated on a small scale before larger investment is made.
I could broadly say that experts who studied the output-DALY values should estimate the cost-effectiveness of the former program types and experts on geography, method, and covered health topics should estimate the cost-effectiveness of the latter type.
That makes sense. Also, especially for the programs that OPP is considering. If the GAVI program that is used as an example is considered, then going through the references to estimate the cost-effectiveness of individual vaccines in different settings, and trying to understand the likely development of pharmaceutical company drug provision commitments, also depending on philanthropic funding of distribution, research, and manufacturing, can be valuable. I further assume that several OPP employees read the collection and use the cited studies for grant evaluation in funded cause areas (such as alcohol taxation). However, I do not believe that the volumes have been read in a way to come up with innovative solutions that cover multiple interventions, such as the four which I am suggesting (that includes, for example surrogate alcohol regulation in addition to taxation of recorded alcohol), which have not been extensively evaluated. So, I think that my greatest marginal contribution can be to suggest these programs.