This is a really impressive report! Looking at 5 different risk-aversion models and applying them to 7 different interventions is an extraordinarily ambitious task, but I think you really succeeded at it. Almost every question or potential objection I came up with as I was reading ended up getting answered within the text, and Iâm very excited to try using some of these models in my own research.
I wanted to also highlight how helpful it was that you took the time to examine simplified cases and ground some of the results in easy-to-understand terminology. To give just a few examples:
The worked example of applying DMREU to a simplified analysis on pages 35-36 was really helpful for actually understanding how DMREU works
Continuing to explain risk aversion levels in terms of the n% chance of saving 1000 lives vs. 100% chance of saving 10 lives example was very helpful for understanding what different results meant in practice
The example in Table 13 was great for building some intuition on what different values of the WLU coefficient implied
I did also have a few minor comments:
You used DALYs as a unit of utility throughout this analysis. I think thatâs fine for this kind of initial report, but itâs something I would be concerned about if it persisted in future inter-cause cost-effectiveness comparison models that were actually used to make funding decisions. DALY weights (since the GBD 2010 update) are designed to be strictly measures of health status, not of utility.[1] My understanding is that interventions that have huge utility benefits for an individual (e.g. pain management in terminal cancer patients) can have little-to-no impact on the DALY weights of their conditions. For that reason, Iâm worried about the potential incoherence of trying to translate the benefits of animal welfare interventions into DALY terms, and I think in general that DALY-based CEAs may be very error-prone. I donât think any of those concerns really impact the conclusions of this particular report, but it would have been nice to see some mention of this issue in the limitations section.
This may just be me, but I found it quite hard to follow some of the acronyms. In particular, I found that not having the W for âweightedâ in REU and DMREU meant that I kept having to go back and look them up. It might be worth considering using RWEU and DMRWEU as the acronyms instead (but definitely see what others say, this is just from n=1).
For the current DALY burden of malaria, you say:
âThe estimate for the annual DALY burden of malaria used in the REU model comes from Our World in Dataâs Burden of Disease dashboard, which places the estimate at 63 million DALYs per year for malaria and neglected tropical diseases. (I assume that this is equal to the DALY burden of just malaria, which is likely inaccurate but probably correct to within an order of magnitude). I add uncertainty to this estimate by modeling the DALY burden of malaria as a normally distributed variable with a mean of 63 million DALYs and a standard deviation of 5 million DALYsâ
The raw data is available from the IHME website, with the point estimate being 46,437,811 DALYs due to malaria and the range being 23,506,933 to 80,116,072.
This is not to say that the pre-2010 DALY weights were good measures of utility, just that the post-2010 weights are explicitly not trying to measure it
This is a really impressive report! Looking at 5 different risk-aversion models and applying them to 7 different interventions is an extraordinarily ambitious task, but I think you really succeeded at it. Almost every question or potential objection I came up with as I was reading ended up getting answered within the text, and Iâm very excited to try using some of these models in my own research.
I wanted to also highlight how helpful it was that you took the time to examine simplified cases and ground some of the results in easy-to-understand terminology. To give just a few examples:
The worked example of applying DMREU to a simplified analysis on pages 35-36 was really helpful for actually understanding how DMREU works
Continuing to explain risk aversion levels in terms of the n% chance of saving 1000 lives vs. 100% chance of saving 10 lives example was very helpful for understanding what different results meant in practice
The example in Table 13 was great for building some intuition on what different values of the WLU coefficient implied
I did also have a few minor comments:
You used DALYs as a unit of utility throughout this analysis. I think thatâs fine for this kind of initial report, but itâs something I would be concerned about if it persisted in future inter-cause cost-effectiveness comparison models that were actually used to make funding decisions. DALY weights (since the GBD 2010 update) are designed to be strictly measures of health status, not of utility.[1] My understanding is that interventions that have huge utility benefits for an individual (e.g. pain management in terminal cancer patients) can have little-to-no impact on the DALY weights of their conditions. For that reason, Iâm worried about the potential incoherence of trying to translate the benefits of animal welfare interventions into DALY terms, and I think in general that DALY-based CEAs may be very error-prone. I donât think any of those concerns really impact the conclusions of this particular report, but it would have been nice to see some mention of this issue in the limitations section.
This may just be me, but I found it quite hard to follow some of the acronyms. In particular, I found that not having the W for âweightedâ in REU and DMREU meant that I kept having to go back and look them up. It might be worth considering using RWEU and DMRWEU as the acronyms instead (but definitely see what others say, this is just from n=1).
For the current DALY burden of malaria, you say:
âThe estimate for the annual DALY burden of malaria used in the REU model comes from Our World in Dataâs Burden of Disease dashboard, which places the estimate at 63 million DALYs per year for malaria and neglected tropical diseases. (I assume that this is equal to the DALY burden of just malaria, which is likely inaccurate but probably correct to within an order of magnitude). I add uncertainty to this estimate by modeling the DALY burden of malaria as a normally distributed variable with a mean of 63 million DALYs and a standard deviation of 5 million DALYsâ
The raw data is available from the IHME website, with the point estimate being 46,437,811 DALYs due to malaria and the range being 23,506,933 to 80,116,072.
This is not to say that the pre-2010 DALY weights were good measures of utility, just that the post-2010 weights are explicitly not trying to measure it