I agree that mental health is an area worth looking into, and you make some good suggestions.
A few comments based on a pretty quick, first-pass reading of the post:
I’m not sure why you use 2013 GBD estimates. They are updated every year and 2017 figures are on the IHME website, and published in the Lancet – though I’d expect the general pattern to be similar. Note that GBD is likely to underestimate the burden of mental illness for various methodological reasons: see Vigo et al (2016).
A ketamine nasal spray and a psilocybin formulation have also been given ‘breakthrough drug’ status by the FDA, and as you mention there is ongoing research in the UK on psilocybin, so it seems like these will become prescribable for mental disorders within the next few years in some countries. I’m unsure how much additional benefit would come from further funding/campaigning on this issue, though rescheduling the substances to Schedule 2 or lower would certainly make research easier, and making them available in developing countries could take a lot of work. (Ketamine is already Schedule 2 in many countries, and there are clinics in the US [including San Fransisco], UK [including Oxford] and Canada that already offer intravenous ketamine therapy for depression).
It’s worth noting that psilocybin and other psychedelics can genuinely cure depression (and anxiety, OCD, addiction...), perhaps in a similar way to CBT, by breaking harmful thought patterns. My inexpert understanding is that ketamine is probably more like standard anti-depressants in that it ‘numbs the pain’ rather than solving the underlying causes (the most recent evidence suggests it works via opioid system activation, making the pain analogy even more apt), though it does lead to permanent remission in some cases, and works much faster with (usually) fewer side-effects. MDMA is sort of inbetween, not directly curing PTSD etc through some biochemical process, but enabling more effective psychological therapy, which can lead to a permanent cure.
You don’t seem to mention electroconvulsive therapy (ECT). My vague understanding is that it can help with schizophrenia and chronicpain as well as depression and is probably underused due to its associations with One Flew Over the Cuckoo’s Nest, etc.
“As the global development charities recommended by GiveWell are widely accepted as the most effective organisations to donate to at present, they are the natural point of comparison.” I suspect a minority of EAs think GiveWell orgs are the most effective, given the interest in far future, animals, etc. But I agree they are the most natural comparators.
Having skimmed the Strong Minds Phase 2 report, I’m skeptical of the claimed effect size, e.g. the control group was not randomly selected from the same population as the intervention group; I think it was from people who didn’t want group therapy, who may be different in all sorts of ways. They did take some steps to adjust for social desirability bias, but I suspect that’s also still an issue. More generally, things tend to look worse the more you examine the evidence, especially when that comes from an interested party. So I agree it’s a promising candidate but would probably assume a smaller effect size when making cost-effectiveness estimates.
GiveWell’s latest model says AMF averts a death for more like $4,000, or $4,500 after accounting for leverage and funging.
GW also says: “Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. For example, our model does not include the short-term impact of non-fatal cases of malaria prevented on health or productivity, prevention of other mosquito-borne diseases, or reductions in health care costs due to LLINs reducing the number of cases of malaria. It also does not include possible offsetting impacts or other harms. We do include possible developmental impacts on children who sleep under an LLIN.” So the true effect of AMF on happiness could be substantially greater than your estimate.
I’m increasingly skeptical of analyses that don’t try to account for long-term indirect effects/‘cluelessness’. It’s plausible that most value created/lost by most interventions is not captured by standard assessments, so the magnitude and even sign of their impact is often unclear. I’m not sure how best to deal with this though; Peter Hurford has made a few suggestions and I think there was a recent series of posts in this forum on the topic.
I wouldn’t assume the mean LS among AMF beneficiaries is the same as the national average – though this assumption may make your analysis more conservative, since I’d assume recipients of bed nets tend to have worse lives than the average.
I’m more skeptical now that mapping from affect to LS is a good approach, because they track different things, e.g. I’d expect someone with low affect because of poverty to report a lower LS than someone with the same level of affect due to mental illness, since (I think) LS responses tend to be more heavily influenced by objective circumstances.
Relatedly: I wouldn’t rely too heavily on LS as it doesn’t seem a great proxy for actual hedonic states, which are probably more important (even if you’re not a strict utilitarian). It might be the best available data so I don’t object to its being used in preliminary analyses, but it adds uncertainty to your claims. That said, I’d expect a shift to measuring affect, or some combination of affect and satisfaction, to favour mental health treatments, since mental illness has even greater impacts on mood than on LS.
Thanks for all these great points (Derek sent these to me privately and I suggested it would be valuable for him to share them here for other interested parties). My brief replies, in order, to those comments that weren’t just informative:
1. fair cop. I think I was lazily using those as I first compiled these numbers back in 2015 (at the start of my PhD).
2. agree it’s unclear what these breakthrough drugs imply for EA
5. it makes sense to compare to GW because that’s who our audience is. People who already think GW is irrelevant and focus on e.g. far future are unlikely to be interested in the analysis here.
6. yes, there are probably flaws in the SM analysis. I look forward to mine being made obsolete in due course. I note that my points on negative spillovers should cause us to downgrade the effectiveness of anti-poverty charities.
8. agree, but this applies to mental health intervention too: their effects could also be larger if we take spillovers into account, e.g. reduced strain on family who care for them.
9. As I’m sympathetic to person-affecting views, I’m not too concerned about the long-term anyway. Even if I were a long-termist, the problem with including indirect effects is that it tends to make the analysis incredibly ‘hand-wavey’ (“ah, saving lives speeds up growth, which is bad for climate change, etc.). I think it makes sense to calculate what can easily be calculated first. If you can’t look anywhere else, at least look under the lamppost.
10. Probably correct. A better analysis would factor in how the LS of AMF recipients would change over their lives (presumably upwards and societal conditions improve)
11. I agree LS is not the ideal thing. If we had affect scores, I would say we use those, but we don’t! (“slaves to the data” etc)
12. I also agree moving to affect would make mental health score better than poverty. I left that out because I thought the analysis was complicated enough already.
I agree that mental health is an area worth looking into, and you make some good suggestions.
A few comments based on a pretty quick, first-pass reading of the post:
I’m not sure why you use 2013 GBD estimates. They are updated every year and 2017 figures are on the IHME website, and published in the Lancet – though I’d expect the general pattern to be similar. Note that GBD is likely to underestimate the burden of mental illness for various methodological reasons: see Vigo et al (2016).
A ketamine nasal spray and a psilocybin formulation have also been given ‘breakthrough drug’ status by the FDA, and as you mention there is ongoing research in the UK on psilocybin, so it seems like these will become prescribable for mental disorders within the next few years in some countries. I’m unsure how much additional benefit would come from further funding/campaigning on this issue, though rescheduling the substances to Schedule 2 or lower would certainly make research easier, and making them available in developing countries could take a lot of work. (Ketamine is already Schedule 2 in many countries, and there are clinics in the US [including San Fransisco], UK [including Oxford] and Canada that already offer intravenous ketamine therapy for depression).
It’s worth noting that psilocybin and other psychedelics can genuinely cure depression (and anxiety, OCD, addiction...), perhaps in a similar way to CBT, by breaking harmful thought patterns. My inexpert understanding is that ketamine is probably more like standard anti-depressants in that it ‘numbs the pain’ rather than solving the underlying causes (the most recent evidence suggests it works via opioid system activation, making the pain analogy even more apt), though it does lead to permanent remission in some cases, and works much faster with (usually) fewer side-effects. MDMA is sort of inbetween, not directly curing PTSD etc through some biochemical process, but enabling more effective psychological therapy, which can lead to a permanent cure.
You don’t seem to mention electroconvulsive therapy (ECT). My vague understanding is that it can help with schizophrenia and chronic pain as well as depression and is probably underused due to its associations with One Flew Over the Cuckoo’s Nest, etc.
“As the global development charities recommended by GiveWell are widely accepted as the most effective organisations to donate to at present, they are the natural point of comparison.” I suspect a minority of EAs think GiveWell orgs are the most effective, given the interest in far future, animals, etc. But I agree they are the most natural comparators.
Having skimmed the Strong Minds Phase 2 report, I’m skeptical of the claimed effect size, e.g. the control group was not randomly selected from the same population as the intervention group; I think it was from people who didn’t want group therapy, who may be different in all sorts of ways. They did take some steps to adjust for social desirability bias, but I suspect that’s also still an issue. More generally, things tend to look worse the more you examine the evidence, especially when that comes from an interested party. So I agree it’s a promising candidate but would probably assume a smaller effect size when making cost-effectiveness estimates.
GiveWell’s latest model says AMF averts a death for more like $4,000, or $4,500 after accounting for leverage and funging.
GW also says: “Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. For example, our model does not include the short-term impact of non-fatal cases of malaria prevented on health or productivity, prevention of other mosquito-borne diseases, or reductions in health care costs due to LLINs reducing the number of cases of malaria. It also does not include possible offsetting impacts or other harms. We do include possible developmental impacts on children who sleep under an LLIN.” So the true effect of AMF on happiness could be substantially greater than your estimate.
I’m increasingly skeptical of analyses that don’t try to account for long-term indirect effects/‘cluelessness’. It’s plausible that most value created/lost by most interventions is not captured by standard assessments, so the magnitude and even sign of their impact is often unclear. I’m not sure how best to deal with this though; Peter Hurford has made a few suggestions and I think there was a recent series of posts in this forum on the topic.
I wouldn’t assume the mean LS among AMF beneficiaries is the same as the national average – though this assumption may make your analysis more conservative, since I’d assume recipients of bed nets tend to have worse lives than the average.
I’m more skeptical now that mapping from affect to LS is a good approach, because they track different things, e.g. I’d expect someone with low affect because of poverty to report a lower LS than someone with the same level of affect due to mental illness, since (I think) LS responses tend to be more heavily influenced by objective circumstances.
Relatedly: I wouldn’t rely too heavily on LS as it doesn’t seem a great proxy for actual hedonic states, which are probably more important (even if you’re not a strict utilitarian). It might be the best available data so I don’t object to its being used in preliminary analyses, but it adds uncertainty to your claims. That said, I’d expect a shift to measuring affect, or some combination of affect and satisfaction, to favour mental health treatments, since mental illness has even greater impacts on mood than on LS.
Thanks for all these great points (Derek sent these to me privately and I suggested it would be valuable for him to share them here for other interested parties). My brief replies, in order, to those comments that weren’t just informative:
1. fair cop. I think I was lazily using those as I first compiled these numbers back in 2015 (at the start of my PhD).
2. agree it’s unclear what these breakthrough drugs imply for EA
5. it makes sense to compare to GW because that’s who our audience is. People who already think GW is irrelevant and focus on e.g. far future are unlikely to be interested in the analysis here.
6. yes, there are probably flaws in the SM analysis. I look forward to mine being made obsolete in due course. I note that my points on negative spillovers should cause us to downgrade the effectiveness of anti-poverty charities.
8. agree, but this applies to mental health intervention too: their effects could also be larger if we take spillovers into account, e.g. reduced strain on family who care for them.
9. As I’m sympathetic to person-affecting views, I’m not too concerned about the long-term anyway. Even if I were a long-termist, the problem with including indirect effects is that it tends to make the analysis incredibly ‘hand-wavey’ (“ah, saving lives speeds up growth, which is bad for climate change, etc.). I think it makes sense to calculate what can easily be calculated first. If you can’t look anywhere else, at least look under the lamppost.
10. Probably correct. A better analysis would factor in how the LS of AMF recipients would change over their lives (presumably upwards and societal conditions improve)
11. I agree LS is not the ideal thing. If we had affect scores, I would say we use those, but we don’t! (“slaves to the data” etc)
12. I also agree moving to affect would make mental health score better than poverty. I left that out because I thought the analysis was complicated enough already.