Generally I really find this research agenda interesting. I have only skimmed this post, but I also like your analysis the way you go about it.
One nitpick:
As it turns out, LSD, psilocybin, and DMT all get rid of Cluster Headaches in a majority of sufferers. Given the safety profile of these agents, it is insane to think that there are millions of people suffering needlessly from this condition who could be nearly-instantly cured with something as simple as growing and eating some magic mushrooms.
I think this is hyperbole. I reviewed the literature a while ago, and while I do agree that there is some suggestive evidence that this is true, I do not think that it is so strong as to warrant the claims you make and there are many qualifications. Also, I think you should cite the relevant studies on this subject (https://scholar.google.com/scholar?as_ylo=2015&q=LSD+cluster+headaches&hl=en&as_sdt=0,5).
Thank you. The survey said that 68% of sufferers who have used psychedelics found gave them a rating of 4 or 5, where 5 means “They have completely eliminated the cluster headaches”. I would certainly stand by the claim that “there are millions of people suffering needlessly from this condition who could be nearly-instantly cured with something as simple as growing and eating some magic mushrooms.” We’ve interviewed people for whom sub-hallucinogenic doses of DMT and psilocybin took a 10⁄10 pain CH all the way to a 1⁄10 or 0⁄10. And in the case of DMT, due to its method of administration, this takes place within seconds (more than one but less than 10). Even if this works only for, say, 20% of the sufferers, it is still millions in absolute terms.
Thanks for saying: “Generally I really find this research agenda interesting”. My experience has been that few people take seriously the long-tails of pleasure and pain. This is precisely the sort of missing piece of information that can add an entire new wing to EA.
For what it’s worth, this seems like a pretty big deal to me if it were true. Is there any quick QALY estimate or similar for how much things can be improved if everyone had quick access to DMT or similar?
One quick thought: it could be neat experiment to make $2k of Facebook ads to target people with these issues, pointing to a specific webpage the discusses how these people could get treatment. That said, I of course realize some may not be legal, so it could be tricky.
In the article specifically about N,N-DMT as a possible treatment for CHs, Quintin added a rough QALY calculation (I should add that any QALY estimate concerning CHs and other ultra-painful disorders will typically severely underestimate the value of the interventions, given the logarithmic nature of pain scales):
While we believe that traditional metrics such as the QALY do not accurately capture the suffering caused by a cluster headache (see upcoming post on the true pain/pleasure scale), a rough QALY calculation would be as follows (focusing on chronic cluster headache sufferers rather than average, since they compromise up to 83% of total headaches[3]):
Facebook AD campaign:
1. An estimated 370,000 Americans suffer from cluster headaches, 68% of whom are on Facebook[4] (=251,000). About 15% of these suffer from chronic cluster headaches (=37,740). According to Sprout Social, the average estimated cost per click of an ad campaign is $1.72. Assuming 1⁄10 who click are cluster headache sufferers, to reach all chronic sufferers would take (ballpark) $650,000.
2. Assuming about 30% of those who view the ad will pursue the treatment (rough estimate-those who put 2 or less on survey results for questions of legality, difficulty to acquire etc.) and that in 68% of cases it cured or nearly cured their clusters (based on survey results), then the resulting increase in QALYs would be (37,740 people * 0.3 * 0.68) * [0.760 (QALY coefficient) * 1 QALY – ( −0.429 (QALY coefficient)* (0.47QALY)) [5]] = $650,000/7, 404QALY = $87.70 per QALY.
3. These ads could also be targeted to users in countries where psilocybin and DMTare legal for use recreationally, increasing conversion rate. Further targeting could be done on Facebook groups (and other social media groups) which are associated with cluster headache treatment.
One quick way to get people to not take you seriously is with a bad cost effectiveness estimate. There’s a much bigger risk of doing a sloppy/overconfident job than benefit of having a high number at the end of it (in EA circles). Also, there is a reputation of these estimates to both produce amazing numbers and also be very wrong, so while I support attempts, I’d also recommend lots of clarification, hedging, and consideration of ways the number could be poor. I think the default expectation is for the number to not be great; but even if the median isn’t good, it’s possible upon further investigation it could be better than expected, which could be quite worthwhile.
“to reach all chronic sufferers” → I’d recommend targeting 30%-60% of sufferers. The last several percent would be much more expensive.
I’m quite skeptical of the click → cure stats in particular. For-profit websites often have a 1% rate of people who go from click → purchase, and this could be a pretty significant amount of work to purchase.
Is this equation taking into account that the “cure” could last for many years? Would the result be in “QALYs per year”?
I’m sure you’ve answered this elsewhere, but why the American focus? Would it be possible in India or similar?
This estimation seems like something that Charity Entrepreneurship would have a lot more experience in. The program seems quite similar to some of their others.
Generally I really find this research agenda interesting. I have only skimmed this post, but I also like your analysis the way you go about it.
One nitpick:
I think this is hyperbole. I reviewed the literature a while ago, and while I do agree that there is some suggestive evidence that this is true, I do not think that it is so strong as to warrant the claims you make and there are many qualifications. Also, I think you should cite the relevant studies on this subject (https://scholar.google.com/scholar?as_ylo=2015&q=LSD+cluster+headaches&hl=en&as_sdt=0,5).
Thank you. The survey said that 68% of sufferers who have used psychedelics found gave them a rating of 4 or 5, where 5 means “They have completely eliminated the cluster headaches”. I would certainly stand by the claim that “there are millions of people suffering needlessly from this condition who could be nearly-instantly cured with something as simple as growing and eating some magic mushrooms.” We’ve interviewed people for whom sub-hallucinogenic doses of DMT and psilocybin took a 10⁄10 pain CH all the way to a 1⁄10 or 0⁄10. And in the case of DMT, due to its method of administration, this takes place within seconds (more than one but less than 10). Even if this works only for, say, 20% of the sufferers, it is still millions in absolute terms.
Thanks for saying: “Generally I really find this research agenda interesting”. My experience has been that few people take seriously the long-tails of pleasure and pain. This is precisely the sort of missing piece of information that can add an entire new wing to EA.
For what it’s worth, this seems like a pretty big deal to me if it were true. Is there any quick QALY estimate or similar for how much things can be improved if everyone had quick access to DMT or similar?
One quick thought: it could be neat experiment to make $2k of Facebook ads to target people with these issues, pointing to a specific webpage the discusses how these people could get treatment. That said, I of course realize some may not be legal, so it could be tricky.
Still seems worth it, FB might just eventually ban. ( I sort of doubt anything would happen if you link to an informational infographic)
In the article specifically about N,N-DMT as a possible treatment for CHs, Quintin added a rough QALY calculation (I should add that any QALY estimate concerning CHs and other ultra-painful disorders will typically severely underestimate the value of the interventions, given the logarithmic nature of pain scales):
Thanks!
Some quick thoughts:
One quick way to get people to not take you seriously is with a bad cost effectiveness estimate. There’s a much bigger risk of doing a sloppy/overconfident job than benefit of having a high number at the end of it (in EA circles). Also, there is a reputation of these estimates to both produce amazing numbers and also be very wrong, so while I support attempts, I’d also recommend lots of clarification, hedging, and consideration of ways the number could be poor. I think the default expectation is for the number to not be great; but even if the median isn’t good, it’s possible upon further investigation it could be better than expected, which could be quite worthwhile.
“to reach all chronic sufferers” → I’d recommend targeting 30%-60% of sufferers. The last several percent would be much more expensive.
I’m quite skeptical of the click → cure stats in particular. For-profit websites often have a 1% rate of people who go from click → purchase, and this could be a pretty significant amount of work to purchase.
Is this equation taking into account that the “cure” could last for many years? Would the result be in “QALYs per year”?
I’m sure you’ve answered this elsewhere, but why the American focus? Would it be possible in India or similar?
This estimation seems like something that Charity Entrepreneurship would have a lot more experience in. The program seems quite similar to some of their others.
I’d suggest reading up on the mini-fiasco of the leafletting research, if you haven’t yet. Just make sure not to make some of the mistakes made around that. Some context: https://animalcharityevaluators.org/blog/ace-highlight-updated-leafleting-intervention-report/ https://acesounderglass.com/2015/04/24/leaflets-are-ineffective-tell-your-friends/ https://medium.com/@harrisonnathan/the-problems-with-animal-charity-evaluators-in-brief-cd56b8cb5908
Consider using Guesstimate for clarity, but I’m biased :)
Kudos for the efforts, and good luck!