Here’s the full exchange between Özler and Haushofer:
JH: Whenever someone meekly suggests that one might not leave those with the lowest incomes entirely alone with their mental health struggles, the “it’s not that simple” brigade shows up and talks about the therapy-industrial complex and it’s so tiresome.
BO: Thanks, Johannes. That thread & and recommendation is outrageous: there’s no good evidence that Strong Minds is effective, let alone most effective. It’s 20-year old studies combined with pre-post data provided by SM itself. People should pay no attention to this 🧵, whatsoever.
BO: Show me one good published study of the impact of SM on the ground at some decent scale...
JH: My point is not that SM is the best available intervention. My point is that people who get upset at HLI for caring about wellbeing on the grounds that this ignores structural interventions are mistaken.
BO: I have zero problems with your point. It’s well taken & that’s why I thanked you for your tweet. My issue is w the unequivocal SM recommendation, the evidence presented, & the conviction with which it is advertised. People (unfortunately) donate based on such recommendations...
JH: Ok, one can disagree about that. Do you think the Bolton et al work is good? If yes what dissuades you from thinking that SM is effective? (Regardless of whether it should be top-ranked)
BO: Bolton et al. (2003) is not about SM. Why did you evaluate GD with an RCT when we could have extrapolated from Progresa?
JH: Of course there’s always room for more studies. But if someone had said at the time that GD is likely to be effective for certain outcomes based on work on Progresa, I wouldn’t have considered that claim outrageous. (Side point, I think IPT/SM more similar than Progresa/GD.)
JH: (Addendum: I’ve always been a bit uneasy with the hype around GD and our study in light of all the other evidence on cash transfers.)
BO: You would have if someone was claiming that it was exactly 6 times more effective per $ than the most promising alternative. Also, no one (give or take) would have donated to GD on the basis of previous evidence, say our study in Malawi...(which is much closer).
JH: GD definitely emphasized early on that cash transfers were very well-researched! Of course not sure if that helped, but I don’t recall any “how can you say that!” reactions. 1⁄2
JH: On the “6 times” claim: I think orgs like Givewell and HLI are in the business of providing guesstimates given limited and mixed-quality evidence. I have much more tolerance for extrapolation in that context than in an academic paper. 2⁄2
BO: I think smt got lost in translation here: my point is you can’t claim your NGO is best because it works on something promising. You have to show it—effectiveness at scale. That was true for GD and it remains true for SM. /1
BO: And you definitely should not make forest plots like that with guesstimates. It’s misleading.../FIN.
BO: I’m going to leave it at this. I don’t care for that type of hype right before the giving season, with shaky evidence. We all know the dangers of assuming effectiveness at scale from efficacy studies, especially 20-year-old ones...
JH: I definitely see that. I also think this shaky evidence is leagues better than what most people base their giving decisions on, so if someone listens, it’s a net improvement.
Here’s the full exchange between Özler and Haushofer:
JH: Whenever someone meekly suggests that one might not leave those with the lowest incomes entirely alone with their mental health struggles, the “it’s not that simple” brigade shows up and talks about the therapy-industrial complex and it’s so tiresome.
BO: Thanks, Johannes. That thread & and recommendation is outrageous: there’s no good evidence that Strong Minds is effective, let alone most effective. It’s 20-year old studies combined with pre-post data provided by SM itself. People should pay no attention to this 🧵, whatsoever.
JH: This dismissal seems much too strong to me. I thought HLI’s discussion of the evidence here was fair and reasonable: https://www.happierlivesinstitute.org/report/strongminds-cost-effectiveness-analysis
BO: Show me one good published study of the impact of SM on the ground at some decent scale...
JH: My point is not that SM is the best available intervention. My point is that people who get upset at HLI for caring about wellbeing on the grounds that this ignores structural interventions are mistaken.
BO: I have zero problems with your point. It’s well taken & that’s why I thanked you for your tweet. My issue is w the unequivocal SM recommendation, the evidence presented, & the conviction with which it is advertised. People (unfortunately) donate based on such recommendations...
JH: Ok, one can disagree about that. Do you think the Bolton et al work is good? If yes what dissuades you from thinking that SM is effective? (Regardless of whether it should be top-ranked)
BO: Bolton et al. (2003) is not about SM. Why did you evaluate GD with an RCT when we could have extrapolated from Progresa?
JH: Of course there’s always room for more studies. But if someone had said at the time that GD is likely to be effective for certain outcomes based on work on Progresa, I wouldn’t have considered that claim outrageous. (Side point, I think IPT/SM more similar than Progresa/GD.)
JH: (Addendum: I’ve always been a bit uneasy with the hype around GD and our study in light of all the other evidence on cash transfers.)
BO: You would have if someone was claiming that it was exactly 6 times more effective per $ than the most promising alternative. Also, no one (give or take) would have donated to GD on the basis of previous evidence, say our study in Malawi...(which is much closer).
JH: GD definitely emphasized early on that cash transfers were very well-researched! Of course not sure if that helped, but I don’t recall any “how can you say that!” reactions. 1⁄2
JH: On the “6 times” claim: I think orgs like Givewell and HLI are in the business of providing guesstimates given limited and mixed-quality evidence. I have much more tolerance for extrapolation in that context than in an academic paper. 2⁄2
BO: I think smt got lost in translation here: my point is you can’t claim your NGO is best because it works on something promising. You have to show it—effectiveness at scale. That was true for GD and it remains true for SM. /1
BO: And you definitely should not make forest plots like that with guesstimates. It’s misleading.../FIN.
BO: I’m going to leave it at this. I don’t care for that type of hype right before the giving season, with shaky evidence. We all know the dangers of assuming effectiveness at scale from efficacy studies, especially 20-year-old ones...
JH: I definitely see that. I also think this shaky evidence is leagues better than what most people base their giving decisions on, so if someone listens, it’s a net improvement.