I had not realized that he was running an RCT on StrongMinds (as mentioned in this post), so possibly had access to insider data on the (lack of) effectiveness.
Hereās the full exchange between Ćzler and Haushofer:
JH: Whenever someone meekly suggests that one might not leave those with the lowest incomes entirely alone with their mental health struggles, the āitās not that simpleā brigade shows up and talks about the therapy-industrial complex and itās so tiresome.
BO: Thanks, Johannes. That thread & and recommendation is outrageous: thereās no good evidence that Strong Minds is effective, let alone most effective. Itās 20-year old studies combined with pre-post data provided by SM itself. People should pay no attention to this š§µ, whatsoever.
BO: Show me one good published study of the impact of SM on the ground at some decent scale...
JH: My point is not that SM is the best available intervention. My point is that people who get upset at HLI for caring about wellbeing on the grounds that this ignores structural interventions are mistaken.
BO: I have zero problems with your point. Itās well taken & thatās why I thanked you for your tweet. My issue is w the unequivocal SM recommendation, the evidence presented, & the conviction with which it is advertised. People (unfortunately) donate based on such recommendations...
JH: Ok, one can disagree about that. Do you think the Bolton et al work is good? If yes what dissuades you from thinking that SM is effective? (Regardless of whether it should be top-ranked)
BO: Bolton et al. (2003) is not about SM. Why did you evaluate GD with an RCT when we could have extrapolated from Progresa?
JH: Of course thereās always room for more studies. But if someone had said at the time that GD is likely to be effective for certain outcomes based on work on Progresa, I wouldnāt have considered that claim outrageous. (Side point, I think IPT/āSM more similar than Progresa/āGD.)
JH: (Addendum: Iāve always been a bit uneasy with the hype around GD and our study in light of all the other evidence on cash transfers.)
BO: You would have if someone was claiming that it was exactly 6 times more effective per $ than the most promising alternative. Also, no one (give or take) would have donated to GD on the basis of previous evidence, say our study in Malawi...(which is much closer).
JH: GD definitely emphasized early on that cash transfers were very well-researched! Of course not sure if that helped, but I donāt recall any āhow can you say that!ā reactions. 1ā2
JH: On the ā6 timesā claim: I think orgs like Givewell and HLI are in the business of providing guesstimates given limited and mixed-quality evidence. I have much more tolerance for extrapolation in that context than in an academic paper. 2ā2
BO: I think smt got lost in translation here: my point is you canāt claim your NGO is best because it works on something promising. You have to show itāeffectiveness at scale. That was true for GD and it remains true for SM. /ā1
BO: And you definitely should not make forest plots like that with guesstimates. Itās misleading.../āFIN.
BO: Iām going to leave it at this. I donāt care for that type of hype right before the giving season, with shaky evidence. We all know the dangers of assuming effectiveness at scale from efficacy studies, especially 20-year-old ones...
JH: I definitely see that. I also think this shaky evidence is leagues better than what most people base their giving decisions on, so if someone listens, itās a net improvement.
Just to clarify, Berk has deleted his entire Twitter profile rather than these specific tweets. Will be interesting to the results from the upcoming RCT.
Another reason is that Berk Ćzler had a scathing review of StrongMinds on Twitter (archived, tweets are now deleted).
I had not realized that he was running an RCT on StrongMinds (as mentioned in this post), so possibly had access to insider data on the (lack of) effectiveness.
Hereās the full exchange between Ćzler and Haushofer:
JH: Whenever someone meekly suggests that one might not leave those with the lowest incomes entirely alone with their mental health struggles, the āitās not that simpleā brigade shows up and talks about the therapy-industrial complex and itās so tiresome.
BO: Thanks, Johannes. That thread & and recommendation is outrageous: thereās no good evidence that Strong Minds is effective, let alone most effective. Itās 20-year old studies combined with pre-post data provided by SM itself. People should pay no attention to this š§µ, whatsoever.
JH: This dismissal seems much too strong to me. I thought HLIās discussion of the evidence here was fair and reasonable: https://āāwww.happierlivesinstitute.org/āāreport/āāstrongminds-cost-effectiveness-analysis
BO: Show me one good published study of the impact of SM on the ground at some decent scale...
JH: My point is not that SM is the best available intervention. My point is that people who get upset at HLI for caring about wellbeing on the grounds that this ignores structural interventions are mistaken.
BO: I have zero problems with your point. Itās well taken & thatās why I thanked you for your tweet. My issue is w the unequivocal SM recommendation, the evidence presented, & the conviction with which it is advertised. People (unfortunately) donate based on such recommendations...
JH: Ok, one can disagree about that. Do you think the Bolton et al work is good? If yes what dissuades you from thinking that SM is effective? (Regardless of whether it should be top-ranked)
BO: Bolton et al. (2003) is not about SM. Why did you evaluate GD with an RCT when we could have extrapolated from Progresa?
JH: Of course thereās always room for more studies. But if someone had said at the time that GD is likely to be effective for certain outcomes based on work on Progresa, I wouldnāt have considered that claim outrageous. (Side point, I think IPT/āSM more similar than Progresa/āGD.)
JH: (Addendum: Iāve always been a bit uneasy with the hype around GD and our study in light of all the other evidence on cash transfers.)
BO: You would have if someone was claiming that it was exactly 6 times more effective per $ than the most promising alternative. Also, no one (give or take) would have donated to GD on the basis of previous evidence, say our study in Malawi...(which is much closer).
JH: GD definitely emphasized early on that cash transfers were very well-researched! Of course not sure if that helped, but I donāt recall any āhow can you say that!ā reactions. 1ā2
JH: On the ā6 timesā claim: I think orgs like Givewell and HLI are in the business of providing guesstimates given limited and mixed-quality evidence. I have much more tolerance for extrapolation in that context than in an academic paper. 2ā2
BO: I think smt got lost in translation here: my point is you canāt claim your NGO is best because it works on something promising. You have to show itāeffectiveness at scale. That was true for GD and it remains true for SM. /ā1
BO: And you definitely should not make forest plots like that with guesstimates. Itās misleading.../āFIN.
BO: Iām going to leave it at this. I donāt care for that type of hype right before the giving season, with shaky evidence. We all know the dangers of assuming effectiveness at scale from efficacy studies, especially 20-year-old ones...
JH: I definitely see that. I also think this shaky evidence is leagues better than what most people base their giving decisions on, so if someone listens, itās a net improvement.
Just to clarify, Berk has deleted his entire Twitter profile rather than these specific tweets. Will be interesting to the results from the upcoming RCT.