Could I add on to Nick’s comment and an ask for clarification about including”Any form of face-to-face psychotherapy delivered to groups or by non-specialists deployed in LMICs.”—it seems in your Appendix B that the studies incorporated in meta-regression include a lot of individually delivered interventions, do you still use them and if so how/any differently?
Yes, we still use individually delivered interventions as general evidence of psychotherapy’s efficacy in low and middle-income countries. We assigned this general evidence 46% of the weight in the StrongMind’s analysis (see Table 2 in the StrongMinds report).
While we found that group-delivered psychotherapy is more impactful, I’m not entirely clear what the causal mechanism for this would be, so I thought it’d be appropriately conservative to leave in that evidence. We showed and discussed this topic in Table 2 of our psychotherapy report (page 16).
This type of generally recruited and potentially partially biased sample seems a little different than a sample that includes women survivors of torture/ violence/ SA/ in post-conflict settings of which you have a number of RCTs.
I discussed the potential issues with the differences in samples and the way I try to address them in my response to Henry, so I won’t repeat myself unless you have a further concern there.
Regarding the risk of bias due to mistaken beliefs about receiving material benefits—this is honestly new to me since Nick Laing brought it up a couple of months ago. Insomuch as this bias exists, I assume for StrongMinds, this will have to go down over time as word travels that they do not, in fact, do much other than mental health treatments.
And to reiterate the crux here: for this to affect our comparison to, say, cash transfers, we need to believe that this bias leads to people over-reporting their benefits more (or less) than it would for the people who receive cash transfers who hope that if they give positive responses, they’ll get even more cash transfers.
I’m not trying to dismiss this concern out of hand, but I’d prefer to collect more evidence before I change my analysis. I will, if possible, try to make that evidence come to be (just as I try to push for the creation of evidence to inform other questions we’re uncertain about) -- if I can do so cost-effectively, but in my position, resources are limited.
Are there baseline mental health scores for all these samples that you could look at?
There are in many cases, but that’s not data we recorded from the samples, but I think for most studies, the sample was selected for having psychological distress above some clinical threshold. That may be worth looking into.
Thanks Joel—I might be wrong but I think the point might be more more that sometimes you seem to say something in the methodology, than do something a bit different. I don’t think Rina was necessarily saying that you shouldn’t have included individual interventions but more clarifying what the study is actually doing, compared to what you said it was doing in the methodology.
Hi Rina! I appreciate the nice words.
Yes, we still use individually delivered interventions as general evidence of psychotherapy’s efficacy in low and middle-income countries. We assigned this general evidence 46% of the weight in the StrongMind’s analysis (see Table 2 in the StrongMinds report).
While we found that group-delivered psychotherapy is more impactful, I’m not entirely clear what the causal mechanism for this would be, so I thought it’d be appropriately conservative to leave in that evidence. We showed and discussed this topic in Table 2 of our psychotherapy report (page 16).
I discussed the potential issues with the differences in samples and the way I try to address them in my response to Henry, so I won’t repeat myself unless you have a further concern there.
Regarding the risk of bias due to mistaken beliefs about receiving material benefits—this is honestly new to me since Nick Laing brought it up a couple of months ago. Insomuch as this bias exists, I assume for StrongMinds, this will have to go down over time as word travels that they do not, in fact, do much other than mental health treatments.
And to reiterate the crux here: for this to affect our comparison to, say, cash transfers, we need to believe that this bias leads to people over-reporting their benefits more (or less) than it would for the people who receive cash transfers who hope that if they give positive responses, they’ll get even more cash transfers.
I’m not trying to dismiss this concern out of hand, but I’d prefer to collect more evidence before I change my analysis. I will, if possible, try to make that evidence come to be (just as I try to push for the creation of evidence to inform other questions we’re uncertain about) -- if I can do so cost-effectively, but in my position, resources are limited.
There are in many cases, but that’s not data we recorded from the samples, but I think for most studies, the sample was selected for having psychological distress above some clinical threshold. That may be worth looking into.
Thanks Joel—I might be wrong but I think the point might be more more that sometimes you seem to say something in the methodology, than do something a bit different. I don’t think Rina was necessarily saying that you shouldn’t have included individual interventions but more clarifying what the study is actually doing, compared to what you said it was doing in the methodology.