Hi Nick. I found more details about the Baird et al. RCT here. I’ve copied the section about the ‘cash alone’ arm below as I know you’ll be interested to read that:
One of the issues that can cause difficulties in the interpretation of the findings from this study comes from the fact that it does not have a classical 2x2 factorial design that includes a “cash alone” arm. While it would have been ideal, from a study design perspective, to have such a design, which would have enabled us to experimentally reject (or not) that cash alone would have been as effective as IPT-G+. Such benchmarking of the (cost-) effectiveness of a group therapy intervention to cash transfers alone would have been nice, the three-arm trial we designed for our setting, leaving out a “cash-only” arm, is an adequate study design for the following three reasons.
First, the extant evidence on the effects of economic interventions in general, or cash transfers in particular, do not support the idea of improved mental health outcomes past the short-run. For example, in their review, Lund et al. (2011) finds that the mental health effects of poverty alleviation programs were inconclusive. Blattman, Jamison, and Sheridan (2017) find no effects of lump-sum cash transfers alone on mental health outcomes in the short- or the medium-run. Short-term effects of cash transfers (monthly or lump-sum) on psychological wellbeing that were observed in the short-run dissipated a couple of years later (Baird, de Hoop, and Özler 2013; Baird, McIntosh and Özler 2019; Haushofer and Shapiro 2016, 2018). Hence, we do not think that there is sufficient equipoise to include a “cash only” arm when it comes to sustained effects on depression two-years after the end of the intervention.
Second, even if one could make a case that there may be a sustained income effect on mental health – for example, such as those indicated by studies of lottery winners (Gardner and Oswald 2007; Lindahl 2005) – the amounts offered to IPT-G participants here are too small to have this kind of an effect two years after they are transferred. A similar argument has been made in another experiment that lacked a pure unconditional cash transfer (UCT) arm: Benhassine et al. (2015) argue that the labelling of the cash transfer as an education support program increased school participation through its effect on the “...parents’ belief that education was a worthwhile investment,” rather than through a pure income effect, because the transfers were too small to cause the observed effects.
Finally, and related to the point above, one of the aims of our trial is to test the efficacy of a low-cost and scalable intervention through two NGOs that have a track record of implementing programs that are being utilized here (BRAC Uganda and StrongMinds Uganda). While BRAC Uganda is interested in taking advantage of its ELA girls’ clubs’ platform to provide mental health services across Uganda, it does not have any plans to provide UCTs, especially not in transfer sizes that might perhaps have sustained effects. Hence, the lack of evidence on the potential effectiveness of UCTs on sustained reductions in depression, combined with a lack of interest from the implementing partners, resulted in the study team designing a trial that has only three arms. Should the trial show that IPT-G+ is significantly more effective than IPT-G alone in reducing depression in the medium-run, our interpretation will be that there is a complementarity between the two interventions, and not that cash is effective on its own for sustained improvements in psychological wellbeing.
Thanks Barry I tried to find this earlier but couldn’t.
I find these arguments rather uncompelling. What do you think Barry and Joel? (I wish I could tag people on this forum haha)
That they feel the need to write 4 paragraphs to defend against this elephant in the room says a lot. The question we are all still asking is how much better (if at all) StrongMinds really is than cash for wellbeing.
My first question is why don’t they reference the 2020 Haushofer study, the only RCT comparing psychotherapy to cash and showing cash is better? https://www.nber.org/papers/w28106
Second, their equipoise argument is very poor. The control arm should have been BRAC ELA club + cash. Then you keep 3 arms and avoid their straw man 4 arm problem. You would lose nothing in equipoise giving cash to the control arm—I don’t understand the equipoise argument perhaps I’m missing something?
Then third there’s this...
“Should the trial show that IPT-G+ is significantly more effective than IPT-G alone in reducing depression in the medium-run, our interpretation will be that there is a complementarity between the two interventions, and not that cash is effective on its own for sustained improvements in psychological wellbeing.”
This is the most telling paragraph. It’s like, we designed our study so that even if we see that cash gives a big boost, we aren’t going to consider the alternative that we don’t like. It seems to me like they are defending poor design post-hoc, rather than that they made good decision made in advance.
The more I see this, the more I suspect that leaving the cash arm out was either a big mistake or an intentional move by the NGOs. What we have now is a million dollar RCT, which doesn’t answer conclusively the most important question we are all asking. This leaves organisations like your HLI having to use substandard data to assess psychotherapy vs. cash because there is no direct gold standard comparison.
It’s pretty sad that a million dollars will be spent on a study that at best fails to address the elephant in the room (while spending 4 paragraphs explaining why they are not). Other than that the design and reasoning in this study seems fantastic.
Hi Nick. I found more details about the Baird et al. RCT here. I’ve copied the section about the ‘cash alone’ arm below as I know you’ll be interested to read that:
Thanks Barry I tried to find this earlier but couldn’t.
I find these arguments rather uncompelling. What do you think Barry and Joel? (I wish I could tag people on this forum haha)
That they feel the need to write 4 paragraphs to defend against this elephant in the room says a lot. The question we are all still asking is how much better (if at all) StrongMinds really is than cash for wellbeing.
My first question is why don’t they reference the 2020 Haushofer study, the only RCT comparing psychotherapy to cash and showing cash is better? https://www.nber.org/papers/w28106
Second, their equipoise argument is very poor. The control arm should have been BRAC ELA club + cash. Then you keep 3 arms and avoid their straw man 4 arm problem. You would lose nothing in equipoise giving cash to the control arm—I don’t understand the equipoise argument perhaps I’m missing something?
Then third there’s this...
“Should the trial show that IPT-G+ is significantly more effective than IPT-G alone in reducing depression in the medium-run, our interpretation will be that there is a complementarity between the two interventions, and not that cash is effective on its own for sustained improvements in psychological wellbeing.”
This is the most telling paragraph. It’s like, we designed our study so that even if we see that cash gives a big boost, we aren’t going to consider the alternative that we don’t like. It seems to me like they are defending poor design post-hoc, rather than that they made good decision made in advance.
The more I see this, the more I suspect that leaving the cash arm out was either a big mistake or an intentional move by the NGOs. What we have now is a million dollar RCT, which doesn’t answer conclusively the most important question we are all asking. This leaves organisations like your HLI having to use substandard data to assess psychotherapy vs. cash because there is no direct gold standard comparison.
It’s pretty sad that a million dollars will be spent on a study that at best fails to address the elephant in the room (while spending 4 paragraphs explaining why they are not). Other than that the design and reasoning in this study seems fantastic.