Should all RCTs now be Intervention X Vs. Cash Transfer?
Should cash transfers now be considered the mininum “standard of development”, and used as the control arm for most RCT trials?
Many development RCTs are still performed vs. no intervention at all. However with evidence of benefits of cash transfers across a range of development spheres (Health, education, income), should every new development intervention should have to be proved better than Cash transfers? Is there any point in seeing whether an intervention is beneficial vs. doing nothing at all, or should the bar now be higher for both practical and Ethical reasons? Should cash transfers be our new standard benchmark for any development intervention?
I was triggered to write this post after checking out this RCT which investigates whether StrongMinds group psychotherapy can help teenagers. I wondered why the control arm gave people nothing at all, rather than a cash transfer of about 150 dollars – the cost of psychotherapy for each person?
Practical considerations
I have developed intervention X, a voice robot which automatically calls pregnant mothers in low income countries to remind them 2 days before each antenatal appointment and then every day a week before their scheduled delivery date. I want to test whether Intervention X improves infant mortality, infant weight at birth and 6 weeks, and the percent of mothers who deliver their baby while cared for by a skilled health worker. Implementing Intervention X will cost 50 dollars per mother. We know that cash transfers to the mother can potentially improve infant outcomes, so why not make the control arm a 50 dollar cash transfer a week before the delivery date rather than no intervention at all? This will give us the practical result we want for real world use. Should we use and scale up intervention X for babies, or would we be better giving a $50 cash transfer?
The cash transfer amount in the trial should be roughly equivalent to the real cost of the intervention to be tested. Giving more or less wouldn’t make sense as you then couldn’t do a meaningful cost-effectiveness comparison—you would be testing apples vs oranges (or perhaps bigger apples with smaller apples)
And fortunately there are few practical barriers to using cash transfers as a control. We can now give almost anyone anywhere a cash transfer through phones. Obviously the transfers will add cost to our study, but apart from cost there aren’t big practical barriers.
Are Development RCTs vs nothing unethical? (high uncertainty)
Given that cash transfers can work on a variety of fronts, is there now an ethical problem in giving people nothing in the control arm of an a development RCT? In medical RCTs (the OG RCTs ;) if you know a medication works, say for High Blood pressure, then any new blood pressure medication is not pitted against no medication at all, it is pitted against the previous “standard of care” as not to deny participants a treatment that we know works just for the sake of our research. Giving them nothing wouldn’t be fair. Is it fair now to give people nothing in control arms of development RCTs, when there is a high likelihood that transfers would provide a benefit?
Also, is it fair to subject people to trials which might not have practical use even if the trial is “successful”? If intervention X improves outcomes by 1%, while we already know that cash transfers improves outcomes by 2%. Is it right to subject people to research that might find a real effect, but an effect that is not practically useful in the real world because cash works better. Ethically, should cash transfer become the standard which every other intervention needs to beat to be considered an ethically sound intervention?
Counter arguments and other comments
There are strong counter arguments – One that there is value in seeing if intervention X works at all, even if it is doesn’t meet the cash transfer benchmark. Perhaps intervention X can later be developed to perform better than cash transfers in future. If we set the “cash transfer standard” fro the start, we may write off good ideas early in their development stage, before they have the chance to develop and become better than cash transfers. I would argue though that even if early studies are vs. a control of nothing, at least the second stage of trials should be vs. Cash transfers. Also there is the possibility of having both a cash transfer control arm, and a no intervention control arm.
Another counter argument is that evidence for efficacy of cash transfers across all spheres of development may not yet strong enough to consider it a standard.
To be clear, many interventions have been shown to probably be more effective than cash transfers – I don’t argue that cash transfers are the best intervention for everything, or that we shouldn’t be innovating and trialling many new interventions. Just that they perhaps should now be the standard control arm for development RCTs
Obviously there will be many exceptions where a control arm of cash transfers doesn’t make sense, like for example comparing a new type of bednet to an old one.
This may have been discussed in detail before, so apologies if there are other posts which cover this which I have missed – I did a search and couldn’t find any on a cursory look. I’m super interested to hear criticisms and thoughts, and will update the post if need be :).
Hi Nick—thanks for the thoughtful post!
I think cash arms make a lot of intuitive sense, my main pushback would be a practical one: cash and intervention X will likely have different impact timelines (e.g. psychotherapy takes a few months to work but delivers sustained benefits, perhaps cash has massive welfare benefits immediately but they diminish quickly over time). This makes the timing of your endline study super important, to the point that when you run the endline is really what determines which intervention comes out on top, rather than the actual differences in the interventions. I have a post on this here with a bit more detail.
Your point on the ethics here is an interesting one, I agree that medical ethics might suggest “control” groups should still receive some kind of intervention. Part of the distinction could be that medical trials give sick patients placebos, which control patients accurately believe might be medicine, which feels perhaps deceptive, whereas control groups in development RCTs are well aware that they aren’t receiving any intervention (i.e. they know they haven’t received psychotherapy or cash), which feels more honest?
The downside is this changes the research question from “What is the impact of X?” to “How much better is X than cash”, and there are lots of cases were the counterfactual really would be inaction. A way around this might be to give control groups an intervention that we know to be “good” but that doesn’t affect the specific outcome of interest. e.g. I’ve worked on an agriculture RCT that gave control groups water/sanitation products that had no plausible way to affect their maize yield but at least meant they weren’t losing out. This might not apply to broad measures like WELBYs
I’m honestly not sure about the ethical side here though, interested to explore further.
Thanks so much Rory and for the links to your earlier post and the USAID stuff!
I think your criticism is a good criticism of RCTs in general, but it seems to me more a criticism comment about RCT design then being a clear argument against comparing with cash transfers. RCTs on development NEED longer term outcome measurement, and surely need at a minimum 2 data points at 2 different times after the study. And of course the most important data point is after many months or even many years as you talked about in your article.
I’m not at all sure about the ethical side either . Medical RCTs compare a new trial treatment against the most up-to-date treatment—not so much because we worry about “tricking” a patient like you say (there are still plenty of RCTs with sugar placebo pills which is deemed ethically OK), we are still OK with a kind of ‘deception’. What we AREN’T OK with is doing a trial where we give the control arm nothing at all, when we know there is a better option than nothing for the medical condition. And I’d argue that cash is usually a better option than nothing for many development conditions.
That’s a great and sobering point about the counterfactual potentially being inaction if cash transfers won the day. Why should the counterfactual be inaction though? I would hope as development people we are good enough that if Cash was equivalent or better than intervention X, this wouldn’t lead us not to inaction but instead to give more cash instead. Maybe I’m naive and idealistic though, and maybe you’re right that there is actually a practical advantage in seeing a positive impact of intervention X, even if it is worse than a cash transfer. I don’t think that should be the case though.
That’s the whole question really—should we spend our millions on RCTs asking “What is the impact of X”, or “Is X better than cash”. What we really want to know, the practical question which underlies the research question is is “Should we be implementing this intervention at scale”. I’d argue that to answer that, the question vs. Cash is the one that matters more.
Thanks so much for your reply, I can see you’ve thought about this far more than me and I loved your original post—weird that searches on the forum didn’t bring it up, maybe they should employ google search on the site haha.