I haven’t read the debate closely, but people who like this post would probably be interested in the authors’ Twitter conversation about this research with Alexander Berger (head of global health and wellbeing at Open Philanthropy).
Hi! Not exactly sure why you disagree, but some context might help. We found 5 studies of hshold spillovers for CTs. Taken at face value, the imply other members of hshold get approx 100% of SWB value as recipient.
Weirdly,we could only find 1 study of intra-hshold therapy effects; that was non RCT and had N<200. It found hshold spillover was 77% size of that to recipient. Taking this at face value too, and assuming hshold size is about 4, therapy goes from 12x to 10x better.
I think the right thing to do here (besides further research) would be to give less weight to the psychotherapy spillover estimate and adjust the effect downwards (or at least more downwards than CTs’ spillover, which has evidence from multiple studies, presumably some better designed and with larger sample sizes), based on a skeptical prior.
PT=psychotherapy, CT=cash transfers
PT= 12 x CT without spillovers
CT’ = 4 x CT with spillovers
PT’ = (1+3s) x PT with spillovers, where s=spillover effect for psychotherapy
PT’= ((1+3s)/4) * 12 x CT’ = 3(1+3s) x CT’.
The worst case, s=0, has PT’=3 x CT’. With s=0.25=25%, PT’=5.25 x CT’, and with s=0.5=50%, PT’=7.5 x CT’.
As a result of Alexander’s feedback, we’ve updated our cost-effectiveness comparison of psychotherapy and cash transfers to include an estimate of the effects on other household members. Our previous analysis only considered the effects on recipients.
For cash transfers, we estimate that each household member experiences 86% of the benefits experienced by the recipient. For psychotherapy, we estimate the spillover ratio to be 53%.
After including the household spillover effects, we estimate that StrongMinds is 9 times more cost-effective than GiveDirectly (a slight reduction from 12 times in our previous analysis).
This new analysis of household effects is based on a small number of studies, eight for cash transfers and three for psychotherapy. The lack of data on household effects is a seriousgap in the literature that should be addressed by further research because it is such a large part—indeed, the majority—of the total effects. The significance of household effects seems plausibly crucial for many interventions, such as poverty alleviation programmes, housing improvement interventions, and air or water quality improvements.
I haven’t read the debate closely, but people who like this post would probably be interested in the authors’ Twitter conversation about this research with Alexander Berger (head of global health and wellbeing at Open Philanthropy).
On Twitter, Michael Plant wrote:
I think the right thing to do here (besides further research) would be to give less weight to the psychotherapy spillover estimate and adjust the effect downwards (or at least more downwards than CTs’ spillover, which has evidence from multiple studies, presumably some better designed and with larger sample sizes), based on a skeptical prior.
PT=psychotherapy, CT=cash transfers
PT= 12 x CT without spillovers
CT’ = 4 x CT with spillovers
PT’ = (1+3s) x PT with spillovers, where s=spillover effect for psychotherapy
PT’= ((1+3s)/4) * 12 x CT’ = 3(1+3s) x CT’.
The worst case, s=0, has PT’=3 x CT’. With s=0.25=25%, PT’=5.25 x CT’, and with s=0.5=50%, PT’=7.5 x CT’.
As a result of Alexander’s feedback, we’ve updated our cost-effectiveness comparison of psychotherapy and cash transfers to include an estimate of the effects on other household members. Our previous analysis only considered the effects on recipients.
You can read a summary of our new analysis here.
For cash transfers, we estimate that each household member experiences 86% of the benefits experienced by the recipient. For psychotherapy, we estimate the spillover ratio to be 53%.
After including the household spillover effects, we estimate that StrongMinds is 9 times more cost-effective than GiveDirectly (a slight reduction from 12 times in our previous analysis).
This new analysis of household effects is based on a small number of studies, eight for cash transfers and three for psychotherapy. The lack of data on household effects is a serious gap in the literature that should be addressed by further research because it is such a large part—indeed, the majority—of the total effects. The significance of household effects seems plausibly crucial for many interventions, such as poverty alleviation programmes, housing improvement interventions, and air or water quality improvements.