Given that this post has been curated, I wanted to follow up with a few points I’d like to emphasise that I forgot to include in the original comment.
To my knowledge, we were the first to attempt to estimate household spillovers empirically. In hindsight, it shouldn’t be too surprising that it’s been a messy enterprise. I think I’ve updated towards “messiness will continue”.
One hope of ours in the original report was to draw more attention to the yawning chasm of good data on this topic.
“The lack of data on household effects seems like a gap in the literature that should be addressed by further research. We show that including household spillovers can change the relative cost-effectiveness of two interventions, which demonstrates the need to account for the impact of interventions beyond the direct recipient.”
Relatedly, spillovers don’t have to be huge to be important. If you have a household of 5, with 1 recipient and 4household non-recipients, household spillovers only need to be 25% that of the recipient effect for the two effects to be equivalent in size. I’m still pretty confident we omit an important parameter when we fail to estimate household spillovers.
So I’m pleased with this conversation and hopeful that spillovers for all outcomes in the global health and wellbeing space will be given more empirical consideration as a consequence.
There are probably relatively cost-effective ways to gather more data regarding psychotherapy spillovers in particular.
I’ve heard that some people working with Vida-Plena are trying to find funding for an RCT that includes spillovers — but I haven’t spoken to Joy about this recently.
StrongMinds could be willing to do more work here. I think they’re planning an RCT — if they get it funded, I think adding a module for household surveys shouldn’t be too expensive.
There’s also a slew of meta-analyses of interventions aimed at families that didn’t always seem jointly to target parents and children that may include more RCTs where we can infer spillovers. Many of these I missed before: Siegenthaler et al. (2012), Thanhäuser et al. (2017), Yap et al. (2016), Loechner et al. (2018), Lannes et al., (2018), and Havinga et al. (2021)
In general, household spillovers should be relatively cheap to estimate if they just involve surveying a randomly selected additional household member and clarifying the relationships between those surveyed.
I still don’t have the Barker et al. RCT spillover results, but will update this comment once I know.
Is it as easy (or easy enough) to enroll participants in RCTs if you need their whole household, rather than just them, to consent to participate? Does it create any bias in the results?
I’d assume that 1. you don’t need the whole household, depending on the original sample size, it seems plausible to randomly select a subset of household members [1](e.g., in house A you interview recipient and son, in B. recipient and partner, etc...) and 2. they wouldn’t need to consent to participate, just to be surveyed, no?
If these assumptions didn’t hold, I’d be more worried that this would introduce nettlesome selection issues.
I recognise this isn’t necessarily simple as I make it out to be. I expect you’d need to be more careful with the timing of interviews to minimise the likelihood that certain household members are more likely to be missing (children at school, mother at the market, father in the fields, etc.).
Given that this post has been curated, I wanted to follow up with a few points I’d like to emphasise that I forgot to include in the original comment.
To my knowledge, we were the first to attempt to estimate household spillovers empirically. In hindsight, it shouldn’t be too surprising that it’s been a messy enterprise. I think I’ve updated towards “messiness will continue”.
One hope of ours in the original report was to draw more attention to the yawning chasm of good data on this topic.
“The lack of data on household effects seems like a gap in the literature that should be addressed by further research. We show that including household spillovers can change the relative cost-effectiveness of two interventions, which demonstrates the need to account for the impact of interventions beyond the direct recipient.”
Relatedly, spillovers don’t have to be huge to be important. If you have a household of 5, with 1 recipient and 4household non-recipients, household spillovers only need to be 25% that of the recipient effect for the two effects to be equivalent in size. I’m still pretty confident we omit an important parameter when we fail to estimate household spillovers.
So I’m pleased with this conversation and hopeful that spillovers for all outcomes in the global health and wellbeing space will be given more empirical consideration as a consequence.
There are probably relatively cost-effective ways to gather more data regarding psychotherapy spillovers in particular.
I’ve heard that some people working with Vida-Plena are trying to find funding for an RCT that includes spillovers — but I haven’t spoken to Joy about this recently.
StrongMinds could be willing to do more work here. I think they’re planning an RCT — if they get it funded, I think adding a module for household surveys shouldn’t be too expensive.
There’s also a slew of meta-analyses of interventions aimed at families that didn’t always seem jointly to target parents and children that may include more RCTs where we can infer spillovers. Many of these I missed before: Siegenthaler et al. (2012), Thanhäuser et al. (2017), Yap et al. (2016), Loechner et al. (2018), Lannes et al., (2018), and Havinga et al. (2021)
In general, household spillovers should be relatively cheap to estimate if they just involve surveying a randomly selected additional household member and clarifying the relationships between those surveyed.
I still don’t have the Barker et al. RCT spillover results, but will update this comment once I know.
Is it as easy (or easy enough) to enroll participants in RCTs if you need their whole household, rather than just them, to consent to participate? Does it create any bias in the results?
I’d assume that 1. you don’t need the whole household, depending on the original sample size, it seems plausible to randomly select a subset of household members [1](e.g., in house A you interview recipient and son, in B. recipient and partner, etc...) and 2. they wouldn’t need to consent to participate, just to be surveyed, no?
If these assumptions didn’t hold, I’d be more worried that this would introduce nettlesome selection issues.
I recognise this isn’t necessarily simple as I make it out to be. I expect you’d need to be more careful with the timing of interviews to minimise the likelihood that certain household members are more likely to be missing (children at school, mother at the market, father in the fields, etc.).