More worringly: what we really care about is not the gap between the effects at a given point in time. What we care about is the difference between the integrals of those curves. The difference in total impact (divided by program cost).
Yes. I don’t think the issue is with cash transfers alone. It’s that most RCTs (I’m most familiar with the subjective wellbeing / mental health literature) don’t perform or afford the analysis of the total impact of an intervention. The general shortcoming is the lack of information about the decay or growth of effects over time.
But I don’t quite share your update away from using RCTs. Instead, we should demand better data and analysis of RCTs.
Despite the limited change-over-time data on many interventions, we often don’t need to guess what happens over time (if by “requires making a lot of assumptions” you mean more of an educated guess rather than modelling something with empirically estimated parameters). At the Happier Lives Institute, we estimate the total effects by first evaluating an initial effect (what’s commonly measured in meta-analyses) and then empirically estimating how the effect changes over time (which is rarely done). You can dig into how we’ve done this by using meta-analyses in our cash transfers and psychotherapy reports here.
While not perfect, if we have two-time points within or between studies, we can use the change in the impact between those time points to inform our view on how long the effect lasts, and thus estimate the total effect of an intervention. The knot is then cut.
FWIW, from our report about cash transfers, we expect the effect on subjective wellbeing to last around a decade or less. Interestingly, some studies of “big push” asset transfers find effects on subjective wellbeing that have not declined or grown after a decade post-intervention. If that holds up to scrutiny, that’s a way in which asset transfers could be more cost-effective than cash transfers.
Note to reader: Michael Plant (who commented elsewhere) and I both work at HLI, which is related to why we’re expressing similar views.
Yes. I don’t think the issue is with cash transfers alone. It’s that most RCTs (I’m most familiar with the subjective wellbeing / mental health literature) don’t perform or afford the analysis of the total impact of an intervention. The general shortcoming is the lack of information about the decay or growth of effects over time.
But I don’t quite share your update away from using RCTs. Instead, we should demand better data and analysis of RCTs.
Despite the limited change-over-time data on many interventions, we often don’t need to guess what happens over time (if by “requires making a lot of assumptions” you mean more of an educated guess rather than modelling something with empirically estimated parameters). At the Happier Lives Institute, we estimate the total effects by first evaluating an initial effect (what’s commonly measured in meta-analyses) and then empirically estimating how the effect changes over time (which is rarely done). You can dig into how we’ve done this by using meta-analyses in our cash transfers and psychotherapy reports here.
While not perfect, if we have two-time points within or between studies, we can use the change in the impact between those time points to inform our view on how long the effect lasts, and thus estimate the total effect of an intervention. The knot is then cut.
FWIW, from our report about cash transfers, we expect the effect on subjective wellbeing to last around a decade or less. Interestingly, some studies of “big push” asset transfers find effects on subjective wellbeing that have not declined or grown after a decade post-intervention. If that holds up to scrutiny, that’s a way in which asset transfers could be more cost-effective than cash transfers.
Note to reader: Michael Plant (who commented elsewhere) and I both work at HLI, which is related to why we’re expressing similar views.