Hi Derek, thank you for your comment and for clarifying a few things.
Time discounting: We will revisit time discounting when looking at interventions with longer time scales. To be clear, we plan to update these analyses for backwards compatibility as we introduce refinements to our models and analyse new interventions.
Costs: You’re right, expenses in an organisation can be lumpy over time. If costs are high in all previous years but low in 2019 and we only use the 2019 figures, we’d probably be making a wrong prediction about future costs. I think a reasonable way to account for this is by treating the cost for an organisation as an average of the previous years, where you give more weight increasingly to years closer to the present.
Depression data: Thanks for the clarification; I think I understand better now. We make a critical assumption that a one-unit improvement in depression scales corresponds to the same improvement in well-being as a one-unit change in subjective well-being scales. If SWB is our gold standard, we can ask if depression scale changes predict SWB scale changes. Our preliminary analyses suggest that the difference here would, in any case, be pretty small. For cash transfers, we found the ‘SWB only’ effect would be about 13% larger than the pooled ‘SWB-and-MH’ effect (see page 10, footnote 16). To assess therapy, we looked at some psychological interventions that had outcome measures in SWB and MH and found the SWB effect was 11% smaller (see p27-8). We’d like to dig further into this in the future. But these are not result-reversing differences.
I strongly agree with Derek’s point about measuring the nonmonetary costs to the recipients and their families. If your benefits are driven mainly by the differences in costs, then omitting potentially relevant costs can invalidate the entire analysis. You absolutely must account for the time that recipients spent in the program, and traveling to and from the program, and any other money or time costs that they or their families incurred as a result of program participation. At minimum, this time should be valued at the local wage rate. Until this is addressed, I will assume that your analysis is junk, and say so to anyone who asks me about it.
Hi Derek, thank you for your comment and for clarifying a few things.
Time discounting: We will revisit time discounting when looking at interventions with longer time scales. To be clear, we plan to update these analyses for backwards compatibility as we introduce refinements to our models and analyse new interventions.
Costs: You’re right, expenses in an organisation can be lumpy over time. If costs are high in all previous years but low in 2019 and we only use the 2019 figures, we’d probably be making a wrong prediction about future costs. I think a reasonable way to account for this is by treating the cost for an organisation as an average of the previous years, where you give more weight increasingly to years closer to the present.
Depression data: Thanks for the clarification; I think I understand better now. We make a critical assumption that a one-unit improvement in depression scales corresponds to the same improvement in well-being as a one-unit change in subjective well-being scales. If SWB is our gold standard, we can ask if depression scale changes predict SWB scale changes. Our preliminary analyses suggest that the difference here would, in any case, be pretty small. For cash transfers, we found the ‘SWB only’ effect would be about 13% larger than the pooled ‘SWB-and-MH’ effect (see page 10, footnote 16). To assess therapy, we looked at some psychological interventions that had outcome measures in SWB and MH and found the SWB effect was 11% smaller (see p27-8). We’d like to dig further into this in the future. But these are not result-reversing differences.
I strongly agree with Derek’s point about measuring the nonmonetary costs to the recipients and their families. If your benefits are driven mainly by the differences in costs, then omitting potentially relevant costs can invalidate the entire analysis. You absolutely must account for the time that recipients spent in the program, and traveling to and from the program, and any other money or time costs that they or their families incurred as a result of program participation. At minimum, this time should be valued at the local wage rate. Until this is addressed, I will assume that your analysis is junk, and say so to anyone who asks me about it.