Any intervention is extremely sensitive to implementation details, whether deworming or nets or psychotherapy.
Yes, I’m sorry if my comment appeared to dismiss this fact as I do strongly agree with this.
Maybe some interventions are easier to implement than others, and there might be more variance in the effectiveness of psychotherapy compared with net distribution (although I doubt that, I would guess less variance than nets) but all are very sensitive to implementation details.
This is pretty much my point
I’d be interested in you backing up this comment with a bit explanation if you have time (all good if not!). I know this isn’t your job and you don’t have the time that Joel has, but what is it that has led you to conclude that the numbers are “on the order (or lower) than cash transfers”? Is this comment based on intuition or have you done some maths?
I haven’t done a bottom up analyses, more I have made my own adjustments to the HLI numbers which get me to about that level:
You use 0.88 as the effect-size for StrongMinds whereas I think it’s more appropriate to use something closer to the 0.4/0.5 you use here. (And in fact I actually skew this number even lower than you do)
You convert SDs of depression-scores directly to SDs of well-being, which I strongly object to. I don’t have exact numbers of how I would discount this, but my guess is there are two reasons I want to discount this:
Non-linearity in severity of depression
No perfect correlation between the measures (when I spoke to Joel we discussed this, and I do think your reasoning is reasonable, but I still disagree with it)
I think the fairest way to resolve this would be to bet on the effect-size of the Ozler trial. Where would you make me 50⁄50 odds in $5k?
Just to clarify, am not part of Strongminds or HLI, maybe you thought I was Joel replying?
Thanks for the clarifications, appreciate that. Seems like we generally agree on implementation sensitivity.
Thanks for your explanation on the HLI numbers which unfortunately I only I partly understand. A quick (and possibly stupid question), what does SD stand for? Usually I would expect standard deviation?
No bet from me on the Ozler trial I’m afraid (not a gambling guy ;) ). Personally I think this trial it will find a fairly large effect due partly to the intervention actually working, but the effect will be inflated compared to the real effect to a due to inflated post-study SBJ scores. This happens due to “demand bias” and “future hope bias” (discussed in another post) but my certainty of any of this is so low it almost touches the floor...
what does SD stand for? Usually I would expect standard deviation?
Yes, that’s exactly right. The HLI methodology consists of polling together a bunch of different studies effect-sizes (measured in standard deviations) and then converting those standard deviations into WELLBYs. (By mulitplying by a number ~2).
No bet from me on the Ozler tria
Fair enough—I’m open to betting on this with anyone* fwiw. * anyone who hasn’t already seen results / involved in the trial ofc
Yes, I’m sorry if my comment appeared to dismiss this fact as I do strongly agree with this.
This is pretty much my point
I haven’t done a bottom up analyses, more I have made my own adjustments to the HLI numbers which get me to about that level:
You use 0.88 as the effect-size for StrongMinds whereas I think it’s more appropriate to use something closer to the 0.4/0.5 you use here. (And in fact I actually skew this number even lower than you do)
You convert SDs of depression-scores directly to SDs of well-being, which I strongly object to. I don’t have exact numbers of how I would discount this, but my guess is there are two reasons I want to discount this:
Non-linearity in severity of depression
No perfect correlation between the measures (when I spoke to Joel we discussed this, and I do think your reasoning is reasonable, but I still disagree with it)
I think the fairest way to resolve this would be to bet on the effect-size of the Ozler trial. Where would you make me 50⁄50 odds in $5k?
Just to clarify, am not part of Strongminds or HLI, maybe you thought I was Joel replying?
Thanks for the clarifications, appreciate that. Seems like we generally agree on implementation sensitivity.
Thanks for your explanation on the HLI numbers which unfortunately I only I partly understand. A quick (and possibly stupid question), what does SD stand for? Usually I would expect standard deviation?
No bet from me on the Ozler trial I’m afraid (not a gambling guy ;) ). Personally I think this trial it will find a fairly large effect due partly to the intervention actually working, but the effect will be inflated compared to the real effect to a due to inflated post-study SBJ scores. This happens due to “demand bias” and “future hope bias” (discussed in another post) but my certainty of any of this is so low it almost touches the floor...
Yes, that’s exactly right. The HLI methodology consists of polling together a bunch of different studies effect-sizes (measured in standard deviations) and then converting those standard deviations into WELLBYs. (By mulitplying by a number ~2).
Fair enough—I’m open to betting on this with anyone* fwiw. * anyone who hasn’t already seen results / involved in the trial ofc