My reading of the Strongminds debate that has taken place is that the strength of the evidence wasn’t sufficient to list Strongminds as a top charity (relevant posts are 1, 2, 3).
One hope of ours in the original report was to draw more attention to the yawning chasm of good data on this topic.
If the data isn’t good enough might it be worth suggesting people fund research studies rather than suggesting people fund the charity itself?
EDIT: I just want to say I would feel uncomfortable if anyone else updated too much based on my comments. I would encourage people to read the critiques I linked for themselves as well as HLI responses.
Hello Jack. I think people can and will have different conceptions of what the criteria to be on a/the ‘top charity’ list are, including what counts as sufficient strength of evidence. If strength of evidence is essential, that may well rule out any interventions focused on the longterm (whose effects we will never know) as well as deworming (the recommendation of which is substantially based on a single long-term study). The evidence relevant for StrongMinds was not trivial though: we drew on 39 studies of mental health interventions in LICs to calibrate our estimates.
We’d be very happy to see further research funded. However, we see part of our job as trying to inform donors who want to fund interventions, rather than research. On the current evidence and analysis we’ve been able to do, StrongMinds was the only organisation we felt comfortable recommending. We are working to update our existing analysis and search for new top interventions.
Thanks Michael. My main concern is that it doesn’t seem that there is enough clarity on the spillovers, and spillovers are likely to be a large component of the total impact. As Joel says there is a lack of data, and James Snowden’s critique implies your current estimate is likely to be an overestimate for a number of reasons. Joel says in a comment“a high quality RCT would be very welcome for informing our views and settling our disagreements”. This implies even Joel accepts that, given the current strength of evidence, there isn’t clarity on spillovers.
Therefore I would personally be more inclined to fund a study estimating spillovers than funding Strongminds. I find it disappointing that you essentially rule out suggesting funding research when it is at least plausible that this is the most effective way to improve happiness as it might enable better use of funds (it just wouldn’t increase happiness immediately).
It can be more challenging to raise money for research than operations, and even without that adjustment: the amount HLI has raised for SM is only a fraction of the high six / low seven figures for a solid RCT. Moreover, I think it would be particularly difficult to get a spillover study funded until the Ozler study results. So it’s not clear to me that this option was or is realistically open to HLI.
I’ve just realized you were probably referring to Strongminds not HLI?
Funding Strongminds to carry out research into the efficacy of their own intervention seems a very bad idea to me given their incentive for the results to be favorable.
The studies I had in mind would be empirical field studies best carried out by academic economists. The studies a charity like HLI would then make use of the results from.
My reading of the Strongminds debate that has taken place is that the strength of the evidence wasn’t sufficient to list Strongminds as a top charity (relevant posts are 1, 2, 3).
With regards to spillovers Joel McGuire says in a separate post:
If the data isn’t good enough might it be worth suggesting people fund research studies rather than suggesting people fund the charity itself?
EDIT: I just want to say I would feel uncomfortable if anyone else updated too much based on my comments. I would encourage people to read the critiques I linked for themselves as well as HLI responses.
Hello Jack. I think people can and will have different conceptions of what the criteria to be on a/the ‘top charity’ list are, including what counts as sufficient strength of evidence. If strength of evidence is essential, that may well rule out any interventions focused on the longterm (whose effects we will never know) as well as deworming (the recommendation of which is substantially based on a single long-term study). The evidence relevant for StrongMinds was not trivial though: we drew on 39 studies of mental health interventions in LICs to calibrate our estimates.
We’d be very happy to see further research funded. However, we see part of our job as trying to inform donors who want to fund interventions, rather than research. On the current evidence and analysis we’ve been able to do, StrongMinds was the only organisation we felt comfortable recommending. We are working to update our existing analysis and search for new top interventions.
Thanks Michael. My main concern is that it doesn’t seem that there is enough clarity on the spillovers, and spillovers are likely to be a large component of the total impact. As Joel says there is a lack of data, and James Snowden’s critique implies your current estimate is likely to be an overestimate for a number of reasons. Joel says in a comment “a high quality RCT would be very welcome for informing our views and settling our disagreements”. This implies even Joel accepts that, given the current strength of evidence, there isn’t clarity on spillovers.
Therefore I would personally be more inclined to fund a study estimating spillovers than funding Strongminds. I find it disappointing that you essentially rule out suggesting funding research when it is at least plausible that this is the most effective way to improve happiness as it might enable better use of funds (it just wouldn’t increase happiness immediately).
It can be more challenging to raise money for research than operations, and even without that adjustment: the amount HLI has raised for SM is only a fraction of the high six / low seven figures for a solid RCT. Moreover, I think it would be particularly difficult to get a spillover study funded until the Ozler study results. So it’s not clear to me that this option was or is realistically open to HLI.
To be fair I didn’t have any idea how much an RCT would cost!
Though I guess the charity could run research studies?
I’ve just realized you were probably referring to Strongminds not HLI?
Funding Strongminds to carry out research into the efficacy of their own intervention seems a very bad idea to me given their incentive for the results to be favorable.
The studies I had in mind would be empirical field studies best carried out by academic economists. The studies a charity like HLI would then make use of the results from.