Nice analysis I agree. Seems hugely overweighted both instinctively and from the evidence.
As a side note, I’m feeling like Strongminds and HLI analysis has taken a perhaps out-of-proportion level of criticism of late (from me included). Not to say that the criticism isn’t important, but perhaps we should be having more of a hack at some of Givewell’s (or others) analysis as well. It may well be largely because HLI are so active on the forum which is great, and lay out their analysis pretty clearl. I would love for Givewell to post their fresh Analysis step by step on the forum as well sometimes—I genuinely find their analysis quite hard to understand and therefore hard to criticise too. Maybe I’m missing the place where they lay out their analysis step by step?
Or maybe the amazing quality analysis of Godly Givewell soars beyond us mere mortals and is beyond reproach ;) (full jokes, I know they even pay people to criticise their approach).
I think you’re in part picking up on a difference between what GiveWell publishes and what HLI publishes. In terms of money influenced, the most analogous GiveWell outcomes are probably grants from its All Grants Fund in the single-digit millions range, [1] rather than its Top Charities designations. There is a writeup for each grant, but for most of them there isn’t a published analysis that has anywhere near the depth of what HLI has put out for StrongMinds.
That isn’t a criticism of GiveWell; it is not recommending that individual donors give to organizations that receive grants (but are not Top Charities). In other words, GiveWell seeks to explain its decisions for its grants; HLI necessarily seeks to persuade donors to act in accordance with its recommendations. So it makes sense that GiveWell doesn’t devote the significant resources necessary to make its more detailed internal work product ready for public consumption for most non-Top Charities.
But the upshot is that we can’t compare apples to apples in a particular sense. In particular, we can’t compare detailed analysis that GiveWell currently thinks is solid enough to move single-digit millions counterfactually to HLI’s analysis that I expect would influence a similar level of money moved.
It is possible that HLI thinks the current StrongMinds analysis is solid enough to move tens or hundreds of millions of dollars counterfactually. However, for purposes of this comment, I am assuming a single-digit millions range based on my (uninformed) best guess of how much funding HLI’s recommendation might counterfactually drive over the next few years. Thus, I think it is fairest to evaluate HLI’s analysis on that basis.
Nice analysis I agree. Seems hugely overweighted both instinctively and from the evidence.
As a side note, I’m feeling like Strongminds and HLI analysis has taken a perhaps out-of-proportion level of criticism of late (from me included). Not to say that the criticism isn’t important, but perhaps we should be having more of a hack at some of Givewell’s (or others) analysis as well. It may well be largely because HLI are so active on the forum which is great, and lay out their analysis pretty clearl. I would love for Givewell to post their fresh Analysis step by step on the forum as well sometimes—I genuinely find their analysis quite hard to understand and therefore hard to criticise too. Maybe I’m missing the place where they lay out their analysis step by step?
Or maybe the amazing quality analysis of Godly Givewell soars beyond us mere mortals and is beyond reproach ;) (full jokes, I know they even pay people to criticise their approach).
I think you’re in part picking up on a difference between what GiveWell publishes and what HLI publishes. In terms of money influenced, the most analogous GiveWell outcomes are probably grants from its All Grants Fund in the single-digit millions range, [1] rather than its Top Charities designations. There is a writeup for each grant, but for most of them there isn’t a published analysis that has anywhere near the depth of what HLI has put out for StrongMinds.
That isn’t a criticism of GiveWell; it is not recommending that individual donors give to organizations that receive grants (but are not Top Charities). In other words, GiveWell seeks to explain its decisions for its grants; HLI necessarily seeks to persuade donors to act in accordance with its recommendations. So it makes sense that GiveWell doesn’t devote the significant resources necessary to make its more detailed internal work product ready for public consumption for most non-Top Charities.
But the upshot is that we can’t compare apples to apples in a particular sense. In particular, we can’t compare detailed analysis that GiveWell currently thinks is solid enough to move single-digit millions counterfactually to HLI’s analysis that I expect would influence a similar level of money moved.
It is possible that HLI thinks the current StrongMinds analysis is solid enough to move tens or hundreds of millions of dollars counterfactually. However, for purposes of this comment, I am assuming a single-digit millions range based on my (uninformed) best guess of how much funding HLI’s recommendation might counterfactually drive over the next few years. Thus, I think it is fairest to evaluate HLI’s analysis on that basis.