I’ve read those comments awhile back and I don’t think they support your view for relying overwhelmingly on explicit quantitative cost-effectiveness analyses. In particular the key parts I got out of Isabel’s comment weren’t what you quoted but instead (emphasis mine not hers)
Cost-effectiveness is the primary driver of our grantmaking decisions. But, “overall estimated cost-effectiveness of a grant” isn’t the same thing as “output of cost-effectiveness analysis spreadsheet.” (This blog post is old and not entirely reflective of our current approach, but it covers a similar topic.)
and
That is, we don’t solely rely on our spreadsheet-based analysis of cost-effectiveness when making grants.
which is in direct contradistinction to your style as I understand it, and aligned with what Holden wrote earlier in that link you quoted (emphasis his this time)
While some people feel that GiveWell puts too much emphasis on the measurable and quantifiable, there are others who go further than we do in quantification, and justify their giving (or other) decisions based on fully explicit expected-value formulas. The latter group tends to critique us – or at least disagree with us – based on our preference for strong evidence over high apparent “expected value,” and based on the heavy role of non-formalized intuition in our decisionmaking. This post is directed at the latter group.
We believe that people in this group are often making a fundamental mistake, one that we have long had intuitive objections to but have recently developed a more formal (though still fairly rough) critique of. The mistake (we believe) is estimating the “expected value” of a donation (or other action) based solely on a fully explicit, quantified formula, many of whose inputs are guesses or very rough estimates. We believe that any estimate along these lines needs to be adjusted using a “Bayesian prior”; that this adjustment can rarely be made (reasonably) using an explicit, formal calculation; and that most attempts to do the latter, even when they seem to be making very conservative downward adjustments to the expected value of an opportunity, are not making nearly large enough downward adjustments to be consistent with the proper Bayesian approach.
This view of ours illustrates why – while we seek to ground our recommendations in relevant facts, calculations and quantifications to the extent possible – every recommendation we make incorporates many different forms of evidence and involves a strong dose of intuition. And we generally prefer to give where we have strong evidence that donations can do a lot of good rather than where we have weak evidence that donations can do far more good – a preference that I believe is inconsistent with the approach of giving based on explicit expected-value formulas (at least those that (a) have significant room for error (b) do not incorporate Bayesian adjustments, which are very rare in these analyses and very difficult to do both formally and reasonably).
Note that I’m not saying CEAs don’t matter, or that CEA-focused approaches are unreliable — I’m a big believer in the measurability of things people often claim can’t be measured, I think in principle EV-maxing is almost always correct but in practice it can be perilous and on the margin people should instead be working a bit more on how different moral conceptions cash out in different recommendations more systematically e.g. with RP’s work, if a CEA-based case can’t be made for a grant I get very skeptical, I in fact also consider CEAs the main input into my thinking on these kinds of things, etc. I am simply wary of single-parameter optimisation taken to the limit in general (for anything, really, not just for donating), and I see your approach as being willing to go much further along that path than I do (and I’m already further along that path than almost anyone I meet IRL).
But I’ve seen enough back-and-forth between people in the cluster and sequence camps to have the sense that nobody really ends up changing their mind substantively and I doubt this will happen here either, sorry, so I will respectfully bow out of the conversation.
I’ve read those comments awhile back and I don’t think they support your view for relying overwhelmingly on explicit quantitative cost-effectiveness analyses. In particular the key parts I got out of Isabel’s comment weren’t what you quoted but instead (emphasis mine not hers)
and
which is in direct contradistinction to your style as I understand it, and aligned with what Holden wrote earlier in that link you quoted (emphasis his this time)
Note that I’m not saying CEAs don’t matter, or that CEA-focused approaches are unreliable — I’m a big believer in the measurability of things people often claim can’t be measured, I think in principle EV-maxing is almost always correct but in practice it can be perilous and on the margin people should instead be working a bit more on how different moral conceptions cash out in different recommendations more systematically e.g. with RP’s work, if a CEA-based case can’t be made for a grant I get very skeptical, I in fact also consider CEAs the main input into my thinking on these kinds of things, etc. I am simply wary of single-parameter optimisation taken to the limit in general (for anything, really, not just for donating), and I see your approach as being willing to go much further along that path than I do (and I’m already further along that path than almost anyone I meet IRL).
But I’ve seen enough back-and-forth between people in the cluster and sequence camps to have the sense that nobody really ends up changing their mind substantively and I doubt this will happen here either, sorry, so I will respectfully bow out of the conversation.
Thanks for the context, Mo!