EA should seek out more criticism of key EA concepts
EA’s efforts to maximise impact uses important concepts / thinking tools, including:
Maximisation
Expected value
The scientific method
Cost-efficiency
Hierarchies of evidence
The ITN Framework
Counterfactual reasoning
Use of probabilities derived from belief and expert opinion
Forecasting and modelling
Comparing point estimates with wide confidence intervals
Measuring and quantifying everything, even if there’s very high uncertainty involved
Cost-effectiveness bars
While these tools are used formally in write-ups and cost-effectiveness models, I think EAs also use these tools informally to make a variety of decisions.
As far as I’m aware, the only concept a where a weakness or criticism is widely known and widely pointed out, is expected value, where Pascal’s mugging comes up often.
Given that expected value has this weakness, I think
EA should seek out weaknesses of other thinking tools we use in EA, and maybe pay philosophers, statisticians and economists to make these weakness clearer to EAs.
More EAs should become aware of the weaknesses of other thinking tools often used in EA.
For example,
Givewell has written about how expected value also fails to consider evidence quality
There’s this Wikipedia section of criticisms of various hierarchies of evidence
I don’t understand what the value of criticizing the Scientific Method in the context of EA would be.
There are criticisms of the idea that a scientific approach is sufficient to gain a more complete understanding of morality but my impression is that that criticism mostly doesn’t apply to EA. EA seems more focused on applying science to the “how to do” of the question of “how to do the most good,” as opposed to the “good” part.
I’ve met EAs who have the conviction a scientific approach to morality is sufficient but they’re an unrepresentative minority. For moral philosophers in EA trying to advance EA as a theory of moral progress, while their work may be informed by science or a scientific mindset, I’ve only ever known most of them to not take a particularly scientistic approach.
As to applying scientific methods to develop practices in EA, there are also criticisms that the scientific method is not sufficient to identify what the best interventions are. I’m aware of almost no EA-aligned organizations that pursue their goals based on that conviction. There may be something to criticize about how EA relies too heavily on the scientific method as a still-imperfect tool. As a matter of kind, though, it seems like there aren’t any potential flaws of applying the scientific method in EA that haven’t already been internalized in the movement.
This was my one outlier in reading the list too.
I like the idea but I’ve got a question. What is a cost-effectiveness “bar?”
Do you mean a pre-existing set of one or more standard metrics that interventions being newly evaluated are measured against?
Or a preconceived threshold for cost-effectiveness an intervention has to clear before it will be seriously considered as plausibly competitive with existing top interventions? Or something else?
The second one, a theshold!