The methods of EBM do absolutely favour formal approaches and concrete results. However—and partly because of some of the pitfalls you describe—it’s relatively common to find you have no high quality evidence that specifically applies to inform your decision. It is also relatively common to find poor quality evidence (such as a badly constructed trial, or very confounded cohort studies). If those constitute the best-available evidence, a strict reading of the phrase ‘to greatest extent possible, decisions and policies should be based on evidence’ would imply that decisions should be founded on that dubious evidence. However in practice I think most doctors who are committed to EBM would not change their practice on the basis of a bad trial.
Regarding tradeoffs between maximising expected good and certainty of results (which I guess is maximising the minimum you achieve), I agree that’s a point where people come down on different sides. I don’t think it strictly divides causes (because as you say, one can lean to maximising expected utility within the global poverty), though the overlap between those who favour maximising expectation and those think existential risk is the best cause to focus on is probably high. I think this is actually going to be a topic of panel discussion at EA Global Oxford if you’re going?
Not to imply that you were implying otherwise, but I don’t think that the ‘evidence camp’ generally sees itself as maximising the minimum you achieve, or as disagreeing with maximising expected good. Instead it often disagrees with specific claims about what does the most good, particularly ones based on a certain sort of expected value calculation.
(In a way this only underscores your point that there isn’t that sharp a divide between the two approaches, and that we need to take into account all the evidence and reasons that we have. As you say, we often don’t have RCTs to settle things, leaving everyone with the tricky job of weighting different forms of evidence. There will be disagreements about that, but they won’t look like a sharp, binary division into two opposed ‘camps’. Describing what actually happens in medicine seems very helpful to understanding this.)
Sorry for being slow to reply James.
The methods of EBM do absolutely favour formal approaches and concrete results. However—and partly because of some of the pitfalls you describe—it’s relatively common to find you have no high quality evidence that specifically applies to inform your decision. It is also relatively common to find poor quality evidence (such as a badly constructed trial, or very confounded cohort studies). If those constitute the best-available evidence, a strict reading of the phrase ‘to greatest extent possible, decisions and policies should be based on evidence’ would imply that decisions should be founded on that dubious evidence. However in practice I think most doctors who are committed to EBM would not change their practice on the basis of a bad trial.
Regarding tradeoffs between maximising expected good and certainty of results (which I guess is maximising the minimum you achieve), I agree that’s a point where people come down on different sides. I don’t think it strictly divides causes (because as you say, one can lean to maximising expected utility within the global poverty), though the overlap between those who favour maximising expectation and those think existential risk is the best cause to focus on is probably high. I think this is actually going to be a topic of panel discussion at EA Global Oxford if you’re going?
Not to imply that you were implying otherwise, but I don’t think that the ‘evidence camp’ generally sees itself as maximising the minimum you achieve, or as disagreeing with maximising expected good. Instead it often disagrees with specific claims about what does the most good, particularly ones based on a certain sort of expected value calculation.
(In a way this only underscores your point that there isn’t that sharp a divide between the two approaches, and that we need to take into account all the evidence and reasons that we have. As you say, we often don’t have RCTs to settle things, leaving everyone with the tricky job of weighting different forms of evidence. There will be disagreements about that, but they won’t look like a sharp, binary division into two opposed ‘camps’. Describing what actually happens in medicine seems very helpful to understanding this.)