Thank you for your very informative response. I must admit that my knowledge of EBM is much more limited than yours and is primarily Wikipedia-based.
The lines which particularly led me to believe that EBM favoured formal approaches rather than doctors’ intuitions were:
“Although all medicine based on science has some degree of empirical support, EBM goes further, classifying evidence by its epistemologic strength and requiring that only the strongest types (coming from meta-analyses, systematic reviews, and randomized controlled trials) can yield strong recommendations; weaker types (such as from case-control studies) can yield only weak recommendations”
“Whether applied to medical education, decisions about individuals, guidelines and policies applied to populations, or administration of health services in general, evidence-based medicine advocates that to the greatest extent possible, decisions and policies should be based on evidence, not just the beliefs of practitioners, experts, or administrators.”
Criticism of EBM: “Research tends to focus on populations, but individual persons can vary substantially from population norms, meaning that extrapolation of lessons learned may founder. Thus EBM applies to groups of people, but this should not preclude clinicians from using their personal experience in deciding how to treat each patient.”
Perhaps the disagreement comes from my unintentional implication that the two camps were diametrically opposed to each other.
I agree that they are “both fundamentally important when you act in the real world” and that evidence based giving / evidence based medicine are not the last word on the matter and need to be supplemented by reason. At the same time though, I think there is an important distinction between maximising expected utility and being averse to ambiguity.
For example, to the best of my knowledge, the tradeoff between donating to SCI ($1.23 per treatment) and Deworm the World Initiative ($0.50 per treatment), is that DWI has demonstrated higher cost effectiveness but with a wider confidence interval (less of a track record). Interestingly, this actually sounds similar to your EGDT example. I therefore donate to SCI because I prefer to be confident in the effect. I think this distinction also applies to XRisk vs. development.
The methods of EBM do absolutely favour formal approaches and concrete results. However—and partly because of some of the pitfalls you describe—it’s relatively common to find you have no high quality evidence that specifically applies to inform your decision. It is also relatively common to find poor quality evidence (such as a badly constructed trial, or very confounded cohort studies). If those constitute the best-available evidence, a strict reading of the phrase ‘to greatest extent possible, decisions and policies should be based on evidence’ would imply that decisions should be founded on that dubious evidence. However in practice I think most doctors who are committed to EBM would not change their practice on the basis of a bad trial.
Regarding tradeoffs between maximising expected good and certainty of results (which I guess is maximising the minimum you achieve), I agree that’s a point where people come down on different sides. I don’t think it strictly divides causes (because as you say, one can lean to maximising expected utility within the global poverty), though the overlap between those who favour maximising expectation and those think existential risk is the best cause to focus on is probably high. I think this is actually going to be a topic of panel discussion at EA Global Oxford if you’re going?
Not to imply that you were implying otherwise, but I don’t think that the ‘evidence camp’ generally sees itself as maximising the minimum you achieve, or as disagreeing with maximising expected good. Instead it often disagrees with specific claims about what does the most good, particularly ones based on a certain sort of expected value calculation.
(In a way this only underscores your point that there isn’t that sharp a divide between the two approaches, and that we need to take into account all the evidence and reasons that we have. As you say, we often don’t have RCTs to settle things, leaving everyone with the tricky job of weighting different forms of evidence. There will be disagreements about that, but they won’t look like a sharp, binary division into two opposed ‘camps’. Describing what actually happens in medicine seems very helpful to understanding this.)
Bernadette,
Thank you for your very informative response. I must admit that my knowledge of EBM is much more limited than yours and is primarily Wikipedia-based.
The lines which particularly led me to believe that EBM favoured formal approaches rather than doctors’ intuitions were:
“Although all medicine based on science has some degree of empirical support, EBM goes further, classifying evidence by its epistemologic strength and requiring that only the strongest types (coming from meta-analyses, systematic reviews, and randomized controlled trials) can yield strong recommendations; weaker types (such as from case-control studies) can yield only weak recommendations”
“Whether applied to medical education, decisions about individuals, guidelines and policies applied to populations, or administration of health services in general, evidence-based medicine advocates that to the greatest extent possible, decisions and policies should be based on evidence, not just the beliefs of practitioners, experts, or administrators.”
Criticism of EBM: “Research tends to focus on populations, but individual persons can vary substantially from population norms, meaning that extrapolation of lessons learned may founder. Thus EBM applies to groups of people, but this should not preclude clinicians from using their personal experience in deciding how to treat each patient.”
Perhaps the disagreement comes from my unintentional implication that the two camps were diametrically opposed to each other.
I agree that they are “both fundamentally important when you act in the real world” and that evidence based giving / evidence based medicine are not the last word on the matter and need to be supplemented by reason. At the same time though, I think there is an important distinction between maximising expected utility and being averse to ambiguity.
For example, to the best of my knowledge, the tradeoff between donating to SCI ($1.23 per treatment) and Deworm the World Initiative ($0.50 per treatment), is that DWI has demonstrated higher cost effectiveness but with a wider confidence interval (less of a track record). Interestingly, this actually sounds similar to your EGDT example. I therefore donate to SCI because I prefer to be confident in the effect. I think this distinction also applies to XRisk vs. development.
Sorry for being slow to reply James.
The methods of EBM do absolutely favour formal approaches and concrete results. However—and partly because of some of the pitfalls you describe—it’s relatively common to find you have no high quality evidence that specifically applies to inform your decision. It is also relatively common to find poor quality evidence (such as a badly constructed trial, or very confounded cohort studies). If those constitute the best-available evidence, a strict reading of the phrase ‘to greatest extent possible, decisions and policies should be based on evidence’ would imply that decisions should be founded on that dubious evidence. However in practice I think most doctors who are committed to EBM would not change their practice on the basis of a bad trial.
Regarding tradeoffs between maximising expected good and certainty of results (which I guess is maximising the minimum you achieve), I agree that’s a point where people come down on different sides. I don’t think it strictly divides causes (because as you say, one can lean to maximising expected utility within the global poverty), though the overlap between those who favour maximising expectation and those think existential risk is the best cause to focus on is probably high. I think this is actually going to be a topic of panel discussion at EA Global Oxford if you’re going?
Not to imply that you were implying otherwise, but I don’t think that the ‘evidence camp’ generally sees itself as maximising the minimum you achieve, or as disagreeing with maximising expected good. Instead it often disagrees with specific claims about what does the most good, particularly ones based on a certain sort of expected value calculation.
(In a way this only underscores your point that there isn’t that sharp a divide between the two approaches, and that we need to take into account all the evidence and reasons that we have. As you say, we often don’t have RCTs to settle things, leaving everyone with the tricky job of weighting different forms of evidence. There will be disagreements about that, but they won’t look like a sharp, binary division into two opposed ‘camps’. Describing what actually happens in medicine seems very helpful to understanding this.)