I actually think thatās fine. You can always look it up if youāre interested in the details, but for the casual consumer of charity-evaluation information, the bottom-line best estimate is the info thatās decision-relevant, not the uncertainty range. I think itās completely fine for people to share core info like this without simultaneously sharing all the fine print. Just like itās OK for public health experts to promote simple pro-vax messaging that doesnāt include all the fine print.
(See moral misdirection for my principled account of when it is or isnāt OK to leave out information.)
Absent these ranges, I see these claims repeated all over the place as if $5000 really is an objectively correct answer and not a rough estimate.
Here you just seem to be repeating the mistake of assuming that presenting a best estimate without also presenting the uncertainty range is thereby to present it as certain. I disagree with that interpretative norm. There is no āas ifā being presented. Thatās on you.
Iāll take an intermediate position: most readers will at least unconsciously infer an uncertainty range when presented with a point estimate only. If my mechanic tells me their best estimate for fixing my car is $1000 without saying more, I should understand from that $1200 is a reasonable possibility but would legitimately be upset if presented with a $2000 bill even if $1000 were provably the mean, median, mode, and likely outcome.
Here, I think the reader is on notice that estimating cost to save a life is likely to involve some imprecision, plus it is presented as an estimate, it is linked to a more detailed explanation, and it is rounded off.
There would be cases in which more should be said about the uncertainty range, for instance if it were between $500 and $50K! In that kind of scenario, you would need to say more to clue the reader into the degree of imprecision.
Yeah. The words āestimatesā and āaboutā are right there in the quote. There is no pretension of certainty here, unless you think mere use of numbers amounts to pretended certainty.
But what is decision relevant is the expected value. So by best estimate do they mean expected value, or maximum likelihood estimate, or something else? To my ear, ābest estimateā sounds like it means the estimate most likely to be right, and not the mean of the probability distribution. For instance, take the (B) option in āWhy it can be OK to predictably loseā, where you have a 1% chance of saving 1000 people, and a 99% chance of saving no one, and the choice is non-repeatable. I would think the ābest estimateā of the effectiveness of option (B) is that you will save 0 lives. But what matters for decision making is the expected value which is 10 lives.
Sorry if this is a stupid question, Iām not very familiar with GiveWell.
Fair question! I donāt know the answer. But Iād be surprised if the two came apart too sharply in this case (even though, as you rightly note, they can drastically diverge in principle). My sense is that GiveWell aims to recommend relatively āsafeā bets, rather than a āhits-basedā EV-maximizing approach. (I think itās important to be transparent when recommending the latter, just because I take it many people are not in fact so comfortable with pursuing that strategy, even if I think they ought to be.)
I actually think thatās fine. You can always look it up if youāre interested in the details, but for the casual consumer of charity-evaluation information, the bottom-line best estimate is the info thatās decision-relevant, not the uncertainty range. I think itās completely fine for people to share core info like this without simultaneously sharing all the fine print. Just like itās OK for public health experts to promote simple pro-vax messaging that doesnāt include all the fine print.
(See moral misdirection for my principled account of when it is or isnāt OK to leave out information.)
Here you just seem to be repeating the mistake of assuming that presenting a best estimate without also presenting the uncertainty range is thereby to present it as certain. I disagree with that interpretative norm. There is no āas ifā being presented. Thatās on you.
Iāll take an intermediate position: most readers will at least unconsciously infer an uncertainty range when presented with a point estimate only. If my mechanic tells me their best estimate for fixing my car is $1000 without saying more, I should understand from that $1200 is a reasonable possibility but would legitimately be upset if presented with a $2000 bill even if $1000 were provably the mean, median, mode, and likely outcome.
Here, I think the reader is on notice that estimating cost to save a life is likely to involve some imprecision, plus it is presented as an estimate, it is linked to a more detailed explanation, and it is rounded off.
There would be cases in which more should be said about the uncertainty range, for instance if it were between $500 and $50K! In that kind of scenario, you would need to say more to clue the reader into the degree of imprecision.
Agreed!
Yeah. The words āestimatesā and āaboutā are right there in the quote. There is no pretension of certainty here, unless you think mere use of numbers amounts to pretended certainty.
But what is decision relevant is the expected value. So by best estimate do they mean expected value, or maximum likelihood estimate, or something else? To my ear, ābest estimateā sounds like it means the estimate most likely to be right, and not the mean of the probability distribution. For instance, take the (B) option in āWhy it can be OK to predictably loseā, where you have a 1% chance of saving 1000 people, and a 99% chance of saving no one, and the choice is non-repeatable. I would think the ābest estimateā of the effectiveness of option (B) is that you will save 0 lives. But what matters for decision making is the expected value which is 10 lives.
Sorry if this is a stupid question, Iām not very familiar with GiveWell.
Fair question! I donāt know the answer. But Iād be surprised if the two came apart too sharply in this case (even though, as you rightly note, they can drastically diverge in principle). My sense is that GiveWell aims to recommend relatively āsafeā bets, rather than a āhits-basedā EV-maximizing approach. (I think itās important to be transparent when recommending the latter, just because I take it many people are not in fact so comfortable with pursuing that strategy, even if I think they ought to be.)