A QALY is a health metric. Most of Scott’s giving targets economic mobility, education, and equity, whose value is largely non-health; a WELLBY or consumption frame would credit those buckets far more. The model is deliberately scoped to one question: how much health does the money buy? It is not a verdict on her choices.
and with your 2nd bullet:
The model also prices the same dollars at the global-health frontier. … At the skeptical defaults, the frontier delivers roughly 1,500× more health per marginal dollar.
That multiple is a marginal comparison — the next dollar, not the whole portfolio. Frontier-priced opportunities are scarce… And the implied frontier counterfactual — ~105 million QALYs at the default settings — would mean averting ~4.2 million child deaths, most of a full year of the world’s under-5 deaths; no amount of money buys that at bed-net prices. Redeploying the full $30 billion would ride up the marginal-cost curve — to a few hundred times rather than ~1,500×, at my guess — softer in magnitude, same in direction. The floor is direct cash, the one option with effectively unbounded capacity, which GiveWell now scores at 3–4× its own historic benchmark: in health-only terms, somewhere under 50–100×.
Thanks for the context but putting these caveats in the disclaimers doesn’t change how the tool works.
The underlying studies are solid. The problem is the layer on top of them. The scores and weights are AI set guesses on a slider, not field data. Again, that’s most true for the civic engagement, equity and justice buckets, which is where most of Scott’s money goes. The author says himself those buckets get “wide skeptical priors” because no health pathway has credible evidence. That isn’t a real effect adjusted for uncertainty. It’s a number filling in for missing data. Since those are the biggest buckets, that number is doing a lot of the work behind the headline frontier multiple. The comparison is weakest exactly where most of her money sits.
That’s my real concern. The tool takes her hardest to measure work and makes it look like it buys almost nothing, when that low number is a measurement gap, not a fact about the work. A model is only as strong as its weakest layer. For most of this portfolio, that layer is a guess standing in for how funding actually works on the ground, like whether a local coalition can absorb a sudden influx of cash, or the fact that structural change doesn’t run in a straight line to a health outcome.
Flagging that in the fine print is good practice, but it doesn’t fix it. The calculator is still running precise maths on top of guesses.
Hm, I empathise with your underlying concern, but don’t the sliders help with that?
(Tangentially, “precise maths on top of guesses” is basically how I’d describe all CEAs, including GiveWell’s which move hundreds of millions annually. The guesses get better in all kinds of ways, but at least when I last checked the AMF CEA it wasn’t using field data like you’d like. So in a sense you seem to be asking for more than whole teams of specialists can deliver, let alone one person and a bunch of subscriptions can do in their limited spare time?)
FWIW the app agrees with your 1st bullet:
and with your 2nd bullet:
Thanks for the context but putting these caveats in the disclaimers doesn’t change how the tool works.
The underlying studies are solid. The problem is the layer on top of them. The scores and weights are AI set guesses on a slider, not field data. Again, that’s most true for the civic engagement, equity and justice buckets, which is where most of Scott’s money goes. The author says himself those buckets get “wide skeptical priors” because no health pathway has credible evidence. That isn’t a real effect adjusted for uncertainty. It’s a number filling in for missing data. Since those are the biggest buckets, that number is doing a lot of the work behind the headline frontier multiple. The comparison is weakest exactly where most of her money sits.
That’s my real concern. The tool takes her hardest to measure work and makes it look like it buys almost nothing, when that low number is a measurement gap, not a fact about the work. A model is only as strong as its weakest layer. For most of this portfolio, that layer is a guess standing in for how funding actually works on the ground, like whether a local coalition can absorb a sudden influx of cash, or the fact that structural change doesn’t run in a straight line to a health outcome.
Flagging that in the fine print is good practice, but it doesn’t fix it. The calculator is still running precise maths on top of guesses.
Hm, I empathise with your underlying concern, but don’t the sliders help with that?
(Tangentially, “precise maths on top of guesses” is basically how I’d describe all CEAs, including GiveWell’s which move hundreds of millions annually. The guesses get better in all kinds of ways, but at least when I last checked the AMF CEA it wasn’t using field data like you’d like. So in a sense you seem to be asking for more than whole teams of specialists can deliver, let alone one person and a bunch of subscriptions can do in their limited spare time?)