Its also likely that $68000 doesn’t buy you anywhere near 20 lives through AMF having looked at their calcs, Would have thought 30-60 QALYs is closer.
Can you elaborate on this? GiveWell estimates $3400/life saved, and $68k/3.4k = 20 lives saved. Do you think GiveWell’s calculation is off by an order of magnitude? If so, I’d love to know why!
I’m conservative when estimating effects. Their spreadsheets let you play around with things like this. There are good reasons to be conservative. Taking two of the main considerations to discount the cost effectiveness estimate outside from givewell’s models:
1). using an outside view, we can see that the cost effectiveness estimates of these charities have decreased over time as greater scrutiny has been brought to bear—if you project going forwards, then we’d expect that estimate to come down further. This is backed up things like insecticide resistance not being in the model.
2). I don’t think we should attribute all of the gain in AMF’s activities to money given—I know that there is a causal argument—but I’d like to give some credit to Rob Mathers, partner funders etc. etc. so am unwilling to say I’ve saved a life by giving $x—its more that I’ve been part of a team that’s done so (so I discount against other activities with different numbers of actions by different people that mean it wouldn’t have happened)
3). Even thinking about causal level changes—AMF might well have little room for funding depending on how the next net distribution agreements go.
Have a play around with these and see for yourself how fragile the estimates are on a number of assumptions. Its all related to how confident you are in the data and the assumptions, and how reliable you want to be when you say you can save a life for $x.
Holden has repeatedly said that these kind of cost effectiveness estimates aren’t enough to hang your hat on.
They do seem to look for reasons that their estimates might be too high a lot more vigorously than they look for reasons that their estimates might be too low. (Though that’s not necessarily a problem).
edit: they look hard for reasons that they might be overestimating the benefits
Huh, my impression is the exact opposite: GiveWell leaves out a number of reasons that would lower their estimated cost-per-life-saved (i.e. the estimate tries to err high right now, i.e. they adjusted for reasons that it would otherwise be too low). For instance, they only count the effect of reduced child mortality, not reduced morbidity. And by counting non-AMF costs towards the cost-per-life-saved, they assume that AMF’s partners would otherwise spend the money on something just as effective.
If you took all these into account, it would lower the denominator in the calculation in the grandparent, which would result in 68k saving more than 20 lives.
Oh, right, I don’t think I used the money from other partners in my calcs in the way you say givewell does—so if I was inadvertantly increasing the cost of a net that would be a mistake—double discounting.
Then, with the morbidity, its really a tiny effect in terms of QALYs so I’m not to concerned about that—even the income shock is relatively small in most places.
Can you elaborate on this? GiveWell estimates $3400/life saved, and $68k/3.4k = 20 lives saved. Do you think GiveWell’s calculation is off by an order of magnitude? If so, I’d love to know why!
I’m conservative when estimating effects. Their spreadsheets let you play around with things like this. There are good reasons to be conservative. Taking two of the main considerations to discount the cost effectiveness estimate outside from givewell’s models: 1). using an outside view, we can see that the cost effectiveness estimates of these charities have decreased over time as greater scrutiny has been brought to bear—if you project going forwards, then we’d expect that estimate to come down further. This is backed up things like insecticide resistance not being in the model. 2). I don’t think we should attribute all of the gain in AMF’s activities to money given—I know that there is a causal argument—but I’d like to give some credit to Rob Mathers, partner funders etc. etc. so am unwilling to say I’ve saved a life by giving $x—its more that I’ve been part of a team that’s done so (so I discount against other activities with different numbers of actions by different people that mean it wouldn’t have happened) 3). Even thinking about causal level changes—AMF might well have little room for funding depending on how the next net distribution agreements go.
Have a play around with these and see for yourself how fragile the estimates are on a number of assumptions. Its all related to how confident you are in the data and the assumptions, and how reliable you want to be when you say you can save a life for $x.
Holden has repeatedly said that these kind of cost effectiveness estimates aren’t enough to hang your hat on.
They do seem to look for reasons that their estimates might be too high a lot more vigorously than they look for reasons that their estimates might be too low. (Though that’s not necessarily a problem).
edit: they look hard for reasons that they might be overestimating the benefits
Huh, my impression is the exact opposite: GiveWell leaves out a number of reasons that would lower their estimated cost-per-life-saved (i.e. the estimate tries to err high right now, i.e. they adjusted for reasons that it would otherwise be too low). For instance, they only count the effect of reduced child mortality, not reduced morbidity. And by counting non-AMF costs towards the cost-per-life-saved, they assume that AMF’s partners would otherwise spend the money on something just as effective.
If you took all these into account, it would lower the denominator in the calculation in the grandparent, which would result in 68k saving more than 20 lives.
Oh, right, I don’t think I used the money from other partners in my calcs in the way you say givewell does—so if I was inadvertantly increasing the cost of a net that would be a mistake—double discounting.
Then, with the morbidity, its really a tiny effect in terms of QALYs so I’m not to concerned about that—even the income shock is relatively small in most places.
Oh, I was trying to say that they seem overly pessimistic, same as you.
Oh, ok. That would make 68k buy more than 20 lives saved, though, not less as tomstocker alleges.