Out of this bunch, let’s say that we get only 1 new EA who becomes a Giving What We Can pledger—a measly conversion rate of less than 0.1%. (Note: I made this conversion rate up as one that could feasibly be far below the conversion rate we might actually get to avoid overselling the project. A less than 0.1% conversion rate would be terrible for most companies.)
A 0.1% (or worse) conversion rate doesn’t seem that implausible to me, because the ask is so large. I think it’s pretty similar to asking people to go vegetarian based on flyering which, though we don’t yet have good evidence, is currently pegged to be a conversion rate somewhere between 0.1% and 3%.
It does seem to be a large and in that respect ill chosen ask and conversion target. I wouldn’t be surprised if finding 50 mailing lists yields no one who gets convinced to donate 10% of their lifetime income as a result of getting some emails. At best it might find some more people who are already on track to do something similar, and there has been value to having their numbers listed publicly on the Giving What We Can website, but it’s less than that of saving 20 lives.
Have you thought of choosing a different conversion target? Even keeping to % pledges (which charities wouldn’t conventionally consider a good ask), The Life You Can Save pledge seems a better pick. Did you consider this alongside the Giving What We Can one, and if so why did you pick Giving What We Can?
(I ought to add that you may have talked about saving 20 lives as standard marketing hyperbole to get people excited, which would be fine! In that case I apologize if I’ve doused that excitement with cold water—treat this as a somewhat separate academic discussion.)
Yes, and its likely Tyler will find a way of populating that list anyway. Its also likely that $68000 doesn’t buy you anywhere near 20 lives through AMF having looked at their calcs, Would have thought 30-60 QALYs is closer.
Its also likely that $68000 doesn’t buy you anywhere near 20 lives through AMF having looked at their calcs, Would have thought 30-60 QALYs is closer.
Can you elaborate on this? GiveWell estimates $3400/life saved, and $68k/3.4k = 20 lives saved. Do you think GiveWell’s calculation is off by an order of magnitude? If so, I’d love to know why!
I’m conservative when estimating effects. Their spreadsheets let you play around with things like this. There are good reasons to be conservative. Taking two of the main considerations to discount the cost effectiveness estimate outside from givewell’s models:
1). using an outside view, we can see that the cost effectiveness estimates of these charities have decreased over time as greater scrutiny has been brought to bear—if you project going forwards, then we’d expect that estimate to come down further. This is backed up things like insecticide resistance not being in the model.
2). I don’t think we should attribute all of the gain in AMF’s activities to money given—I know that there is a causal argument—but I’d like to give some credit to Rob Mathers, partner funders etc. etc. so am unwilling to say I’ve saved a life by giving $x—its more that I’ve been part of a team that’s done so (so I discount against other activities with different numbers of actions by different people that mean it wouldn’t have happened)
3). Even thinking about causal level changes—AMF might well have little room for funding depending on how the next net distribution agreements go.
Have a play around with these and see for yourself how fragile the estimates are on a number of assumptions. Its all related to how confident you are in the data and the assumptions, and how reliable you want to be when you say you can save a life for $x.
Holden has repeatedly said that these kind of cost effectiveness estimates aren’t enough to hang your hat on.
They do seem to look for reasons that their estimates might be too high a lot more vigorously than they look for reasons that their estimates might be too low. (Though that’s not necessarily a problem).
edit: they look hard for reasons that they might be overestimating the benefits
Huh, my impression is the exact opposite: GiveWell leaves out a number of reasons that would lower their estimated cost-per-life-saved (i.e. the estimate tries to err high right now, i.e. they adjusted for reasons that it would otherwise be too low). For instance, they only count the effect of reduced child mortality, not reduced morbidity. And by counting non-AMF costs towards the cost-per-life-saved, they assume that AMF’s partners would otherwise spend the money on something just as effective.
If you took all these into account, it would lower the denominator in the calculation in the grandparent, which would result in 68k saving more than 20 lives.
Oh, right, I don’t think I used the money from other partners in my calcs in the way you say givewell does—so if I was inadvertantly increasing the cost of a net that would be a mistake—double discounting.
Then, with the morbidity, its really a tiny effect in terms of QALYs so I’m not to concerned about that—even the income shock is relatively small in most places.
Yes, the effect would be more to prevent him from dipping into the several hundreds of thousands of UK pounds of income CEA brings in each year (I’m using this figure.)
Or indeed many times less than 0.1% - it seems like you have a quite good sense of what the conversion rate is pegged to be, but don’t some people think vegan-creation leafleting can healthily be treated as not working for practical purposes?
The difference between this and vegan flyering is that you’re already targeting groups that have already self-selected for one aspect of EA. That said, I could definitely see a much lower than .1% rate being the case. Though the cost-effectiveness still seems competitive even at a conversion rate of .01% or even .001%. That’s 10 days and 100 days, respectively, of work for a year of earn-to-give.
That said, as Peter alluded earn-to-give still seems competitive if, e.g., you’re funding that much more of this work happens. Unless, by doing the work, you’re recruiting EtGers that will fund the work. Unless… [mind explodes]
Do vegan leafleters ever try to target groups they think’d be responsive? Does anyone (e.g. Peter Hurford) know what conversion rate do they get from those, on average?
A 0.1% (or worse) conversion rate doesn’t seem that implausible to me, because the ask is so large. I think it’s pretty similar to asking people to go vegetarian based on flyering which, though we don’t yet have good evidence, is currently pegged to be a conversion rate somewhere between 0.1% and 3%.
It does seem to be a large and in that respect ill chosen ask and conversion target. I wouldn’t be surprised if finding 50 mailing lists yields no one who gets convinced to donate 10% of their lifetime income as a result of getting some emails. At best it might find some more people who are already on track to do something similar, and there has been value to having their numbers listed publicly on the Giving What We Can website, but it’s less than that of saving 20 lives.
Have you thought of choosing a different conversion target? Even keeping to % pledges (which charities wouldn’t conventionally consider a good ask), The Life You Can Save pledge seems a better pick. Did you consider this alongside the Giving What We Can one, and if so why did you pick Giving What We Can?
(I ought to add that you may have talked about saving 20 lives as standard marketing hyperbole to get people excited, which would be fine! In that case I apologize if I’ve doused that excitement with cold water—treat this as a somewhat separate academic discussion.)
Yes, and its likely Tyler will find a way of populating that list anyway. Its also likely that $68000 doesn’t buy you anywhere near 20 lives through AMF having looked at their calcs, Would have thought 30-60 QALYs is closer.
Can you elaborate on this? GiveWell estimates $3400/life saved, and $68k/3.4k = 20 lives saved. Do you think GiveWell’s calculation is off by an order of magnitude? If so, I’d love to know why!
I’m conservative when estimating effects. Their spreadsheets let you play around with things like this. There are good reasons to be conservative. Taking two of the main considerations to discount the cost effectiveness estimate outside from givewell’s models: 1). using an outside view, we can see that the cost effectiveness estimates of these charities have decreased over time as greater scrutiny has been brought to bear—if you project going forwards, then we’d expect that estimate to come down further. This is backed up things like insecticide resistance not being in the model. 2). I don’t think we should attribute all of the gain in AMF’s activities to money given—I know that there is a causal argument—but I’d like to give some credit to Rob Mathers, partner funders etc. etc. so am unwilling to say I’ve saved a life by giving $x—its more that I’ve been part of a team that’s done so (so I discount against other activities with different numbers of actions by different people that mean it wouldn’t have happened) 3). Even thinking about causal level changes—AMF might well have little room for funding depending on how the next net distribution agreements go.
Have a play around with these and see for yourself how fragile the estimates are on a number of assumptions. Its all related to how confident you are in the data and the assumptions, and how reliable you want to be when you say you can save a life for $x.
Holden has repeatedly said that these kind of cost effectiveness estimates aren’t enough to hang your hat on.
They do seem to look for reasons that their estimates might be too high a lot more vigorously than they look for reasons that their estimates might be too low. (Though that’s not necessarily a problem).
edit: they look hard for reasons that they might be overestimating the benefits
Huh, my impression is the exact opposite: GiveWell leaves out a number of reasons that would lower their estimated cost-per-life-saved (i.e. the estimate tries to err high right now, i.e. they adjusted for reasons that it would otherwise be too low). For instance, they only count the effect of reduced child mortality, not reduced morbidity. And by counting non-AMF costs towards the cost-per-life-saved, they assume that AMF’s partners would otherwise spend the money on something just as effective.
If you took all these into account, it would lower the denominator in the calculation in the grandparent, which would result in 68k saving more than 20 lives.
Oh, right, I don’t think I used the money from other partners in my calcs in the way you say givewell does—so if I was inadvertantly increasing the cost of a net that would be a mistake—double discounting.
Then, with the morbidity, its really a tiny effect in terms of QALYs so I’m not to concerned about that—even the income shock is relatively small in most places.
Oh, I was trying to say that they seem overly pessimistic, same as you.
Oh, ok. That would make 68k buy more than 20 lives saved, though, not less as tomstocker alleges.
Yes, the effect would be more to prevent him from dipping into the several hundreds of thousands of UK pounds of income CEA brings in each year (I’m using this figure.)
Or indeed many times less than 0.1% - it seems like you have a quite good sense of what the conversion rate is pegged to be, but don’t some people think vegan-creation leafleting can healthily be treated as not working for practical purposes?
The difference between this and vegan flyering is that you’re already targeting groups that have already self-selected for one aspect of EA. That said, I could definitely see a much lower than .1% rate being the case. Though the cost-effectiveness still seems competitive even at a conversion rate of .01% or even .001%. That’s 10 days and 100 days, respectively, of work for a year of earn-to-give.
That said, as Peter alluded earn-to-give still seems competitive if, e.g., you’re funding that much more of this work happens. Unless, by doing the work, you’re recruiting EtGers that will fund the work. Unless… [mind explodes]
Do vegan leafleters ever try to target groups they think’d be responsive? Does anyone (e.g. Peter Hurford) know what conversion rate do they get from those, on average?