Thanks for writing this up – the ceiling-cost estimate seems like a valuable tool for comparing interventions across different cause areas.
Second, I also assumed it would have an effect of 0.1 HALYs per person per year. This might be high, so let’s assume it has a 1/10th of that impact.
0.01 HALY/person/year on average still seems quite high. We’re estimating the average impact across a billion people, and any sort of systemic reform is going to have an enormous number of impacts (of varying magnitudes, in both directions). Attributing 0.01 HALY on average sorta assumes there aren’t any really big negative impacts (more precisely, that any negative impacts are insubstantial compared to the positive impacts).
It also seems difficult to separate out the impacts that are appropriate to attribute to the systemic reform from all the other effects that are going on in the background.
All this is to say that I think arriving at believable average-impact estimates for systemic interventions is tricky. It’s probably one of the harder parts of making good ceiling-cost estimates.
These numbers are just illustrative and to get people thinking, rather than to be taken literally.
Nevertheless, in some sense, it’s not the 0.01 that’s so important, it’s the ratio between that and the Give Directly score. I’m amusing the intervention, whatever it is, has a 1/50th of a effect Give Directly does. That seems pretty believable: massive campaign to restucture the trade and subsidy system could do quite a bit of shift people out of poverty.
We could make the average effect a 1/500th of the GD average effect and the mystery campaign would be cost-effective up to $14.6bn. That’s still a lot of sauce.
But yes, if you don’t think the intervention would do good, that would be a substantial reason to dodge it (presumably in favour of another systemic intervention).
You seem to be assuming that the “bad case” for systematic reform is that it’s, say, 1/500th of the benefit of the GD average effort. But I don’t think that’s the bad case for most systematic reforms: the bad case is that they’re actively harmful.
For me, at least, the core of my problem with “systematic reform” is that we’re “clueless” about its effects—it could have good effects, but could also have quite bad effects, and it’s extremely hard for us to tell which.
I think the ceiling cost estimate is a nice way of framing the comparison, but I agree with Milan that the hard bit is working out the expected effect.
There are some systemic reforms that seem easier reason about that others. Getting governments to be able to agree a tax scheme such that the Google’s and Facebook’s of the world can’t hide their profits, seems like a pretty good idea. Their money piles suggest that they aren’t hurting for cash to invest in innovation. It is hard to see the downside.
The upside is going to be less in developing world than the developed (due to more profits occurring in the developed world). So it may not be ideal. The tax justice network is something I want to follow more. They had a conversation with givewell.pdf)
There’s a sliding scale of what people consider “systematic reform”. Often people mean things like “replace capitalism”. I probably wouldn’t even have classed drug policy reform or tax reform as “systematic reform”, but it’s a vague category. Of course the simpler ones will be easier to analyze.
the core of my problem with “systematic reform” is that we’re “clueless” about its effects—it could have good effects, but could also have quite bad effects, and it’s extremely hard for us to tell which.
I think this can also apply for the atomic interventions EAs tend to like, namely those from GW. You can tell a story about how Give Directly increases meat consumption, so that’s bad. For life saving charities, there’s the same worry about meat, in addition to concerns about overpopulation. I’m not claiming we can’t sensible work through these and concude they all do more good than bad, only that cluelessness isn’t just a systemic intervention worry.
FWIW, I think this is way too broad. Even if, a priori, systemic interventions are more clueness-ny (?) than atomic interventions ones, it’s not that useful to talk about them as a category. It’d would be more useful to argue the toss on particular cases.
Sure—I don’t think “systematic change” is a well-defined category. The relevant distinction is “easy to analyze” vs “hard to analyze”. But in the post you’ve basically just stipulated that your example is easy to analyze, and I think that’s doing most of the work.
So I don’t think we should conclude that “systematic changes look much more effective”—as you say, we should look at them case by case.
Thanks for writing this up – the ceiling-cost estimate seems like a valuable tool for comparing interventions across different cause areas.
0.01 HALY/person/year on average still seems quite high. We’re estimating the average impact across a billion people, and any sort of systemic reform is going to have an enormous number of impacts (of varying magnitudes, in both directions). Attributing 0.01 HALY on average sorta assumes there aren’t any really big negative impacts (more precisely, that any negative impacts are insubstantial compared to the positive impacts).
It also seems difficult to separate out the impacts that are appropriate to attribute to the systemic reform from all the other effects that are going on in the background.
All this is to say that I think arriving at believable average-impact estimates for systemic interventions is tricky. It’s probably one of the harder parts of making good ceiling-cost estimates.
These numbers are just illustrative and to get people thinking, rather than to be taken literally.
Nevertheless, in some sense, it’s not the 0.01 that’s so important, it’s the ratio between that and the Give Directly score. I’m amusing the intervention, whatever it is, has a 1/50th of a effect Give Directly does. That seems pretty believable: massive campaign to restucture the trade and subsidy system could do quite a bit of shift people out of poverty.
We could make the average effect a 1/500th of the GD average effect and the mystery campaign would be cost-effective up to $14.6bn. That’s still a lot of sauce.
But yes, if you don’t think the intervention would do good, that would be a substantial reason to dodge it (presumably in favour of another systemic intervention).
You seem to be assuming that the “bad case” for systematic reform is that it’s, say, 1/500th of the benefit of the GD average effort. But I don’t think that’s the bad case for most systematic reforms: the bad case is that they’re actively harmful.
For me, at least, the core of my problem with “systematic reform” is that we’re “clueless” about its effects—it could have good effects, but could also have quite bad effects, and it’s extremely hard for us to tell which.
I think the ceiling cost estimate is a nice way of framing the comparison, but I agree with Milan that the hard bit is working out the expected effect.
There are some systemic reforms that seem easier reason about that others. Getting governments to be able to agree a tax scheme such that the Google’s and Facebook’s of the world can’t hide their profits, seems like a pretty good idea. Their money piles suggest that they aren’t hurting for cash to invest in innovation. It is hard to see the downside.
The upside is going to be less in developing world than the developed (due to more profits occurring in the developed world). So it may not be ideal. The tax justice network is something I want to follow more. They had a conversation with givewell.pdf)
There’s a sliding scale of what people consider “systematic reform”. Often people mean things like “replace capitalism”. I probably wouldn’t even have classed drug policy reform or tax reform as “systematic reform”, but it’s a vague category. Of course the simpler ones will be easier to analyze.
I think this can also apply for the atomic interventions EAs tend to like, namely those from GW. You can tell a story about how Give Directly increases meat consumption, so that’s bad. For life saving charities, there’s the same worry about meat, in addition to concerns about overpopulation. I’m not claiming we can’t sensible work through these and concude they all do more good than bad, only that cluelessness isn’t just a systemic intervention worry.
Frame it as a matter of degree if you like: I think we’re drastically more clueless about systematic reform than we are about atomic interventions.
FWIW, I think this is way too broad. Even if, a priori, systemic interventions are more clueness-ny (?) than atomic interventions ones, it’s not that useful to talk about them as a category. It’d would be more useful to argue the toss on particular cases.
Sure—I don’t think “systematic change” is a well-defined category. The relevant distinction is “easy to analyze” vs “hard to analyze”. But in the post you’ve basically just stipulated that your example is easy to analyze, and I think that’s doing most of the work.
So I don’t think we should conclude that “systematic changes look much more effective”—as you say, we should look at them case by case.