Thanks for writing this up—I found it helpful. I’m just trying to summarise this in my head and have some questions.
To get the claim that the best interventions are much better than the rest, don’t you need to claim that interventions follow a (very) fat-tailed distribution, rather than the claim there are lots of interventions? If they were normally distributed, then (say) bednets would be a massive outlier in terms of effectiveness, right? Do you (or does someone else) have an argument that interventions should be heavy-tailed?
About predicting effectiveness, it seems your conclusion should be one of epistemic modesty relating to hard-to-quantify interventions, not that we should never think they are better. The thought seems to be people are bad at predicting interventions in general, but we can at least check for the easy-to-quantify predictions to overcome our bias; whereas we cannot do this for the hard ones. It seems the implication is that we should discount the naive cost-effectiveness of systemic interventions to account for this bias. But ‘sophisticated’ estimates of cost-effectiveness for hard-to-quantify interventions might still turn out to be better than those for estimates of simple interventions. Hence it’s a note of caution about estimations, not a claim that, in fact, hard to quantify interventions are (always or generally) less cost-effective.
You say that the distribution needs to be “very” fat tailed—implying that we have a decent chance of finding interventions order of mangitude more eefective than bed-nets. I disagree. The very most effective possible interventions, where the cost-benefit ratio is insanely large, are things that we don’t need to run as interventions. For instance, telling people to eat when they have food so they don’t starve would be really impactful if it weren’t unnecessary because of how obviously beneficial it is.
So I don’t think bednets are a massive outlier—they just have a relatively low saturation compared to most comparably effective interventions. The implication of my model is that most really effective interventions are saturated, often very quickly. Even expensive systemic efforts like vaccinations for smallpox got funded fairly rapidly after such universal eradication was possible, and the less used vaccines are either less effective, for less critical diseases, or are more expensive and/or harder to distribute. (And governments and foundations are running those campaigns, successfully, without needing EA pushing or funding.) And that’s why we see few very effective outliers—and since the underlying distribution isn’t fat tailed, even more effective interventions are even rarer, and those that did exist are gone very quickly.
On prediction, I agree that the conclusion is one of epistemic modesty rather than confident claims of non-effectiveness. But the practical implication of that modesty is that for any specific intervention, if we fund it thinking it may be really impactful, we’re incredibly unlikely to be correct.
Also, I’m far more skeptical than you about ‘sophisticated’ estimates. Having taken graduate courses in econometrics, I’ll say that the methods are sometimes really useful, but the assumptions never apply, and unless the system model is really fantastic, the prediction error once accounting for model specification uncertainty is large enough that most such econometric analyses of these sorts of really complex, poorly understood systems like corruption or poverty simply don’t say anything.
About predicting effectiveness, it seems your conclusion should be one of epistemic modesty relating to hard-to-quantify interventions, not that we should never think they are better.
This is where I’m at too – e.g. the impact of Bletchley Park would have been hard to quantify prospectively, and in retrospect was massively positive.
Curious if OP is actually saying the other thing (that hard-to-quantify implies lower cost-effectiveness).
See my comment above. Bletchley park was exactly the sort of intervention that doesn’t need any pushing. It was funded immediately because of how obvious the benefit was. That″s not retrospective.
If you were to suggest something similar now that were politically feasible and similarly important to a country, I’d be shocked if it wasn’t already happening. Invest in AI and advanced technologies? Check. Invest in Global Health Security? Also check. So the things left to do are less obviously good ideas.
“Turing decided to tackle the particularly difficult problem of German naval Enigma ‘because no one else was doing anything about it and I could have it to myself’.”
If you were to suggest something similar now that were politically feasible and similarly important to a country, I’d be shocked if it wasn’t already happening. Invest in AI and advanced technologies?...
What about AI alignment work circa 2010?
Quick examples from the present day: preparing for risks from nanotechnology; working on geoengineering safety
Bletchley park as an intervention wasn’t mostly focused on enigma, at least in the first part of the war. It was tremendously effective anyways, as should have been expected. The fact that new and harder codes were being broken was obviously useful as well, and from what I understood, was being encouraged by the leadership alongside the day-to-day codebreaking work.
And re: AI alignment, it WAS being funded. Regarding nanotech risks and geoengineering safety now, it’s been a focus of discussion at CSER and FHI, at least—and there is agreement about the relatively low priority of each compared to other work. (But if someone qualified and aligned with EA goals wanted to work on it more, there’s certainly funding available.)
I feel confused about whether there’s actually a disagreement here. Seems possible that we’re just talking past each other.
I agree that Bletchley Park wasn’t mostly focused on cracking Enigma.
I don’t know enough about Bletchley’s history to have an independent view about whether it was underfunded or not. I’ll follow your view that it was well supported.
It does seem like Turing’s work on Enigma wasn’t highly prioritized when he started working on it (”...because no one else was doing anything about it and I could have it to myself”), and this work turned out to be very impactful. I feel confident claiming that Bletchley wasn’t prioritizing Enigma highly enough before Turing decided to work on it. (Curious whether you disagree about this.)
On the present-day stuff:
My claim is that circa 2010 AI alignment work was being (dramatically) underfunded by institutions, not that it wasn’t being funded at all.
It wouldn’t surprise me if 20 years from now the consensus view was “Oh man, we totally should have been putting more effort towards figuring out what safe geoengineering looks like back in 2019.”
I believe Drexler had a hard time getting support to work on nanotech stuff (believe he’s currently working mostly on AI alignment), but I don’t know the full story there. (I’m holding Drexler as someone who is qualified and aligned with EA goals.)
I thought this was a really good comment – well written and well structured.
I feel confident claiming that Bletchley wasn’t prioritizing Enigma highly enough before Turing decided to work on it.
This is obvious from hindsight, but to make that claim you need to show that they could predict that the expected value was high in advance, which does seem to be the whole game.
I think we can drop the Bletchley park discussion. On the present-day stuff, I think they key point is that future-focused interventions have a very different set of questions than present-day non-quantifiable interventions, and you’re plausibly correct that they are underfunded—but I was trying to focus on the present-day non-quantifiable interventions.
I think we can drop the Bletchley park discussion.
Okay, I take it that you agree with my view.
… future-focused interventions have a very different set of questions than present-day non-quantifiable interventions
How are you separating out “future-focused interventions” from “present-day non-quantifiable interventions”?
Plausibly geoengineering safety will be very relevant in 15-30 years. Assuming that’s true, would you categorize geoengineering safety research as future-focused or present-day non-quantifiable?
I think my example of corruption reduction captures most of the types of interventions that people have suggested are useful but hard-to quantify, but other examples would be happiness focused work, or pushing for systemic change of various sorts.
Tech risks involving GCRs that are a decade or more away are much more future-focused in the sense that different arguments apply, as I said in the original post.
Thanks for writing this up—I found it helpful. I’m just trying to summarise this in my head and have some questions.
To get the claim that the best interventions are much better than the rest, don’t you need to claim that interventions follow a (very) fat-tailed distribution, rather than the claim there are lots of interventions? If they were normally distributed, then (say) bednets would be a massive outlier in terms of effectiveness, right? Do you (or does someone else) have an argument that interventions should be heavy-tailed?
About predicting effectiveness, it seems your conclusion should be one of epistemic modesty relating to hard-to-quantify interventions, not that we should never think they are better. The thought seems to be people are bad at predicting interventions in general, but we can at least check for the easy-to-quantify predictions to overcome our bias; whereas we cannot do this for the hard ones. It seems the implication is that we should discount the naive cost-effectiveness of systemic interventions to account for this bias. But ‘sophisticated’ estimates of cost-effectiveness for hard-to-quantify interventions might still turn out to be better than those for estimates of simple interventions. Hence it’s a note of caution about estimations, not a claim that, in fact, hard to quantify interventions are (always or generally) less cost-effective.
You say that the distribution needs to be “very” fat tailed—implying that we have a decent chance of finding interventions order of mangitude more eefective than bed-nets. I disagree. The very most effective possible interventions, where the cost-benefit ratio is insanely large, are things that we don’t need to run as interventions. For instance, telling people to eat when they have food so they don’t starve would be really impactful if it weren’t unnecessary because of how obviously beneficial it is.
So I don’t think bednets are a massive outlier—they just have a relatively low saturation compared to most comparably effective interventions. The implication of my model is that most really effective interventions are saturated, often very quickly. Even expensive systemic efforts like vaccinations for smallpox got funded fairly rapidly after such universal eradication was possible, and the less used vaccines are either less effective, for less critical diseases, or are more expensive and/or harder to distribute. (And governments and foundations are running those campaigns, successfully, without needing EA pushing or funding.) And that’s why we see few very effective outliers—and since the underlying distribution isn’t fat tailed, even more effective interventions are even rarer, and those that did exist are gone very quickly.
On prediction, I agree that the conclusion is one of epistemic modesty rather than confident claims of non-effectiveness. But the practical implication of that modesty is that for any specific intervention, if we fund it thinking it may be really impactful, we’re incredibly unlikely to be correct.
Also, I’m far more skeptical than you about ‘sophisticated’ estimates. Having taken graduate courses in econometrics, I’ll say that the methods are sometimes really useful, but the assumptions never apply, and unless the system model is really fantastic, the prediction error once accounting for model specification uncertainty is large enough that most such econometric analyses of these sorts of really complex, poorly understood systems like corruption or poverty simply don’t say anything.
This is where I’m at too – e.g. the impact of Bletchley Park would have been hard to quantify prospectively, and in retrospect was massively positive.
Curious if OP is actually saying the other thing (that hard-to-quantify implies lower cost-effectiveness).
See my comment above. Bletchley park was exactly the sort of intervention that doesn’t need any pushing. It was funded immediately because of how obvious the benefit was. That″s not retrospective.
If you were to suggest something similar now that were politically feasible and similarly important to a country, I’d be shocked if it wasn’t already happening. Invest in AI and advanced technologies? Check. Invest in Global Health Security? Also check. So the things left to do are less obviously good ideas.
Pretty sure that’s not right, at least for Turing’s work on Enigma:
“Turing decided to tackle the particularly difficult problem of German naval Enigma ‘because no one else was doing anything about it and I could have it to myself’.”
What about AI alignment work circa 2010?
Quick examples from the present day: preparing for risks from nanotechnology; working on geoengineering safety
Bletchley park as an intervention wasn’t mostly focused on enigma, at least in the first part of the war. It was tremendously effective anyways, as should have been expected. The fact that new and harder codes were being broken was obviously useful as well, and from what I understood, was being encouraged by the leadership alongside the day-to-day codebreaking work.
And re: AI alignment, it WAS being funded. Regarding nanotech risks and geoengineering safety now, it’s been a focus of discussion at CSER and FHI, at least—and there is agreement about the relatively low priority of each compared to other work. (But if someone qualified and aligned with EA goals wanted to work on it more, there’s certainly funding available.)
I feel confused about whether there’s actually a disagreement here. Seems possible that we’re just talking past each other.
I agree that Bletchley Park wasn’t mostly focused on cracking Enigma.
I don’t know enough about Bletchley’s history to have an independent view about whether it was underfunded or not. I’ll follow your view that it was well supported.
It does seem like Turing’s work on Enigma wasn’t highly prioritized when he started working on it (”...because no one else was doing anything about it and I could have it to myself”), and this work turned out to be very impactful. I feel confident claiming that Bletchley wasn’t prioritizing Enigma highly enough before Turing decided to work on it. (Curious whether you disagree about this.)
On the present-day stuff:
My claim is that circa 2010 AI alignment work was being (dramatically) underfunded by institutions, not that it wasn’t being funded at all.
It wouldn’t surprise me if 20 years from now the consensus view was “Oh man, we totally should have been putting more effort towards figuring out what safe geoengineering looks like back in 2019.”
I believe Drexler had a hard time getting support to work on nanotech stuff (believe he’s currently working mostly on AI alignment), but I don’t know the full story there. (I’m holding Drexler as someone who is qualified and aligned with EA goals.)
I thought this was a really good comment – well written and well structured.
This is obvious from hindsight, but to make that claim you need to show that they could predict that the expected value was high in advance, which does seem to be the whole game.
I think we can drop the Bletchley park discussion. On the present-day stuff, I think they key point is that future-focused interventions have a very different set of questions than present-day non-quantifiable interventions, and you’re plausibly correct that they are underfunded—but I was trying to focus on the present-day non-quantifiable interventions.
Okay, I take it that you agree with my view.
How are you separating out “future-focused interventions” from “present-day non-quantifiable interventions”?
Plausibly geoengineering safety will be very relevant in 15-30 years. Assuming that’s true, would you categorize geoengineering safety research as future-focused or present-day non-quantifiable?
I think my example of corruption reduction captures most of the types of interventions that people have suggested are useful but hard-to quantify, but other examples would be happiness focused work, or pushing for systemic change of various sorts.
Tech risks involving GCRs that are a decade or more away are much more future-focused in the sense that different arguments apply, as I said in the original post.