It might be worth looking at the domains where it might be less worthwhile (formal chaotic systems, or systems with many sign flipping crucial considerations). If you can show that trying to make cost-effectiveness based decisions in such environments is not worth it, that might strengthen your case.
...systems with many sign flipping crucial considerations
Yeah, I’m continuing to think about this, and would like to get more specific about which domains are most amiable to cost-effectiveness analysis (some related thinking here).
I think it’s very hard to identify which domains have the most crucial considerations, because such considerations are unveiled over long time frames.
A hypothesis that seems plausible: cost-effectiveness is good for deciding about which interventions to focus on within a given domain (e.g. “want to best reduce worldwide poverty in the next 20 years? These interventions should yield the biggest bang for buck...”)
But not so good for deciding about which domain to focus on, if you’re trying to select the domain that most helps the world over the entire course of the future. For that, comparing theories of change probably works better.
Aren’t there interventions that could be considered (with relatively high probability) robustly positive with regards to the long term future? Somewhat more abstract things such as “increasing empathy” or “improving human rationality” come to mind, but I guess one could argue how they could have a negative impact on the future in some plausible way. Another one certainly is “reduce existencial risks”—unless you weigh suffering risks so heavily that it’s unclear whether preventing existential risk is good or bad in the first place.
Regarding such causes—given we can identify robust ones—it then may still be valuable to analyze cost-effectiveness, as there would likely be a (high?) correlation between cost-effectiveness and positive impact on the future.
If you were to agree with that, then maybe we could reframe your argument from “cost-effectiveness may be of low value” to “cause areas outside of far future considerations are overrated (and hence their cost-effectiveness is measured in a way that is of little use)” or something like that.
Aren’t there interventions that could be considered (with relatively high probability) robustly positive with regards to the long term future?
I agree that interventions like this exist, and I think we identify them by making theoretical cases for & against.
Regarding such causes—given we can identify robust ones—it then may still be valuable to analyze cost-effectiveness
As above, I think cost-effectiveness can useful for determining which intervention to focus on within a specific domain (e.g. “which intervention most increases empathy?” could benefit from a cost-effect analysis).
But for questions about which domain to focus on, I don’t think cost-effectiveness gives much lift (e.g. “is it better to focus on increasing empathy or improving nuclear security?” is the kind of question that seems intractable to cost-effect analysis).
Sure, but I don’t think those are the only options.
Possible alternative option: come up with a granular theory of change; use that theory to inform decision-making.
I think this is basically what MIRI does. As far as I know, MIRI didn’t use cost-effectiveness analysis to decide on its research agenda (apart from very zoomed-out astronomical waste considerations).
Instead, it used a chain of theoretical reasoning to arrive at the intervention it’s focusing on.
I’m not sure I understand the distinction you’re making. In what sense is this compatible with your contention that “Any model that includes far-future effects isn’t believable because these effects are very difficult to predict accurately”? Is this “chain of theoretical reasoning” a “model that includes far-future effects”?
We do have a fair amount of documentation regarding successful forecasters, see e.g. the book Superforecasting. The most successful forecasters tend to rely less on a single theoretical model and more on an ensemble of models (hedgehogs vs foxes, to use Phil Tetlock’s terminology). Ensembles of models are also essential for winning machine learning competitions. (A big part of the reason I am studying machine learning, aside from AI safety, is its relevance to forecasting. Several of the top forecasters on Metaculus seem to be stats/ML folks, which makes sense because stats/ML is the closest thing we have to “the math of forecasting”.)
I’m not sure I understand the distinction you’re making...
I’m trying to distinguish between cost-effectiveness analyses (quantitative work that takes a bunch of inputs and arrives at a output, usually in the form of a best-guess cost-per-outcome), and theoretical reasoning (often qualitative, doesn’t arrive at a numerical cost-per-outcome, instead arrives at something like ”...and so this thing is probably best”).
Perhaps all theoretical reasoning is just a kind of imprecise cost-effect analysis, but I think they’re actually using pretty different mental processes.
The most successful forecasters tend to rely less on a single theoretical model and more on an ensemble of models...
Sure, but forecasters are working with pretty tight time horizons. I’ve never heard of a forecaster making predictions about what will happen 1000 years from now. (And even if one did, what could we make of such a prediction?)
My argument is that what we care about (the entire course of the future) extends far beyond what we can predict (the next few years, perhaps the next few decades).
“Anything you need to quantify can be measured in some way that is superior to not measuring it at all.”
My post is basically contesting the claim that any measurement is superior to no measurement in all domains.
It might be worth looking at the domains where it might be less worthwhile (formal chaotic systems, or systems with many sign flipping crucial considerations). If you can show that trying to make cost-effectiveness based decisions in such environments is not worth it, that might strengthen your case.
Yeah, I’m continuing to think about this, and would like to get more specific about which domains are most amiable to cost-effectiveness analysis (some related thinking here).
I think it’s very hard to identify which domains have the most crucial considerations, because such considerations are unveiled over long time frames.
A hypothesis that seems plausible: cost-effectiveness is good for deciding about which interventions to focus on within a given domain (e.g. “want to best reduce worldwide poverty in the next 20 years? These interventions should yield the biggest bang for buck...”)
But not so good for deciding about which domain to focus on, if you’re trying to select the domain that most helps the world over the entire course of the future. For that, comparing theories of change probably works better.
Aren’t there interventions that could be considered (with relatively high probability) robustly positive with regards to the long term future? Somewhat more abstract things such as “increasing empathy” or “improving human rationality” come to mind, but I guess one could argue how they could have a negative impact on the future in some plausible way. Another one certainly is “reduce existencial risks”—unless you weigh suffering risks so heavily that it’s unclear whether preventing existential risk is good or bad in the first place.
Regarding such causes—given we can identify robust ones—it then may still be valuable to analyze cost-effectiveness, as there would likely be a (high?) correlation between cost-effectiveness and positive impact on the future.
If you were to agree with that, then maybe we could reframe your argument from “cost-effectiveness may be of low value” to “cause areas outside of far future considerations are overrated (and hence their cost-effectiveness is measured in a way that is of little use)” or something like that.
I agree that interventions like this exist, and I think we identify them by making theoretical cases for & against.
As above, I think cost-effectiveness can useful for determining which intervention to focus on within a specific domain (e.g. “which intervention most increases empathy?” could benefit from a cost-effect analysis).
But for questions about which domain to focus on, I don’t think cost-effectiveness gives much lift (e.g. “is it better to focus on increasing empathy or improving nuclear security?” is the kind of question that seems intractable to cost-effect analysis).
Another way of saying it is “Sometimes pulling numbers out of your arse and using them to make a decision is better than pulling a decision out of your arse.” It’s taken from http://slatestarcodex.com/2013/05/02/if-its-worth-doing-its-worth-doing-with-made-up-statistics/ which is relevant here.
Sure, but I don’t think those are the only options.
Possible alternative option: come up with a granular theory of change; use that theory to inform decision-making.
I think this is basically what MIRI does. As far as I know, MIRI didn’t use cost-effectiveness analysis to decide on its research agenda (apart from very zoomed-out astronomical waste considerations).
Instead, it used a chain of theoretical reasoning to arrive at the intervention it’s focusing on.
I’m not sure I understand the distinction you’re making. In what sense is this compatible with your contention that “Any model that includes far-future effects isn’t believable because these effects are very difficult to predict accurately”? Is this “chain of theoretical reasoning” a “model that includes far-future effects”?
We do have a fair amount of documentation regarding successful forecasters, see e.g. the book Superforecasting. The most successful forecasters tend to rely less on a single theoretical model and more on an ensemble of models (hedgehogs vs foxes, to use Phil Tetlock’s terminology). Ensembles of models are also essential for winning machine learning competitions. (A big part of the reason I am studying machine learning, aside from AI safety, is its relevance to forecasting. Several of the top forecasters on Metaculus seem to be stats/ML folks, which makes sense because stats/ML is the closest thing we have to “the math of forecasting”.)
I’m trying to distinguish between cost-effectiveness analyses (quantitative work that takes a bunch of inputs and arrives at a output, usually in the form of a best-guess cost-per-outcome), and theoretical reasoning (often qualitative, doesn’t arrive at a numerical cost-per-outcome, instead arrives at something like ”...and so this thing is probably best”).
Perhaps all theoretical reasoning is just a kind of imprecise cost-effect analysis, but I think they’re actually using pretty different mental processes.
Sure, but forecasters are working with pretty tight time horizons. I’ve never heard of a forecaster making predictions about what will happen 1000 years from now. (And even if one did, what could we make of such a prediction?)
My argument is that what we care about (the entire course of the future) extends far beyond what we can predict (the next few years, perhaps the next few decades).