I don’t think this is true. But let’s say I did—what makes you think our expected value calculations will ever become good enough, and how will you know if they do?
what makes you think our expected value calculations will ever become good enough, and how will you know if they do?
Agree with this: it seems unclear to me that they’ll become good enough in many cases since our reasoning capabilities are fairly limited and the world is really complicated. I think this point is what Eliezer is trying to communicate with the tweet pictured in the post:
In my field (operations research), which is literally all about using optimization to make ‘optimal’ decisions, one way in which we account for issues like these is with robust optimization (RO).
In RO, you account for uncertainty by assuming unknown parameters (e.g. the weights between different possible objectives) lie within some predefined uncertainty set. You then maximize your objective over the worst case values of the uncertainty (ie, maximin optimization). In this way, you protect yourself against being wrong in a bounded worst-case way. Of course, this punts the problem to choosing a good uncertainty set, but I still think the heuristic “prefer to take actions which are good even under some mistaken assumptions” should be used more often.
The difference is, though, that in robust optimization you don’t really predict the future. You create an algorithm that you control, which uses some randomness or noisy estimates, and you prove that after enough time you can expect it to wind up somewhere. This is different than straight up giving an estimate of what a single choice will lead to, in a complex non-convex system where you have no control over anything else.
I wouldn’t quite say that. In any sort of multi-period optimization you must model the consequences of actions and are (at least implicitly) predicting something about the future.
Regardless I was mostly gesturing at the intuition, I agree this doesn’t solve the problem.
Well, this depends on what we are talking about, and what the alternatives are.
So for example, 80k hours’ ranking of careers is pretty great, and people do defer to it a whole lot, but they still make adjustments to it based on personal fit, personal values, etc. And you can imagine that as it gets better and better (e.g., as it gets more years poured into its development), the argument for deferring to it becomes more forceful.
That still doesn’t answer the question − 5 or 10 years from now, how do you know what the correct level of deference should be? And why do you expect it to approach 100% and not be bounded from above by something much smaller?
I don’t think this is true. But let’s say I did—what makes you think our expected value calculations will ever become good enough, and how will you know if they do?
Agree with this: it seems unclear to me that they’ll become good enough in many cases since our reasoning capabilities are fairly limited and the world is really complicated. I think this point is what Eliezer is trying to communicate with the tweet pictured in the post:
In my field (operations research), which is literally all about using optimization to make ‘optimal’ decisions, one way in which we account for issues like these is with robust optimization (RO).
In RO, you account for uncertainty by assuming unknown parameters (e.g. the weights between different possible objectives) lie within some predefined uncertainty set. You then maximize your objective over the worst case values of the uncertainty (ie, maximin optimization). In this way, you protect yourself against being wrong in a bounded worst-case way. Of course, this punts the problem to choosing a good uncertainty set, but I still think the heuristic “prefer to take actions which are good even under some mistaken assumptions” should be used more often.
The difference is, though, that in robust optimization you don’t really predict the future. You create an algorithm that you control, which uses some randomness or noisy estimates, and you prove that after enough time you can expect it to wind up somewhere. This is different than straight up giving an estimate of what a single choice will lead to, in a complex non-convex system where you have no control over anything else.
I wouldn’t quite say that. In any sort of multi-period optimization you must model the consequences of actions and are (at least implicitly) predicting something about the future.
Regardless I was mostly gesturing at the intuition, I agree this doesn’t solve the problem.
Well, this depends on what we are talking about, and what the alternatives are.
So for example, 80k hours’ ranking of careers is pretty great, and people do defer to it a whole lot, but they still make adjustments to it based on personal fit, personal values, etc. And you can imagine that as it gets better and better (e.g., as it gets more years poured into its development), the argument for deferring to it becomes more forceful.
That still doesn’t answer the question − 5 or 10 years from now, how do you know what the correct level of deference should be? And why do you expect it to approach 100% and not be bounded from above by something much smaller?