Mmh, not sure how many of these problems can be solved by “just maximizing correctly”, or, as I like to call it, “better implementing utilitarianism”. E.g., some of the problems you gesture at seem like they could be solved by better evaluation and better expected value calculations.
Arguendo, once our expected value calculations become good enough, we should just switch to them, even if there is some “valley of naïve utilitarianism”.
I don’t think this is true. But let’s say I did—what makes you think our expected value calculations will ever become good enough, and how will you know if they do?
what makes you think our expected value calculations will ever become good enough, and how will you know if they do?
Agree with this: it seems unclear to me that they’ll become good enough in many cases since our reasoning capabilities are fairly limited and the world is really complicated. I think this point is what Eliezer is trying to communicate with the tweet pictured in the post:
In my field (operations research), which is literally all about using optimization to make ‘optimal’ decisions, one way in which we account for issues like these is with robust optimization (RO).
In RO, you account for uncertainty by assuming unknown parameters (e.g. the weights between different possible objectives) lie within some predefined uncertainty set. You then maximize your objective over the worst case values of the uncertainty (ie, maximin optimization). In this way, you protect yourself against being wrong in a bounded worst-case way. Of course, this punts the problem to choosing a good uncertainty set, but I still think the heuristic “prefer to take actions which are good even under some mistaken assumptions” should be used more often.
The difference is, though, that in robust optimization you don’t really predict the future. You create an algorithm that you control, which uses some randomness or noisy estimates, and you prove that after enough time you can expect it to wind up somewhere. This is different than straight up giving an estimate of what a single choice will lead to, in a complex non-convex system where you have no control over anything else.
I wouldn’t quite say that. In any sort of multi-period optimization you must model the consequences of actions and are (at least implicitly) predicting something about the future.
Regardless I was mostly gesturing at the intuition, I agree this doesn’t solve the problem.
Well, this depends on what we are talking about, and what the alternatives are.
So for example, 80k hours’ ranking of careers is pretty great, and people do defer to it a whole lot, but they still make adjustments to it based on personal fit, personal values, etc. And you can imagine that as it gets better and better (e.g., as it gets more years poured into its development), the argument for deferring to it becomes more forceful.
That still doesn’t answer the question − 5 or 10 years from now, how do you know what the correct level of deference should be? And why do you expect it to approach 100% and not be bounded from above by something much smaller?
Yes, there’s this weird class of arguments against consequentialism where people say “but following decision procedure X (e.g. blindly maximising short term consequences) will lead to worse outcomes!”. Yes, and that’s a good consequentialist argument for using a different decision procedure!
Perhaps someone needs to write the “Maximisation is not easy and if you think it is you will probably do it very badly” post.
Would you say that incorporating the “advice” of one of my common sense moral intuitions (e.g. “you should be nice”) can be considered part of a process called “better evaluating the EV”?
Mmh, not sure how many of these problems can be solved by “just maximizing correctly”, or, as I like to call it, “better implementing utilitarianism”. E.g., some of the problems you gesture at seem like they could be solved by better evaluation and better expected value calculations.
Arguendo, once our expected value calculations become good enough, we should just switch to them, even if there is some “valley of naïve utilitarianism”.
I don’t think this is true. But let’s say I did—what makes you think our expected value calculations will ever become good enough, and how will you know if they do?
Agree with this: it seems unclear to me that they’ll become good enough in many cases since our reasoning capabilities are fairly limited and the world is really complicated. I think this point is what Eliezer is trying to communicate with the tweet pictured in the post:
In my field (operations research), which is literally all about using optimization to make ‘optimal’ decisions, one way in which we account for issues like these is with robust optimization (RO).
In RO, you account for uncertainty by assuming unknown parameters (e.g. the weights between different possible objectives) lie within some predefined uncertainty set. You then maximize your objective over the worst case values of the uncertainty (ie, maximin optimization). In this way, you protect yourself against being wrong in a bounded worst-case way. Of course, this punts the problem to choosing a good uncertainty set, but I still think the heuristic “prefer to take actions which are good even under some mistaken assumptions” should be used more often.
The difference is, though, that in robust optimization you don’t really predict the future. You create an algorithm that you control, which uses some randomness or noisy estimates, and you prove that after enough time you can expect it to wind up somewhere. This is different than straight up giving an estimate of what a single choice will lead to, in a complex non-convex system where you have no control over anything else.
I wouldn’t quite say that. In any sort of multi-period optimization you must model the consequences of actions and are (at least implicitly) predicting something about the future.
Regardless I was mostly gesturing at the intuition, I agree this doesn’t solve the problem.
Well, this depends on what we are talking about, and what the alternatives are.
So for example, 80k hours’ ranking of careers is pretty great, and people do defer to it a whole lot, but they still make adjustments to it based on personal fit, personal values, etc. And you can imagine that as it gets better and better (e.g., as it gets more years poured into its development), the argument for deferring to it becomes more forceful.
That still doesn’t answer the question − 5 or 10 years from now, how do you know what the correct level of deference should be? And why do you expect it to approach 100% and not be bounded from above by something much smaller?
Yes, there’s this weird class of arguments against consequentialism where people say “but following decision procedure X (e.g. blindly maximising short term consequences) will lead to worse outcomes!”. Yes, and that’s a good consequentialist argument for using a different decision procedure!
Perhaps someone needs to write the “Maximisation is not easy and if you think it is you will probably do it very badly” post.
Would you say that incorporating the “advice” of one of my common sense moral intuitions (e.g. “you should be nice”) can be considered part of a process called “better evaluating the EV”?