And I think that, even when one is extremely uncertain, the optimizer’s curse doesn’t mean you should change your preference ordering (just that you should be far less certain about it, as you’re probably greatlyoverestimating the value of best-seeming option).
I’m not very sure, but I imagine that the Optimizer’s curse might result in a reason against maximizing expected utility (though I’d distinguish it from using explicit probability models in general) if we’re dealing with a bounded budget—in which case, one might prefer a suboptimal option with low variance...?
(Plus, idk if this is helpful: in social contexts, a decision rule might incorporate the distribution of the cognitive burdens—I’m thinking about Prudence in Accounting, or maybe something like a limited precautionary principle. If you use an uninformative prior to assess a risk / liability / asset of a company, it might be tempted to hide information)
I now believe the statement of mine you quote was incorrect, and I’ve updated the optimizer’s curse section, primarily to remove the sentence you quoted (as I think it’s unnecessary in any case) and to alter an earlier part where I made a very similar claim so that it now says:
As best I can tell:
*The optimizer’s curse is likely to be a pervasive problem and is worth taking seriously.
*In many situations, the curse will just indicate that we’re probably overestimating how much better (compared to the alternatives) the option we estimate is best is—it won’t indicate that we should actually change what option we pick.
*But the curse can indicate that we should pick an option other than that which we estimate is best, if we have reason to believe that our estimate of the value of the best option is especially uncertain, and we don’t model that information.
(I think I already knew this but just previously didn’t explain it properly, leaving the conditions I had in mind as assumed, even though they often won’t hold in practice.)
But I think this updated version doesn’t address the points you make. From “if we’re dealing with a bounded budget—in which case, one might prefer a suboptimal option with low variance”, it sounds to me like maybe what you’re getting at is risk-aversion and/or diminishing returns to a particular thing?
For example, let’s say I can choose either A, which gives me $1 thousand in expectation, or B, which gives me $1 million in expectation. So far, B obviously seems way better. But what if B is way higher uncertainty (or way higher risk, if one prefers that phrasing)? Then maybe I’d prefer A.
I’d personally consider this biased if it’s pure risk-aversion, and the dollar values perfectly correspond to my “utility” from this. But in reality, each additional dollar is less valuable. For example, perhaps I’m broke, and by far the most important thing is that I get $1000 to get myself out of a real hole—a quite low chance of much higher payoffs isn’t worth it, because I get far less than 1000 times as much value out of 1000 times as much money.
If that’s what you were getting at, I think that’s all valid, and I think the optimizer’s curse does probably magnify those reasons to sometimes not go with what you estimate will give you, in expectation, the most of some thing you value. But I think really that doesn’t depend on the optimizer’s curse, and is more about uncertainty in general. Also, I think it’s really important to distinguish “maximising expected utility” from “maximising expected amount of some particular thing I value”. My understanding is that “risk-aversion” based on diminishing returns to dollars, for example, can 100% make sense within expected utility maximisation—it’s only pure risk-aversion (in terms of utility itself) that can’t.
(Let me know if I was totally misunderstanding you.)
I am very satisfied with the new text. I think you understood me pretty well; the problem is, I was a little bit unclear and ambiguous.
I’m not sure if this impacts your argument: I think diminishing returns accounts pretty well for saturation (ie., gaining $1 is not as important as losing $1); but it’s plausible to complement subjective expected utility theory with pure risk-aversion, like Lara Buchak does.
But what I actually had in mind is something like, in the extreme for unbounded utility, St. Petersburg paradox: if you’re willing to constantly bet all your budget, you’ll sure end up with $0 and bankrupt. In real life, I guess that if you were constantly updating your marginal utility per dollar, this wouldn’t be a problem (so I agree with you—this is not a challenge to expected utility maximisation).
Yeah, I’ve seen mentions of Buchak’s work and one talk from her, but didn’t really get it, and currently (with maybe medium confidence?) still think that, when talking about utility itself, and thus having accounted for diminishing returns and all that, one should be risk-neutral.
I hadn’t heard of martingales, and have relatively limited knowledge of the St Petersburg paradox. It seems to me (low confidence) that:
things like the St Petersburg paradox and Pascal’s mugging are plausible candidates for reasons to reject standard expected utility maximisation, at least in certain edge cases, and maybe also expected value reasoning
Recognising that there are diminishing returns to many (most?) things at least somewhat blunts the force of those weird cases
Things like accepting risk aversion or rounding infinitemal probabilities to 0 may solve the problems without us having to get rid of expected value reasoning or entirely get rid of expected utility maximisation (just augment it substantially)
There are some arguments for just accepting as rational what expected utility maximisation says in these edge cases—it’s not totally clear that our aversion to the “naive probabilistic” answer here is valid; maybe that aversion just reflects scope neglect, or the fact that, in the St Petersburg case, there’s the overlooked cost of it potentially taking months of continual play to earn substantial sums
I don’t think these reveal problems with using EPs specifically. It seems like the same problems could occur if you talked in qualitative terms about probabilities (e.g., “at least possible”, “fairly good odds”), and in either case the “fix” might look the same (e.g., rounding down either a quantitative or qualitative probability to 0 or to impossibility).
But it does seem that, in practice, people not using EPs are more likely to round down low probabilities to 0. This could be seen as good, for avoiding Pascal’s mugging, and/or as bad, for a whole host of other reasons (e.g., ignoring many x-risks).
Maybe a fuller version of this post would include edge cases like that, but I know less about them, and I think they could create “issues” (arguably) even when one isn’t using explicit probabilities anyway.
I mostly agree with you. I subtracted the reference to martingales from my previous comment because: a) not my expertise, b) this discussion doesn’t need additional complexity.
I’m sorry for having raised issues about paradoxes (perhaps there should be a Godwin’s Law about them); I don’t think we should mix edge cases like St. Petersburg (and problems with unbounded utility in general) with the optimizer’s curse – it’s already hard to analyze them separately.
when talking about utility itself, and thus having accounted for diminishing returns and all that, one should be risk-neutral.
Pace Buchak, I agree with that, but I wouldn’t say it aloud without adding caveats: in the real world, our problems are often of dynamic choice (and so one may have to think about optimal stopping and strategies, information gathering, etc.), we don’t observe utility-functions, we have limited cognitive resources, and we are evaluated and have to cooperate with others, etc. So I guess some “pure” risk-aversion might be a workable satisficing heuristics to [signal you] try to avoid the worst outcomes when you can’t account for all that. But that’s not talking about utility itself—and certainly not talking probability / uncertainty itself.
I subtracted the reference to martingales from my previous comment because: a) not my expertise, b) this discussion doesn’t need additional complexity.
I’m sorry for having raised issues about paradoxes (perhaps there should be a Godwin’s Law about them); I don’t think we should mix edge cases like St. Petersburg (and problems with unbounded utility in general) with the optimizer’s curse – it’s already hard to analyze them separately.
In line with the spirit of your comment, I believe, I think that it’s useful to recognise that not all discussions related to pros and cons of probabilities or how to use them or that sort of thing can or should address all potential issues. And I think that it’s good to recognise/acknowledge when a certain issue or edge case actually applies more broadly than just to the particular matter at hand (e.g., how St Petersburg is relevant even aside from the optimizer’s curse). An example of roughly the sort of reasoning I mean with that second sentence, from Tarsney writing on moral uncertainty:
The third worry suggests a broader objection, that content-based normalization approach in general is vulnerable to fanaticism. Suppose we conclude that a pluralistic hybrid of Kantianism and contractarianism would give lexical priority to Kantianism, and on this basis conclude that an agent who has positive credence in Kantianism, contractarianism, and this pluralistic hybrid ought to give lexical priority to Kantianism as well. [...]
I am willing to bite the bullet on this objection, up to a point: Some value claims may simply be more intrinsically weighty than others, and in some cases absolutely so. In cases where the agent’s credence in the lexically prioritized value claim approaches zero, however, the situation begins to resemble Pascal’s Wager (Pascal, 1669), the St. Petersburg Lottery (Bernoulli, 1738), and similar cases of extreme probabilities and magnitudes that bedevil decision theory in the context of merely empirical uncertainty. It is reasonable to hope, then, that the correct decision-theoretic solution to these problems (e.g. a dismissal of “rationally negligible probabilities” (Smith, 2014, 2016) or general rational permission for non-neutral risk attitudes (Buchak, 2013)) will blunt the force of the fanaticism objection.
But I certainly don’t think you need to apologise for raising those issues! They are relevant and very worthy of discussion—I just don’t know if they’re in the top 7 issues I’d discuss in this particular post, given its intended aims and my current knowledge base.
Oh, I only apologised because, well, if we start discussing about catchy paradoxes, we’ll soon lose the track of our original point.
But if you enjoy it, and since it is a relevant subject, I think people use 3 broad “strategies” to tackle St. Petersburg paradoxes and the like:
[epistemic status: low, but it kind makes sense]
a) “economist”: “if you use a bounded version, or takes time into account, the paradox disappears: just apply a logarithmic function for diminishing returns...”
b) “philosopher”: “unbounded utility is weird” or “beware, it’s Pascal’s Wager with objective probabilities!”
c) “statistician”: “the problem is this probability distribution, you can’t apply central limit / other theorem, or the indifference principle, or etc., and calculate its expectation”
I’m not very sure, but I imagine that the Optimizer’s curse might result in a reason against maximizing expected utility (though I’d distinguish it from using explicit probability models in general) if we’re dealing with a bounded budget—in which case, one might prefer a suboptimal option with low variance...?
(Plus, idk if this is helpful: in social contexts, a decision rule might incorporate the distribution of the cognitive burdens—I’m thinking about Prudence in Accounting, or maybe something like a limited precautionary principle. If you use an uninformative prior to assess a risk / liability / asset of a company, it might be tempted to hide information)
I now believe the statement of mine you quote was incorrect, and I’ve updated the optimizer’s curse section, primarily to remove the sentence you quoted (as I think it’s unnecessary in any case) and to alter an earlier part where I made a very similar claim so that it now says:
(I think I already knew this but just previously didn’t explain it properly, leaving the conditions I had in mind as assumed, even though they often won’t hold in practice.)
But I think this updated version doesn’t address the points you make. From “if we’re dealing with a bounded budget—in which case, one might prefer a suboptimal option with low variance”, it sounds to me like maybe what you’re getting at is risk-aversion and/or diminishing returns to a particular thing?
For example, let’s say I can choose either A, which gives me $1 thousand in expectation, or B, which gives me $1 million in expectation. So far, B obviously seems way better. But what if B is way higher uncertainty (or way higher risk, if one prefers that phrasing)? Then maybe I’d prefer A.
I’d personally consider this biased if it’s pure risk-aversion, and the dollar values perfectly correspond to my “utility” from this. But in reality, each additional dollar is less valuable. For example, perhaps I’m broke, and by far the most important thing is that I get $1000 to get myself out of a real hole—a quite low chance of much higher payoffs isn’t worth it, because I get far less than 1000 times as much value out of 1000 times as much money.
If that’s what you were getting at, I think that’s all valid, and I think the optimizer’s curse does probably magnify those reasons to sometimes not go with what you estimate will give you, in expectation, the most of some thing you value. But I think really that doesn’t depend on the optimizer’s curse, and is more about uncertainty in general. Also, I think it’s really important to distinguish “maximising expected utility” from “maximising expected amount of some particular thing I value”. My understanding is that “risk-aversion” based on diminishing returns to dollars, for example, can 100% make sense within expected utility maximisation—it’s only pure risk-aversion (in terms of utility itself) that can’t.
(Let me know if I was totally misunderstanding you.)
I am very satisfied with the new text. I think you understood me pretty well; the problem is, I was a little bit unclear and ambiguous.
I’m not sure if this impacts your argument: I think diminishing returns accounts pretty well for saturation (ie., gaining $1 is not as important as losing $1); but it’s plausible to complement subjective expected utility theory with pure risk-aversion, like Lara Buchak does.
But what I actually had in mind is something like, in the extreme for unbounded utility, St. Petersburg paradox: if you’re willing to constantly bet all your budget, you’ll sure end up with $0 and bankrupt. In real life, I guess that if you were constantly updating your marginal utility per dollar, this wouldn’t be a problem (so I agree with you—this is not a challenge to expected utility maximisation).
Yeah, I’ve seen mentions of Buchak’s work and one talk from her, but didn’t really get it, and currently (with maybe medium confidence?) still think that, when talking about utility itself, and thus having accounted for diminishing returns and all that, one should be risk-neutral.
I hadn’t heard of martingales, and have relatively limited knowledge of the St Petersburg paradox. It seems to me (low confidence) that:
things like the St Petersburg paradox and Pascal’s mugging are plausible candidates for reasons to reject standard expected utility maximisation, at least in certain edge cases, and maybe also expected value reasoning
Recognising that there are diminishing returns to many (most?) things at least somewhat blunts the force of those weird cases
Things like accepting risk aversion or rounding infinitemal probabilities to 0 may solve the problems without us having to get rid of expected value reasoning or entirely get rid of expected utility maximisation (just augment it substantially)
There are some arguments for just accepting as rational what expected utility maximisation says in these edge cases—it’s not totally clear that our aversion to the “naive probabilistic” answer here is valid; maybe that aversion just reflects scope neglect, or the fact that, in the St Petersburg case, there’s the overlooked cost of it potentially taking months of continual play to earn substantial sums
I don’t think these reveal problems with using EPs specifically. It seems like the same problems could occur if you talked in qualitative terms about probabilities (e.g., “at least possible”, “fairly good odds”), and in either case the “fix” might look the same (e.g., rounding down either a quantitative or qualitative probability to 0 or to impossibility).
But it does seem that, in practice, people not using EPs are more likely to round down low probabilities to 0. This could be seen as good, for avoiding Pascal’s mugging, and/or as bad, for a whole host of other reasons (e.g., ignoring many x-risks).
Maybe a fuller version of this post would include edge cases like that, but I know less about them, and I think they could create “issues” (arguably) even when one isn’t using explicit probabilities anyway.
I mostly agree with you. I subtracted the reference to martingales from my previous comment because: a) not my expertise, b) this discussion doesn’t need additional complexity.
I’m sorry for having raised issues about paradoxes (perhaps there should be a Godwin’s Law about them); I don’t think we should mix edge cases like St. Petersburg (and problems with unbounded utility in general) with the optimizer’s curse – it’s already hard to analyze them separately.
Pace Buchak, I agree with that, but I wouldn’t say it aloud without adding caveats: in the real world, our problems are often of dynamic choice (and so one may have to think about optimal stopping and strategies, information gathering, etc.), we don’t observe utility-functions, we have limited cognitive resources, and we are evaluated and have to cooperate with others, etc. So I guess some “pure” risk-aversion might be a workable satisficing heuristics to [signal you] try to avoid the worst outcomes when you can’t account for all that. But that’s not talking about utility itself—and certainly not talking probability / uncertainty itself.
In line with the spirit of your comment, I believe, I think that it’s useful to recognise that not all discussions related to pros and cons of probabilities or how to use them or that sort of thing can or should address all potential issues. And I think that it’s good to recognise/acknowledge when a certain issue or edge case actually applies more broadly than just to the particular matter at hand (e.g., how St Petersburg is relevant even aside from the optimizer’s curse). An example of roughly the sort of reasoning I mean with that second sentence, from Tarsney writing on moral uncertainty:
But I certainly don’t think you need to apologise for raising those issues! They are relevant and very worthy of discussion—I just don’t know if they’re in the top 7 issues I’d discuss in this particular post, given its intended aims and my current knowledge base.
Oh, I only apologised because, well, if we start discussing about catchy paradoxes, we’ll soon lose the track of our original point.
But if you enjoy it, and since it is a relevant subject, I think people use 3 broad “strategies” to tackle St. Petersburg paradoxes and the like:
[epistemic status: low, but it kind makes sense]
a) “economist”: “if you use a bounded version, or takes time into account, the paradox disappears: just apply a logarithmic function for diminishing returns...”
b) “philosopher”: “unbounded utility is weird” or “beware, it’s Pascal’s Wager with objective probabilities!”
c) “statistician”: “the problem is this probability distribution, you can’t apply central limit / other theorem, or the indifference principle, or etc., and calculate its expectation”