MichaelA🔸 comments on Potential downsides of using explicit probabilities

[MichaelA🔸]

Yeah, I think all of that’s right. I ended up coincidentally finding my way to a bunch of stuff about Goodhart on LW that I think is what you were referring to in another comment, and I’ve realised my explanation of the curse moved too fast and left out details. I think I was implicitly imagining that we’d already adjusted for what we know about the uncertainties of the estimates of the different options—but that wasn’t made clear.

I’ve now removed the sentence you quote (as I think it was unnecessary there anyway), and changed my earlier claims to:

The implications of, and potential solutions to, the optimizer’s curse seem to be complicated and debatable. For more detail, see this post, Smith’s post, comments on Smith’s post, and discussion of the related problem of Goodhart’s law.

As best I can tell:

*The optimizer’s curse is likely to be a pervasive problem and is worth taking seriously.

*In many situations, the curse will just indicate that we’re probably overestimating how much better (compared to the alternatives) the option we estimate is best is—it won’t indicate that we should actually change what option we pick.

*But the curse can indicate that we should pick an option other than that which we estimate is best, if we have reason to believe that our estimate of the value of the best option is especially uncertain, and we don’t model that information.

I’ve deliberately kept the above points brief (again, see the sources linked to for further explanations and justifications). This is because those claims are only relevant to the question of when to use EPs if the optimizer’s curse is a larger problem when using EPs than when using alternative approaches, and I don’t think it necessarily is.

Now, that’s not very clear, but I think it’s more accurate, at least :D

[Ozzie Gooen]

I think that makes sense. Some of it is a matter of interpretation.

From one perspective, the optimizer’s curse is a dramatic and challenging dilemma facing modern analysis. From another perspective, it’s a rather obvious and simple artifact from poorly-done estimates.

I.E. they sometimes say that if mathamaticians realize something is possible, they consider the problem trivial. Here the optimizer’s curse is considered a reasonably-well-understood phenomena, unlike some other estimation-theory questions currently being faced.