I really like idea here and think it’s presented well. (A+ use of illustrative graphs.) The tradeoff of “invest more in pondering vs invest in exploring object-level options” is very common.
I think that the credal ranges are not monotonically narrowing, mostly because we’re imperfect/bounded reasoners.
There are events in peoples lives / observations / etc that cause us to realize that we’ve incorrectly narrowed credence in the past and must now re-expand our uncertainty.
This theory still makes a lot of sense in that world—where termination might be followed up by re-initiation in the future, and uncertainty-expanding events would constitute a clear trigger for that re-initiation.
Value of information for updating our meta-uncertainty
Given the above point about our judgement on when/how to narrow credal ranges being flawed, I think we should care about improving at that meta-judgement.
This adds an additional value of information to pondering more—that we improve our judgement for when to stop pondering.
I think this is important to call out because this update is highly asymmetric—it’s much easier to get feedback that you pondered too long (by doing extra ponding for very little update) than to get feedback that you pondered too short (because you don’t know what you’d think if you pondered longer).
In cases where there is this very asymmetric value of information, I think a useful heuristic is “if in doubt, ponder too long, rather than too short” (this doesn’t really account for that fact that its not Yes/No as much as it is opportunity cost of other actions, but hopefully the heuristic can be adapted to be useful)
(Coda: this seems more like rationality than the modal EA forum post—maybe would get additional useful/insightful comments on LW)
I really like idea here and think it’s presented well. (A+ use of illustrative graphs.) The tradeoff of “invest more in pondering vs invest in exploring object-level options” is very common.
Two thoughts I’d like to add to this post:
re-initating deliberation & non-monotonic credence
I think that the credal ranges are not monotonically narrowing, mostly because we’re imperfect/bounded reasoners.
There are events in peoples lives / observations / etc that cause us to realize that we’ve incorrectly narrowed credence in the past and must now re-expand our uncertainty.
This theory still makes a lot of sense in that world—where termination might be followed up by re-initiation in the future, and uncertainty-expanding events would constitute a clear trigger for that re-initiation.
Value of information for updating our meta-uncertainty
Given the above point about our judgement on when/how to narrow credal ranges being flawed, I think we should care about improving at that meta-judgement.
This adds an additional value of information to pondering more—that we improve our judgement for when to stop pondering.
I think this is important to call out because this update is highly asymmetric—it’s much easier to get feedback that you pondered too long (by doing extra ponding for very little update) than to get feedback that you pondered too short (because you don’t know what you’d think if you pondered longer).
In cases where there is this very asymmetric value of information, I think a useful heuristic is “if in doubt, ponder too long, rather than too short” (this doesn’t really account for that fact that its not Yes/No as much as it is opportunity cost of other actions, but hopefully the heuristic can be adapted to be useful)
(Coda: this seems more like rationality than the modal EA forum post—maybe would get additional useful/insightful comments on LW)