FWIW, I didn’t get the impression there’s a very principled justification for softmax in this post, if that’s what you intended by “highly principled”. That it might work better than naive argmax in practice on some counts isn’t really enough, and there wasn’t really much comparison to enlightened argmax, which is optimal in theory.
I’d probably require being provably (approximately) optimal for a principled justification. Quickly checking bandits and the Gittins index on Wikipedia, bandits are general problems and the Gittins index is just the value of the aggregate reward. I guess you could say “maximize Gittins index” (use the Gittins index policy), but that’s, imo, just a formal characterization of what enlightened argmax should be under certain problem assumptions, and doesn’t provide much useful guidance on its own. Like what procedure should we follow to maximize the Gittins index? Is it just calculate really hard?
Also, according to the Wikipedia page, the Gittins index policy is optimal if the projects are independent, but not necessarily if they aren’t, and the problem is NP-hard in general if they can be dependent.
Not in this post, we just link to this one. By “principled” I just mean “not arbitrary, has a nice short derivation starting with something fundamental (like the entropy)”.
Yeah, the Gittins stuff would be pitched at a similar level of handwaving.
Looking back two weeks later, this post really needs
to discuss of the cost of prioritisation (we use softmax because we are boundedly rational) and the Price of Anarchy;
to have separate sections for individual prioritisation and collective prioritisation;
to at least mention bandits and the Gittins index, which is optimal where softmax is highly principled suboptimal cope.
FWIW, I didn’t get the impression there’s a very principled justification for softmax in this post, if that’s what you intended by “highly principled”. That it might work better than naive argmax in practice on some counts isn’t really enough, and there wasn’t really much comparison to enlightened argmax, which is optimal in theory.
I’d probably require being provably (approximately) optimal for a principled justification. Quickly checking bandits and the Gittins index on Wikipedia, bandits are general problems and the Gittins index is just the value of the aggregate reward. I guess you could say “maximize Gittins index” (use the Gittins index policy), but that’s, imo, just a formal characterization of what enlightened argmax should be under certain problem assumptions, and doesn’t provide much useful guidance on its own. Like what procedure should we follow to maximize the Gittins index? Is it just calculate really hard?
Also, according to the Wikipedia page, the Gittins index policy is optimal if the projects are independent, but not necessarily if they aren’t, and the problem is NP-hard in general if they can be dependent.
Not in this post, we just link to this one. By “principled” I just mean “not arbitrary, has a nice short derivation starting with something fundamental (like the entropy)”.
Yeah, the Gittins stuff would be pitched at a similar level of handwaving.