Pablo comments on I’m Buck Shlegeris, I do research and outreach at MIRI, AMA

[Pablo]

See also bmg’s LW post, Realism and rationality. Relevant excerpt:

A third point of tension is the community’s engagement with normative decision theory research. Different normative decision theories pick out different necessary conditions for an action to be the one that a given person should take, with a focus on how one should respond to uncertainty (rather than on what ends one should pursue).

A typical version of CDT says that the action you should take at a particular point in time is the one that would cause the largest expected increase in value (under some particular framework for evaluating causation). A typical version of EDT says that the action you should take at a particular point in time is the one that would, once you take it, allow you to rationally expect the most value. There are also alternative versions of these theories—for instance, versions using risk-weighted expected value maximization or the criterion of stochastic dominance—that break from the use of pure expected value.

I’ve pretty frequently seen it argued within the community (e.g. in the papers “Cheating Death in Damascus” and “Functional Decision Theory”) that CDT and EDT are not “correct” and that some other new theory such as functional decision theory is. But if anti-realism is true, then no decision theory is correct.

Eliezier Yudkowsky’s influential early writing on decision theory seems to me to take an anti-realist stance. It suggests that we can only ask meaningful questions about the effects and correlates of decisions. For example, in the context of the Newcomb thought experiment, we can ask whether one-boxing is correlated with winning more money. But, it suggests, we cannot take a step further and ask what these effects and correlations imply about what it is “reasonable” for an agent to do (i.e. what they should do). This question—the one that normative decision theory research, as I understand it, is generally about—is seemingly dismissed as vacuous.

If this apparently anti-realist stance is widely held, then I don’t understand why the community engages so heavily with normative decision theory research or why it takes part in discussions about which decision theory is “correct.” It strikes me a bit like an atheist enthustiastically following theological debates about which god is the true god. But I’m mostly just confused here.