The demon case shows that there are cases where FDT loses, as is true of all decision theories. IF the question is which decision theory will programming into an AI generate most utility, then that’s an empirical question that depends on facts about the world. If it’s once you’re in a situation which will get the most utility, well, that’s causal decision theory.
Decision theories are intended as theories of what is rational for you to do. So it describes what choices are wise and which choices are foolish. I think Eliezer is confused about what a decision theory is, but that is a reason to trust his judgment less.
In the demon case, we can assume it’s only almost infallible, so every million times it makes a mistake. The demon case is a better example, because I have some credence in EVT, and EVT entails you should one box. I am waaaaaaaaaaaay more confident FDT is crazy than I am that you should two box.
I thought we already agreed the demon case showed that FDT wins in real life, since FDT agents will consistently end up with more utility than other agents.
Eliezer’s argument is that you can become the kind of entity that is programmed to do X, by choosing to do X. This is in some ways a claim about demons (they are good enough to predict even the choices you made with “your free will”). But it sounds like we’re in fact positing that demons are that good—I don’t know how to explain how they have 999,999/million success rate otherwise—so I think he is right.
I don’t think the demon being wrong one in a million times changes much. 999,999 of the people created by the demon will be some kind of FDT decision theorist with great precommitment skills. If you’re the one who isn’t, you can observe that you’re the demon’s rare mistake and avoid cutting off your legs, but this just means you won the lottery—it’s not a generally winning strategy.
Decision theories are intended as theories of what is rational for you to do. So it describes what choices are wise and which choices are foolish.
I don’t understand why you think that the choices that get you more utility with no drawbacks are foolish, and the choices that cost you utility for no reason are wise.
On the Newcomb’s Problem post, Eliezer explicitly said that he doesn’t care why other people are doing decision theory, he would like to figure out a way to get more utility. Then he did that. I think if you disagree with his goal, you should be arguing “decision theory should be about looking good, not about getting utility” (so we can all laugh at you) rather than saying “Eliezer is confidently and egregiously wrong” and hiding the fact that one of your main arguments is that he said we should try to get utility instead of failing all the time and then came up with a strategy that successfully does that.
We all agree that you should get utility. You are pointing out that FDT agents get more utility. But once they are already in the situation where they’ve been created by the demon, FDT agents get less utility. If you are the type of agent to follow FDT, you will get more utility, just as if you are the type of agent to follow CDT while being in a scenario that tortures FDTists, you’ll get more utility. The question of decision theory is, given the situation you are in, what gets you more utility—what is the rational thing to do. Eliezer’s turns you into the type of agent who often gets more utility, but that does not make it the right decision theory. The fact that you want to be the type of agent who does X doesn’t make doing X rational if doing X is bad for you and not doing X is rewarded artificially.
Again, there is no dispute about whether on average one boxers or two boxers get more utility or which kind of AI you should build.
The demon case shows that there are cases where FDT loses, as is true of all decision theories. IF the question is which decision theory will programming into an AI generate most utility, then that’s an empirical question that depends on facts about the world. If it’s once you’re in a situation which will get the most utility, well, that’s causal decision theory.
Decision theories are intended as theories of what is rational for you to do. So it describes what choices are wise and which choices are foolish. I think Eliezer is confused about what a decision theory is, but that is a reason to trust his judgment less.
In the demon case, we can assume it’s only almost infallible, so every million times it makes a mistake. The demon case is a better example, because I have some credence in EVT, and EVT entails you should one box. I am waaaaaaaaaaaay more confident FDT is crazy than I am that you should two box.
I thought we already agreed the demon case showed that FDT wins in real life, since FDT agents will consistently end up with more utility than other agents.
Eliezer’s argument is that you can become the kind of entity that is programmed to do X, by choosing to do X. This is in some ways a claim about demons (they are good enough to predict even the choices you made with “your free will”). But it sounds like we’re in fact positing that demons are that good—I don’t know how to explain how they have 999,999/million success rate otherwise—so I think he is right.
I don’t think the demon being wrong one in a million times changes much. 999,999 of the people created by the demon will be some kind of FDT decision theorist with great precommitment skills. If you’re the one who isn’t, you can observe that you’re the demon’s rare mistake and avoid cutting off your legs, but this just means you won the lottery—it’s not a generally winning strategy.
I don’t understand why you think that the choices that get you more utility with no drawbacks are foolish, and the choices that cost you utility for no reason are wise.
On the Newcomb’s Problem post, Eliezer explicitly said that he doesn’t care why other people are doing decision theory, he would like to figure out a way to get more utility. Then he did that. I think if you disagree with his goal, you should be arguing “decision theory should be about looking good, not about getting utility” (so we can all laugh at you) rather than saying “Eliezer is confidently and egregiously wrong” and hiding the fact that one of your main arguments is that he said we should try to get utility instead of failing all the time and then came up with a strategy that successfully does that.
We all agree that you should get utility. You are pointing out that FDT agents get more utility. But once they are already in the situation where they’ve been created by the demon, FDT agents get less utility. If you are the type of agent to follow FDT, you will get more utility, just as if you are the type of agent to follow CDT while being in a scenario that tortures FDTists, you’ll get more utility. The question of decision theory is, given the situation you are in, what gets you more utility—what is the rational thing to do. Eliezer’s turns you into the type of agent who often gets more utility, but that does not make it the right decision theory. The fact that you want to be the type of agent who does X doesn’t make doing X rational if doing X is bad for you and not doing X is rewarded artificially.
Again, there is no dispute about whether on average one boxers or two boxers get more utility or which kind of AI you should build.