Are you familiar with MIRI’s work on this? One recent iteration is Functional Decision Theory, though it is unclear to me if they made more recent progress since then.
It took me a long time to come around to it, but I currently buy that FDT is superior to CDT in the twin prisoner’s dilemma case, while not falling to evidential blackmail (the way EDT does), as well as being notably superior overall in the stylized situation of “how should an agent relate to a world where other smarter agents can potentially read the agent’s source code”
Are you familiar with MIRI’s work on this? One recent iteration is Functional Decision Theory, though it is unclear to me if they made more recent progress since then.
It took me a long time to come around to it, but I currently buy that FDT is superior to CDT in the twin prisoner’s dilemma case, while not falling to evidential blackmail (the way EDT does), as well as being notably superior overall in the stylized situation of “how should an agent relate to a world where other smarter agents can potentially read the agent’s source code”
Thanks that’s interesting, I’ve heard of it but I haven’t looked into it.