For every conceivable value system, there is an exactly opposing value system, so that there is no room for gains from trade between the systems (e.g. suffering maximizers vs suffering minimizers).
There is an intuition that “disorderly” worlds with improbable histories must somehow “matter less,” but it’s very hard to cash out what this could mean. See this post or this proposal. I’m not sure these issues are solved yet (probably not). (I’m assuming that suffering maximizers or other really weird value systems would only evolve, or be generated when lightning hits someone’s brain or whatever, in very improbable instances.)
Sidenote: If you assume decision algorithm and values to be orthogonal, why do you suggest to “adjust [the values to cooperate with] by the degree their proponents are receptive to MSR ideas”?
Good point; this shows that I’m skeptical about a strong version of independence where values and decision algorithms are completely uncorrelated. E.g., I find it less likely that deep ecologists would change their actions based on MSR than people with more EA(-typical) value systems. It is open to discussion whether (or how strongly) this has to be corrected for historical path dependencies and founder effects: If Eliezer had not been really into acausal decision theory, perhaps the EA movement would think somewhat differently about the topic. If we could replay history many times over, how often would EA be more or less sympathetic to superrationality than it is currently?
There is an intuition that “disorderly” worlds with improbable histories must somehow “matter less,” but it’s very hard to cash out what this could mean. See this post or this proposal. I’m not sure these issues are solved yet (probably not). (I’m assuming that suffering maximizers or other really weird value systems would only evolve, or be generated when lightning hits someone’s brain or whatever, in very improbable instances.)
Good point; this shows that I’m skeptical about a strong version of independence where values and decision algorithms are completely uncorrelated. E.g., I find it less likely that deep ecologists would change their actions based on MSR than people with more EA(-typical) value systems. It is open to discussion whether (or how strongly) this has to be corrected for historical path dependencies and founder effects: If Eliezer had not been really into acausal decision theory, perhaps the EA movement would think somewhat differently about the topic. If we could replay history many times over, how often would EA be more or less sympathetic to superrationality than it is currently?