I think that there’s an inevitable tradeoff between wanting a reflection process to have certain properties and worries about this violating goal preservation for at least some people. This blogpost is not about MCE directly, but if you think of “BAAN thought experiment” as “we do moral reflection and the outcome is such a wide circle that most people think it is extremely counterintuitive” then the reasoning in large parts of the blogpost should apply perfectly to the discussion here.
That is not to say that trying to fine tune reflection processes is pointless: I think it’s very important to think about what our desiderata should be for a CEV-like reflection process. I’m just saying that there will be tradeoffs between certain commonly mentioned desiderata that people don’t realize are there because they think there is such a thing as “genuinely free and open-ended deliberation.”
Thanks for commenting, Lukas. I think Lukas, Brian Tomasik, and others affiliated with FRI have thought more about this, and I basically defer to their views here, especially because I haven’t heard any reasonable people disagree with this particular point. Namely, I agree with Lukas that there seems to be an inevitable tradeoff here.
I think that there’s an inevitable tradeoff between wanting a reflection process to have certain properties and worries about this violating goal preservation for at least some people. This blogpost is not about MCE directly, but if you think of “BAAN thought experiment” as “we do moral reflection and the outcome is such a wide circle that most people think it is extremely counterintuitive” then the reasoning in large parts of the blogpost should apply perfectly to the discussion here.
That is not to say that trying to fine tune reflection processes is pointless: I think it’s very important to think about what our desiderata should be for a CEV-like reflection process. I’m just saying that there will be tradeoffs between certain commonly mentioned desiderata that people don’t realize are there because they think there is such a thing as “genuinely free and open-ended deliberation.”
Thanks for commenting, Lukas. I think Lukas, Brian Tomasik, and others affiliated with FRI have thought more about this, and I basically defer to their views here, especially because I haven’t heard any reasonable people disagree with this particular point. Namely, I agree with Lukas that there seems to be an inevitable tradeoff here.