The one view that seems unusually prevalent within FRI, apart from people self-identifying with suffering-focused values, is a particular anti-realist perspective on morality and moral reasoning where valuing open-ended moral reflection is not always regarded as the by default “prudent” thing to do.
Thanks for pointing this out. I’ve noticed this myself in some of FRI’s writings, and I’d say this, along with the high amount of certainty on various object-level philosophical questions that presumably cause the disvaluing of reflection about them, are what most “turns me off” about FRI. I worry a lot about potential failures of goal preservation (i.e., value drift) too, but because I’m highly uncertain about just about every meta-ethical and normativequestion, I see no choice but to try to design some sort of reflection procedure that I can trust enough to hand off control to. In other words, I have nothing I’d want to “lock in” at this point and since I’m by default constantly handing off control to my future self with few safeguards against value drift, doing something better than that default is one of my highest priorities. If other people are also uncertain and place high value on (safe/correct) reflection as a result, that helps with my goal (because we can then pool resources together to work out what safe/correct reflection is), so it’s regrettable to see FRI people sometimes argue for more certainty than I think is warranted and especially to see them argue against reflection.
That makes sense. I do think as a general policy, valuing reflection is more positive-sum, and if one does not feel like much is “locked in” yet then it becomes very natural too. I’m not saying that people who value reflection more than I do are doing it wrong; I think I would even argue for reflection being very important and recommend it to new people, if I felt more comfortable that they’d end up pursuing things that are beneficial from all/most plausible perspectives. Though what I find regrettable is that the “default” interventions that are said to be good from as many perspectives as possible oftentimes do not seem great from a suffering-focused perspective.
I’d also agree that designing trustworthy reflection procedures is important. My intuitions here are:
(1) value-drift is a big potential problem with FRI’s work (even if they “lock in” caring about suffering, if their definition of ‘suffering’ drifts, their tacit values do too);
(2) value-drift will be a problem for any system of ethics that doesn’t cleanly ‘compile to physics’. (This is a big claim, centering around my Objection 6, above.)
Perhaps we could generalize this latter point as “if information is physical, and value is informational, then value is physical too.”
Thanks for pointing this out. I’ve noticed this myself in some of FRI’s writings, and I’d say this, along with the high amount of certainty on various object-level philosophical questions that presumably cause the disvaluing of reflection about them, are what most “turns me off” about FRI. I worry a lot about potential failures of goal preservation (i.e., value drift) too, but because I’m highly uncertain about just about every meta-ethical and normative question, I see no choice but to try to design some sort of reflection procedure that I can trust enough to hand off control to. In other words, I have nothing I’d want to “lock in” at this point and since I’m by default constantly handing off control to my future self with few safeguards against value drift, doing something better than that default is one of my highest priorities. If other people are also uncertain and place high value on (safe/correct) reflection as a result, that helps with my goal (because we can then pool resources together to work out what safe/correct reflection is), so it’s regrettable to see FRI people sometimes argue for more certainty than I think is warranted and especially to see them argue against reflection.
That makes sense. I do think as a general policy, valuing reflection is more positive-sum, and if one does not feel like much is “locked in” yet then it becomes very natural too. I’m not saying that people who value reflection more than I do are doing it wrong; I think I would even argue for reflection being very important and recommend it to new people, if I felt more comfortable that they’d end up pursuing things that are beneficial from all/most plausible perspectives. Though what I find regrettable is that the “default” interventions that are said to be good from as many perspectives as possible oftentimes do not seem great from a suffering-focused perspective.
I really enjoyed your linked piece on meta-ethics. Short but insightful. I believe I’d fall into the second bucket.
If you’re looking for what (2) might look like in practice, and how we might try to relate it to the human brain’s architecture/drives, you might enjoy this: http://opentheory.net/2017/05/why-we-seek-out-pleasure-the-symmetry-theory-of-homeostatic-regulation/
I’d also agree that designing trustworthy reflection procedures is important. My intuitions here are: (1) value-drift is a big potential problem with FRI’s work (even if they “lock in” caring about suffering, if their definition of ‘suffering’ drifts, their tacit values do too); (2) value-drift will be a problem for any system of ethics that doesn’t cleanly ‘compile to physics’. (This is a big claim, centering around my Objection 6, above.)
Perhaps we could generalize this latter point as “if information is physical, and value is informational, then value is physical too.”