If I can try to rephrase your beliefs: economic rationality tells us that tradeoffs do in fact exist, and therefore rational agents must be able to make a comparison in every case. There has to be some amount of every value that you’d trade for another amount of every other value, otherwise you’ll end up paralyzed and decisionless.
You’re saying that, although we’d like to have this coherent total utility function, realistically it’s impossible to do so. We run into the theoretical problems you mention, and more fundamentally, some of our goals simply are not maximizing goals, and there is no rule that can accurately describe the relationship between those goals. Do we end up paralyzed and decisionless, with no principled way to tradeoff between the different goals? Yes, that’s unavoidable.
And one clarification: Would you say that this non-comparability is a feature more of human preferences, where we biologically have desires that aren’t integrated into a single utility function, or morality, where there are independent goals with independent moral worth?
I’m curious, what more specifically do you find it persuasive of? I generally feel that people do not easily want to bite the bullets that experiences have no independent positive value (only interdependent value), and that value doesn’t aggregate, and that outweighing “does not compute” in a physical balancing kind of sense. (I haven’t yet read the 2016 Oxford Handbook of Hypo-egoic Phenomena, but I expect that hypo-egoic systems, like many or most kinds of Buddhism, may be an angle from which it’d be easier to bite these bullets, though I don’t know much about the current intersection of Buddhism and EA.)
If I can try to rephrase your beliefs: economic rationality tells us that tradeoffs do in fact exist, and therefore rational agents must be able to make a comparison in every case. There has to be some amount of every value that you’d trade for another amount of every other value, otherwise you’ll end up paralyzed and decisionless.
You’re saying that, although we’d like to have this coherent total utility function, realistically it’s impossible to do so. We run into the theoretical problems you mention, and more fundamentally, some of our goals simply are not maximizing goals, and there is no rule that can accurately describe the relationship between those goals. Do we end up paralyzed and decisionless, with no principled way to tradeoff between the different goals? Yes, that’s unavoidable.
And one clarification: Would you say that this non-comparability is a feature more of human preferences, where we biologically have desires that aren’t integrated into a single utility function, or morality, where there are independent goals with independent moral worth?
Yes, we’ll unavoidably face quantitative resource splits between goals, none of which we can fully satisfy as long as we have even one infinitely hungry goal like “minimize suffering”, “maximize happiness”, or “maximize survival”. In practice, we can resolve conflicts between these goals by coming up with a common language to mediate trade between them, but how could they settle on an agreement if they were all independent and infinite goals? (My currently preferred solution is that happiness and survival are not such goals, compared to compassion.)
Alternatively, they could split from being a unified agent into being three agents, each independently unified, but they’d eventually run into conflicts again down the line (if they’re competing over control, space, energy, etc.). I’m interested in internal unity from the “minimize suffering” perspective, because violent competition from non-negotiating splitting causes suffering. In other words, I suppose my self-compassion wants my goals to play in harmony, and “self-compassion aligned with omnicompassion” is the unification that results in that harmony in the most robust way I can imagine.
More biological needs are more clearly in the domain of self-compassion, while omnicompassion is the theoretical attractor, asymptotically approximated ideal, or “gravity” that pulls self-compassion towards extended self-compassion, which is like a process of gradually importing more and more needs of others as my own, which increases expected harmony so long as it’s done slowly enough to maintain the harmony within the initial small self. For people with a chaotic life situation or who are working on a lot of unmet needs, this domain of self-compassion could occupy a lot of their attention for years while still being aligned with eventually extending self-compassion, which means that a seemingly non-helping person may be working on precisely the need areas that are long-term aligned with their role in minimizing suffering (or maximizing harmony or however one likes to think of it).
(I may have sidestepped your question of morality and moral worth, because I prefer to think in terms of needs and motivating tensions, and to see if the theoretical implications of unified consequentialism could be derived from our needs instead of abstract shoulds, obligations, imperatives, or duties.)
As a parallel comment, here is more (from a previous discussion) of why I am gravitating towards suffering as the only independent (dis)value and everything else as interdependently valuable in terms of preventing suffering:
“Life, consciousness, and activity; health and strength; pleasures and satisfactions of all or certain kinds; happiness, beatitude, contentment, etc.; truth; knowledge and true opinions of various kinds, understanding, wisdom; beauty, harmony, proportion in objects contemplated; aesthetic experience; morally good dispositions or virtues; mutual affection, love, friendship, cooperation; just distribution of goods and evils; harmony and proportion in one’s own life; power and experiences of achievement; self-expression; freedom; peace, security; adventure and novelty; and good reputation, honor, esteem, etc.”
but I don’t know how to ultimately prioritize between them unless they are commensurable. I make them commensurable by weighing their interdependent value in terms of the one thing we all(?) agree is an independent motivation: preventable suffering. (If preventable suffering is not worth preventing for its own sake, what is it worth preventing for, and is this other thing agreeable to someone undergoing the suffering as the reason for its motivating power?) This does not mean that I constantly think of them in these terms (that would be counterproductive), but in conflict resolution I do not assign them independent positive numerical values, which pluralism would imply one way or another.
Any pluralist theory begs the question of outweighing suffering with enough of any independently positive value. If you think about it for five minutes, aggregate happiness (or any other experience) does not exist. If our first priority is to prevent preventable suffering, that alone is an infinite game; it does not help to make a detour to boost/copy positive states unless this is causally connected to preventing suffering. (Aggregates of suffering do not exist either, but each moment of suffering is terminally worth preventing, and we have limited attention, so aggregates and chain-reactions of suffering are useful tools of thought for preventing as many as we can. So are many other things without requiring our attaching them independent positive value, or else we would be tiling Mars with them whenever it outweighed helping suffering on Earth according to some formula.)
My experience so far with this kind of unification is that it avoids many (or even all) of the theoretical problems that are still considered canonical challenges for pluralist utilitarianisms that assign both independent negative value to suffering and independent positive value to other things. I do not claim that this would be simple or intuitive – that would be analogous to reading about some Buddhist system, realizing its theoretical unity, and teleporting past its lifelong experiential integration – but I do claim that a unified theory with grounding in a universally accepted terminal value might be worth exploring further, because we cannot presuppose that any kind of CEV would be intuitive or easy to align oneself with.
[...]
People also differ along their background assumptions on whether AGI makes the universally life-preventing button a relevant question, because for many, the idea of an AGI represents an omnipotent optimizer that will decide everything about the future. If so, we want to be careful about assigning independent positive value to all the things, because each one of those invites this AGI to consider {outweighing suffering} with {producing those things}, since pluralist theories do not require a causal connection between the things being weighed.
Really cool thought, this is persuasive to me.
If I can try to rephrase your beliefs: economic rationality tells us that tradeoffs do in fact exist, and therefore rational agents must be able to make a comparison in every case. There has to be some amount of every value that you’d trade for another amount of every other value, otherwise you’ll end up paralyzed and decisionless.
You’re saying that, although we’d like to have this coherent total utility function, realistically it’s impossible to do so. We run into the theoretical problems you mention, and more fundamentally, some of our goals simply are not maximizing goals, and there is no rule that can accurately describe the relationship between those goals. Do we end up paralyzed and decisionless, with no principled way to tradeoff between the different goals? Yes, that’s unavoidable.
And one clarification: Would you say that this non-comparability is a feature more of human preferences, where we biologically have desires that aren’t integrated into a single utility function, or morality, where there are independent goals with independent moral worth?
I’m curious, what more specifically do you find it persuasive of? I generally feel that people do not easily want to bite the bullets that experiences have no independent positive value (only interdependent value), and that value doesn’t aggregate, and that outweighing “does not compute” in a physical balancing kind of sense. (I haven’t yet read the 2016 Oxford Handbook of Hypo-egoic Phenomena, but I expect that hypo-egoic systems, like many or most kinds of Buddhism, may be an angle from which it’d be easier to bite these bullets, though I don’t know much about the current intersection of Buddhism and EA.)
Yes, we’ll unavoidably face quantitative resource splits between goals, none of which we can fully satisfy as long as we have even one infinitely hungry goal like “minimize suffering”, “maximize happiness”, or “maximize survival”. In practice, we can resolve conflicts between these goals by coming up with a common language to mediate trade between them, but how could they settle on an agreement if they were all independent and infinite goals? (My currently preferred solution is that happiness and survival are not such goals, compared to compassion.)
Alternatively, they could split from being a unified agent into being three agents, each independently unified, but they’d eventually run into conflicts again down the line (if they’re competing over control, space, energy, etc.). I’m interested in internal unity from the “minimize suffering” perspective, because violent competition from non-negotiating splitting causes suffering. In other words, I suppose my self-compassion wants my goals to play in harmony, and “self-compassion aligned with omnicompassion” is the unification that results in that harmony in the most robust way I can imagine.
More biological needs are more clearly in the domain of self-compassion, while omnicompassion is the theoretical attractor, asymptotically approximated ideal, or “gravity” that pulls self-compassion towards extended self-compassion, which is like a process of gradually importing more and more needs of others as my own, which increases expected harmony so long as it’s done slowly enough to maintain the harmony within the initial small self. For people with a chaotic life situation or who are working on a lot of unmet needs, this domain of self-compassion could occupy a lot of their attention for years while still being aligned with eventually extending self-compassion, which means that a seemingly non-helping person may be working on precisely the need areas that are long-term aligned with their role in minimizing suffering (or maximizing harmony or however one likes to think of it).
(I may have sidestepped your question of morality and moral worth, because I prefer to think in terms of needs and motivating tensions, and to see if the theoretical implications of unified consequentialism could be derived from our needs instead of abstract shoulds, obligations, imperatives, or duties.)
As a parallel comment, here is more (from a previous discussion) of why I am gravitating towards suffering as the only independent (dis)value and everything else as interdependently valuable in terms of preventing suffering:
––––
I experience all of the things quoted in Complexity of value,
but I don’t know how to ultimately prioritize between them unless they are commensurable. I make them commensurable by weighing their interdependent value in terms of the one thing we all(?) agree is an independent motivation: preventable suffering. (If preventable suffering is not worth preventing for its own sake, what is it worth preventing for, and is this other thing agreeable to someone undergoing the suffering as the reason for its motivating power?) This does not mean that I constantly think of them in these terms (that would be counterproductive), but in conflict resolution I do not assign them independent positive numerical values, which pluralism would imply one way or another.
Any pluralist theory begs the question of outweighing suffering with enough of any independently positive value. If you think about it for five minutes, aggregate happiness (or any other experience) does not exist. If our first priority is to prevent preventable suffering, that alone is an infinite game; it does not help to make a detour to boost/copy positive states unless this is causally connected to preventing suffering. (Aggregates of suffering do not exist either, but each moment of suffering is terminally worth preventing, and we have limited attention, so aggregates and chain-reactions of suffering are useful tools of thought for preventing as many as we can. So are many other things without requiring our attaching them independent positive value, or else we would be tiling Mars with them whenever it outweighed helping suffering on Earth according to some formula.)
My experience so far with this kind of unification is that it avoids many (or even all) of the theoretical problems that are still considered canonical challenges for pluralist utilitarianisms that assign both independent negative value to suffering and independent positive value to other things. I do not claim that this would be simple or intuitive – that would be analogous to reading about some Buddhist system, realizing its theoretical unity, and teleporting past its lifelong experiential integration – but I do claim that a unified theory with grounding in a universally accepted terminal value might be worth exploring further, because we cannot presuppose that any kind of CEV would be intuitive or easy to align oneself with.
[...]
People also differ along their background assumptions on whether AGI makes the universally life-preventing button a relevant question, because for many, the idea of an AGI represents an omnipotent optimizer that will decide everything about the future. If so, we want to be careful about assigning independent positive value to all the things, because each one of those invites this AGI to consider {outweighing suffering} with {producing those things}, since pluralist theories do not require a causal connection between the things being weighed.