Lila, the future may not be controlled by a singleton but by a plurality of people implementing diverse values. But even if it is, the singleton may not maximize one value, but a mix of different values different people care about—a compromise “value handshake” as Scott Alexander called it.
Thus, it is best to emphasize you are not paperclip minimizers. The same goes for hedonium, unaided scientific insight, longevity, or art, to name just a few things some transhumanists value while others don’t.
There are two kinds of value conflicts: Ones where the values are merely orthogonal, and ones where values are either diametrically opposed or at least strongly negatively correlated in practice.
The orthogonal ones are still in conflict when limited resources are concerned—but not otherwise. It is much easier to find a compromise between them than between the opposed or practically negatively correlated ones.
There is no reason why a sigleton could not spend some resources on paperclips, some on hedonium, some on bigger happy minds, some on Fun, some on art, some on biodiversity, etc., if this increases the probability that people will compromise on letting the singleton come into being and be functional.
Lila, the future may not be controlled by a singleton but by a plurality of people implementing diverse values. But even if it is, the singleton may not maximize one value, but a mix of different values different people care about—a compromise “value handshake” as Scott Alexander called it.
Thus, it is best to emphasize you are not paperclip minimizers. The same goes for hedonium, unaided scientific insight, longevity, or art, to name just a few things some transhumanists value while others don’t.
There are two kinds of value conflicts: Ones where the values are merely orthogonal, and ones where values are either diametrically opposed or at least strongly negatively correlated in practice.
The orthogonal ones are still in conflict when limited resources are concerned—but not otherwise. It is much easier to find a compromise between them than between the opposed or practically negatively correlated ones.
There is no reason why a sigleton could not spend some resources on paperclips, some on hedonium, some on bigger happy minds, some on Fun, some on art, some on biodiversity, etc., if this increases the probability that people will compromise on letting the singleton come into being and be functional.