Thanks for writing this! I think you touch upon a few facts that I’ve been thinking about a lot recently:
1. If the future consists of an ASI-enabled singleton (whether US-led, China-led, jointly-led, etc.), the moral values and posture of the singleton towards the rest of the world matter a lot. Given enough concentration of power, no outside force could realistically impose better morality on the singleton. 2. The present-day moral failures of current powerful people will not necessarily carry over in an ASI-enabled-abundance world. Dictator’s hands are often forced into oppressing their people. Dictator games in the experimental economics literature suggest unconditional altruism is frequent, although I don’t know what would occur if these games were run with extremely large sums of money. I suspect, however, that with extreme abundance, giving away a small fraction of this abundance to make the world a very good place to live (compared to where we are now, at least) seems like something almost anyone would do. Perhaps I have a naively optimistic view of human morality, however. Taken together, these suggest that the future we live in might be one that is essentially reliant on the sheer kindness of a singleton. This is more of a hope than a solution. I would rather not risk the singleton being absolutely terrible. I’m currently trying to think of ways in which a singleton could be organized to maximize the chances of things like moral innovation and a genuinely positive posture towards the rest of the world being present. If anyone knows of any work that has already been done on this, pointing this out to me would be much appreciated!
Thanks for writing this! I think you touch upon a few facts that I’ve been thinking about a lot recently:
1. If the future consists of an ASI-enabled singleton (whether US-led, China-led, jointly-led, etc.), the moral values and posture of the singleton towards the rest of the world matter a lot. Given enough concentration of power, no outside force could realistically impose better morality on the singleton.
2. The present-day moral failures of current powerful people will not necessarily carry over in an ASI-enabled-abundance world. Dictator’s hands are often forced into oppressing their people. Dictator games in the experimental economics literature suggest unconditional altruism is frequent, although I don’t know what would occur if these games were run with extremely large sums of money. I suspect, however, that with extreme abundance, giving away a small fraction of this abundance to make the world a very good place to live (compared to where we are now, at least) seems like something almost anyone would do. Perhaps I have a naively optimistic view of human morality, however.
Taken together, these suggest that the future we live in might be one that is essentially reliant on the sheer kindness of a singleton. This is more of a hope than a solution. I would rather not risk the singleton being absolutely terrible. I’m currently trying to think of ways in which a singleton could be organized to maximize the chances of things like moral innovation and a genuinely positive posture towards the rest of the world being present. If anyone knows of any work that has already been done on this, pointing this out to me would be much appreciated!
Nice, seems like important work, keen to hear what comes of this!