Agreed. It also doesn’t seem to me that even a successful alignment of AGI to “human values”—the same values which gave us war, slavery, and even today still torture billions of sentient beings per year before killing them—is prima facie a good thing.
Does AI alignment imply that the AGI must care about blind spots in today’s “human values”, and seek to expand its circle to include moral patients who would otherwise have been ignored? Not necessarily.
Agreed. It also doesn’t seem to me that even a successful alignment of AGI to “human values”—the same values which gave us war, slavery, and even today still torture billions of sentient beings per year before killing them—is prima facie a good thing.
Does AI alignment imply that the AGI must care about blind spots in today’s “human values”, and seek to expand its circle to include moral patients who would otherwise have been ignored? Not necessarily.