I think that there is a radical solution: have the AGI aligned to a certain treaty that requires the AGI, instead of obeying all orders except for the ones determined by the Spec, to harvest at most a certain share of resources and to help humans only in certain ways that amplify humanity and don’t cause it to degrade, like teaching humans about the facts that mankind has already discovered or pointing out mistakes in humans’ works. Or protecting mankind from some other existential risks that are hard to deal with, like a nuclear war that might be caused by an accident. An even more radical idea close to hopium is that this type of alignment is even easier than the ones that induce the Curse or force people “to work makeshift government jobs” or to collect a generous basic income.
I think that there is a radical solution: have the AGI aligned to a certain treaty that requires the AGI, instead of obeying all orders except for the ones determined by the Spec, to harvest at most a certain share of resources and to help humans only in certain ways that amplify humanity and don’t cause it to degrade, like teaching humans about the facts that mankind has already discovered or pointing out mistakes in humans’ works. Or protecting mankind from some other existential risks that are hard to deal with, like a nuclear war that might be caused by an accident. An even more radical idea close to hopium is that this type of alignment is even easier than the ones that induce the Curse or force people “to work makeshift government jobs” or to collect a generous basic income.