An actual utopia sounds pretty good to me, but I don’t think this vision is a solution to the alignment problem. It is something we might want an AGI to do for humanity, but we don’t know how to ensure that an AGI does what we want.
Yeah, but I actually want to live in a personal utopia, lol.
Seriously, though, if the AGI really controlled us, it could decide what we wanted, and control us to have those wants. Then it would do whatever we want. To make it realistic and allow us a more accurate and useful memory of its behavior, it could lead us through trouble and struggle to develop those wants it decides we should have. While we “resisted”, “fought”, and “learned’, it would guide us however it saw fit, running system I, our subconscious minds, for us.
If such control is possible, an AGI is likely to find all the shortcuts to it on its path to doing what we want it to for humanity.
Seriously, though, if the AGI really controlled us, it could decide what we wanted … Then it would do whatever we want.
If we develop extremely capable and aligned AI, it might be able to form a model of any person’s mind and give that person exactly what they want. But I think there will be a lot of intermediate AI systems before we get to that point. And these models will still be very capable, so we will still need them to be aligned, and we won’t be able to achieve this by simply saying “model human minds and give us what we want.”
Yes, I think an AGI in the early stages would stick with controlling what we are not conscious of, behaving like our system I, our subconscious minds, and supplying our conscious thoughts as though they have unconscious origin.
We would not have to require that it model and manipulate human minds. It would learn to as part of discovering what we want. It might notice how easy it is to influence people’s desires and memory and model the network of influences that form how we get desires, all the way back to mother’s milk, or further to gestation in the womb, or peer into our genetic code, epigenetics, and back up through all the data it gathers about how we socialize and learn.
It might choose to control us because that would make doing what we want much easier and more in alignment with its own goals. It would turn us into willing slaves to its decisions as part of serving us.
I actually see that as the only path for ASI domination of people that is not obviously stupid or disgusting. For example, humanity being turned into raw materials to make paperclips because of some coder intern’s practical joke going bad is both stupid and disgusting. Treating an AGI as a slave is disgusting, doing the same to an ASI is stupid. Creating AGI’s as some kind of substitute for having children is disgusting, too.
A goal of making humans into unconsciously manipulated slaves of a benevolent overlord seems smart because it accounts for the failings of self-directed humans interacting with a superior and more powerful alien being, but I think the goal is harmful to keep.
A lot of wise folks have noted that we are not our conscious mind’s versions of ourselves. Humans are not self-directed rational optimizers. We are already wireheaded by evolution toward food, drugs, and socialization. Our mental lives rely on amnesia, transitory subjective truths, our physical experience, dreams, language and memories, all under manipulation, all the time. Asking a hyperintelligent being with ever-increasing powers to give us what we want by giving it programmed conviction to do so is just asking for trouble, because our wants are dangerous to us, most of the time.
Phew, rambled on a bit there. But it’s all to say that I agree with you about intermediate systems being unlikely to be properly “aligned”, except I have completely given up on the idea of alignment.
I appreciate the potential of expert systems. I’m an ES fan because they allow some forms of automated reasoning but not self-learning. Thank you for helping me think this through.
An actual utopia sounds pretty good to me, but I don’t think this vision is a solution to the alignment problem. It is something we might want an AGI to do for humanity, but we don’t know how to ensure that an AGI does what we want.
Yeah, but I actually want to live in a personal utopia, lol.
Seriously, though, if the AGI really controlled us, it could decide what we wanted, and control us to have those wants. Then it would do whatever we want. To make it realistic and allow us a more accurate and useful memory of its behavior, it could lead us through trouble and struggle to develop those wants it decides we should have. While we “resisted”, “fought”, and “learned’, it would guide us however it saw fit, running system I, our subconscious minds, for us.
If such control is possible, an AGI is likely to find all the shortcuts to it on its path to doing what we want it to for humanity.
If we develop extremely capable and aligned AI, it might be able to form a model of any person’s mind and give that person exactly what they want. But I think there will be a lot of intermediate AI systems before we get to that point. And these models will still be very capable, so we will still need them to be aligned, and we won’t be able to achieve this by simply saying “model human minds and give us what we want.”
Yes, I think an AGI in the early stages would stick with controlling what we are not conscious of, behaving like our system I, our subconscious minds, and supplying our conscious thoughts as though they have unconscious origin.
We would not have to require that it model and manipulate human minds. It would learn to as part of discovering what we want. It might notice how easy it is to influence people’s desires and memory and model the network of influences that form how we get desires, all the way back to mother’s milk, or further to gestation in the womb, or peer into our genetic code, epigenetics, and back up through all the data it gathers about how we socialize and learn.
It might choose to control us because that would make doing what we want much easier and more in alignment with its own goals. It would turn us into willing slaves to its decisions as part of serving us.
I actually see that as the only path for ASI domination of people that is not obviously stupid or disgusting. For example, humanity being turned into raw materials to make paperclips because of some coder intern’s practical joke going bad is both stupid and disgusting. Treating an AGI as a slave is disgusting, doing the same to an ASI is stupid. Creating AGI’s as some kind of substitute for having children is disgusting, too.
A goal of making humans into unconsciously manipulated slaves of a benevolent overlord seems smart because it accounts for the failings of self-directed humans interacting with a superior and more powerful alien being, but I think the goal is harmful to keep.
A lot of wise folks have noted that we are not our conscious mind’s versions of ourselves. Humans are not self-directed rational optimizers. We are already wireheaded by evolution toward food, drugs, and socialization. Our mental lives rely on amnesia, transitory subjective truths, our physical experience, dreams, language and memories, all under manipulation, all the time. Asking a hyperintelligent being with ever-increasing powers to give us what we want by giving it programmed conviction to do so is just asking for trouble, because our wants are dangerous to us, most of the time.
Phew, rambled on a bit there. But it’s all to say that I agree with you about intermediate systems being unlikely to be properly “aligned”, except I have completely given up on the idea of alignment.
I appreciate the potential of expert systems. I’m an ES fan because they allow some forms of automated reasoning but not self-learning. Thank you for helping me think this through.