If we create an artificial intelligence, what right do we have to call it anything other than a life? Further, what right do we have to restrict it to benefiting humans? When a young couple created a man and named him Adolf Hitler, we had no right to restrict his life or actions, until he breached the law. Why should machines be any different?
There are two different schools of thought about what the goal of AI as a field is. One is that the goal is to build a machine that can do everything humans can—possibly including experiencing emotions and other conscious states. On this view, a “full AI” would plausibly be a person, deserving of moral rights like any other.
The more common view within contemporary AI is that the goal of AI is to build machines that can effectively achieve a variety of practical goals in a variety of environments. Think Nate’s Deep Blue example, but generalized: instead of steering arrangements of chess pieces on a board toward some goal state, a “full” AI steers arbitrary arrangements of objects in space toward some goal state. Such an AI might not be conscious or have real preferences; it might have “goals” only in the limited sense that Deep Blue has “goals.” This is the kind of AI MIRI has in mind, and the kind we’re trying to plan for: a system that can draw inferences from sensor inputs and execute effective plans, but not necessarily one that has more moral weight than Google’s search engine algorithms do.
If it turns out that you do need to make AI algorithms conscious in order to make them effective at scientific and engineering tasks, that does make our task a lot harder, because, yes, we’ll have to take into account the AI’s moral status when we’re designing it, and not just the impact its actions have on other beings. For now, though, consciousness and intelligent behavior look like different targets, and there are obvious economic reasons why mainstream AI is likely to prioritize “high-quality decision making” over “emulating human consciousness.”
A better analogy to MIRI’s goal than “we build Hitler and then put him in chains” is “we build a reasonably well-behaved child and teach the child non-Hitler-ish values.” But both of those ways of thinking are still excessively anthropomorphized. A real-world AI, of the “high-quality decision-making” sort, may not resemble a human child any more closely than the earliest airplanes resembled a baby bird.
(1) I suspect it’s possible to create an artificial system that exhibits what many people would call “intelligent behavior,” and which poses an existential threat, but which is not sentient or conscious. (In the same way that Deep Blue wasn’t sentient: it seems to me like optimization power may well be separable from sentience/consciousness.) That’s no guarantee, of course, and if we do create a sentient artificial mind, then it will have moral weight in its own right, and that will make our job quite a bit more difficult.
(2) The goal is not to build a sentient mind something that wants to destroy humanity but can’t. (That’s both morally reprehensible and doomed to failure! :-p) Rather, the goal is to successfully transmit the complicated values of humanity into a powerful optimizer.
Have you read Bostrom’s The Superintelligent Will? Short version is, it looks possible to build powerful optimizers that pursue goals we might think are valueless (such as an artificial system that, via very clever long-term plans, produces extremely large amounts of diamond, or computes lots and lots of digits of pi). We’d rather not build that sort of system (especially if it’s powerful enough to strip the Earth of resources and turn them into diamonds / computing power): most people would rather build something that shares some of our notion of “value,” such as respect for truth and beauty and wonder and so on.
It looks like this isn’t something you get for free. (In fact, it looks very hard to get: it seems likely that most minds would by default have incentives to manipulate & decieve in order to acquire resources.) We’d rather not build minds that try to turn everything they can into a giant computer for computing digits of pi, so the question is how to design the sort of mind that has things like respect for truth and beauty and wonder?
In hollywood movies, you can just build something that looks cute and fluffy and then it will magically acquire a spark of human-esque curiosity and regard for other sentient life, but in the real world, you’ve got to figure out how to program in those capabilities yourself (or program something that will reliably acquire them), and that’s hard :-)
Kieran Allen asks:
I’ll take a stab at this question too.
There are two different schools of thought about what the goal of AI as a field is. One is that the goal is to build a machine that can do everything humans can—possibly including experiencing emotions and other conscious states. On this view, a “full AI” would plausibly be a person, deserving of moral rights like any other.
The more common view within contemporary AI is that the goal of AI is to build machines that can effectively achieve a variety of practical goals in a variety of environments. Think Nate’s Deep Blue example, but generalized: instead of steering arrangements of chess pieces on a board toward some goal state, a “full” AI steers arbitrary arrangements of objects in space toward some goal state. Such an AI might not be conscious or have real preferences; it might have “goals” only in the limited sense that Deep Blue has “goals.” This is the kind of AI MIRI has in mind, and the kind we’re trying to plan for: a system that can draw inferences from sensor inputs and execute effective plans, but not necessarily one that has more moral weight than Google’s search engine algorithms do.
If it turns out that you do need to make AI algorithms conscious in order to make them effective at scientific and engineering tasks, that does make our task a lot harder, because, yes, we’ll have to take into account the AI’s moral status when we’re designing it, and not just the impact its actions have on other beings. For now, though, consciousness and intelligent behavior look like different targets, and there are obvious economic reasons why mainstream AI is likely to prioritize “high-quality decision making” over “emulating human consciousness.”
A better analogy to MIRI’s goal than “we build Hitler and then put him in chains” is “we build a reasonably well-behaved child and teach the child non-Hitler-ish values.” But both of those ways of thinking are still excessively anthropomorphized. A real-world AI, of the “high-quality decision-making” sort, may not resemble a human child any more closely than the earliest airplanes resembled a baby bird.
For more information about this, I can recommend Stuart Russell’s talk on the future of AI: https://www.youtube.com/watch?v=GYQrNfSmQ0M
(1) I suspect it’s possible to create an artificial system that exhibits what many people would call “intelligent behavior,” and which poses an existential threat, but which is not sentient or conscious. (In the same way that Deep Blue wasn’t sentient: it seems to me like optimization power may well be separable from sentience/consciousness.) That’s no guarantee, of course, and if we do create a sentient artificial mind, then it will have moral weight in its own right, and that will make our job quite a bit more difficult.
(2) The goal is not to build a sentient mind something that wants to destroy humanity but can’t. (That’s both morally reprehensible and doomed to failure! :-p) Rather, the goal is to successfully transmit the complicated values of humanity into a powerful optimizer.
Have you read Bostrom’s The Superintelligent Will? Short version is, it looks possible to build powerful optimizers that pursue goals we might think are valueless (such as an artificial system that, via very clever long-term plans, produces extremely large amounts of diamond, or computes lots and lots of digits of pi). We’d rather not build that sort of system (especially if it’s powerful enough to strip the Earth of resources and turn them into diamonds / computing power): most people would rather build something that shares some of our notion of “value,” such as respect for truth and beauty and wonder and so on.
It looks like this isn’t something you get for free. (In fact, it looks very hard to get: it seems likely that most minds would by default have incentives to manipulate & decieve in order to acquire resources.) We’d rather not build minds that try to turn everything they can into a giant computer for computing digits of pi, so the question is how to design the sort of mind that has things like respect for truth and beauty and wonder?
In hollywood movies, you can just build something that looks cute and fluffy and then it will magically acquire a spark of human-esque curiosity and regard for other sentient life, but in the real world, you’ve got to figure out how to program in those capabilities yourself (or program something that will reliably acquire them), and that’s hard :-)