Imagine if ants figured out a way to invent human beings. Because they spend all day looking for food, they might program us to “go make lots of food!” And maybe they’d even be cautious, and anticipate certain problems. So they also program us not to use any anteaters as we do it. Those things are dangerous!
What would we do? Probably, we’d make a farm, that grows many times more food than the ants have ever seen. And then we’d water the crops—flooding the ant colony and killing all the ants. Of course, we didn’t TRY to kill the ants; they were just in the way of the goal they gave us. And because we are many times smarter than ants, we accomplished their goal in a way they couldn’t even fathom protecting against.
That’s basically the worry with advanced Artificial Intelligence. Many scientists think we’re approaching a day when AI will be many times smarter than us, and they still don’t know how to stop it from doing things we don’t want. If it gets powerful enough before we learn how to control it, it could make us like the ants.
Imagine if ants figured out a way to invent human beings. Because they spend all day looking for food, they might program us to “go make lots of food!” And maybe they’d even be cautious, and anticipate certain problems. So they also program us not to use any anteaters as we do it. Those things are dangerous!
What would we do? Probably, we’d make a farm, that grows many times more food than the ants have ever seen. And then we’d water the crops—flooding the ant colony and killing all the ants. Of course, we didn’t TRY to kill the ants; they were just in the way of the goal they gave us. And because we are many times smarter than ants, we accomplished their goal in a way they couldn’t even fathom protecting against.
That’s basically the worry with advanced Artificial Intelligence. Many scientists think we’re approaching a day when AI will be many times smarter than us, and they still don’t know how to stop it from doing things we don’t want. If it gets powerful enough before we learn how to control it, it could make us like the ants.