The step that’s missing for me is the one where the paperclip maximiser gets the opportunity to kill everyone.
Your talk of “plans” and the dangers of executing them seems to assume that the AI has all the power it needs to execute the plans. I don’t think the AI crowd has done enough to demonstrate how this could happen.
If you drop a naked human in amongst some wolves I don’t think the human will do very well despite its different goals and enormous intellectual advantage. Similarly, I don’t see how a fledgling sentient AGI on OpenAI servers can take over enough infrastructure that it poses a serious threat. I’ve not seen a convincing theory for how this would happen. Mailorder nanobots seem unrealistic (too hard to simulate the quantum effects in protein chemistry), the AI talking itself out of its box is another suggestion that seems far-fetched (main evidence seems to be some chat games that Yudkowsky played a few times?), a gradual takeover by its voluntary uptake into more an more of our lives seems slow enough to stop.
Is your question basically how an AGI would gain power in the beginning in order to get to a point where it could execute on a plan to annihilate humans?
I would argue that:
Capitalists would quite readily give the AGI all the power it wants, in order to stay competitive and drive profits.
Some number of people would deliberately help the AGI gain power just to “see what happens” or specifically to hurt humanity. Think ChaosGPT, or consider the story of David Charles Hahn.
Some number of lonely, depressed, or desperate people could be persuaded over social media to carry out actions in the real world.
Considering these channels, I’d say that a sufficiently intelligent AGI with as much access to the real world as ChatGPT has now would have all the power needed to increase its power to the point of being able to annihilate humans.
The step that’s missing for me is the one where the paperclip maximiser gets the opportunity to kill everyone.
Your talk of “plans” and the dangers of executing them seems to assume that the AI has all the power it needs to execute the plans. I don’t think the AI crowd has done enough to demonstrate how this could happen.
If you drop a naked human in amongst some wolves I don’t think the human will do very well despite its different goals and enormous intellectual advantage. Similarly, I don’t see how a fledgling sentient AGI on OpenAI servers can take over enough infrastructure that it poses a serious threat. I’ve not seen a convincing theory for how this would happen. Mailorder nanobots seem unrealistic (too hard to simulate the quantum effects in protein chemistry), the AI talking itself out of its box is another suggestion that seems far-fetched (main evidence seems to be some chat games that Yudkowsky played a few times?), a gradual takeover by its voluntary uptake into more an more of our lives seems slow enough to stop.
Is your question basically how an AGI would gain power in the beginning in order to get to a point where it could execute on a plan to annihilate humans?
I would argue that:
Capitalists would quite readily give the AGI all the power it wants, in order to stay competitive and drive profits.
Some number of people would deliberately help the AGI gain power just to “see what happens” or specifically to hurt humanity. Think ChaosGPT, or consider the story of David Charles Hahn.
Some number of lonely, depressed, or desperate people could be persuaded over social media to carry out actions in the real world.
Considering these channels, I’d say that a sufficiently intelligent AGI with as much access to the real world as ChatGPT has now would have all the power needed to increase its power to the point of being able to annihilate humans.