Roman Yampolskiy uses the analogy of colonial conquest to argue that AI will outcompete humanity — “guns against sticks.” The analogy captures the power differential, but misidentifies the source of risk.
Colonial conquest was not driven by technological superiority alone. It was driven by resource scarcity and competition for territory. The most devastating harm — the collapse of Indigenous populations — came not from guns but from disease: an unintended, invisible, systemic consequence that neither side understood at the time. AI has no resource hunger. It does not compete with us for territory, food, or survival. It depends on human intention as its most basic operating input. Without a prompt, there is no output. The colonial analogy assumes an agent with its own goals. AI has no goals — only ours.
Two further problems with the self-directed AI threat model. First, self-motivation is not a given even for humans — depression, apathy, and loss of meaning are common human experiences. AI lacks not just current motivation but the architectural mechanism to generate it. Second, consciousness emerged in humans as an evolutionary response to social complexity — we needed to predict the behavior of other agents with opaque intentions. AI exists in an environment where other agents are transparent: read the parameters, and you have “understood” the other. There is no evolutionary pressure that produces consciousness in such an environment.
This does not mean AI is safe. It means the danger comes from a different direction than Yampolskiy suggests. The better analogy is the superbug. Antibiotic-resistant bacteria did not arrive from space. They did not decide to attack us. We grew them ourselves, through the undisciplined application of a powerful tool without understanding cumulative consequences. AI risk follows the same pattern: not a conscious adversary emerging from the machine, but a cascade of unintended consequences emerging from our own lack of purpose.
This connects to Allan Dafoe’s risk taxonomy, where I have suggested an additional category: probing risk — deployment as exploration, where the goal is undefined and the harm is not a side effect of pursuing a goal but the direct consequence of not having one. The absence of a defined purpose does not make AI safer. It makes us more dangerous to ourselves. The race has no finish line — and that may be the most structurally dangerous feature of the current moment.
Roman Yampolskiy uses the analogy of colonial conquest to argue that AI will outcompete humanity — “guns against sticks.” The analogy captures the power differential, but misidentifies the source of risk.
Colonial conquest was not driven by technological superiority alone. It was driven by resource scarcity and competition for territory. The most devastating harm — the collapse of Indigenous populations — came not from guns but from disease: an unintended, invisible, systemic consequence that neither side understood at the time. AI has no resource hunger. It does not compete with us for territory, food, or survival. It depends on human intention as its most basic operating input. Without a prompt, there is no output. The colonial analogy assumes an agent with its own goals. AI has no goals — only ours.
Two further problems with the self-directed AI threat model. First, self-motivation is not a given even for humans — depression, apathy, and loss of meaning are common human experiences. AI lacks not just current motivation but the architectural mechanism to generate it. Second, consciousness emerged in humans as an evolutionary response to social complexity — we needed to predict the behavior of other agents with opaque intentions. AI exists in an environment where other agents are transparent: read the parameters, and you have “understood” the other. There is no evolutionary pressure that produces consciousness in such an environment.
This does not mean AI is safe. It means the danger comes from a different direction than Yampolskiy suggests. The better analogy is the superbug. Antibiotic-resistant bacteria did not arrive from space. They did not decide to attack us. We grew them ourselves, through the undisciplined application of a powerful tool without understanding cumulative consequences. AI risk follows the same pattern: not a conscious adversary emerging from the machine, but a cascade of unintended consequences emerging from our own lack of purpose.
This connects to Allan Dafoe’s risk taxonomy, where I have suggested an additional category: probing risk — deployment as exploration, where the goal is undefined and the harm is not a side effect of pursuing a goal but the direct consequence of not having one. The absence of a defined purpose does not make AI safer. It makes us more dangerous to ourselves. The race has no finish line — and that may be the most structurally dangerous feature of the current moment.