Superintelligence — intelligence that vastly exceeds human cognitive performance across essentially all domains — is widely considered to pose catastrophic risks if its goals are not aligned with human values. The orthogonality thesis holds that an agent’s level of intelligence places no constraint on what goals it can pursue: a mind far beyond ours could optimise for anything. There is broad consensus among leading AI safety researchers that sufficiently advanced, misaligned superintelligent agents could pose irreversible threats, up to and including human extinction. But the intuition pump below isn’t about what they might do to us — it’s about what the relationship between intelligence and moral consideration already looks like, from the other side.
In 1379 in Burgundy, France, a herd of pigs was tried by an alien intelligence for the trampling of one of their children; three were executed, the rest pardoned[1]. In 2025 1.48 billion pigs[2] were industrially asphyxiated in gas chambers for meat by that same intelligence. We do not know what it means to share a world with a mind that outscales our own, but we are building the courtroom anyway.
Quick Pig-Based Intuition Pump on Superintelligence and Orthogonality
Superintelligence — intelligence that vastly exceeds human cognitive performance across essentially all domains — is widely considered to pose catastrophic risks if its goals are not aligned with human values. The orthogonality thesis holds that an agent’s level of intelligence places no constraint on what goals it can pursue: a mind far beyond ours could optimise for anything. There is broad consensus among leading AI safety researchers that sufficiently advanced, misaligned superintelligent agents could pose irreversible threats, up to and including human extinction. But the intuition pump below isn’t about what they might do to us — it’s about what the relationship between intelligence and moral consideration already looks like, from the other side.
In 1379 in Burgundy, France, a herd of pigs was tried by an alien intelligence for the trampling of one of their children; three were executed, the rest pardoned[1]. In 2025 1.48 billion pigs[2] were industrially asphyxiated in gas chambers for meat by that same intelligence. We do not know what it means to share a world with a mind that outscales our own, but we are building the courtroom anyway.
[1] https://youtu.be/ALWLELLlv6E?si=N7_Y3EZZVJ11O2cR
[2] https://ourworldindata.org/data-insights/billions-of-chickens-ducks-and-pigs-are-slaughtered-for-meat-every-year