In the same way that two human super powers can’t simply make a contract to guarantee world peace, two AI powers could not do so either.
(Assuming an AI safety worldview and the standard, unaligned, agentic AIs) in the general case, each AI will always weigh/consider/scheme at getting the other’s proportion of control, and expect the other is doing the same.
based on their relative power and initial utility functions
It’s possible that peace/agreement might come from some sort of “MAD” or game theory sort of situation. But it doesn’t mean anything to say it will come from “relative power”.
Also, I would be cautious about being too specific about utility functions. I think an AI’s “utility function” generally isn’t a literal, concrete, thing, like a Python function that gives comparisons , but might be far more abstract, and could only appear from emergent behavior. So it may not be something that you can rely on to contract/compare/negotiate.
This seems basic and wrong.
In the same way that two human super powers can’t simply make a contract to guarantee world peace, two AI powers could not do so either.
(Assuming an AI safety worldview and the standard, unaligned, agentic AIs) in the general case, each AI will always weigh/consider/scheme at getting the other’s proportion of control, and expect the other is doing the same.
It’s possible that peace/agreement might come from some sort of “MAD” or game theory sort of situation. But it doesn’t mean anything to say it will come from “relative power”.
Also, I would be cautious about being too specific about utility functions. I think an AI’s “utility function” generally isn’t a literal, concrete, thing, like a Python function that gives comparisons , but might be far more abstract, and could only appear from emergent behavior. So it may not be something that you can rely on to contract/compare/negotiate.