Individual humans are usually quite selfish, frequently lie to each other, and are often cruel, and yet the world mostly gets along despite this. This is true even when there are vast differences in power and wealth between humans. For example some groups in the world have almost no power relative to the United States, and residents in the US don’t particularly care about them either, and yet they survive anyway.
Okay so these are two analogies: individual humans & groups/countries.
First off, “surviving” doesn’t seem like the right thing to evaluate, more like “significant harm”/”being exploited ”
Can you give some examples where individual humans have a clear strategic decisive advantage (i.e. very low risk of punishment), where the low-power individual isn’t at a high risk of serious harm? Because the examples I can think of are all pretty bad: dictators, slaveholders, husbands in highly patriarchal societies.. Sexual violence is extremely prevalent and is pretty much always in a high power difference context.
I find the US example unconvincing, because I find it hard to imagine the US benefiting more from aggressive use it force, than trade and soft economic exploitation. The US doesn’t have the power to successfully occupy countries anymore. When there were bigger power differences due to technology, we had the age of colonialism.
Can you give some examples where individual humans have a clear strategic decisive advantage (i.e. very low risk of punishment), where the low-power individual isn’t at a high risk of serious harm?
Why are we assuming a low risk of punishment? Risk of punishment depends largely on social norms and laws, and I’m saying that AIs will likely adhere to a set of social norms.
I think the central question is whether these social norms will include the norm “don’t murder humans”. I think such a norm will probably exist, unless almost all AIs are severely misaligned. I think severe misalignment is possible; one can certainly imagine it happening. But I don’t find it likely, since people will care a lot about making AIs ethical, and I’m not yet aware of any strong reasons to think alignment will be super-hard.
Okay so these are two analogies: individual humans & groups/countries.
First off, “surviving” doesn’t seem like the right thing to evaluate, more like “significant harm”/”being exploited ”
Can you give some examples where individual humans have a clear strategic decisive advantage (i.e. very low risk of punishment), where the low-power individual isn’t at a high risk of serious harm? Because the examples I can think of are all pretty bad: dictators, slaveholders, husbands in highly patriarchal societies.. Sexual violence is extremely prevalent and is pretty much always in a high power difference context.
I find the US example unconvincing, because I find it hard to imagine the US benefiting more from aggressive use it force, than trade and soft economic exploitation. The US doesn’t have the power to successfully occupy countries anymore. When there were bigger power differences due to technology, we had the age of colonialism.
Why are we assuming a low risk of punishment? Risk of punishment depends largely on social norms and laws, and I’m saying that AIs will likely adhere to a set of social norms.
I think the central question is whether these social norms will include the norm “don’t murder humans”. I think such a norm will probably exist, unless almost all AIs are severely misaligned. I think severe misalignment is possible; one can certainly imagine it happening. But I don’t find it likely, since people will care a lot about making AIs ethical, and I’m not yet aware of any strong reasons to think alignment will be super-hard.