Suppose we have 10 different MAD mechanisms. Wouldn’t the defense become impossible at some point? Wouldn’t the AI think at one point, “It is simpler to just complete the task without killing anybody”?
EDIT: okay, if the task given to AI is “calculate the 2^256 th digit of Pi”, then perhaps it’d rather risk the easier task of “before they certainly interrupt my Pi calculations, defeat 50 MAD threats and kill all people, then hopelessly proceed”
Perhaps, but now you’re substituting one risk for another, which is the risk that the MAD mechanisms trigger by accident, causing unintentional destruction of the earth.
An entity powerful enough to threaten all humans on earth would most likely also be powerful enough to defeat the MAD mechanism.
Suppose we have 10 different MAD mechanisms. Wouldn’t the defense become impossible at some point? Wouldn’t the AI think at one point, “It is simpler to just complete the task without killing anybody”?
EDIT: okay, if the task given to AI is “calculate the 2^256 th digit of Pi”, then perhaps it’d rather risk the easier task of “before they certainly interrupt my Pi calculations, defeat 50 MAD threats and kill all people, then hopelessly proceed”
Perhaps, but now you’re substituting one risk for another, which is the risk that the MAD mechanisms trigger by accident, causing unintentional destruction of the earth.
Yeah, thanks for replying. I did now realize that its more complex than that.