What other methods are there that would in principle allow iteration?
If it is true that “a failed AGI attempt could result in unrecoverable loss of human potential within the bounds everything that it can affect”, then our options are to A) not fail or B) limit the bounds of everything that it can affect. In this sense any strategy that hopes to allow for iteration is abstractly equivalent to a box/simulation/sandbox whatever you may call it.
I’m not excited about this particular idea, but finding some way to iterate on alignment solutions is a hugely important problem.
What other methods are there that would in principle allow iteration?
If it is true that “a failed AGI attempt could result in unrecoverable loss of human potential within the bounds everything that it can affect”, then our options are to A) not fail or B) limit the bounds of everything that it can affect. In this sense any strategy that hopes to allow for iteration is abstractly equivalent to a box/simulation/sandbox whatever you may call it.