It’s hard to imagine more general and capability-demanding activity as doing good (superhuman!) science in such an absurdly cross-disciplinary field as AI safety (and among the disciplines that are involved there are those that are notoriously not very scientific yet: psychology, sociology, economics, the studies of consciousness, ethics, etc.). So if there is an AI that can do that but still is not counted as AGI, I don’t know what the heck ‘AGI’ should even refer to. Compare with chess, which is a very narrow problem which can be formally defined and doesn’t require AI to operate with any science (and world models) whatsoever.
It’s hard to imagine more general and capability-demanding activity as doing good (superhuman!) science in such an absurdly cross-disciplinary field as AI safety (and among the disciplines that are involved there are those that are notoriously not very scientific yet: psychology, sociology, economics, the studies of consciousness, ethics, etc.). So if there is an AI that can do that but still is not counted as AGI, I don’t know what the heck ‘AGI’ should even refer to. Compare with chess, which is a very narrow problem which can be formally defined and doesn’t require AI to operate with any science (and world models) whatsoever.