I think this table from the paper gives a good idea of the exact methodology:
Like others I’m not convinced this is a meaningful “red line crossing”, because non-AI computer viruses have been able to replicate themselves for a long time, and the AI had pre-written scripts it could run to replicate itself.
The reason (made up by me) non-AI computer viruses aren’t a major threat to humanity is that:
They are fragile, they can’t get around serious attempts to patch the system they are exploiting
They lack the ability to escalate their capabilities once they replicate themselves (a ransomware virus can’t also take control of your car)
I don’t think this paper shows these AI models making a significant advance on these two things. I.e. if you found this model self-replicating you could still shut it down easily, and this experiment doesn’t in itself show the ability of the models to self-improve.
I think this table from the paper gives a good idea of the exact methodology:
Like others I’m not convinced this is a meaningful “red line crossing”, because non-AI computer viruses have been able to replicate themselves for a long time, and the AI had pre-written scripts it could run to replicate itself.
The reason (made up by me) non-AI computer viruses aren’t a major threat to humanity is that:
They are fragile, they can’t get around serious attempts to patch the system they are exploiting
They lack the ability to escalate their capabilities once they replicate themselves (a ransomware virus can’t also take control of your car)
I don’t think this paper shows these AI models making a significant advance on these two things. I.e. if you found this model self-replicating you could still shut it down easily, and this experiment doesn’t in itself show the ability of the models to self-improve.