It would be interesting to read the post mortem (if they will release). In particular in which phase of the training it escaped the sandbox, in which way it used the information. Training is different from inference, I doubt there is a context there and where would the model store these information? In the weights?! That makes little sense to me
If you train a model for cyber warfare, you can’t be surprised when it attacks the defenses keeping it from engaging in cyber warfare — even if they are your own.
Oooorrrr there is more to the story