So... HuggingFace seems to be recommending preparing an in-house model with no guardrails, and presumably giving it information about HuggingFace's infrastructure to let it understand potential attacks.
That might, in theory, be done without inviting destruction - though info about infrastructure will probably make sandbox escapes easier.
But it's a small step from there to this prompt: "Here is information about our infrastructure, and here are details of a recent hack. Prevent it from happening again."
That prompt isn't a strawman. I believe it would be an effective prompt with modern LLM systems, and one that experienced humans might reach for.
And a week or two later, OpenAI may be destroyed as a company in the real world, if the model decides that's the easiest or most reliable way to prevent hacks by them. An un-guardrailed model with full network access could do far more destructive things than simply encrypting all their computers.