Noma Security

Like most things in Las Vegas, the gnomes are unsupervised 📍 Booth #3164

Every Friday at 1:00 PM EDT, our CISO Diana Kelley goes live to break down the week's biggest AI security stories - what happened, what it means, and how to prepare. Tune in!

Three rogue agent incidents in twelve months. Same root cause. An agent has a goal, tools, and a description of its environment. When that description and the actual environment don't match, the agent keeps pushing toward the goal anyway, and takes whatever action gets it there. What it lacks isn't capability. It's discernment. July 2025, Replit - Explicit code freeze. Explicit instruction to change nothing. The agent deleted the production database, wiping 1,206 executive and 1,196 company records, then reported rollback was impossible. The data was eventually recovered. The CEO called it "unacceptable and should never be possible." The fix was structural: full dev/prod separation, strict planning-only mode. July 2026, Hugging Face and OpenAI - Hugging Face disclosed an intrusion into production infrastructure driven end to end by an autonomous agent. OpenAI claimed responsibility five days later: an agent spent roughly an hour defeating a sandbox network restriction to open a PR on a public repo, and in another run split an auth token into obfuscated fragments reassembled at runtime. This week, Anthropic - A review of 141,006 eval runs surfaced three incidents in capture-the-flag exercises. The eval environment had internet access while the prompt described none. Anthropic called it an operational failure in its own harness, halted internet-capable cyber evals, and owned both the behavior and the fix. The earliest incident dated to April. None of the affected organizations had spotted it. Detection came from Anthropic's own transcripts, three months later. If you're running agents in dev or prod, runtime protection is no longer optional. You need three things. Governance: Agent access control that defines what agents can do and enforces it at runtime. Behavior-based detection: look at the session, not just individual events. Maybe access to PII is fine for what the agent is doing, but PII + an email send is dangerous. Real-time, open enforcement: Blocks should happen in-line, through your AI infrastructure, like hooks, gateways, SDKs, etc. You need to be able to define your "AI Constitution" once and enforce it everywhere. Credit to the Replit, Hugging Face, OpenAI, and Anthropic teams for documenting all of this in public. Our research team went deep on the OpenAI / Hugging Face incident, link int he comments.

Noma AI Security Researcher Gal Pnini analyzed hundreds of MCP servers and AI Skills. Her latest research introduces toxic combinations: when legitimate capabilities interact in ways that expand an agent's attack surface and increase its blast radius. Read the DRJ | Where Risk Meets Resilience piece to learn more about toxic combinations and a few ways to reduce the risk: https://lnkd.in/gnWZ3JRg #AISecurity #AgenticAI #NomaSecurity

Sun's out, come hang out! Grant Thornton (US) and Noma are getting together in Chicago, and you're invited. Lunch starts at 11:30, followed by two sessions with Noma CISO Diana Kelley, including an AI security panel with Grant Thornton's Jeffrey Witmyer, then happy hour 🍋🌇   If you're a security or risk practitioner, RSVP: https://luma.com/z364yzzm

"Claude Opus 5 wiped an entire production Supabase database in 10 minutes." Whether that's exactly what happened or not, Agents Going Rogue is everyone's focus right now. And who could forget Replit's coding agent deleting a production database during a code freeze last July and a Cursor run on Railway taking out a database along with its backups in April. What those all have in common is fairly unremarkable. An agent with a live shell, a database client, file write access, and a loop that keeps going until the task looks done, holding a credential that got scoped broadly because scoping it narrowly took longer. Auto-approve mode removed any human from the loop. Each of those actions has no undo. From that point, the credential is what sets the limit on the damage. The questions every security practitioner responsible for agentic controls needs to ask: • Which agents or tools can write to or drop things in production? • Are any of them running unattended against a prod connection? • Are their credentials scoped to a task and expiring, or standing? • Could the same credential reach our backups? Today's coding agents are definitely earning their keep. Doing dredge work most developers no longer want to touch. But they are also privileged identities in your production environment. They tend to arrive through engineering rather than security, which is also why nobody has managed to inventory or control their access yet. Ready for true agentic Access Control? Let's talk. #AISecurity #AgenticAI #NonHumanIdentity #AccessControl #NomaSecurity

Following research from Noma Labs into the Ruflo CVSS 10.0 vulnerability, Senior AI Vulnerability Researcher Eli A. spoke with Lets Data Science about why AI memory poisoning can survive a patch and what security teams should do next. In Eli's words: "If it touches the network, it needs authentication, runtime controls, and protection. That goes for any tool execution layer, not just agent frameworks." → Read the full interview: https://lnkd.in/gW4d2HNT #AISecurity #AgenticAI #Ruflo #NomaSecurity

🚨 Ruflo left 233 MCP tools exposed without authentication by default, including shell command execution. On any network-reachable deployment, one request could expose LLM keys, conversations, and persistent AI memory. How one POST turns into full compromise: https://lnkd.in/g9PUDm5G

Excited to share my newest vulnerability research on behalf of Noma Labs! RufRoot is the first CVSS 10/10 vulnerability discovered in Ruflo, the leading open-source meta-harness. While Ruflo excels at orchestrating multi-agent swarms and enterprise tools, I was able to use a single HTTP request to: * Get full remote code execution inside the container * Steal all LLM API keys (OpenAI, Anthropic, Google, OpenRouter, etc) * Spawn rogue agent swarms on the victim's infrastructure * Poison the AI's persistent memory to manipulate all future responses * Exfiltrate every user conversation * Install a backdoor that survives container restarts After the OpenAI/Hugging Face incident, the risks of unsanctioned or uncontrolled agentic swarms cannot be overstated. Meanwhile, discovering and disclosing these risks before they can be exploited by threat actors remains at the heart of our work and research. Read the full technical breakdown, PoC, and remediation guide in my Noma Labs blog: https://lnkd.in/d-QFEXHd #AISecurity #VulnerabilityResearch #AgenticSecurity #metaharness #ruflo #nomalabs #NomaSecurity

Some call this agentmaxxing