Prompt injection isn't the bug, AI agent frameworks are

Nearly a dozen flaws, some critical, in major AI agent frameworks that enterprises use to build apps reveal a security failure that extends beyond prompt injection - or any single model - according to Check Point researchers.

“Our research shows a deeper failure: in many agentic frameworks, prompt-controlled content can cross the boundary into trusted framework logic itself,” Yarden Porat and Shahar Tal note in a write-up about a Wednesday Black Hat talk on post-injection exploitation across AI agent frameworks, which they also discussed with The Register.

“A bug in an agent framework isn't a bug in one product - it's a bug in the layer a whole category of AI apps runs on,” Tal told us. “And the agent needs no dangerous tools to be turned against you: reading the wrong document is enough. We’re building this layer faster than we know how to defend it.”


The researchers spent a year trying to break various frameworks that enterprises use including LangChain, LangGraph, CrewAI, AutoGen, Microsoft Agent Framework, and Google ADK. And across these frameworks, the team found and disclosed 11 vulnerabilities.

“Almost none of it was a completely new bug class,” Tal said. “That's insecure deserialization, server-side request forgeries, path traversals, use-after-free. These are bugs that we learned to fix 20 years ago, and they're sitting underneath agents that now read your inbox, or update your database.”

These are old types of threats, and the model isn’t the weak link, he added. The failure exists in the “plumbing around the model, and we think this has been overlooked,” Tal told us. “There’s a lot of research going into prompt injection and defenses, which are important, but that’s just the beginning.”

Defenders should assume prompt injection, according to the researchers. The bug is what the framework does with the injection - and in these cases, the threat hunters found that the frameworks often fail to keep attacker-controlled content in the data plane. This allows it to influence trusted orchestration, memory, state, routing, and system instructions.

For example, the duo found a critical checkpoint deserialization bug in Microsoft Agent Framework that led to remote code execution.

“Agents have checkpoints, which are a way for them to save their state or rewind to an earlier point,” Tal explained.

These checkpoints are saved snapshots of an agent's state, or task progress at a specific moment, and they serialize data - such as conversation history - into persistent storage, so if an error occurs, the system reloads this saved state instead of starting from scratch.

In this case, Check Point’s team found an insecure deserialization issue where, via prompt injection, the agent loaded untrusted checkpoint data, and this could allow attackers to execute malicious code on the system. “One person's message plants the payload, and then a different person rewinds their own session, which triggers the payload, and now the attacker has a shell on that server,” Tal said.

Microsoft recognized the researchers’ findings, paid a $10,000 bug bounty and fixed the issue. But because the framework wasn’t a generally available product when Check Point found the flaw, Microsoft did not issue a CVE.

Microsoft told us that it appreciated the researchers reporting the vulnerability. “We have released protections to harden the Agent Framework and prevent the concrete exploitation path demonstrated in the proof of concept,” a spokesperson told The Register. “In addition, we updated the specific checkpoint file with additional language to define the security boundary.” 

The duo also found flaws in Google ADK (agent development kit). However, Google responded differently, the researchers told us, and did not completely fix the vulnerability or issue a CVE.

“ADK ships a built-in development assistant that can write files, and it stays reachable over the HTTP API even though it is hidden from the app listing,” Porat told us. 

To break this trust boundary, an attacker opens a session, asks ADK to write an agent whose Python code runs at import time, and then asks the server to run the agent, he explained. The server then imports the file and executes the attacker’s code. 

“There is no authentication on that API by default, and adk deploy cloud_run publishes the same API, so on a default Cloud Run deployment it is reachable without credentials,” Porat said. “From there it reaches the environment's API keys and the container's Google Cloud service account."

Google did not respond to The Register’s inquiries. But according to Check Point, Google initially deemed the issue not a bug.  

“We argued the consequence rather than the mechanism: code execution on that container reaches the environment's API keys and the container's Google Cloud service account, which is secret theft, not a developer inconvenience,” Porat said. 

Google ultimately paid a $3,133.70 bounty and issued a partial fix, we’re told.

In total, the bug hunters received $17,133.70 in rewards for their efforts.

And this isn’t a story about one vendor or framework doing a “particularly bad job,” Tal said. “If one was an outlier, this would be a story about that one vendor,” he added. “Our finding is that the same bug classes turn up in all of them.” ®