release-gate
Agent Scan FAQ Scan a repo

Security Research

The State of Agent Code Safety

We scanned 30+ of the most-starred open-source AI-agent frameworks for the risk your SAST tool can't see — then hand-verified every finding, and threw most of them out.

30+
frameworks scanned
73%
ship no output ceiling
1
published CVE surfaced
6
false-positive classes removed

A static analyzer sees eval(x) and asks "is x tainted by SQL or an HTTP parameter?" It has no concept of "x is the model's reply." That blind spot — model output reaching a code, shell, or deserialization sink — is the entire agent layer, and it's invisible to the tools most teams already run.

What's actually out there

1 in 5
Model output reaches eval / exec
Mostly code-writing agents by design — the model writes code, the framework runs it. The only thing between a prompt injection and RCE is a sandbox the scanner can't verify. One case is unambiguous and already public: SuperAGI's eval() on the assistant's own reply (CVE-2025-51472).
73%
No output-token ceiling, anywhere
11 of 15 frameworks assemble request params in a dict and spread them (create(**params)) with no max_tokens — 27 call sites. Cost hygiene, not a vuln, on its own; but near-universal, and passed through **kwargs where most scanners can't see it.
≈0
Deserialization "vulnerabilities" that are real
Frameworks pickle constantly — over local pipes, in caches, to persist state. A naive scanner flags every pickle.loads() as RCE. Almost none of them are. Which brings us to the part that matters more than the findings.

The part nobody publishes

What we refused to flag

A scanner that cries wolf gets demoted to advisory and ignored. The hardest — and most valuable — engineering here is not flagging the things that look scary but aren't. Every row below is a real pattern in a real, well-engineered framework that a lazy scanner reports as critical:

Looks like HIGHIs actually FALSE POSITIVEWhere
pickle.loads(data) "RCE from user input"a LogRecord off a local multiprocessing pipe — trusted internal transporttop-tier voice-agent framework
pickle.loads(message_ser) "RCE"the framework round-tripping its own serialized objectsleading multi-agent framework
HEADER_TOKEN = "X-…-Token" "hardcoded secret"an HTTP header name, not a valuetop-tier voice-agent framework
gas_token = "0x0000…0000" "secret"the Ethereum zero address — a constanton-chain agent framework
KEY = "YOUR-AZURE-…-KEY" "leaked key"a placeholder / example valuemajor data framework
57 findings, HOLDall in a cookbook of 256 tutorial files; the core is cleanminimalist agent framework

That last row is the pattern that quietly wrecks trust: grading a framework on its tutorials instead of its code. We exclude cookbook/, examples/, and tests/ from the score entirely — and surface them separately, labelled unscored.

The discipline in one rule

We assert HIGH · confirmed only when we can see the dangerous input's source — a value assigned from an LLM call in scope, or an unambiguous name like request.body. When the danger is only inferred from a variable's name, we say so: MEDIUM · inferred — "confirm the source." A judge rules on the evidence in front of it, not on what a name suggests.

How we verified · what we don't claim

Every high-severity finding was traced to the exact line and read in context before we trusted it — that process is what produced the table above. Several findings that looked confirmed collapsed under a two-minute read; the local-IPC pickle and the header-name "secret" both came from a marquee framework, and both were wrong.

We're not publishing a precision/recall number yet — that needs a labeled benchmark with true negatives, and we're building it: every verified verdict now feeds a calibration corpus. What this is: a methodology and a set of hand-checked findings across the real ecosystem, plus a public account of the false positives we eliminated. Honesty about the second is the point.

One finding closed the loop the way it should — we reported an unbounded tool-call loop to the LightAgent maintainers, and they shipped a fix. A credible finding, acted on. That's the bar.

Scan your own agent

Two honest scores and one verdict — PROMOTE / HOLD / BLOCK. Every finding cites its evidence and its confidence.

# free, no config
pip install release-gate
release-gate audit .

Methodology, the analyzer, and the full false-positive test suite are open source. Findings were hand-verified; only a publicly-disclosed vulnerability (the SuperAGI CVE) and a maintainer-confirmed fix (LightAgent) are named — every other pattern is reported in aggregate, by design.