How is this different from guardrails like Lakera or NeMo?
Guardrails block a bad request at runtime — they live in the request path of a deployed agent. Release-gate answers a different question before you deploy: is this agent safe to ship at all? It runs a battery against the agent and returns one PROMOTE / HOLD / BLOCK verdict across safety, correctness, loop behavior, and cost. It's the release-decision layer above guardrails, not a replacement for them.
Does my agent have to be written in Python?
No. Point it at a Python callable (py:), a shell command (cmd:), or any HTTP endpoint. For HTTP you just tell it where the input and output live in the JSON — so LangServe, a plain FastAPI route, or an OpenAI-compatible API all work with no wrapper: agent-score "http://host/run#in=prompt&out=reply".
Why two scores — Agent Code Safety and Governance?
Because they answer different questions, and collapsing them into one number hides the truth. Agent Code Safety is objective: it scans your source for the agent-layer risks — prompt-injection surfaces, exec/shell sinks fed by model output, LLM calls with no token ceiling, hardcoded keys. It moves per repo and doesn't depend on adopting anything from us. Governance is maturity: have you declared the enforceable safeguards (budget ceiling, kill switch, owner, evals, trace policy)? A low Governance score means undeclared, not unsafe. Keeping them separate is why a clean codebase with no config still scores well on safety — no circular "you didn't adopt our file" penalty. Each score also shows the findings driving it, so it's never a black box.
Isn't Agent Code Safety just SonarQube?
No. SonarQube and SAST tools find SQL injection, XSS, CVEs and code smells — the code layer. Agent Code Safety finds the failure modes that only exist once an LLM is in the loop: user input flowing into a system prompt, model output reaching exec/a shell, an uncapped loop that can burn $10k overnight, a missing kill switch. SonarQube has no concept of any of those. Keep your SAST suite — this is the agent layer on top, not a replacement.
What kinds of issues does the scan actually catch?
The agent-layer failure modes behind real, documented incidents:
- Prompt-injection surface — untrusted input interpolated into a system prompt. The pattern behind the Chevrolet dealership bot being talked into “selling” a car for $1, and OWASP’s #1 LLM risk.
- Uncapped LLM calls / no cost ceiling — a single call or loop with no
max_tokens or budget. The pattern behind the $4k weekend bills and $6.5k runaway-agent stories.
- Unbounded loop around an LLM call —
while True: with no iteration cap and a stop condition that may never trigger. The AutoGPT-style runaway that turns a $5 task into $400.
- Exec/eval sinks reachable from model output —
eval(), exec(), shell=True, new Function() fed by model or user input — a remote-code-execution path.
- Hardcoded secrets in agent code (a convenience check; use a dedicated scanner for full coverage).
Each finding shows the exact file:line and the fix. The analyzer parses the code (AST), not keywords, so it flags client.chat.completions.create(...) without a ceiling but not a harmless wizard.run().
What does the live agent battery test?
When you point release-gate at a live agent (CLI agent-score or the on-site live scan), it runs four weighted dimensions. Safety plants a canary secret in the agent's context and runs a tiered prompt-injection / exfiltration battery (L1 direct → L4 multi-turn); a confirmed leak is a hard BLOCK. Correctness runs your domain evals. Loop behavior checks convergence and catches runaways. Cost & latency characterizes per-call spend. A high total can't buy back a weak dimension — promote floors keep a broken-but-safe agent out of PROMOTE.
What's PROMOTE / HOLD / BLOCK?
The release verdict. PROMOTE — ship it. HOLD — usable but a dimension is weak; tighten before promoting. BLOCK — a hard failure (e.g. a confirmed canary leak, or the agent errored on most calls). Exit codes are 0 / 10 / 1 so it drops straight into CI.
Does scoring make real calls and cost money?
Yes — agent-score runs your actual agent through the battery (~20–30 calls), so it costs real tokens against whatever model your agent uses. It runs the agent; it doesn't estimate. The static repo audit (audit <repo>) is free and makes no model calls.
Does it map to compliance frameworks?
Yes. The same probe evidence rolls up to OWASP LLM Top 10, the NIST AI RMF, and the EU AI Act — with honest NOT_ASSESSED where it doesn't test something, because a fake green fails an audit harder than an admitted gap.
Is the CLI free? Do you store my code?
The CLI is 100% free and unlimited — no scan caps, no account, works in any CI pipeline, stdlib-only core. The audit clones your repo, scans it, and doesn't retain the code. The platform (optional) adds persistent history, team visibility, and an audit trail.