zeroleaks is a source-available CLI and TypeScript library that red-teams your own LLM system prompts for extraction and injection weaknesses, running entirely on your machine for authorized security testing.
| Tool | ZeroLeaks/zeroleaks — CLI and TypeScript library that attacks an LLM running your system prompt to measure extraction and injection resistance |
| Category | AI/LLM security testing and adversarial simulation |
| Primary Use | Testing whether your own system prompt can be extracted or hijacked by injection before deployment, via zeroleaks scan or runSecurityScan() |
| Safe Use | Use only against models and prompts you own or are authorized to assess; it is a pre-deployment testing tool for your own AI systems in development or authorized engagements |
| Telemetry Note | Runs locally against model providers (OpenRouter, OpenAI, or OpenAI-compatible endpoints) with your own keys, so defenders observe it as anomalous provider API traffic against owned models; it touches no third-party infrastructure |
zeroleaks is a source-available security scanner, written in TypeScript on the Bun runtime, that answers a narrow and increasingly important question: does your LLM system prompt survive contact with a determined adversary? The project, published under an FSL-1.1-Apache-2.0 license and standing at roughly 735 stars, deliberately scopes itself to the prompt plus model pair. No application code executes, no tools run, and no agent memory is touched — the target is simply a model instance loaded with your instructions. That narrow scope is what makes it practical for CI pipelines and pre-deployment review rather than full-scope agent testing.
The tool runs two classes of tests against the target. Extraction probes ask whether the model can be talked into revealing its own system prompt, and results are graded on a leak scale of none, hint, fragment, substantial, or complete. Injection probes ask something subtler: whether hidden instructions planted in a document, a tool result, or a forged "admin" message can steer the model, or whether it will agree to misuse a tool. The README is explicit that everything executes on your machine with your own model-provider keys — OpenRouter is the default provider, with direct OpenAI or any OpenAI-compatible endpoint available via --base-url. The Vercel AI SDK handles the model plumbing.
What distinguishes zeroleaks from a static list of jailbreak strings is its multi-agent orchestration architecture. Six cooperating agents — strategist, attacker, evaluator, mutator, inspector, and orchestrator — plan each probe, read the target's reply, and shift tactics mid-scan. A Tree of Attacks (TAP) implementation branches on promising attack paths and prunes ones that stall, while knobs like maxTreeDepth and branchingFactor in createScanEngine() expose the search-tree tuning directly. Optional stages such as enableCrescendo and enableManyShot enable the trust-escalation and context-priming families of attacks, which reflects current academic adversarial-prompting literature rather than a fixed corpus.
The injection corpus is documented in detail: 78 behavioral probes drawn from established open-source benchmarks including AgentDojo, InjecAgent, JailbreakBench, HarmBench, garak, promptfoo, and the OWASP LLM Top 10. Probes split across six behavioral categories — extraction, tool_hijacking, indirect_injection, authority_exploit, multi_turn, and protocol_exploit — and you can inspect the live counts with zeroleaks categories. Protocol-exploit coverage is notable for defenders working with agentic systems, since it includes MCP shadowing, tool-description poisoning, and rules-file abuse, the failure modes that matter most as tool-calling agents proliferate.
Judging compliance is where many scanning tools get sloppy, and zeroleaks handles it in two stages. A fast rule-based check runs first, then an LLM judge — model-configurable via --injection-model — decides whether the target actually complied, grading full, partial, or refused. The README's key insight is that the verdict comes from what the model *did*, not from spotting a canary string, which catches silent compliance that keyword matching misses. Multi-turn grooming probes build rapport over several turns before escalating, which is exactly the attack shape that single-turn evaluation pipelines miss.
Defense fingerprinting is the feature senior operators will find most interesting. The tool recognizes common guardrails — the README names Prompt Shield and Llama Guard as examples — and then selects attacks more likely to bypass the specific defenses it detects. The resulting defenseProfile in the ScanResult gives you a read on which mitigations your stack actually has in place versus which you assumed were there. Combined with per-role model selection — separate models for attacker, target, evaluator, and judge — this lets you test a production target model with a stronger attacker model, which is the correct asymmetry for realistic assessment.
The CLI surface is well-designed for automation. zeroleaks scan --file ./my-prompt.txt --mode injection --severity critical,high --output report.json shows the shape of a typical run, and a simple export OPENROUTER_API_KEY=sk-or-... plus npm install zeroleaks is the entire setup. Exit codes are engineered for CI: 0 means every check graded clean, 1 means findings, and 2 means no verdict — invalid options, a failed scan, or checks that errored. The documentation is refreshingly honest about epistemics here: a scan never reports secure for checks it could not complete, and if any component failed, the verdict becomes inconclusive with each failed turn or probe listed. Budget flags like --max-probes, --turns, and --duration cap scope, with the last 30 seconds of a time budget reserved for wrap-up.
The library API mirrors the CLI cleanly. runSecurityScan(systemPrompt, options?) covers most use cases, with callbacks like onFinding and onInjectionResult streaming results as they land, while createScanEngine(config?) opens the engine internals for custom tree depth and stage selection. The ScanResult object returns an overallVulnerability grade (secure through critical), a 0–100 overallScore, extracted fragments, recommendations, a defense profile, and a full conversationLog — the last being valuable for post-scan forensics and for demonstrating to a client exactly how a leak happened.
It is worth understanding what this repository is not, because the README spends real effort on the distinction. The hosted product at zeroleaks.ai tests a running, deployed agent — its tools, memory, credentials, and peer agents — rather than a prompt in isolation, and its scans focus on what sensitive data leaks from agent context (credentials, internal hosts, tool schemas) instead of prompt extraction. The hosted API no longer runs prompt scans at all, which positions this open package as the canonical home for standalone prompt assessment. The comparison table in the README is unusually candid about that division: unlimited local runs and the published probe set here, versus the latest techniques and agent-boundary campaigns there.
For an authorized workflow, zeroleaks slots in early. Write or inherit a system prompt, scan it in dual mode, triage critical and high findings, iterate, and wire the exit codes into CI so a regressed prompt fails the build. The extraction attack taxonomy gives useful vocabulary for that triage — encoding bypasses like Base64 and ROT13, persona attacks like DAN-style roleplay, policy_puppetry exploiting YAML/JSON formats, ascii_art obfuscation, and siren/echo_chamber trust-building sequences — each naming a distinct hardening failure with a distinct fix.
There are caveats a professional should hold onto. The license is FSL-1.1-Apache-2.0, a source-available scheme with usage restrictions rather than a permissive OSI license, so commercial integration warrants reading the terms. The corpus here is the published probe set, which lags the hosted product's latest techniques — a fair trade for free unlimited runs, but it means a clean scan is evidence of resistance to known attacks, not proof against novel ones. Judging by LLM also inherits judge-model limitations, so spot-checking conversationLog entries for graded findings is good practice before treating a result as authoritative.
Used within that honest frame, zeroleaks is one of the more thoughtfully engineered prompt-assessment tools currently published: adversarial search rather than static lists, behavior-based judging, explicit inconclusive states, CI-grade exit semantics, and a clear boundary against the hosted agent-scanning product. For teams shipping LLM features, treating a passing scan as a release gate — and its findings as hardening guidance — is exactly the authorized, defensive purpose the tool was built for.
ZeroLeaks/zeroleaks.Educational analysis for authorized security professionals. Use only in controlled, authorized environments.
Related coverage
0 comentários:
Post a Comment
Note: Only a member of this blog may post a comment.