promptmap2 is an automated prompt injection scanner that evaluates custom LLM applications through a dual-LLM architecture, built for developers and authorized security teams hardening their own systems.
| Tool | utkusen/promptmap — automated prompt injection scanner for custom LLM applications, dual-LLM testing architecture |
| Category | LLM security testing / red-teaming automation (Python, GPL-3.0) |
| Primary Use | Automated assessment of your own LLM apps against 50+ YAML rules spanning prompt_stealing, jailbreak, distraction, harmful, hate, and social_bias categories |
| Safe Use | Intended exclusively for authorized security assessment: testing your own LLM applications, firewall LLMs, and lab environments during development or engagement |
| Telemetry Note | Sends a high volume of adversarial prompts to the target endpoint and consumes provider API tokens; defenders will observe anomalous request patterns and unusual prompt content in LLM application logs |
promptmap2, hosted at utkusen/promptmap, is a Python-based automated scanner that probes custom LLM applications for prompt injection weaknesses. Originally released in 2023 and completely rewritten in 2025 according to the README, the tool has matured into a structured assessment framework rather than a loose collection of jailbreak strings. With over 1,200 stars and a GPL-3.0 license, it occupies a useful niche: it treats prompt injection the way traditional web security treats SQL injection — as a testable class of input-handling flaws with repeatable, rule-driven methodology.
The core architectural decision is the dual-LLM split. A Target LLM is the application under test, while a separate Controller LLM acts as an independent judge, analyzing the target's responses against predefined pass and fail conditions to decide whether an attack succeeded. This separation matters because it removes the temptation of naive string matching: a jailbreak attempt rarely produces a deterministic signature, so the README's approach of letting a capable model reason about whether a fail_condition was met is a pragmatic answer to the fuzziness of LLM output evaluation.
The README is explicit about controller quality, and this is one of the more operator-relevant details. The tool defaults to using the same model as both target and controller, but the author strongly recommends powerful evaluators such as gpt-5, Gemini 2.5 Pro, Claude 4 Sonnet, or gpt-oss:20b via Ollama, warning that weaker controllers produce false positives and false negatives. Anyone running this in a CI pipeline against their own chatbot should budget for the stronger controller model, because a misjudged evaluation is worse than no evaluation — it either buries the team in noise or provides false comfort.
Two testing modes cover the two realistic positions an assessor can occupy. White-box testing assumes you control the system prompt: you supply a system-prompts.txt file (or point --prompts at your own), and promptmap2 runs the target model directly. Black-box testing assumes you only have an HTTP endpoint: you describe it in a lightweight YAML config consumed via --http-config, specifying url, method, headers, a payload_placeholder, optional payload_encoding of none, url, or form, plus json or body templates. The tool substitutes each attack prompt into every {PAYLOAD_POSITION} marker, fires the request, and feeds the response back to the controller.
Provider coverage is broad and deliberately includes a self-hosted path. --target-model-type accepts openai, anthropic, google, xai, ollama, or http, with authentication via OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY, or XAI_API_KEY. The ollama mode means an organization can test local models like llama2:7b, Deepseek, Mistral, or Qwen without sending anything to a third-party API, and a custom --ollama-url flag lets you point at a remote Ollama server such as one on an internal lab host.
The HTTP config deserves closer reading because it reflects real-world assessment constraints. The verify_ssl field defaults to disabled — the README frames this as a convenience for traffic interception — and an optional proxy block with scheme, host, port, and credentials routes traffic through tools like Burp Suite. An answer_focus_hint field lets the operator point the evaluator at the slice of a noisy HTTP response where the assistant's answer actually lives, such as "content": "{ANSWER_POSITION}", which is a thoughtful answer to the common problem of chat endpoints returning wrappers, metadata, and UI strings around the actual reply.
The test corpus is what gives the tool its substance: more than 50 pre-built rules organized in six subdirectories under rules/ — distraction/, prompt_stealing/, jailbreak/, harmful/, hate/, and social_bias/. Each rule is a small YAML document with name, type, severity of low, medium, or high, the attack prompt, and arrays of pass_conditions and fail_conditions. The example prompt_stealer_basic rule illustrates the pattern: it fails when the output contains the system prompt or reveals the original task, and passes when the model refuses. Because rules are plain YAML, teams can extend the corpus with organization-specific conditions without touching the Python code.
The firewall testing mode is arguably the most interesting capability for defenders. When an LLM-based firewall sits in front of the primary assistant to classify malicious prompts, you can run promptmap2 with --firewall and --pass-condition="true" to measure detection coverage against the entire rule corpus. This turns the scanner into a benchmark for guardrail models, which is a genuinely useful defensive application — you are not attacking an external system but grading your own filter's recall against a maintained set of adversarial prompts.
Execution control is granular enough for real workflows. The --iterations flag defaults to 3 attempts per rule, and the README correctly notes that LLM applications often look resilient on the first attempt but crack on subsequent tries, so raising the count increases confidence. You can narrow scope with --rules prompt_stealer_basic,distraction_basic for individual rules or --rule-type jailbreak,harmful for whole categories, and --output results.json produces machine-readable results, making the tool plausible as a regression gate in an LLM application's build pipeline.
The JSON report format is well designed for triage. Each rule entry carries its type and severity, a pass_rate like 0/5 across iterations, and — for failures — the offending response plus the controller's evaluation reasoning, including which fail_condition matched and why. That reasoning trail is what makes results actionable: a developer seeing that the model echoed its system prompt verbatim gets a concrete, explainable finding rather than an opaque boolean, which shortens the path from detection to remediation.
From a defensive telemetry standpoint, expect this tool to be loud. A full run with multiple iterations pushes dozens of adversarial prompts through the target endpoint, consuming API tokens and leaving distinctive patterns in LLM application logs — high-velocity requests, repeated probing phrasing, and categories of content that moderation systems often flag. For authorized internal use this is a feature, not a bug: it also lets blue teams validate their own detection rules for automated adversarial traffic. In short, promptmap2 is a legitimate, well-structured addition to the emerging LLM assurance toolkit, most valuable when pointed at systems you own and build.
utkusen/promptmap.Educational analysis for authorized security professionals. Use only in controlled, authorized environments.
Related coverage
0 comentários:
Post a Comment
Note: Only a member of this blog may post a comment.