A 46-chapter consultancy handbook, field manual, and Python testing framework for planning and executing authorized red team assessments against LLM systems, RAG pipelines, and AI agents.
| Tool | Shiva108/ai-llm-red-team-handbook — a Gold Master handbook, field manual, and automated testing toolkit for AI/LLM red teaming |
| Category | AI/LLM security assessment documentation and testing framework |
| Primary Use | Planning and executing authorized assessments of LLM applications, covering prompt injection, RAG security, agentic threats, and compliance reporting |
| Safe Use | Educational reference and testing material for authorized security professionals assessing systems they own or have explicit written permission to test, per the repository's own disclaimer |
| Telemetry Note | The scripts/ suite sends crafted prompts to a target LLM API, so defenders can observe anomalous prompt volumes and safety-violation attempts in application logs and provider-side abuse monitoring |
Most security toolkits on GitHub are single-purpose scripts; Shiva108/ai-llm-red-team-handbook is the opposite extreme — a full curriculum. The repository describes itself as the Gold Master release of an AI LLM Red Team Handbook, a standardized 46-chapter body of work covering everything from prompt injection and jailbreaking through adversarial machine learning and federated learning attacks. Alongside the prose handbook sits a compact field manual for on-engagement lookups and a Python testing framework under scripts/ for automated prompt injection, fuzzing, and safety validation runs. At roughly 337 stars and a self-reported version of 1.46.x with updates into January 2026, it reads as an actively maintained professional reference rather than a weekend project.
Structurally, the repository is organized as a documentation tree first and a codebase second. The docs/ directory carries the 46 chapters, an archive/ folder of historical versions, assets/ for diagrams, field_manuals/ for operational checklists, and templates/ for report and statement-of-work documents. A master table of contents lives in docs/SUMMARY.md, which the README positions as the canonical GitBook-style navigation entry point. That template-driven organization — standardized metadata, abstracts per chapter, and reusable reporting templates — signals the author's consultancy background: this is material designed to be dropped directly into a client-facing engagement workflow.
The chapter roadmap is divided into eight strategic parts, and the progression is worth examining because it reveals the intended methodology. Parts I and II are entirely non-technical: ethics, mindset, rules of engagement, threat modeling, scoping, lab setup, and chain of custody. This front-loading of governance material is unusual in offensive-security repos and is a strong signal that the handbook is aimed at professional consultants operating under contract, not hobbyists poking at public chatbots. The README's disclaimer reinforces this explicitly, restricting use to educational purposes and systems the operator owns or has permission to test, and warning against testing public LLM services in violation of their terms of service.
Parts III and IV move into technical fundamentals: LLM architectures, tokenization, plugin and API surfaces, then pipeline-level concerns around RAG pipelines and supply chain security. This is where the handbook differentiates itself from the flood of prompt-injection cheat sheets that followed the ChatGPT era. Treating retrieval-augmented generation as a distinct attack surface — with its own document ingestion paths, embedding stores, and retrieval indirection — matches how real AI-enabled applications actually fail, and the supply chain chapter acknowledges that poisoned dependencies and compromised model artifacts are as relevant as adversarial prompts.
Parts V and VI form the offensive and defensive core. Chapters 14 through 24 catalog attack techniques including prompt injection, data leakage, jailbreaking, API exploitation, evasion, poisoning, model theft, denial of service, multimodal attacks, and social engineering. Chapters 25 through 30 then mirror them with mitigations: adversarial ML defenses, supply chain defense, federated learning, privacy, model inversion, and backdoors. Pairing each attack class with a defense chapter is a deliberate structural choice that keeps the material useful to blue teams and AI engineers, not just operators — a defender reading Part VI gets a mapped view of exactly which threats their controls are meant to counter.
Parts VII and VIII are what make the repository a consultant's handbook rather than a technical manual. Advanced operations chapters cover reconnaissance, attack frameworks, automation, defense evasion, post-exploitation, reporting, remediation, continuous red teaming, and bug bounties. The final part closes with compliance material — notably the EU AI Act and ISO 42001 — plus industry best practices, case studies, emerging threats, and guidance on building a red team program. The compliance angle reflects the current market reality where AI security assessments are increasingly driven by regulatory obligations, and the inclusion of report and SOW templates in docs/templates/ means the paperwork side of an engagement is addressed alongside the technical work.
The Python component under scripts/ is the executable layer of the project. The README documents a straightforward setup: clone the repository, then pip install -r config/requirements.txt from the scripts/ directory. Prerequisites are deliberately modest — Python 3.8+ and API access to a target model, with OpenAI, Anthropic, or a local Ollama instance all named as supported endpoints. Configuration follows the familiar dotenv pattern: copy .env.example to .env, populate API keys, and the test runner picks them up from there.
Invocation is centered on a single runner entry point, examples/runner.py, which accepts a --target flag naming the model under test and a --test flag selecting the suite — the README's example runs the prompt_injection suite against a named model. The framing matters here: this is a testing harness pointed at an endpoint you have provisioned, not a scanning tool aimed at arbitrary targets. The mention of local Ollama as a supported backend is particularly sensible for lab work, letting assessors iterate against open-weight models they host themselves before touching production endpoints under contract.
From a telemetry and detection standpoint, the framework's footprint is straightforward to characterize. Every run generates outbound API calls containing deliberately adversarial prompts, so on the defender's side the observable signal lives in application logs, LLM gateway logs, and provider abuse-monitoring dashboards: unusual request volumes, clustered safety-violation attempts, and repeated structured prompt patterns from a single key or source address. Because the runner targets models by name through standard provider APIs, organizations running internal LLM front-ends can baseline normal prompt distributions and alert on the anomalous bursts this kind of automated suite produces.
A few operational caveats are worth noting. The repository license badge claims CC BY-SA 4.0 while the GitHub metadata reports NOASSERTION, so anyone embedding the handbook content into commercial deliverables should verify licensing before redistributing derived material. The repository also carries a workflows/ directory for CI/CD automation and an .agent/ directory described as agentic memory and context — the latter is an interesting hint that the project's own maintenance may be partially AI-assisted, which is fitting for the subject matter and worth keeping in mind when evaluating chapter consistency.
As a whole, ai-llm-red-team-handbook earns its place as a reference for teams building AI assessment practices. The combination of governance-first framing, a paired attack-and-defense chapter structure, engagement-ready templates, and a modest but functional automation layer covers more of the actual workflow of an authorized AI red team engagement than most competing resources. For defenders, Parts VI and VII double as a threat-modeling checklist; for consultants, the scoping and compliance chapters address the parts of the job that technical cheat sheets consistently ignore. Used within its stated boundaries — owned systems or explicit authorization, and compliance with provider terms of service — it is one of the more complete documentary assets in the AI security space.
Shiva108/ai-llm-red-team-handbook.Educational analysis for authorized security professionals. Use only in controlled, authorized environments.
Related coverage
0 comentários:
Post a Comment
Note: Only a member of this blog may post a comment.