A lab-scoped purple-team harness that drives MITRE Caldera through a chained credential-access path in an Active Directory domain to verify Sigma detections and expose documented gaps.
| Tool | joshuagodwin7929/Purple-Team-Automation — automated purple-team validation of Sigma rules using Caldera adversary emulation in an isolated AD lab |
| Category | detection validation / adversary emulation automation |
| Primary Use | Executing chained credential-access techniques against a private Active Directory lab to confirm Sigma detections fire and to map results into a MITRE ATT&CK Navigator heatmap |
| Safe Use | Designed exclusively for authorized security practitioners operating their own isolated lab environments; every action described targets a self-built domain, with results validated in Kibana for detection-engineering purposes |
| Telemetry Note | Emulation activity is intentionally loud in lab telemetry — Sysmon and Windows event logs feed Sigma rules and Kibana, and the project documents that LLMNR/NBT-NS poisoning currently goes undetected, which is exactly the kind of blind spot defenders should reproduce |
Most detection-rule testing stops at the atomic level: fire a single technique, check whether a rule matched, move on. joshuagodwin7929/Purple-Team-Automation takes the opposite stance. It is a documentation-heavy automation project that uses MITRE Caldera to run a multi-step credential-access chain against a purpose-built Active Directory lab, then verifies that a companion Sigma detection-as-code repository actually catches each stage in Kibana. The output is not a tool you install so much as a reproducible methodology plus the artifacts — custom ability YAMLs, an adversary profile, validation evidence, and an ATT&CK Navigator layer — that make the whole exercise auditable.
The README opens with an explicit design rationale: why Caldera rather than Atomic Red Team. The author's argument is that Atomic Red Team executes techniques in isolation, while Caldera is a full adversary-emulation framework with agents, adversary profiles, and chained operations. The distinction matters for detection engineering. The question this project asks is not "did this single technique get detected" but "does a realistic, ordered attack chain survive the current detection stack from start to finish." That framing places the work squarely in purple-team territory, where red-side execution exists to generate telemetry for the blue side.
Structurally, the repository is organized into a handful of directories with clear intent. abilities/ holds the custom Caldera ability YAMLs, adversary-profiles/ contains the chained profile driving the operation, and validation/ stores per-technique Kibana evidence along with a deliberately documented known-gap writeup on LLMNR poisoning. At the root sit attack-navigator-heatmap.json, loadable into the ATT&CK Navigator at mitre-attack.github.io/attack-navigator, and purple-team-automation-report.md, the full scope-methodology-results writeup. The repo rounds out at roughly 30 stars and carries topical tags spanning adversary-emulation, detection-engineering, siem, soc, and threat-hunting, accurately reflecting its dual audience.
The infrastructure section is unusually candid about operational friction, which is where much of the practical value lives for anyone replicating the setup. The author deployed a dedicated Caldera server via Docker Compose on a separate Ubuntu VM, deliberately isolated from the ELK/SIEM stack, and notes that Caldera v5.0.0 ships with 2000 stock abilities and 29 stock adversaries out of the box. From there, the README walks through a genuinely useful sequence of root-caused failures: a silent no-image build, a container crash-looping because a full-directory Docker volume mount in docker-compose.yml overwrote the compiled Vue frontend in plugins/magma/dist/assets/, and a non-functional UI caused by localhost:8888 hardcoded at Vue build time via plugins/magma/.env (VITE_CALDERA_URL) — notably not controlled by the runtime app.frontend.api_base_url in conf/local.yml.
Those build notes read like war stories, but they encode transferable lessons about container image hygiene and the difference between build-time and runtime configuration. The author also hit a disk-space failure traced to an LVM logical volume using only half the allocated disk, fixed with lvextend and resize2fs, and reclaimed build-cache layers with docker system prune -a --volumes. For defenders building similar lab infrastructure, this section doubles as a checklist of failure modes that waste hours: verify images actually built, distrust broad volume mounts, and check whether your frontend's API base URL was baked in at compile time.
The more technically interesting contribution is the custom ability work. The README observes that Caldera's Stockpile plugin only ships a native ability for LSASS/Mimikatz credential dumping (T1003.001), while the four techniques this project needed — Kerberoasting, AS-REP Roasting, password spraying, and DCSync — required custom abilities built around Impacket utilities (GetUserSPNs.py, GetNPUsers.py, secretsdump.py) and Kerbrute. The stated reason is compatibility: Stockpile's existing Kerberoasting abilities target Windows via Rubeus and WinPwn, which are incompatible with this lab's Linux-based Sandcat agent. That constraint-driven engineering is exactly the kind of detail that makes the project reusable by others running Linux emulation agents.
Two ability-authoring pitfalls are documented with equal precision. First, Caldera v5's ability schema uses purpose-built per-tool parser modules — the README cites plugins.stockpile.app.parsers.katz with source/edge/target fields — rather than a generic regex pattern parser, and assuming otherwise produced a silent TypeError('ParserConfig.__init__()') on load. Because no built-in parser exists for raw Impacket output, the author dropped the parsers: block entirely and validates results manually against Kibana. Second, two ability YAMLs were silently saved as 0-byte files after a failed nano paste, caught only by sanity-checking files with cat and wc -l before restarting the container. Both are small failures with outsized debugging costs, and documenting them is a service to the community.
On the red-side mechanics, the README describes deploying a Sandcat agent on a Kali VM with the group red, running as root with the proc/sh executor, and using the process name splunkd for OPSEC masquerading. This is worth noting for defenders reading the project from the blue perspective: an emulation agent masquerading as a legitimate SIEM process is realistic tradecraft, and the fact that it succeeded in the lab without tripping a detection is itself a finding about process-name-based trust. The chained adversary profile, AD Credential Access Chain, orders the four abilities the way an opportunistic internal attacker would: password spray via Kerbrute, then Kerberoasting via GetUserSPNs.py, then AS-REP roasting via GetNPUsers.py, then DCSync via secretsdump.py.
The results section is the payoff, and it is refreshingly honest. Four techniques were confirmed detected against the companion Sigma repository and cross-checked directly in Kibana: password spraying (T1110.003), Kerberoasting (T1558.003), AS-REP roasting (T1558.004), and DCSync (T1003.006) all earned green checkmarks. LLMNR/NBT-NS poisoning (T1557.001) remains a red-flagged gap. Critically, that gap was not accidental — the technique was deliberately excluded from the Caldera profile and is maintained as a documented negative control in validation/llmnr-known-gap.md. Treating a known blind spot as a first-class, versioned artifact rather than an embarrassment is a mature detection-engineering practice.
The remediation roadmap closes the loop in a way that models good purple-team hygiene. The next steps are concrete: enable Sysmon Event IDs 3 and 22, write and tune a new Sigma rule for LLMNR/NBT-NS poisoning, re-run the same operation to confirm the fix, and update the heatmap — turning detection coverage into a measurable, iterable pipeline rather than a one-time exercise. The attack-navigator-heatmap.json layer makes the coverage state portable and presentable to stakeholders.
For authorized professionals, this repository is best understood as a template rather than a product. It demonstrates end-to-end purple-team automation in a controlled lab: isolated infrastructure, custom Caldera abilities adapted to a Linux agent, a chained adversary profile, evidence-backed validation in Kibana, and honest gap accounting. Anyone running detection-as-code programs in their own environments — corporate security labs, training ranges, or research setups — can lift the structure wholesale and substitute their own techniques and rules. Nothing here should be pointed at infrastructure you do not own; the value lies entirely in the disciplined loop of emulate, detect, document, and fix.
joshuagodwin7929/Purple-Team-Automation.Educational analysis for authorized security professionals. Use only in controlled, authorized environments.
0 comentários:
Post a Comment
Note: Only a member of this blog may post a comment.