thalesgroup-cert/Watcher is a Django and React platform that applies AI-driven summarization, NER-based IOC extraction, and Certificate Transparency monitoring to emerging-threat detection for authorized security teams.
| Tool | thalesgroup-cert/Watcher — AI-powered threat detection and monitoring platform built on Django and React |
| Category | Threat intelligence platform / OSINT monitoring, ~1380 stars, AGPL-3.0, JavaScript |
| Primary Use | Continuous monitoring of emerging threats, suspicious domains, data leaks, and ransomware/CVE intelligence for defenders, via certstream, dnstwist, and Hugging Face models |
| Safe Use | Defensive intelligence and situational awareness for authorized security teams, CERTs, and incident responders protecting their own organizations; not an attack tool |
| Telemetry Note | Watcher is a consumer of external data: its queries hit cve.circl.lu, ransomware.live, ransomlook.io, certstream, SearxNG, and scraping targets like Pastebin — network operators can observe this outbound collection traffic from the host running it |
Watcher, published by thalesgroup-cert at roughly 1,380 stars under AGPL-3.0, is a defensive threat intelligence platform that combines traditional OSINT collection with modern language-model analysis. Built as a Django backend with a React/MUI frontend and shipped as a Docker Compose stack, it positions itself as a self-hosted alternative to commercial TI portals: the README pitches deployment on a webserver or a ten-minute make up spin-up. What distinguishes it from earlier generation watchtower scripts is that it does not just collect signals — it summarizes, correlates, and exports them into an existing SOC workflow.
The AI layer is the most interesting architectural choice. Rather than calling a hosted LLM API, Watcher runs local models from Hugging Face: google/flan-t5-base handles text-to-text generation for threat summaries, while dslim/bert-base-NER performs named entity recognition to extract IOCs and threat-actor references from incoming text automatically. This is a meaningful design decision for a CERT-developed tool — local inference keeps potentially sensitive keyword watch rules and monitoring context inside the operator's infrastructure instead of leaking them to a third-party API. It also means the AI features are reproducible and free to run, at the cost of requiring the compute to load transformer models.
On the collection side, Watcher aggregates from a broad set of public sources. CVE data comes from cve.circl.lu, ransomware intelligence — both victim lists and group tracking — is pulled from ransomware.live and ransomlook.io, and advisory monitoring ingests RSS feeds from CERT-FR, CERT-EU, US-CERT, and the Australian Cyber Security Centre. The AI layer sits on top of this feed: automated weekly digests rank the top five trending cybersecurity topics, breaking-news alerts fire in real time when significant threats emerge, and on-demand keyword summaries include related CVEs and threat actor context generated by the local models.
Domain surveillance is handled through two complementary mechanisms. Suspicious domain detection uses dnstwist as its permutation engine, catching typosquatting, homograph, and DGA-adjacent variants of legitimate brands, while certstream provides real-time Certificate Transparency monitoring to flag newly registered suspicious domains as certificates are issued. Once a domain is flagged as malicious, the monitoring module tracks changes to its IP addresses, MX records, and web content, using TLSH fuzzy hashing from Trend Micro to detect content modifications that would evade simple diffing, plus automated RDAP/WHOIS checks with registrar and expiry alerts.
The information leak module extends collection to paste sites and developer platforms: Pastebin, StackOverflow, GitHub, GitLab, Bitbucket, APKMirror, and npm registries are all watched for leaked credentials, API keys, and confidential data. One practical compliance note from the README deserves attention: Pastebin's scraping API requires a paid pro account and whitelisting of Watcher's public IP, so operators should budget for that and expect their collection source to be visible to the upstream service — a consideration for teams that prefer their monitoring to be low-signal.
Where Watcher earns its place in an enterprise stack is integration. TheHive synchronization includes automated alert creation, case management, IOC enrichment, and preconfigured Cortex Analyzers and Responders — effectively turning Watcher findings into playbook-ready cases. MISP export via PyMISP supports smart UUID tracking, automatic object creation, and manual attribute updates, so IOCs feed collaborative intelligence sharing without duplicate pollution. A ticketing integration pushes findings into whatever system the SOC already runs, and notifications fan out through email, Slack, or Citadel on critical findings and threshold violations.
Authentication and access control are notably mature for an open source project. Beyond local and LDAP auth, Watcher supports full SSO through any OpenID Connect provider — Keycloak, Azure AD, or otherwise — with PKCE and Knox token issuance, configured per-instance via .env. The README also documents a superuser-only /connectors dashboard where every external integration (SMTP, Slack, Citadel, TheHive, MISP, CyberWatch feeds) can be viewed, edited, health-checked, and tested from one page, with encrypted credential storage. For teams that have inherited TI tools with secrets scattered across config files, this consolidation is a real operational win.
The API surface is properly documented: drf-spectacular auto-generates a Swagger UI at /api/docs/ and an OpenAPI 3 schema at /api/schema/, which makes scripted enrichment or dashboard embedding straightforward. The frontend adds deck.gl-powered geospatial visualization for a WorldMap view, resizable dashboard panels, saved filter sets, and multiple themes — cosmetic features, but they signal sustained engineering investment rather than a weekend project. Django's admin panel remains the backstop for user management, group permissions, and configuration, and granular per-module permissions support multi-analyst teams.
Legitimate domain management is an unusual but sensible module: security teams can maintain a centralized list of approved domains with expiry, repurchase status, registrar info, and contacts, convert monitored malicious domains into legitimate ones when they change hands, and track UDRP cases automatically. This reflects Watcher's CERT heritage — it is built by people who deal with the full lifecycle of abusive domains, not just the detection moment. Combined with shadow-useragent for User-Agent rotation during collection and SearxNG as a privacy-respecting metasearch layer, the dependency list reads like a coherent defensive toolkit rather than an accumulation of convenience libraries.
Installation follows a conventional Docker path: git clone the repo, cd into deployment, run make init to generate the environment and directory structure, then make up to start the stack. First-run setup is make migrate, make superuser, and make populate-db, after which the UI listens on port 9002 by default. The README claims roughly ten minutes from clone to console, which is plausible given the Makefile-driven provisioning, and a full installation guide plus hosted documentation are maintained separately on GitHub Pages.
For defenders evaluating Watcher, the operational footprint is worth understanding. The platform generates continuous outbound queries to its feed sources — CIRCL, ransomware trackers, certstream, RSS endpoints, scraping targets — which is benign but observable, and the Pastebin requirement explicitly ties your IP to your monitoring subscription. On the positive side, everything the platform does is consumption and correlation of public data; it ships no offensive capability. The licensing cuts both ways: AGPL-3.0 permits free internal use, but organizations exposing it over a network should review their obligations. As a self-hosted, AI-augmented intelligence hub that plugs into TheHive, MISP, and enterprise SSO, Watcher is one of the more complete open source offerings in its category and a strong fit for authorized security teams building situational awareness around their own organization's exposure.
thalesgroup-cert/Watcher.Educational analysis for authorized security professionals. Use only in controlled, authorized environments.
Related coverage
0 comentários:
Post a Comment
Note: Only a member of this blog may post a comment.