WhatWeb identifies the CMS, web server, JavaScript libraries, and embedded devices behind a website through a plugin-driven fingerprinting engine, built for authorized reconnaissance and inventory work.
| Tool | urbanadventurer/WhatWeb — plugin-driven web technology fingerprinting scanner written in Ruby |
| Category | Web reconnaissance and fingerprinting scanner |
| Primary Use | Identifying CMS platforms, web servers, JavaScript libraries, and embedded devices, including version numbers, during authorized assessments and asset inventory |
| Safe Use | Intended for penetration tests and security assessments against systems you are explicitly authorized to test, internal asset inventories, labs, and defensive research |
| Telemetry Note | Default --aggression 1 stealthy mode makes a single HTTP request per target, but the default User-Agent string (WhatWeb/0.6.x) is trivially detectable in web server and WAF logs unless overridden with --user-agent; higher aggression levels generate distinctive request patterns for blue teams |
WhatWeb, hosted at urbanadventurer/WhatWeb and developed by Andrew Horton (urbanadventurer) and Brendan Coles (bcoles), answers a deceptively simple question: what is that website? The tool, currently at stable release 0.6.4 and licensed under GPLv2, identifies the web technologies powering a target — content management systems, blogging platforms, analytics packages, JavaScript libraries, web servers, and even embedded devices. With over 1800 plugins in the current release and roughly 6846 stars on GitHub, it has become one of the standard reference tools in the recon phase of authorized web assessments, and its packaging across distributions (visible via the repology badge in the README) confirms how widely deployed it is, including in Kali Linux per the repository topics.
The architecture is plugin-driven and that is the core of what makes WhatWeb durable. Each of the 1800+ plugins recognizes something different — a specific product, framework, or artifact — and most plugins are deliberately layered from subtle to obvious cues. The README's own example is the WordPress plugin: most WordPress sites announce themselves through the meta HTML generator tag, but a minority strip it. Rather than stopping at that single test, the plugin carries over 15 checks, including the favicon, default installation files, login pages, and the presence of /wp-content/ in relative links. This defensive-in-depth approach to matching is what separates a serious fingerprinter from a naive grep over page source.
The most conceptually interesting design decision is the aggression system, controlled with --aggression or -a. The tool explicitly frames this as a trade-off between speed/stealth and reliability. Level 1, the default and named stealthy, makes exactly one HTTP request per target and follows redirects — the README notes this is suitable for scanning public websites. Level 3, aggressive, fires additional requests whenever a level 1 plugin matches, effectively using a preliminary identification to select follow-up interrogation. Level 4, heavy, unleashes aggressive tests from all plugins against all URLs and makes, in the tool's own words, a lot of HTTP requests. The README is candid that the more aggressive modes were developed for penetration tests, which is exactly the framing an operator should internalize: heavy scanning against infrastructure you do not own or are not authorized to test is both abusive and noisy.
Beyond raw fingerprinting, WhatWeb extracts secondary intelligence from the same transactions: version numbers, email addresses, account IDs, web framework modules, and SQL errors. Its output also captures structural signals like cookies, Strict-Transport-Security headers, X-Frame-Options, Open-Graph-Protocol markers, uncommon headers, HttpOnly cookie attributes, and redirect chains, as demonstrated in the README's ./whatweb reddit.com example, which surfaces everything from the HTTPServer[snooserv] banner to Via-Proxy[1.1 varnish]. In practice this means a single stealthy pass produces not just a stack fingerprint but a lightweight security-header audit, which is why the tool doubles nicely as a defensive inventory scanner for organizations enumerating their own exposure.
Target selection is flexible. WhatWeb accepts URLs, hostnames, IP addresses, filenames, and IP ranges in CIDR, x.x.x-x, or x.x.x.x-x.x.x.x formats, plus --input-file (-i) for bulk lists — the README notes you can pipe hostnames directly with -i /dev/stdin. Target modification flags like --url-prefix, --url-suffix, and --url-pattern let you rewrite targets on the fly, for example inserting each hostname from an input file into a URL template to scan a consistent path such as robots.txt across an estate. Dual-protocol scanning automatically tests both HTTP and HTTPS for simple hostnames, and IDN support handles internationalized domain names.
HTTP-level ergonomics are thorough for an authorized workflow. --user-agent (-U) sets the identification string, --header (-H) adds or replaces arbitrary headers — including removing one by passing an empty value like User-Agent: — and --follow-redirect offers fine-grained control with never, http-only, meta-only, same-site, and always modes, capped by --max-redirects. Authentication is supported through --user for HTTP basic auth and a full cookie subsystem: --cookie for manual cookies, --cookiejar for file-based cookies, and --no-cookies to disable automatic handling. The README's performance note is worth heeding — automatic cookie handling across redirects improves fingerprinting accuracy on session-managed sites, but at thread counts above 100 it becomes a bottleneck, so --no-cookies is the right call for large-scale internal sweeps.
Output plumbing is arguably WhatWeb's quietest strength for professional use. Beyond brief greppable one-liners and verbose human-readable logs, it writes XML, JSON, JSON-verbose, SQL INSERT statements (with --log-sql-create to bootstrap the schema), MagicTree XML for reporting workflows, Ruby object inspection format, and direct streaming into MongoDB and ElasticSearch via --log-mongo-* and --log-elastic-* options. That ElasticSearch integration in particular means an authorized team can run continuous asset-discovery scans and query results in Kibana alongside other telemetry — a genuinely defensive application of a tool most people file under offense.
The plugin system is also extensible at runtime. --list-plugins and --info-plugins enumerate and describe the catalog, --search-plugins filters by keyword, and --plugins (-p) selects a subset — accepting directories, files, or names, with + and - modifiers to compose sets and -p + as a shortcut for enabling the disabled-plugins directory. The --custom-plugin flag defines an inline plugin straight from the command line using :text, :version, :ghdb, or :md5 matchers, which is the fastest way to fingerprint an internal application whose generator string only your organization knows. There is even --dorks to list Google dorks associated with a plugin, and --grep (-g) to filter results against strings or regexps.
Performance tuning is conventional but complete: --max-threads defaults to 25, --open-timeout to 15 seconds, --read-timeout to 30, and --wait introduces deliberate spacing between connections when running single-threaded — a useful throttle when scanning fragile authorized targets. Invocation is straightforward (./whatweb example.com), and the project is packaged in most security distributions, so many operators will already have it available as whatweb on their path.
From a defender's perspective, WhatWeb leaves a distinctive signature unless configured otherwise. The default User-Agent identifies itself as WhatWeb/0.6.x, which shows up plainly in access logs, WAF alerts, and SIEM correlation rules; anyone serious about stealth overrides it, but defenders should treat that default string as a reliable IOC of active reconnaissance. Aggression levels 3 and 4 generate escalating request volumes and characteristic probes for default installation files and login paths, all of which are straightforward to detect with rate-based and path-coverage analytics. Used as intended — on assets you own or are contractually authorized to assess — it remains one of the cleanest ways to answer what is actually running in your environment.
urbanadventurer/WhatWeb.Educational analysis for authorized security professionals. Use only in controlled, authorized environments.
0 comentários:
Post a Comment
Note: Only a member of this blog may post a comment.