SECURITY EDUCATION, PRIVACY GUIDANCE, THREAT AWARENESS, OPEN SOURCE TOOLS, RESEARCH NOTES, AND RESPONSIBLE TECHNOLOGY CONTENT

  • Penetration Testing Distribution - BackBox

    BackBox is a penetration test and security assessment oriented Ubuntu-based Linux distribution providing a network and informatic systems analysis toolkit. It includes a complete set of tools required for ethical hacking and security testing...
  • Pentest Distro Linux - Weakerth4n

    Weakerth4n is a penetration testing distribution which is built from Debian Squeeze.For the desktop environment it uses Fluxbox...
  • The Amnesic Incognito Live System - Tails

    Tails is a live system that aims to preserve your privacy and anonymity. It helps you to use the Internet anonymously and circumvent censorship...
  • Penetration Testing Distribution - BlackArch

    BlackArch is a penetration testing distribution based on Arch Linux that provides a large amount of cyber security tools. It is an open-source distro created specially for penetration testers and security researchers...
  • The Best Penetration Testing Distribution - Kali Linux

    Kali Linux is a Debian-based distribution for digital forensics and penetration testing, developed and maintained by Offensive Security. Mati Aharoni and Devon Kearns rewrote BackTrack...
  • Friendly OS designed for Pentesting - ParrotOS

    Parrot Security OS is a cloud friendly operating system designed for Pentesting, Computer Forensic, Reverse engineering, Hacking, Cloud pentesting...

Monday, January 4, 2016

IVRE - A Python network recon framework, based on Nmap, Bro & p0f



IVRE (Instrument de veille sur les réseaux extérieurs) or DRUNK (Dynamic Recon of UNKnown networks) is a network recon framework, including two modules for passive recon (one p0f-based and one Bro-based) and one module for active recon (mostly Nmap-based, with a bit of ZMap).
The advertising slogans are:
  • (in French): IVRE, il scanne Internet.
  • (in English): Know the networks, get DRUNK!
The names IVRE and DRUNK have been chosen as a tribute to "Le Taullier".

External programs / dependencies

IVRE relies on:
  • Python 2, version 2.6 minimum
  • Nmap & ZMap
  • Bro & p0f
  • MongoDB, version 2.6 minimum
  • a web server (successfully tested with Apache and Nginx, should work with anything capable of serving static files and run a Python-based CGI), although a test web server is now distributed with IVRE (httpd-ivre)
  • a web browser (successfully tested with recent versions of Firefox and Chromium)
  • Maxmind GeoIPfree databases
  • optionally Tesseract, if you plan to add screenshots to your Nmap scan results
  • optionally Docker & Vagrant (version 1.6 minimum)
IVRE comes with (refer to the LICENSE-EXTERNALfile for the licenses):

Passive recon

The following steps will show some examples of passive network recon with IVRE. If you only want active (for example, Nmap-based) recon, you can skip this part.

Using Bro

You need to run bro (2.3 minimum) with the option -b and the location of the passiverecon.bro file. If you want to run it on the eth0 interface, for example, run:
# mkdir logs
# bro -b /usr/local/share/ivre/passiverecon/passiverecon.bro -i eth0

If you want to run it on the capture file (capture needs to a PCAP file), run:
$ mkdir logs
$ bro -b /usr/local/share/ivre/passiverecon/passiverecon.bro -r capture

This will produce log files in the logs directory. You need to run a passivereconworker to process these files. You can try:
$ passivereconworker --directory=logs
This program will not stop by itself. You can (p)kill it, it will stop gently (as soon as it has finished to process the current file).

Using p0f

To start filling your database with information from the eth0interface, you just need to run (passiverecon is just a sensor name here):
# p0f2db -s passiverecon iface:eth0
And from the same capture file:
$ p0f2db -s passiverecon capture

Using the results

You have two options for now:
  • the ipinfo command line tool
  • the db.passive object of the ivre.db Python module
For example, to show everything stored about an IP address or a network:
$ ipinfo 1.2.3.4
$ ipinfo 1.2.3.0/24

See the output of ipinfo --help.
To use the Python module, run for example:
$ python
>>> from ivre.db import db
>>> db.passive.get(db.passive.flt_empty)[0]

For more, run help(db.passive) from the Python shell.

Active recon

Scanning

The easiest way is to install IVRE on the "scanning" machine and run:
# runscans --routable --limit 1000 --output=XMLFork
This will run a standard scan against 1000 random hosts on the Internet by running 30 nmap processes in parallel. See the output of runscans --help if you want to do something else.
When it's over, to import the results in the database, run:
$ nmap2db -c ROUTABLE-CAMPAIGN-001 -s MySource -r scans/ROUTABLE/up
Here, ROUTABLE-CAMPAIGN-001 is a category (just an arbitrary name that you will use later to filter scan results) and MySource is a friendly name for your scanning machine (same here, an arbitrary name usable to filter scan results; by default, when you insert a scan result, if you already have a scan result for the same host address with the same source, the previous result is moved to an "archive" collection (fewer indexes) and the new result is inserted in the database).
There is an alternative to installing IVRE on the scanning machine that allows to use several agents from one master. See the AGENT file, the program runscans-agent for the master and the agent/ directory in the source tree.

Using the results

You have three options:
  • the scancli command line tool
  • the db.nmap object of the ivre.db Python module
  • the web interface

CLI: scancli

To get all the hosts with the port 22 open:
$ scancli --port 22
See the output of scancli --help.

Python module

To use the Python module, run for example:
$ python
>>> from ivre.db import db
>>> db.nmap.get(db.nmap.flt_empty)[0]

For more, run help(db.nmap) from the Python shell.

Web interface

The interface is meant to be easy to use, it has its own documentation.


Share:

Passgen - Random Character Generator Crunch to Crack WPA/WPA2




Passgen is an alternative for the random character generator crunch which attempts to solve cracking WPA/WPA2 keys by randomizing the output opposed to generating a list like so, (aaaaaaaa, aaaaaaab, aaaaaac, etc).

Example usuage with aircrack-ng
python passgen.py -l | sudo aircrack-ng --bssid 00:11:22:33:44:55 -w- WiFi.cap)

Argument switches are as followed:
-l lowercase ascii
-l1 lowercase ascii + digits(0-9)
-U uppercase ascii
-U1 uppercase ascii + digits
-lU lowercase + uppercase ascii
-lU1 lowercase + uppercase ascii + digits
-C [char] [length] custom character set + length


Share:

Lynis 2.1.1 - Security Auditing Tool for Unix/Linux Systems





Lynis is an open source security auditing tool. Commonly used by system administrators, security professionals and auditors, to evaluate the security defenses of their Linux/Unix based systems. It runs on the host itself, so it can perform very extensive security scans.

Supported operating systems

The tool has almost no dependencies, therefore it runs on almost all Unix based systems and versions, including:
  • AIX
  • FreeBSD
  • HP-UX
  • Linux
  • Mac OS
  • NetBSD
  • OpenBSD
  • Solaris
  • and others
It even runs on systems like the Raspberry Pi and several storage devices!

No installation required

The tool is very flexible and easy to use. It is one of the few tools, in which installation is optional. Just place it on the system, give it a command like "audit system", and it will run. It is written in shell script and released as open source software (GPL).

How it works

Lynis performs hundreds of individual tests, to determine the security state of the system. The security scan itself consists of performing a set of steps, from initialization the program, up to the report.

Steps
  1. Determine operating system
  2. Search for available tools and utilities
  3. Check for Lynis update
  4. Run tests from enabled plugins
  5. Run security tests per category
  6. Report status of security scan
During the scan, technical details about the scan are stored in a log file. At the same time findings (warnings, suggestions, data collection), are stored in a report file.

Opportunistic scanning

Lynis scanning is opportunistic: it uses what it can find.
For example if it sees you are running Apache, it will perform an initial round of Apache related tests. When during the Apache scan it also discovers a SSL/TLS configuration, it will perform additional auditing steps on that. While doing that, it then will collect discovered certificates, so they can be scanned later as well.

In-depth security scans

By performing opportunistic scanning, the tool can run with almost no dependencies. The more it finds, the deeper the audit will be. In other words, Lynis will always perform scans which are customized to your system. No audit will be the same!

Use cases

Since Lynis is flexible, it is used for several different purposes. Typical use cases for Lynis include:
  • Security auditing
  • Compliance testing (e.g. PCI, HIPAA, SOx)
  • Vulnerability detection and scanning
  • System hardening

Resources used for testing

Many other tools use the same data files for performing tests. Since Lynis is not limited to a few common Linux distributions, it uses tests from standards and many custom ones not found in any other tool.
  • Best practices
  • CIS
  • NIST
  • NSA
  • OpenSCAP data
  • Vendor guides and recommendations (e.g. Debian Gentoo, Red Hat)

Parameters
--auditor "Given name Surname"     Assign an auditor name to the audit (report)
--checkall -c Start the check
--check-update Check if Lynis is up-to-date
--cronjob Run Lynis as cronjob (includes -c -Q)
--help -h Shows valid parameters
--manpage View man page
--nocolors Do not use any colors
--pentest Perform a penetration test scan (non-privileged)
--quick -Q Don't wait for user input, except on errors
--quiet Only show warnings (includes --quick, but doesn't wait)
--reverse-colors Use a different color scheme for lighter backgrounds
--version -V Check program version (and quit)

Changelog
Lynis 2.1.1
=  Lynis 2.1.1 (2015-07-22)  =

This release adds a lot of improvements, with focus on performance, and
additional support for common Linux distributions and external utilities.
We recommend to use this latest version.

* Operating system enhancements
-------------------------------
Support for systems like CentOS, openSUSE, Slackware is improved.

* Performance
-------------
Performance tuning has been applied, to speed up execution of the audit on
systems with many files. This also includes code cleanups.

* Automatic updates
-------------------
Initial work on an automatic updater has been implemented. This way Lynis
can be scheduled for automatic updating from a trusted source.

* Internal functions
--------------------
Not all systems have readlink, or the -f option of readlink. The
ShowSymlinkPath function has been extended with a Python based check, which
is often available.

* Software support
------------------
Apache module directory /usr/lib64/apache has been added, which is used on
openSUSE.

Support for Chef has been added.

Added tests for CSF's lfd utility for integrity monitoring on directories and
files. Related tests are FINT-4334 and FINT-4336.

Added support for Chrony time daemon and timesync daemon. Additionally NTP
sychronization status is checked when it is enabled.

Improved single user mode protection on the rescue.service file.

* Other
-------
Check for user permissions has been extended.
Python binary is now detected, to help with symlink detection.
Several new legal terms have been added, which are used for usage in banners.
In several files old tests have been removed, to further clean up the code.

* Bug fixes
---------
Nginx test showed error when access_log had multiple parameters.
Tests using locate won't be performed if not present.
Fix false positive match on Squid unsafe ports [SQD-3624].
The hardening index is now also inserted into the report if it is not displayed
on screen.

* Functions
---------
Added AddSystemGroup function

* New tests
---------
Several new tests have been added:

[PKGS-7366] Scan for debsecan utility on Debian systems
[PKGS-7410] Determine amount of installed kernel packages
[TIME-3106] Check synchronization status of NTP on systemd based systems
[CONT-8102] Docker daemon status and gather basic details
[CONT-8104] Check docker info for any Docker warnings
[CONT-8106] Check total, running and unused Docker containers

* Plugins
---------

[PLGN-2602] Disabled by default, as it may be too slow for some machines
[PLGN-3002] Extended with /sbin/nologin

* Documentation
---------------
A new document has been created to help with the process of upgrading Lynis.
It is available at https://cisofy.com/documentation/lynis/upgrading/

--------------------------------------------------------------


Share:

Egress-Assess - Tool used to Test Egress Data Detection Capabilities




Egress-Assess is a tool used to test egress data detection capabilities.

Setup

To setup, run the included setup script, or perform the following:
  1. Install pyftpdlib
  2. Generate a server certificate and store it as "server.pem" on the same level as Egress-Assess. This can be done with the following command:
"openssl req -new -x509 -keyout server.pem -out server.pem -days 365 -nodes"

Usage

Typical use case for Egress-Assess is to copy this tool in two locations. One location will act as the server, the other will act as the client. Egress-Assess can send data over FTP, HTTP, and HTTPS.
To extract data over FTP, you would first start Egress-Assess’s FTP server by selecting “--server ftp” and providing a username and password to use:
./Egress-Assess.py --server ftp --username testuser --password pass123
Now, to have the client connect and send data to the ftp server, you could run...
./Egress-Assess.py --client ftp --username testuser --password pass123 --ip 192.168.63.149 --datatype ssn
Also, you can setup Egress-Assess to act as a web server by running....
./Egress-Assess.py --server https
Then, to send data to the FTP server, and to specifically send 15 megs of credit card data, run the following command...
./Egress-Assess.py --client https --data-size 15 --ip 192.168.63.149 --datatype cc


Share:

Dharma - A generation-based, context-free grammar fuzzer



A generation-based, context-free grammar fuzzer.

Requirements

None

Examples

Generate a single test-case.
% ./dharma.py -grammars grammars/webcrypto.dg
Generate a single test case with multiple grammars.
% ./dharma.py -grammars grammars/canvas2d.dg grammars/mediarecorder.dg
Generating test-cases as files.
% ./dharma.py -grammars grammars/webcrypto.dg -storage . -count 5
Generate test-cases, send each over WebSocket to Firefox, observe the process for crashes and bucket them.
% ./dharma.py -server -grammars grammars/canvas2d.dg -template grammars/var/templates/html5/default.html
% ./framboise.py -setup inbound64-release -debug -worker 4 -testcase ~/dev/projects/fuzzers/dharma/grammars/var/index.html

Benchmark the generator.
% time ./dharma.py -grammars grammars/webcrypto.dg -count 10000 > /dev/null

Grammar Cheetsheet

Comment
%%% comment

Controls
%const% name := value

Sections
%section% := value
%section% := variable
%section% := variance


Extension methods
%range%(0-9)
%range%(0.0-9.0)
%range%(a-z)
%range%(!-~)
%range%(0x100-0x200)

%repeat%(+variable+)
%repeat%(+variable+, ", ")

%uri%(path)
%uri%(lookup_key)

%block%(path)

%choice%(foo, "bar", 1)


Assigning values
digit :=
%range%(0-9)

sign :=
+
-

value :=
+sign+%repeat%(+digit+)

Using values
+value+

Assigning variables
variable :=
@variable@ = new Foo();


Using variables
value :=
!variable!.bar();

Referencing values from common.dg
value :=
attribute=+common:number+


Calling javascript library functions
foo :=
Random.pick([0,1]);



Share:

KeyBox - A web-based SSH console that centrally manages administrative access to systems




KeyBox is a web-based SSH console that centrally manages administrative access to systems. Web-based administration is combined with management and distribution of user's public SSH keys. Key management and administration is based on profiles assigned to defined users.

Administrators can login using two-factor authentication with FreeOTP or Google Authenticator. From there they can manage their public SSH keys or connect to their systems through a web-shell. Commands can be shared across shells to make patching easier and eliminate redundant command execution.

KeyBox layers TLS/SSL on top of SSH and acts as a bastion host for administration. Protocols are stacked (TLS/SSL + SSH) so infrastructure cannot be exposed through tunneling / port forwarding. More details can be found in the following whitepaper: The Security Implications of SSH. Also, SSH key management is enabled by default to prevent unmanaged public keys and enforce best practices.

Prerequisites

To Run Bundled with Jetty

If you're not big on the idea of building from source...
Download keybox-jetty-vXX.XX.tar.gz
https://github.com/skavanagh/KeyBox/releases
Export environment variables
for Linux/Unix/OSX
 export JAVA_HOME=/path/to/jdk
export PATH=$JAVA_HOME/bin:$PATH

for Windows
 set JAVA_HOME=C:\path\to\jdk
set PATH=%JAVA_HOME%\bin;%PATH%

Start KeyBox
for Linux/Unix/OSX
    ./startKeyBox.sh
for Windows
    startKeyBox.bat
How to Configure SSL in Jetty (it is a good idea to add or generate your own unique certificate)
http://wiki.eclipse.org/Jetty/Howto/Configure_SSL

Using KeyBox

Open browser to https://<whatever ip>:8443
Login with
username:admin
password:changeme

Steps:
  1. Create systems
  2. Create profiles
  3. Assign systems to profile
  4. Assign profiles to users
  5. Users can login to create sessions on assigned systems
  6. Start a composite SSH session or create and execute a script across multiple sessions
  7. Add additional public keys to systems
  8. Disable any adminstrative public key forcing key rotation.
  9. Audit session history

Share:

BetterCap - A complete, modular, portable and easily extensible MITM framework




BetterCap is an attempt to create a complete, modular, portable and easily extensible MITM framework with every kind of features could be needed while performing a man in the middle attack.
It's currently able to sniff and print from the network the following informations:
  • URLs being visited.
  • HTTPS host being visited.
  • HTTP POSTed data.
  • FTP credentials.
  • IRC credentials.
  • POP, IMAP and SMTP credentials.
  • NTLMv1/v2 ( HTTP, SMB, LDAP, etc ) credentials.

DEPENDS
  • colorize (gem install colorize)
  • packetfu (gem install packetfu)
  • pcaprub (gem install pcaprub) [sudo apt-get install ruby-dev libpcap-dev]

Share:

SIMP - System Integrity Management Platform Lydecker Black




SIMP is a framework that aims to provide a reasonable combination of security compliance and operational flexibility.

The ultimate goal of the project is to provide a complete management environment focused on compliance with the various profiles in the SCAP Security Guide Project and industry best practice.

Though it is fully capable out of the box, the intent of SIMP is to be molded to your target environment in such a way that deviations are easily identifiable to both Operations Teams and Security Officers.


Supported Operating Systems

The following Operating Systems are supported:
  • Red Hat Enterprise Linux
    • 6.6
    • 7.1
  • CentOS
    • 6.6
    • 7.1-1503-01

Technology components

SIMP uses Puppet to manage and maintain the configuration of the various component systems.
Though there are many possible configurations, out of the box SIMP provides:
  • Management
    • Puppet Server
    • PuppetDB
    • MCollective
  • Authentication
    • OpenLDAP
  • Kickstart/Update
    • YUM
    • DNS
    • DHCP
    • TFTP

SIMP Provided Materials

Build Materials

Puppet Modules

Forked External Modules

Most forks are simply to fit the materials into our build processes but some have modifications that we are looking to push back upstream when possible.


Share:

yarGen - A Generator for Yara Rules (for malware researchers)




yarGen is a generator for Yara rules.

What does yarGen do?

The main principle is the creation of yara rules from strings found in malware files while removing all strings that also appear in goodware files.

Since version 0.14.0 it uses naive-bayes-classifier by Mustafa Atik and Nejdet Yucesoy in order to classify the string and detect useful words instead of compression/encryption garbage.

Since version 0.12.0 yarGen does not completely remove the goodware strings from the analysis process but includes them with a very low score. The rules will be included if no better strings can be found and marked with a comment /* Goodware rule */. Force yarGen to remvoe all goodware strings with --excludegood. Also since version 0.12.0 yarGen allows to place the "strings.xml" from PEstudio in the program directory in order to apply the blacklist definition during the string analysis process. You'll get better results.

The rule generation process tries to identify similarities between the files that get analyzed and then combines the strings to so called "super rules". Up to now the super rule generation does not remove the simple rule for the files that have been combined in a single super rule. This means that there is some redundancy when super rules are created. You can supress a simple rule for a file that was already covered by super rule by using --nosimple.

Installation
  1. Make sure you have at least 2GB of RAM on the machine you plan to use yarGen
  2. Clone the git repository
  3. Install all dependancies with sudo pip install pickle scandir lxml naiveBayesClassifier
  4. Unzip the goodware database (e.g. 7z x good-strings.db.zip.001)
  5. See help with python yarGen.py --help

Memory Requirements

Warning: yarGen pulls the whole goodstring database to memory and uses up to 2 GB of memory for a few seconds.

Command Line Parameters

usage: yarGen.py [-h] [-m M] [-g G] [-u] [-c] [-o output_rule_file]
[-p prefix] [-a author] [-r ref] [-l min-size] [-z min-score]
[-s max-size] [-rc maxstrings] [-nr] [-oe] [-fs size-in-MB]
[--score] [--inverse] [--nodirname] [--noscorefilter]
[--excludegood] [--nosimple] [--nomagic] [--nofilesize]
[-fm FM] [--noglobal] [--nosuper] [--debug]

yarGen

optional arguments:
-h, --help show this help message and exit
-m M Path to scan for malware
-g G Path to scan for goodware (dont use the database
shipped with yaraGen)
-u Update local goodware database (use with -g)
-c Create new local goodware database (use with -g)
-o output_rule_file Output rule file
-p prefix Prefix for the rule description
-a author Author Name
-r ref Reference
-l min-size Minimum string length to consider (default=8)
-z min-score Minimum score to consider (default=5)
-s max-size Maximum length to consider (default=128)
-rc maxstrings Maximum number of strings per rule (default=20,
intelligent filtering will be applied)
-nr Do not recursively scan directories
-oe Only scan executable extensions EXE, DLL, ASP, JSP,
PHP, BIN, INFECTED
-fs size-in-MB Max file size in MB to analyze (default=3)
--score Show the string scores as comments in the rules
--inverse Show the string scores as comments in the rules
--nodirname Don't use the folder name variable in inverse rules
--noscorefilter Don't filter strings based on score (default in
'inverse' mode)
--excludegood Force the exclude all goodware strings
--nosimple Skip simple rule creation for files included in super
rules
--nomagic Don't include the magic header condition statement
--nofilesize Don't include the filesize condition statement
-fm FM Multiplier for the maximum 'filesize' condition
(default: 5)
--noglobal Don't create global rules
--nosuper Don't try to create super rules that match against
various files
--debug Debug output

Best Practice

See the following blog post for a more detailed description on how to use yarGen for YARA rule creation: How to Write Simple but Sound Yara Rules

Examples

Use the shipped database (FAST) to create some rules
python yarGen.py -m X:\MAL\Case1401

Use the shipped database of goodware strings and scan the malware directory "X:\MAL" recursively. Create rules for all files included in this directory and below. A file named 'yargen_rules.yar' will be generated in the current directory.

Show the score of the strings as comment

yarGen will by default use the top 20 strings based on their score. To see how a certain string in the rule scored, use the "--score" parameter.
python yarGen.py --score -m X:\MAL\Case1401

Use only strings with a certain minimum score

In order to use only strings for your rules that match a certain minimum score use the "-z" parameter. It is a good pratice to first create rules with "--score" and than perform a second run with a minimum score set for you sample set via "-z".
python yarGen.py --score -z 5 -m X:\MAL\Case1401

Preset author and reference
python yarGen.py -a "Florian Roth" -r "http://goo.gl/c2qgFx" -m /opt/mal/case_441 -o case441.yar

Exclude strings from Goodware samples
python yarGen.py --excludegood -m /opt/mal/case_441

Supress simple rule if alreay covered by a super rules
python yarGen.py --nosimple -m /opt/mal/case_441

Show debugging output
python yarGen.py --debug -m /opt/mal/case_441

Create a new goodware strings database
python yarGen.py -c -g C:\Windows\System32

Update the goodware strings database (append new strings to the old ones)
python yarGen.py -u -g "C:\Program Files"

Inverse rule creation (still beta)

In order to create some inverse rules on goodware, you have to prepare a directory with subdirectories in which you include all versions of the files you want to create inverse rules for with their original name and in their original folder. If that sounds strange, let me give you an example.
E.g. you want to create inverse rules for all Windows executables in the System32 folder, you have to create a goodware archive with the following directory structure:
  • G:\goodware
    • WindowsXP
      • System32 - all files
    • Windows2003
      • System32 - all files
    • Windows2008R2
      • System32 - all files
yarGen than creates rules that identify e.g. file name "cmd.exe" in path ending with "System32" and checks if the file contains certain necessary strings. If the strings don't show up, the rule will fire. This indicates a replaced system file or malware file that tries to masquerade as a system file.
python yarGen.py --inverse -oe -m G:\goodware\

You can also instruct yarGen not to include the file path but solely rely on the filename.
python yarGen.py --inverse -oe --nodirname -m G:\goodware\


Share:
Established in 2015. Offensive Sec Blog has been sharing security research, hacking tools, threat intelligence, and offensive security content since 2015.
Copyright © OffSec Blog | Powered by OffensiveSec
Design by OffSec | Built for the security community