Moloch FPC

Moloch is an open source project providing full packet capture. It's been around for a while now and has matured to the point where deployment is simple and it pretty much manages itself. Moloch has an amazingly good UI and search is powered by the hugely capable Elasticsearch.
In this post I'm going to run through the installation and configuration on a CentOS 7 VM using the Moloch 0.18.2 RPM available here

Requirements For a small lab environment such as I have a VM based solution will work just fine. Moloch will scale; more information can be found on the Readme and in the FAQ. I use a SPAN port on a Cisco switch that sends traffic into a virtual switch on VMware allowing any VM Guest with an interface on that vswitch to sniff traffic. I created a VM Guest with the following specification:
CentOS 7 minimal install8 Cores 400GB Disk 1 100GB Disk 2 12GB Ram 2 NICs Build First I installed CentOS on a 50GB logical volume and then created a second logical volume of 350GB mounte…

IDSUtil and Wireshark Alert plugin

I recently came across a really neat Wireshark plugin for displaying IDS alerts inside of Wireshark. I find this a really useful way of doing historical packet capture analysis as I have the complete detail of the alert right there inside of Wireshark. I installed the IDSUtil on a VM running the Centrych Linux distro which I have found to be one of the most pleasant to install and use. Centrych, the IDSUtil and the Wireshark Alert plugin were all created by Jack Radigan and I highly recommend them to anyone who needs to do historical packet analysis.
There is a great demonstration of the Wireshark plugin and the IDSUtil here.
Once everything is installed and configured all that is required is to update the rules and then run the ids-pcap command with the packet capture:
ids-rules ./snort/default --list
ids-pcap ./snort/default vrouter2.pcap Once the pcap has been read by Snort or Suricata the alerts are available in Wireshark when the same pcap is opened.

Where there are more t…

From Bro to Log Parser Lizard to Security Visualisation

Recently I had to do some work with packet captures and system logs and decided to use Log Parser Lizard to examine the syslog files and the Bro logs I got from parsing the pcap's. Log Parser Lizard is a GUI for the brilliant MS Log Parser utility. I know a lot of us of complain that Windows doesn't have our favourite text processing utilities like grep/sed/awk etc but the addition of MS Log Parser more than makes up for the loss. Adding Log Parser Lizard provides a really cool way of analysing data for forensics and much more. For anyone new to MS Log Parser there is a great book entitled Microsoft Log Parser Toolkit available on Amazon. This is a great solution for ad-hoc data analysis when you don't have the data in ELSA or logstash, but more than that, it provides a minimal capability for exploratory data analysis without requiring the 'R' statistical language or Python with the SciPy stack. Even if ultimately you need to use either of those, these techniques c…

Password Cracking with CUDA 2 ways

A few weeks ago I decided to generate Rainbow Tables for LM hash password cracking. The Rainbowcrack project provides Windows and Linux software that can be used to generate the tables and do the actual cracking. I also wanted to leverage the CUDA GPU support to make the cracking as fast as possible. The first thing I needed to do was to generate the actual rainbow tables. In my lab I have two Proliant ML350 servers running ESXi 5.1 (dual Xeon E5645 in each) so rather than running the table generation on my laptop I created a Windows VM on one of the servers gave it 8 vcpu's and cut 'n' paste the commands for the table generation into a batch file. I set the batch file running and went to bed. The next morning I checked on progress and calculated how long it was going to take to complete. With a bit of rough math I reckoned about six weeks!

Six Weeks Later... After running at a near constant 100% CPU utilisation for the full six weeks my rainbow tables were finally ready. …

Using CIF with SiLK

The Collective Intelligence Framework or CIF for short provides a variety of security intelligence feeds that you can use in your environment. CIF requires a server to collect the information from a variety of sources and a client program that can be used to access the intelligence data. CIF has feeds for malware, botnets, suspicious IP addresses, and scanning IP addresses etc. Installing the CIF client on my SiLK server makes using CIF intelligence data with SiLK commands easy, simply follow the instructions for installing the client here. CIF has a number of ways you can output the intelligence data dependent on how you want to use it. For example the following command produces snort rules as output.
The -p snort part causes the output of CIF to be in snort rule format but all that's required for use with SiLK is CSV format. One of the clever things that CIF does is to provide IP addresses for source data even when the source data does not provide IP addressing such as for UR…

ELSA with Sagan

Sagan is essentially a snort-like rule based detection engine for log data. Sagan is very easy to integrate with ELSA. All the logs sent to ELSA can be examined by Sagan rules. Every rule that fires produces an alert which is passed into ELSA. Sagan is easy to configure, build and install I just followed the instructions here. As barnyard2 will be used to take the unified2 output from Sagan I built Sagan without native database support. Create a Sagan user, the directories and set the permissions. The Sagan user will be used by barnyard2 and Sagan. Configure syslog-ng to send events via the sagan.fifo to Sagan. Edit the /usr/local/syslog-ng-3.2.4/etc/syslog-ng.conf on ELSA to add the configuration that sends all the inbound logs through Sagan. This configuration sends all logs received over the network to Sagan via the fifo. Now we have
Network Syslog -> Syslog-ng -> Sagan The logs are still received by ELSA of course, but now the Sagan rules can inspect the log as w…

Creating a Vyatta parser for ELSA

ELSA or Enterprise Log Search and Archive to give it it's full title is a centralised log management solution. Similar to Splunk in principal; it provides google-like searching, graphing, dashboards and alerts. ELSA is extraordinarily easy to install and get up and running, just follow the quick-start guide.

Searching in ELSA is incredibly fast; Martin Holste, ELSA's creator, went to amazing lengths to ensure that it was not only fast but exceptionally scalable as well. In order to meet those design goals Martin knew that using regex to parse log data would be way too slow, so ELSA uses Syslog-ng and Pattern DB to receive and categorise events.

For more information watch Martin's presentation entitled Perl for Big data on youtube.

ELSA has numerous parsers built into it including Snort, Windows, Apache, Cisco, Check Point and many others, so you might find that your log source is already supported. However, I wanted to add Vyatta Community Edition firewall logs to ELSA…