Using CIF with SiLK

The Collective Intelligence Framework or CIF for short provides a variety of security intelligence feeds that you can use in your environment. CIF requires a server to collect the information from a variety of sources and a client program that can be used to access the intelligence data. CIF has feeds for malware, botnets, suspicious IP addresses, and scanning IP addresses etc. Installing the CIF client on my SiLK server makes using CIF intelligence data with SiLK commands easy, simply follow the instructions for installing the client here. CIF has a number of ways you can output the intelligence data dependent on how you want to use it. For example the following command produces snort rules as output.
The -p snort part causes the output of CIF to be in snort rule format but all that's required for use with SiLK is CSV format. One of the clever things that CIF does is to provide IP addresses for source data even when the source data does not provide IP addressing such as for URLs.
As can be seen from the above command, I needed to remove some localhost IP addresses, headings, blank lines and other extraneous output. The -c parameter allows you to specify the degree of confidence for the intel data. See the CIF homepage for a more detailed description of command line options. SiLK has the ability to create an IP SET and these can be used with rwfilter as source or destination or either addresses:
  • --anyset=IP_SET_FILENAME
  • --not-anyset=IP_SET_FILENAME
  • --dipset=IP_SET_FILENAME
  • --not-dipset=IP_SET_FILENAME
  • --sipset=IP_SET_FILENAME
  • --not-sipset=IP_SET_FILENAME

To create a SiLK IP set from the CIF output simply run the output through rwsetbuild.
The following output uses the previously created malware.set as destination IP addresses:
The output also shows another feature of SiLK to randomize IP addresses. The rwrandomizeip, command randomizes all the IP addresses in a specific IP set. In this case I'm using it to obscure the external IP addressing of my test lab.
The Collective Intelligence Framework is an amazing addition to network security monitoring. CIF can also be integrated into ELSA and even into commercial SIEM platforms. Give it try.


  1. Interesting stuff here! I should look into CIF and SiLK pronto. I currently only use nfsen/nfdump. Thank you.

    Please add "Follow by email" and "Search" widgets to your blog(s).


Post a Comment

Popular posts from this blog

Squid Proxy with SOF-ELK Part 1

Netflow analysis with SiLK - Part 1 Installation

Password Cracking with CUDA 2 ways