Sguil on RedHat HOWTO

From NSMWiki
Revision as of 07:17, 21 February 2007 by Bianco (Talk | contribs) (Snort)

Jump to: navigation, search

Purpose of This Document

The purpose of this document is to show how to compile, install and configure Sguil 0.6.0 on a RedHat Enterprise Linux 3 (RHEL) system. By following these instructions, you'll be able to put together your own Network Security Monitoring (NSM) infrastructure very quickly.

This document is part of the InstantNSM project.

Designing Your NSM Infrastructure

Before installing, you should have a clear view of what you want your NSM infrastructure to look like. A full discussion of this is outside the scope of this document, but the key questions to be answered are:

  • What sorts of things will I be looking for (eg., Insider attacks, Internet-based compromises, Virus/Worm activity, access to critical information assets)
  • Which parts of my network should I monitor?
  • Where can I place my sensors in order to best achieve my goals?
  • How can I get access to the traffic (eg, SPAN ports, taps or other methods)
  • How much data travels over these network links, and how much can I afford to capture?

Sguil supports many types of deployments, from the ýall-in-oneý where the sensor, database and sguil server are deployed on the same hardware, to the distributed configuration where multiple sensors report back to a dedicated database/server system. This document will allow you to install either of the above, or anything in between. Any piece of the sguil infrastructure can share a computer system with any other piece, so even though we refer to separate database machines, sensors and consoles, these can easily be placed on a single machine if enough computing and storage resources are available.

A note on multiple sensors: Sguil is quite capable of running multiple sensors (sniffing different networks) on the same system. Provided you have enough system resources (network cards, CPU, RAM, HD space and PCI bandwidth) this is no problem. If you install sensors as indicated in this document, you can easily configure multiple sensors on a single computer.

In this document, we will assume that the central database and the sguil server daemon are installed on the same system (as is recommended for all but exceptionally large installations). Thus, the terms database server and sguil server are used interchangeably. If this is not the case for your environment, adjust accordingly.

Prerequisites

Sguil runs on a variety of Unix and Unix like systems, both *BSD and Linux based. For the purposes of this document, we will assume that you are going to install and run sguil on Red Hat Enterprise Linux (RHEL) 3. The Workstation version is fine (and certainly less expensive), but if you have AS or ES versions, these will work as well. Newer versions of RHEL may also be compatible but have not been tested. Free clones of RHEL (e.g., CentOS 3 or Whitebox Linux) should also be compatible.

Server Hardware

The hardware necessary to run sguil depends entirely upon the amount of network traffic you plan to monitor, how much of it you store and how long you store it. The public demonstration server at demo.sguil.net gets by with a 133MHz Pentium system, though for production systems you'll want something beefier.

For the sguil server, try to find something with a lot of memory and a fast disk. The MySQL database will need a lot of RAM for caching results from its queries, and a fast disk will speed up the access time while doing sequential searches.

You may find it helpful to place the MySQL storage on a separate disk partition, or even a separate disk. It doesn't matter where you mount this storage area, but for convenience, I'll assume that that you're going to mount it on the same directory path you're using for the sensor data directory (see below), referred to in this document as $NSM.

The following table is a useful guideline for a good server hardware configuration. Feel free to adjust it up or down depending on the size of your installation.

Recommended Server Hardware
CPU RAM Disk Storage
3.0GHz 2GB 150GB (Consider a RAID5 array for speed & reliability)

Estimating the Size of the Database

The disk space required to host the database varies depending on how many sensors you have and how busy your network is. Most of the space is taken up by session records. Each session record takes up approximately 170 bytes of disk space, including database overhead like indices. If you have 3 million records per day, that comes to about 486MB/day of space.

By contrast, an alert consumes around 625 bytes on average, including packet information and other supporting data, but there are usually far fewer alerts than network sessions on a reasonably well-tuned sensor. In fact, the ratio of alerts to sessions is nearly 0, therefore you can safely ignore everything but the session data when sizing your database partition. For a production sensor with a several months of session and alert data, you'll probably want anywhere from 50GB - 100GB of free disk space devoted to MySQL, depending on your data retention policy.

Sensor Hardware

The sensors typically store PCAP files of all network traffic they monitor, so they usually require more disk space than the central database server. A large, fast disk is the most important thing for a sensor; they need to write raw packet data to the disk as fast as it comes off the wire, and there can be quite a lot of it. It would not be out of the question to consider large RAID5 arrays here, at least for the systems at your network perimeter. At the very least, buy a single big disk to use just for the data storage. With 500GB disks available now, storage for the sensors should be no problem. If you're running multiple sensors on the same machine, you'll probably want to double the RAM and multiply the storage space by the number of sensors on the system.

Recommended Sensor Hardware
CPU RAM Disk Storage
2.0GHz 512MB 300GB or more

Whatever you choose for storage, you'll need to mount it somewhere in order for it to be useful. The sguil documentation recommends mounting it as /snort_data, but at least one other well-known installation guide[1] recommends using /nsm instead. You can mount it wherever you like. In this document, we'll refer to the mountpoint as $NSM.

If you're deploying multiple sensors on the same system, be sure to give each of the sensors their own data partition. Each of the sensors has a unique name, and will have a directory named $NSM/$SENSORNAME (e.g., /nsm/sensor1, /nsm/sensor2, etc). Sguil uses this space to store packet capture data (among other things), and when the partition starts to get full, it deletes the oldest packet data for the sensor in order to make room for newer captures. The algorithm it uses assumes that each sensor has it's own partition, so they each need to be mounted separately, otherwise one sensor will eventually take up all the storage space and others won't get any.

Another thing to take into account when preparing the data partitions is how they are mounted. Sguil is constantly writing data to the disk, but it rarely reads it back. Therefore, it makes sense to optimize the filesystem for write performance. This can be accomplished by using the async and noatime mount options, as in the following /etc/fstab entry:

LABEL=nsm  /nsm  ext3 defaults,async,noatime 1 2

As for network interfaces, you'll need one for the system itself to use (the ýmanagement interfaceý) and one or more to sniff network traffic (the ýmonitoring interfacesý). While the management interface obviously requires an IP address and full network setup, the monitoring interfaces should be configured without addresses or network parameters of any sort. The number and type of monitoring interfaces you need will be dictated by the type of data collection you use and the number of sensors you will install on the system. Switch SPAN ports usually require only a single monitoring NIC, while most taps require two NICs (one for each direction of traffic on the monitored link).

Software

Sguil depends on several different pieces of software in order to do its job. Many of them come with RHEL, but some do not, and in the case of MySQL, you might want to use your own version anyway. The following table is a list of the required software, versions and download sites.

Supported Software Versions Software Version Location Download Notes
MySQL 5.x Server http://dev.mysql.com/downloads/mysql 4.1.x versions also work well
Libpcap 0.9.4 Sensor http://www.tcpdump.org/
Libnet 1.0.2a Sensor http://www.packetfactory.net/libnet/ Neither newer nor older versions may be used.
Snort 2.4.3 (or newer) Sensor http://www.snort.org/dl/

You will also need to register for a free snort.org account in order to download IDS rules

SANCP 1.6.1 Sensor http://www.metre.net/sancp.html Be sure to also fetch the two patches, fix200511.a and fix200511.b.
Barnyard 0.2.0 Sensor http://www.snort.org/dl/barnyard/
P0f 2.0.5 Server http://lcamtuf.coredump.cx/p0f.shtml
Tcpflow 0.21 Server http://www.circlemud.org/~jelson/software/tcpflow/
Tcltls 1.5 Server, Client http://sourceforge.net/projects/tls Provides for an encrypted data channel between the sguil server and the analyst consoles or sensors
Mysqltcl 2.51 Server http://www.xdobry.de/mysqltcl/ Be sure to get the mysqltcl-2.51.tar.gz version. Newer versions are incompatible with Sguil.
Tcllib 1.8 Server http://tcllib.sourceforge.net/
Sguil 0.6.0 Server, Sensor, Client http://sguil.sourceforge.net/index.php?page=download Be sure to fetch the 0.6.0 release, and not the development version from CVS!
Sguil startup scripts 0.6.0 Server, Sensor http://sourceforge.net/projects/instantnsm I've put together a set of prepackaged startup scripts for the various components. These files come with the InsantNSM distribution.

Gathering the Network Data

Before you can consider creating your first sguil sensor, you should already know how that sensor will acquire its data. The sensor needs to be able to sniff the network traffic on the monitored link, a topic which is outside the scope of this document. For more reading on this topic, I recommend The Tao of Network Security Monitoring.[2]

If you are using a network tap to gather your data, you'll notice that you will need to connect two different network cables to the sensor system. Most taps split the output into two streams, corresponding to the transmit and receive leads on the monitored network link. Although the process is not as well documented as it should be, Linux allows you to create bonded network interfaces, or virtual interfaces which correspond to two or more physical interfaces. When monitoring taps, bonded interfaces are extremely useful because they allow you to treat both physical interfaces as a single virtual interface, which greatly simplifies the configuration of the various data acquisition tools. If you are planning to use channel bonding on your sensor, see Appendix 1: Configuring Bonded Interfaces.

Compiling the Software

This section shows how to compile all the required software. Since you should under no circumstances have a C compiler installed on a sensor box, compile these on another RHEL3 box and copy them over to the server and sensor later.

Most packages use the standard GNU configure script. The instructions will place each of these in their own directory under /usr/local/package-version, so once you have compiled them you can simply copy the directories to the appropriate server or sensor system.

In the following sections, I will assume that you have already downloaded the source distributions for each package and have extracted them each to their own directory.

MySQL

Issue the following commands to compile the MySQL server and client.

# cd mysql-5.0.15
# ./configure ýprefix=/usr/local/mysql-5.0.15 \
  --enable-assembler --enable-local-infile \
  --enable-thread-safe-client ýwithout-debug \
  --with-openssl
# make 
# make install

libpcap

Issue the following commands to compile libpcap:

# cd libpcap-0.9.4
# ./configure --prefix=/usr/local/libpcap-0.9.4
# make 
# make install
# ln -s /usr/local/libpcap-0.9.4 /usr/local/libpcap

libnet

Issue the following commands to compile libnet:

# cd Libnet-1.0.2a
# ./configure --prefix=/usr/local/libnet-1.0.2a
# make
# make install
# ln -s /usr/local/libnet-1.0.2a /usr/local/libnet

Snort

In order to compile snort, you must first install the pcre-devel RPM on your system. RHEL comes with this RPM, and it can easily be installed using the up2date command (or your distribution's equivalent), like so:

# up2date pcre-devel
A note on snort patches: In previous releases of snort and sguil, it was necessary to apply patches to the snort source code to integrate the flow and portscan preprocessors. With snort 2.4.x and sguil 0.6.0 and later, this is no longer necessary. Instead of using the flow preprocessor, we'll be using SANCP to capture network session information. Also, sguil 0.6.0 now supports snort's newer sfportscan preprocessor, which is both more accurate and easier to configure than the older method.

Issue the following commands to compile snort:

# export PATH=$PATH:/usr/local/libnet/bin
# cd snort-2.4.3
# ./configure --prefix=/usr/local/snort-2.4.3 \
  --with-libpcap-includes=/usr/local/libpcap/include \
  --with-libpcap-libraries=/usr/local/libpcap/lib \
  --with-libnet-includes=/usr/local/libnet/include \
  --with-libnet-libraries=/usr/local/libnet/lib \
  --enable-flexresp
# make
# make install

SANCP

SANCP deviates from the standard installation method by using a simple makefile instead of the GNU automatic configuration. Before you can compile it, you must edit the file called Makefile and change the CFLAGS= line to look like the following (all on a single line):

CFLAGS= -O3 -s -I/usr/local/libpcap/include -L/usr/local/libpcap/lib

After you have edited the makefile, you can then issue the following commands to apply the accompanying patches and to compile SANCP:

# patch < sancp-1.6.1.fix200511.a.patch
# patch < sancp-1.6.1.fix200511.b.patch
# make
# mkdir -p /usr/local/sancp-1.6.1b/bin
# cp sancp /usr/local/sancp-1.6.1b/bin

Barnyard

Barnyard requires the Tcl development package be installed. RHEL comes with this RPM, and it can easily be installed using the up2date command (or your distribution's equivalent), like so:

# up2date tcl-devel

Before compiling barnyard, you must patch it to include the newest version of the sguil support (the included sguil support is very old, and is incompatible with the sguil 0.6.0). You will need access to the sguil source tree, so make sure you've already downloaded and extracted it to the disk. Issue the following commands to patch and compile barnyard. In this example, $SGUIL refers to the location of the top of the downloaded source tree:

# cd barnyard-0.2.0
# cp $SGUIL/sensor/barnyard_mods/op_sguil.* \
  src/output_plugins
# cp $SGUIL/sensor/barnyard_mods/configure.in .
# cd src/output_plugins
# patch -p0 <  \
  $SGUIL/sensor/barnyard_mods/op_plugbase.c.patch
# cd ../..
# autoconf
# ./configure --prefix=/usr/local/barnyard-0.2.0+sguil \
  --enable-tcl
  --with-tcl=/usr/lib
# make
# make install
# cp -r ./etc /usr/local/barnyard-0.2.0+sguil/etc

Don't remove the tcl-devel RPM yet. You'll need it again when compiling the other Tcl-related packages.

p0f

P0f is another tool that deviates from the standard GNU automatic configuration. Before compiling it, you must edit the mk/Linux file. Find the line that starts with LIBS= and change it something like the following (all on one line):

LIBS = -lpcap -I/usr/local/libpcap/include -L/usr/local/libpcap/lib

You'll also need to make a small change to the p0f.c file. Find the line that says

#include <net/bpf.h>

and change it to read

#include <pcap-bpf.h>

After you have made these changes, issue the following commands to compile p0f:

# make
# mkdir -p /usr/local/p0f-2.0/sbin
# cp p0f /usr/local/p0f-2.0/sbin

Tcpflow

Issue the following commands to compile Tcpflow:

# ./configure --prefix=/usr/local/tcpflow-0.21 \
  --with-pcap=/usr/local/libpcap
# make
# make install

Tcllib

Issue the following commands to install Tcllib:

# ./configure --prefix=/usr/local/tcllib-1.8
# make install

Mysqltcl

Issue the following commands to install Mysqltcl:

# ./configure --prefix=/usr/local/mysqltcl-2.51
# make
# make install

TclTLS

Compiling TclTLS is very similar to compiling the other packages that use the GNU automatic configuration script, except that it uses the OpenSSL library to provide the TLS functionality. This library is included by default on most RHEL systems, but Red Hat links it with Kerberos, and thus the compiler must be instructed where to find the necessary Kerberos include files in addition to everything else. Issue the following commands to compile TclTLS.

# export CC=ýgcc -I/usr/kerberos/includeý
# ./configure --prefix=/usr/local/tcltls-1.5 \
  --with-ssl-dir=/usr
# make
# make install

If you like, you can now remove the tcl-devel RPM, like so:

# rpm -e tcl-devel

Sguil

Sguil itself is just a set of Tcl scripts, so technically you don't need to compile it. You will find it convenient, however, to move the directory you downloaded into /usr/local like all the other software in this section. The distribution already comes as a tarball that you can simply unpack. If you downloaded it to /tmp/sguil-0.6.0.tar.gz, you can use something like:

# cd /usr/local
# tar xfvpz /tmp/sguil-0.6.0.tar.gz

Installing the Software

Once you have compiled the software, installing it on the various systems is straightforward. The only hard part is remembering which packages are installed on which systems. For example, only the server system needs MySQL, but only the sensors need snort or any of the other data gathering tools. The following sections tell you which packages to install on which hosts.

How to copy the software: Remember that we compiled each package to reside in it's own directory (typically named like /usr/local/software-version). This makes it very easy to copy from system to system. The easiest way to copy a package is to use the scp command (secure copy), part of the SSH command suite. To copy a directory from the local machine to a machine called rsystem, use a command like the following:

# scp -rp /usr/local/snort-2.4.3 rsystem:/usr/local

Each of the directories you copy will have a software name and a version number embedded in it. To make it easier to upgrade individual software packages later, also make sure to create a symbolic link like so:

# ln -s /usr/local/software-version /usr/local/software

This is important because the configuration examples in this document are going to refer to software with these version-less pathnames. Then, for example, if you need to upgrade snort to include a security patch, you won't have to change any of your other software or sguil configuration files, you'll just change the /usr/local/snort symlink to point to the newest version.

On your server

Before installing the packages you just compiled, you should install a few RPMs that the sguil server needs. You will need to install the tclx package. This can be conveniently installed in one step as follows:

# up2date tclx

If the MySQL server is already installed via RPM, you should also remove it with the following command:

# rpm -e mysql-server

Now copy over the MySQL, Mysqltcl, Tcllib, TclTLS, Tcpflow, p0f and sguil software directories to the server system (see the notes at the beginning of this section, and don't forget to make the version-less symbolic links).

For convenience (and because we'll use these later), also create links to some of the important individual binaries, like so:

# ln -s /usr/local/tcpflow/bin/tcpflow /usr/local/bin
# ln -s /usr/local/p0f/sbin/p0f /usr/local/bin
# ln -s /usr/local/mysql/bin/mysql /usr/local/bin
# ln -s /usr/local/mysql/bin/mysqladmin /usr/local/bin
# ln -s /usr/local/mysql/bin/mysqlcheck /usr/local/bin
# ln -s /usr/local/mysql/bin/mysqld_safe /usr/local/bin
# ln -s /usr/local/mysql/bin/mysqldump /usr/local/bin
# ln -s /usr/local/mysql/bin/mysql_install_db /usr/local/bin

In addition, you'll need to make a few symbolic links to make the Tcllib and Mysqltcl packages available in a directory where the Tcl interpreter will be able to find them. Use the following two commands to hook these into the standard location:

# ln -s /usr/local/mysqltcl/lib/mysqltcl-2.51 /usr/lib
# ln -s /usr/local/tcllib/lib/tcllib1.8 /usr/lib

On your sensor

The sguil sensor needs to be able to find the tcpdump binary when extracting data from the full packet logs. It also needs the tclx package, which provides various Tcl programming extensions. You can easily install these as follows:

# up2date tcpdump tclx

Now you can install the software you just compiled. Copy over the barnyard, sancp, libnet, libpcap, snort, and sguil software directories to the server system (see the notes at the beginning of this section, and don't forget to make the version-less symbolic links).

For convenience (and because we'll use these later), also create symbolic links to some of the important individual binaries, like so:

# ln -s /usr/local/barnyard/bin/barnyard /usr/local/bin
# ln -s /usr/local/sancp/bin/sancp /usr/local/bin
# ln -s /usr/local/snort/bin/snort /usr/local/bin

On your client system

You don't need much software to support the analyst consoles, but there are a few RPMs to install, as shown below:

# up2date ethereal-gnome tclx tcllib itcl

Now just copy over the sguil directory and add a symbolic link to the client:

# ln -s /usr/local/sguil/client/sguil.tk /usr/local/bin

Configuration

Since most of the components of sguil are really independent programs, special care is required in order to configure them such that they all work properly together. Once you've got the hang of it, it's not difficult, but there are a lot of files to edit, and things can be pretty confusing. In this section, you'll learn how to configure the pieces to work together smoothly.

Configuring the Server

To configure the sguil server, log on to the server system and perform the following procedures as root.

First, you need to add a new local user (called sguil) who will own all the files and processes associated with the sguil server. I recommend using the -u option to specify a particular UID. The exact value isn't important (choose one that's not already in use) but specifying it explicitly is convenient when you want to transfer sguil-owned files to and from your sensor systems. (Note: useradd will also create a sguil group in addition to the user account).

# useradd -u 400 -d /home/sguil -c ýSGUIL Userý sguil

Next, you must populate your local sguil directory structure. Remember, the root of your local storage can be anywhere, but in these instructions I'm going to refer to it as $NSM.

# mkdir -p $NSM/sguild_data/archive
# mkdir $NSM/sguild_data/rules
# chown -R sguil.sguil $NSM/sguild_data

If you've ever had the RPM version of the MySQL server installed on this system, you may already have a mysql user account to go with it. By default, this will use a UID and GID of 27. If this account already exists, you can skip this step, but if not, you'll need to create one. The following command will create a mysql account and group using compatible settings. (Note: useradd will also create a mysql group in addition to the user account).

# useradd -u 27 -d /var/lib/mysql -s /bin/bash \
  -c ýMySQL Serverý mysql

Most of the local storage requirement on the server is for the MySQL database. Using the commands below, we'll relocate the database directory from the default /var/lib/mysql to the $NSM/mysql directory. We'll also add a symbolic link to the old location for compatibility.

# mkdir $NSM/mysql
# chown -R mysql.mysql $NSM/mysql
# chmod 755 $NSM/mysql
# rm -rf /var/lib/mysql
# ln -s $NSM/mysql /var/lib/mysql

Now you should install the MySQL startup script and the configuration file that came in the package you downloaded earlier. In addition to copying the startup script into place, you'll need to use the chkconfig command to set it to start at boot time, but you won't actually start the service yet. That comes a little later.

You should have already unpacked the startup files into their own directory when you unpacked the rest of the downloaded software. Assuming the startup files are in $STARTUPFILES, you can use the following commands to install the default MySQL config file and cause the daemon to run at boot time:

# cp $STARTUPFILES/my.cnf /etc
# cp $STARTUPFILES/mysqld /etc/init.d
# chkconfig --add mysqld
# chkconfig mysqld on

Next, create the initial database structure, like so:

# /usr/local/mysql/bin/mysql_install_db --user=mysql

Now we can finally start MySQL. We follow this with a database ping just to make sure it started properly.

# service mysqld start
# mysqladmin ping

For security, the first order of business now that the server is running is to change the passwords for the MySQL root account (the database administrator).

# mysql -u root mysql
mysql> update user set Password=OLD_PASSWORD("new_password") where User = "root";
mysql> flush privileges;
mysql> exit

MySQL authentication and backwards compatibility: You'll notice here and elsewhere in this document that we are using the MySQL OLD_PASSWORD() macro instead of the newer PASSWORD() macro when setting database passwords. This is to ensure compatibility with legacy MySQL systems, especially the 3.2x versions that ship with many Linux systems. If you know that you are using 4.x or newer versions everywhere, you can safely replace OLD_PASSWORD() with PASSWORD().

With the basic MySQL infrastructure up and running, we can now concentrate on creating the sguil database itself. First, lets create a sguil user in MySQL so the sguil server on this machine can log into the database. You'll need to log in with the MySQL root password. When you issue the GRANT command to create a new database account, you'll also specify the password for that account. Pick something different from the MySQL root password.

# mysql -u root -p mysql

Enter password: password

mysql> GRANT ALL PRIVILEGES ON sguildb.* TO sguil@localhost IDENTIFIED BY "sguil_password" WITH GRANT OPTION;
mysql> GRANT FILE ON *.* to sguil@localhost;
mysql> update user set Password = OLD_PASSWORD(sguil_password) where User = "sguil";
mysql> FLUSH PRIVILEGES;

Now you must create the empty database (called sguildb) to store all the data:

# mysql -u sguil -p -e ýCREATE DATABASE sguildbý
# mysql -u sguil -p -D sguildb < /usr/local/sguil/server/sql_scripts/create_sguildb.sql
# mysql -u sguil -p -D sguildb -e "show tables"

This last show tables command should show you a list of all the tables in your new database, similar to the following. If you don't see something like this, your database has not been created.

# mysql -u sguil -p -D sguildb -e "show tables"
+-------------------+
| Tables_in_sguildb |
+-------------------+
| history           |
| nessus            |
| nessus_data       |
| portscan          |
| sensor            |
| sessions          |
| status            |
| user_info         |
| version           |
+-------------------+

At this point, we're done with MySQL configuration, so we move on to the sguil server. First, we need to create a directory to hold the PID file:

# mkdir /var/run/sguil
# chown sguil.sguil /var/run/sguil

Next we need to create a directory structure to hold the sguil server's configuration files. We'll also copy the defaults from the sguil source directory so we'll have a baseline to customize.

# mkdir -p /etc/sguild/certs
# cp /usr/local/sguil/server/sguild.conf /etc/sguild
# cp /usr/local/sguil/server/autocat.conf /etc/sguild
# cp /usr/local/sguil/server/sguild.users /etc/sguild
# cp /usr/local/sguil/server/sguild.queries /etc/sguild
# cp /usr/local/sguil/server/sguild.access /etc/sguild
# cp /usr/local/sguil/server/sguild.email /etc/sguild
# cp /usr/local/sguil/server/sguild.reports /etc/sguild
# chown -R sguil.sguil /etc/sguild

Edit the /etc/sguild/sguild.conf file you just created to make the following changes. In each case, find the line that sets the indicated variable name (shown in UPPER_CASE) and change it to the indicated value. Remember to replace the $NSM variable with the location of your local storage directory!

set SGUILD_LIB_PATH /usr/local/sguil/server/lib
set DEBUG 0
set SENSOR_AGGREGATION_ON 0
set RULESDIR $NSM/rules
set DBPASS "your_sguil_db_password"
set DBUSER sguil
set LOCAL_LOG_DIR $NSM/archive
set TCPFLOW /usr/local/bin/tcpflow
set P0F 1
set P0F_PATH /usr/local/bin/p0f

Now we need to create an SSL certificate to be used for encrypting communications between the sguil server and the analyst consoles. RHEL automates most process, as shown below:

# cd /usr/share/ssl/certs
# make sguild.pem

This will create a file called /usr/share/ssl/certs/sguild.pem. The PEM file is actually just a text file that contains both the private key and the public certificate. It looks like this:

-----BEGIN RSA PRIVATE KEY-----
MIICWwIBAAKBgQCxFTbTzpeU2z1u7PinG0TjLq6UZUT4TWUfyj5DJogwOXteHQkd
lYvwUZgtUmQmyYBmTMbFGeHSJ/TXnFUUETfG3FHel/zeWer5QqJoezERT2RvZ7CK
[DELETED]
5C7aUaFqAsUCj4rOnQJAK46SuoVYYM8/eZLSP6iOYZK1dj74+8ScM8KYzxS9xeTD
phabxc/FbbuCrcDyxlfFW91RbFVCNFFFe2qpDRB1Nw==
-----END RSA PRIVATE KEY-----

-----BEGIN CERTIFICATE-----
MIID0zCCAzygAwIBAgIBADANBgkqhkiG9w0BAQQFADCBqDELMAkGA1UEBhMCVVMx
ETAPBgNVBAgTCFZpcmdpbmlhMRUwEwYDVQQHEwxXaWxsaWFtc2J1cmcxFTATBgNV
[DELETED]
CSEb3wRxje9WILnkpNOf5ETPW4x/WSFFNfcqzeeuNDFzE2zBv9S2dLeGtHc3JvKL
Q808wzYJvS9FeilVKwyxDIRplELRjrI=
-----END CERTIFICATE-----

Sguil needs these to be in separate files, so edit it with your text editor. Copy everything between the "BEGIN RSA PRIVATE KEY" line and the "END RSA PRIVATE KEY" line (inlcuding the "BEGIN" and "END" lines!) to a file in the sguil daemon configuration directory called /etc/sguild/certs/sguild.key. Similarly, copy everything between the "BEGIN CERTIFICATE" and "END CERTIFICATE" lines (including the "BEGIN" and "END" lines) to a file called /etc/sguild/certs/sguild.pem.

Lets add some sguil accounts for the security analysts. Remember, analysts don't necessarily have to have local OS accounts in order to use sguil. Sguil has it's own account database. Use the command below to create an account named sguil.

# /usr/local/sguil/server/sguild -adduser sguil

Finally, copy over the sguild startup file from the unpacked sample startup file directory ($STARTUPFILES), and configure sguild to run at boot time.

# cp $STARTUPFILES/sguild /etc/init.d
# chkconfig --add sguild
# chkconfig sguild on
# service sguild start

Sguild should now be running. Use a command like ps -eaf | grep sguil to ensure that you see three sguild processes (sguild forks off a couple of helper processes when it starts).

At this point, I recommend that you reboot the server system to double check that the MySQL daemon and the sguil server both start properly.

Configuring a Client

Configuring a sguil client console is pretty easy. The software is already installed on your analyst workstation (you did this in a previous section), so all you need to do is copy the default client configuration file and make a few changes to tailor it to your local environment:

# cp /usr/local/sguil/client/sguil.conf /etc/sguil.conf

Make the following changes. In each case, find the line that sets the indicated variable name (shown in UPPER_CASE) and change it to the indicated value.

set SERVERHOST sguil_server_hostname
set SGUILLIB /usr/local/sguil/client/lib
set DEBUG 0
set OPENSSL 1
set TLS_PATH /usr/local/tcltls/lib/libtls1.50.so
set STATUS_UPDATES 10
set ETHEREAL_STORE_DIR /var/tmp

There are numerous other settings you may want to play around with, but these are the most important ones to get the console working properly. Now you are finally ready to try out your sguil system! Launch the client by running the sguil.tk script.

# /usr/local/bin/sguil.tk

You should see a log in screen with the "Sguild Host" set to your sguil server. Log in with the username and password you created earlier and make sure the "Enable SSL" button is checked. If all goes well, you should see the sguil main screen, but there will be no data because you have not yet created any sensors. But we can take care of that!

Configuring a Sensor

Configuring a sensor is probably the trickiest part of this whole document, but by following these instructions, a little practice can make it a quick and (fairly) painless event.

First, choose a name for your sensor. Each sensor must have a unique name, usually something to remind the analyst which link the sensor monitors. For example, you might call your Internet-facing sensor border if it's at the ýborderý of your network. Maybe it'll be finance if it's monitoring the financial subnet. Whatever you choose, just make sure it's meaningful. Also avoid special characters. In fact, it's best if you restrict it to alphanumeric only, and avoid all punctuation, dashes, underscores, spaces or other characters.

For the rest of this section, we'll refer to the name of the sensor as $SENSORNAME.

Just as with the server, you need to add a new local user (called sguil) who will own all the files and processes associated with the sguil sensor. (Note: useradd will also create a sguil group in addition to the user account).

# useradd -u 400 -d /home/sguil -c ýSGUIL Userý sguil

Next, you must populate your local sguil directory structure. Remember, the root of your local storage can be anywhere, but in these instructions I'm going to refer to it as $NSM. Also create a symlink from /var/log/snort-$SENSORNAME to $NSM/snort-logs/$SENSORNAME, just for convenience

# mkdir -p $NSM/snort-logs/$SENSORNAME/OLD
# mkdir -p $NSM/snort_data/$SENSORNAME/dailylogs
# mkdir -p $NSM/snort_data/$SENSORNAME/sancp/today
# chown -R sguil.sguil $NSM/snort-logs $NSM/snort_data
# ln -s $NSM/snort-logs/$SENSORNAME \
  /var/log/snort-$SENSORNAME

Now we need to create a directory to hold the PID files. If this system has already had a sguil server or sensor configured on it, you can skip this step.

# mkdir /var/run/sguil
# chown sguil.sguil /var/run/sguil

Now create a directory to hold the snort rules for this sensor. Note that this should be owned by root and set to group root, not owned by sguil. This is just a little extra security measure to help keep you from accidentally modifying and/or deleting files in this directory later:

# mkdir /usr/local/snortrules-$SENSORNAME
# chown -R root.root /usr/local/snortrules-$SENSORNAME

Now extract the snort rules you downloaded from snort.org into /usr/local/snortrules-$SENSORNAME and configure them to your taste. This part is outside the scope of this document; just be sure that when you finish, all the configuration files are in a flat directory structure, and not in subdirectories (i.e., you have /usr/local/snortrules-$SENSORNAME/snort.conf, /usr/local/snortrules-$SENSORNAME/sid-msg.map, etc).

Once you've got the ruleset that you want, be sure to copy them to the sguil server system as a directory called $NSM/rules/$SENSORNAME. This is important because the sguil server will need to reference the rules in its local directory in order to show them to you in the analyst console. If the sensor and the server are on the same system, you can shortcut this by just creating a symbolic link:

# ln -s /usr/local/snortrules-$SENSORNAME \
  $NSM/rules/$SENSORNAME

Now comes the tricky part: configuring the different information gathering programs to work together.

Start by creating a directory in which all the various configuration files will live. Since we're using these tools as part of a sguil sensor, we'll keep them all in the same directory, /etc/sguil.

# mkdir /etc/sguil

Now copy the various default configuration files from their original locations into the new directory. Note that we will also rename most of these file to include the $SENSORNAME, so that we can easily have more than one configuration for each sensor on the machine. The sancp.conf file is an exception, since all sensors on the machine can share the same configuration file.

# cp /usr/local/sguil/sensor/sancp/sancp.conf \
  /etc/sguil/sancp.conf
# cp /usr/local/sguil/sensor/sensor_agent.conf \
  /etc/sguil/sensor_agent-$SENSORNAME.conf
# cp /usr/local/barnyard/etc/barnyard.conf \
  /etc/sguil/barnyard-$SENSORNAME.conf

Now edit the /etc/sguil/sensor_agent-$SENSORNAME.conf file to configure it for this sensor. Find the following lines and set them to the indicated values.

set SERVER_HOST ip_addr_of_sguild_server
set DEBUG 0
set HOSTNAME $SENSORNAME
set LOG_DIR $NSM
set S4_KEEP_STATS 0

Next, edit /etc/sguil/barnyard-$SENSORNAME.conf and make the changes indicated below. You're going to set the "hostname" to the sensor's name, delete the default filter, comment out the alert_fast and log_dump output plugins and set the sguil sensor name.

config hostname: $SENSORNAME
config filter:
#output alert_fast
#output log_dump
output sguil: sensor_name $SENSORNAME
For systems with more than one sensor: Barnyard communicates with the local sensor agent on a fixed port. If two sensors are sharing the same hardware, you also need to select a new port number for them to communicate over. If you only have a single sensor, the default is fine, otherwise assign each sensor a unique port number. Starting at 7740 and incrementing by one for each sensor is probably safe.

To change the port this instance of the agent listens on, make the following additional modification to /etc/sguil/sensor_agent-$SENSORNAME.conf:

set BY_PORT $UNIQUE_PORT

You'll also need to tell this instance of barnyard what port to use. Do this by editing the /etc/sguil/barnyard-$SENSORNAME.conf like so:

output sguil: sensor_name $SENSORNAME agent_port $UNIQUE_PORT

Now you should install the startup scripts for the sensor processes, using the ones that came in the package you downloaded earlier. In addition to copying the startup scripts into place, you'll need to use the chkconfig command to set them to start at boot time. Don't start them yet, though. You'll do that later, after you edit the scripts to change some of the defaults.

You should have already unpacked the startup files into their own directory when you unpacked the rest of the downloaded software. Assuming the startup files are in $STARTUPFILES, you can use the following commands to install the startup files and cause them to run at boot time. Notice that the sample script names all end in ý-sensorý. You'll replace this with "-$SENSORNAME" when you copy the files into the system directory.

# cp $STARTUPFILES/barnyard-sensor \ 
  /etc/init.d/barnyard-$SENSORNAME
# cp $STARTUPFILES/sancp-sensor \
  /etc/init.d/sancp-$SENSORNAME
# cp $STARTUPFILES/sguil_agent-sensor \
  /etc/init.d/sguil_agent-$SENSORNAME
# cp $STARTUPFILES/sguil_logger-sensor \
  /etc/init.d/sguil_logger-$SENSORNAME
# cp $STARTUPFILES/snort-sensor \
  /etc/init.d/snort-$SENSORNAME
# chkconfig --add barnyard-$SENSORNAME
# chkconfig --add sancp-$SENSORNAME
# chkconfig --add sguil_agent-$SENSORNAME
# chkconfig --add sguil_logger-$SENSORNAME
# chkconfig --add snort-$SENSORNAME
# chkconfig barnyard-$SENSORNAME on
# chkconfig sancp-$SENSORNAME on
# chkconfig sguil_agent-$SENSORNAME on
# chkconfig sguil_logger-$SENSORNAME on
# chkconfig snort-$SENSORNAME on

In /etc/init.d/barnyard-$SENSORNAME, find the line that says "SENSOR=SENSORNAME" and change it to replace "SENSORNAME" with the name of your sensor.

In /etc/init.d/snort-$SENSORNAME, find the line that says "SENSOR=SENSORNAME" and change it to replace "SENSORNAME" with the name of your sensor. Find the line that says "NSMDIR=NSMDIR" and change the it to reflect the path to your local $NSM/snort_data directory. Also find the line that says "INTERFACE=INTERFACE" and change it to list the name of the interface on which you will be monitoring (e.g., "INTERFACE=eth1").

Make the same edits to /etc/init.d/sancp-$SENSORNAME as you made to /etc/init.d/snort-$SENSORNAME.

In /etc/init.d/sguil_agent-$SENSORNAME, find the line that says "SENSOR=SENSORNAME" and change it to replace "SENSORNAME" with the name of your sensor.

Finally, edit /etc/init.d/sguil_logger-$SENSORNAME to change "SENSOR=SENSORNAME" to replace "SENSORNAME" with the name of your sensor. This component requires a bit of extra configuration. First, each must have it's own copy of the script that collects the full packet data.

# cp /usr/local/sguil/sensor/log_packets.sh \
  /usr/local/sguil/sensor/log_packets-$SENSORNAME.sh

Now edit /usr/local/sguil/sensor/log_packets-$SENSORNAME.sh to make the following changes. Remember to set INTERFACE to the monitor interface (e.g., eth1, bond0, etc).

HOSTNAME=ý$SENSORNAMEý
LOG_DIR=ý$NSMý
INTERFACE=ýmonitor_interfaceý
OPTIONS=ý-u sguil -g sguil -m 122ý

You'll also need to add the following cronjob for root to start/restart the logging process on a regular schedule. For most links, once an hour is good, though if you have a particularly high volume link, you may wish to run the job more frequently.

# Restart the sguil packet logger on a regular basis
0 * * * * /etc/init.d/sguil_logger-$SENSORNAME restart

Now reboot the sensor to make sure everything starts up properly. If it does, you should see the following five processes running (in addition to whatever other sguil components you may have configured on this machine):

# ps -eaf | grep -i sguil
sguil     3482     1  5 13:41 ?        00:09:01 /usr/local/bin/sancp -d /local/
sguil     4241     1  8 14:07 ?        00:11:50 /usr/local/bin/snort -u sguil
sguil     4386     1  0 14:08 pts/0    00:01:02 tclsh/usr/local/sguil/sensor/s
sguil     4780     1  0 14:26 pts/0    00:00:03 /usr/local/bin/barnyard -c /us
sguil     6502     1  8 16:25 pts/0    00:00:00 /usr/local/bin/snort -u sguil

Your sensor is now configured! Log in with your analyst console and you should start seeing alerts. If so, you're done!

More Information

The first place to look for more information on sguil is the project's web page at http://www.sguil.org/. Specifically, take a look at the FAQ and the wiki.

The project also maintains mailing lists, which are invaluable sources of information. You can search the archives or subscribe to the lists from the mailing list page.

Finally, if you need more immediate help, or if you just want to chat with the developers and other sguil users, please join us on the project's IRC channel, #snort-gui on irc.freenode.net.

Appendix A:Configuring Bonded Interfaces

Configuring bonded interfaces in RHEL is straightforward, though it does involve editing a few configuration files. In this example, I will assume that you are trying to bond two physical interfaces (eth1 and eth2) into a single virtual interface (bond0). If you're going to use different physical interfaces, or if you are creating more than one bonded interface, adjust the filenames by changing the numbers at the end (eg, you might meld eth3 and eth4 into a second bonded interface called bond1).

RHEL3 Update 6 Caveat: In Update 6, Red Hat introduced a bug in the iputils RPM (iputils-20020927-11.30.2, which makes it impossible to create a bonded interface without an IP address. The next release of the iputils RPM should correct this, but in the meantime, you can overcome this problem by downgrading to the previous release of this RPM (iputils-20020927-11.30.1).

All RHEL network interfaces are configured via files in the /etc/sysconfig/network-scripts directory, so first become root, then change your directory to that location:

% su -
# cd /etc/sysconfig/network-scripts

Next, using your favorite editor, create the file ifcfg-bond0 with the following contents:

DEVICE=bond0
BOOTPROTO=none
ONBOOT=yes

Each of your physical interfaces should already have an ifcfg-ethx file in that directory, so edit the files that go with the cards you are bonding to look something like the following (note that the DEVICE= line will be different in each file):

DEVICE=eth1
HWADDR=XX:XX:XX:XX:XX:XX
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes

You will probably already have other configuration lines in these files. If so, delete them. They will mostly likely contain networking parameters that don't apply to cards used as packet sniffers. I definitely recommend making sure that you have the HWADDR= line, though, followed by the network card's MAC address (AKA the ethernet address). This associates the logical ethx device with a specific hardware card, which will be important if you later upgrade your kernel or introduce new network interfaces, which can reorder the cards.

Now that the network configuration is done, you also need to ensure that the channel bonding kernel driver is automatically loaded whenever this interface is brought up. To do this, add a line like the following to the /etc/modules.conf directory:

alias bond0 bonding

If you are configuring more than one bonded interface, you'll need one of these lines for each. Remember to change bond0 on each line as appropriate.

Now reboot the system and, if everything went well, you should see an additional network interface called bond0:

sensor:/> ifconfig
bond0     Link encap:Ethernet  HWaddr 00:00:00:00:00:00
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

eth0      Link encap:Ethernet  HWaddr 00:0C:29:C9:0F:B7
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4978 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2848 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4386932 (4.1 Mb)  TX bytes:386103 (377.0 Kb)
          Interrupt:10 Base address:0x1400

More information on configuring bonded interfaces under RHEL can be found in the Red Hat Linux Enterprise 3 Reference Guide.[3]

Appendix B:Monitoring VLAN Traffic

With more and more organizations relying on VLANs to provide logical traffic separation over a shared physical network, it's important to be able to monitor these packets as well. Some of the software we're using understands these natively (Snort), some need configuration tweaks (sguil, SANCP) and some require some special patching (tcpflow). This section will describe extra steps you should take if your sensors are monitoring traffic with 802.1Q VLAN tags.

Should I care about VLAN tags? In a typical VLAN environment, most sensors will still not see VLAN tagged traffic. VLAN tags are most commonly seen on trunk lines between switches, and not on network links to which actual computers are attached. If you are not monitoring a trunk line, the chances are you don't have to do anything special. If you are monitoring a trunk, then you probably need to follow these procedures.

First, make sure SANCP is set to start with the ý--strip-80211ý command line option. Knowledgeable readers will notice that 802.11 has nothing to do with 802.1Q tags, but nevertheless this is the correct option name. You can do this editing your /etc/init.d/sancp-$SENSORNAME file and uncommenting the line that starts with "VLAN=", then restarting that service.

Next, edit your /etc/sguil/sensor_agent-$SENSORNAME file to enable VLAN support in the sguil sensor agent. Simply find the line that says "set VLAN 0" and change it to "set VLAN 1" and restart the sensor agent.

Finally, the tcpflow software needs to be patched to include VLAN tag support. The patch is included in the patches/tcpflow-0.21-vlan-patch file in the InstantNSM distribution. Switch into your existing tcpflow source directory, apply the patch, then continue compiling according to the directions given elsewhere in this document.

# cd tcpflow-0.21
# patch < tcpflow-0.21-vlan-patch

Since tcpflow resides on the sguil server, if any of your sensors monitor VLAN trunks, you need to install this patch. Don't worry, it will still work with your non-VLAN sensors as well.

Once you've performed these procedures, you should be able to monitor VLAN traffic with ease.

References

  1. Bejtlich, Richard, Sguil Installation Guide v0.5.3_05, http://sguil.sourceforge.net/index.php?page=latest_docs
  2. Bejtlich, Richard, The Tao of Network Security Monitoring: Beyond Intrusion Detection, Addison-Wesley Professional, 2004
  3. http://www.redhat.com/docs/manuals/enterprise/RHEL-3-Manual/ref-guide/s1-networkscripts-interfaces.html#S2-NETWORKSCRIPTS-INTERFACES-CHAN