NetFlow: installation and configuration of NFDUMP and NfSen on Debian
March 29, 2010 11 Comments
After the brief overview about the installation of flow-tools and FlowViewer, in this post I’d like to share my experience about the setup of a basic solution based on another pair of tools: NFDUMP and NfSen. As always on my posts, the starting point is a fresh Debian 5.0 setup.
Components used in the solution
As it was for the previous solution, two main components are involved, one to acquire NetFlow data, the other to analyze them in an easy way.
NFDUMP is a suite of tools composed by many programs; nfcapd is one of them and it is the daemon which listens for incoming NetFlow data. It is launched by NfSen.
NfSen is the web based front end we use to analyze NFDUMP NetFlow data. It is splitted up in two parts: a Perl program which runs in the background and launches nfcapd, and some PHP web pages.
The biggest difference between a NFDUMP/NfSen based solution and a flow-tools/FlowViewer solution is that the former does support NetFlow v9, while the latter may result in a simpler and easier tool to work with. Anyway, there is not one perfect solution for all the needs!
In this post I’ll use NFDUMP 1.6.1 and NfSen 1.3.2, the latest stable releases I can find on SourceForge while I’m writing this blog entry; you can find their web pages in the References section of this post. I’ll build them from source code.
NFDUMP
Let’s add some tools we’ll need to build the package:
apt-get install gcc flex librrd-dev make
We are ready to start! The steps are very simple: download source code and extract the tar:
cd /usr/local/src/ wget http://sourceforge.net/projects/nfdump/files/stable/nfdump-1.6.1/nfdump-1.6.1.tar.gz/download gzip -dc nfdump-1.6.1.tar.gz | tar -xf - cd nfdump-1.6.1
In order to use NFDUMP with NfSen we need to enable nfprofile in the configure step; then we have to make && make install it:
./configure --enable-nfprofile make make install
Now it is ready to be ran.
If we want to use it without NfSen we can stop here and create a simple init.d script to launch NFDUMP at startup; it will acquire NetFlow data and we can use command line tools (such as nfdump) to analyze them. You can find a basic init.d script at the end of this post (“Annex A: stand-alone NFDUMP”).
But that’s not our goal, we want NfSen!
NfSen
As said, NfSen has a PHP front-end so it needs a web server with PHP support in order to be used. It also uses RRD to store data and paint nice graphs and some Perl modules to produce alerts: let’s satisfy its prerequisites:
apt-get install apache2 libapache2-mod-php5 php5-common libmailtools-perl rrdtool librrds-perl
Now it’s time to download and configure the program:
cd /usr/local/src/ wget http://sourceforge.net/projects/nfsen/files/stable/nfsen-1.3.2/nfsen-1.3.2.tar.gz/download gzip -dc nfsen-1.3.2.tar.gz | tar -xf - cd nfsen-1.3.2
Once extracted, we have to copy the config file into /etc and modify it:
cp etc/nfsen-dist.conf /etc/nfsen.conf nano /etc/nfsen.conf
As you can see from the support web page it presents a lot of parameters, but many of them may be left unchanged; in this example I chose to leave as many parameters as possible at their default settings, so we have all the files in a single directory: /data/nfsen. Here the parameters I changed:
$USER = "www-data";
$WWWUSER = "www-data";
$WWWGROUP = "www-data";
%sources = (
'MYROUTER' => { 'port' => '9995', 'col' => '#0000ff', 'type' => 'netflow' },
);
$MAIL_FROM = 'MYEMAIL@MYDOMAIN.COM';
$SMTP_SERVER = 'MY.SMTPSERVER.COM';
For a better setup I suggest you to read the configuration guide on the project’s web site.
Let’s make the destination directory and complete the installation by running the install.pl script:
mkdir -p /data/nfsen ./install.pl /etc/nfsen.conf
We are ready to run the nfsen program and let it executes nfcapd daemon:
cd /data/nfsen/bin ./nfsen start
We can also schedule it to be ran at startup:
ln -s /data/nfsen/bin/nfsen /etc/init.d/nfsen update-rc.d nfsen defaults 20
It’s ready: point your browser at http://YOUR_IP_ADDRESS/nfsen/nfsen.php and enjoy! Of course, this is just a very basic setup, both NFDUMP and NfSen have many options you can set and use to have better performances and to improve stability and scalability. If you want to go deep into their configuration I suggest you to use the links you can find at the end of this post.
P.S.: Don’t forget to configure your routers to export NetFlow data! 
Annex A: stand-alone NFDUMP
As said, NFDUMP may be used in a stand-alone way, without the help of NfSen.
We just have to build a destination directory for NetFlow data…
mkdir -p /var/flows/MYROUTER mkdir -p /var/flows/MYSECONDROUTER
and add an init.d script like the following one:
#! /bin/sh
### BEGIN INIT INFO
# Provides: nfcapd
# Required-Start: $network
# Required-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: netflow capture daemon
# Description: nfcapd is the netflow capture daemon of the nfdump tools.
### END INIT INFO
# Author: Erik Wenzel <erik@debian.org>
# Edited by Pierky for the blog post "NetFlow: installation and configuration of
# NFDUMP and NfSen on Debian" on http://pierky.wordpress.com
# Do NOT "set -e"
# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/bin
DESC="netflow capture daemon"
NAME=nfcapd
DAEMON=/usr/local/bin/$NAME
DATA_BASE_DIR="/var/flows"
PIDFILE=/var/run/$NAME.pid
DAEMON_ARGS="-D -w -S 1 -P $PIDFILE -n MYROUTER,192.168.0.1,$DATA_BASE_DIR/MYROUTER"
SCRIPTNAME=/etc/init.d/nfdump
# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0
# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh
# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions
#
# Function that starts the daemon/service
#
do_start()
{
# Return
# 0 if daemon has been started
# 1 if daemon was already running
# 2 if daemon could not be started
start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
|| return 1
start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
$DAEMON_ARGS \
|| return 2
# Add code here, if necessary, that waits for the process to be ready
# to handle requests from services started subsequently which depend
# on this one. As a last resort, sleep for some time.
}
#
# Function that stops the daemon/service
#
do_stop()
{
# Return
# 0 if daemon has been stopped
# 1 if daemon was already stopped
# 2 if daemon could not be stopped
# other if a failure occurred
start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
RETVAL="$?"
[ "$RETVAL" = 2 ] && return 2
# Wait for children to finish too if this is a daemon that forks
# and if the daemon is only ever run from this initscript.
# If the above conditions are not satisfied then add some other code
# that waits for the process to drop all resources that could be
# needed by services started subsequently. A last resort is to
# sleep for some time.
#
# Disabled second call, because is kills nfsen controlled nfcapd
#start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
#[ "$?" = 2 ] && return 2
# Many daemons don't delete their pidfiles when they exit.
rm -f $PIDFILE
return "$RETVAL"
}
#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
#
# If the daemon can reload its configuration without
# restarting (for example, when it is sent a SIGHUP),
# then implement that here.
#
start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME
return 0
}
case "$1" in
start)
[ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
do_start
case "$?" in
0|1) [ q"$VERBOSE" != qno ] && log_end_msg 0 ;;
2) [ p"$VERBOSE" != pno ] && log_end_msg 1 ;;
esac
;;
stop)
[ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
do_stop
case "$?" in
0|1) [ "r$VERBOSE" != rno ] && log_end_msg 0 ;;
2) [ "s$VERBOSE" != sno ] && log_end_msg 1 ;;
esac
;;
#reload|force-reload)
#
# If do_reload() is not implemented then leave this commented out
# and leave 'force-reload' as an alias for 'restart'.
#
#log_daemon_msg "Reloading $DESC" "$NAME"
#do_reload
#log_end_msg $?
#;;
restart|force-reload)
#
# If the "reload" option is implemented then remove the
# 'force-reload' alias
#
log_daemon_msg "Restarting $DESC" "$NAME"
do_stop
case "$?" in
0|1)
do_start
case "$?" in
0|1) log_end_msg 0 ;;
*) log_end_msg 1 ;; # Failed to start
esac
;;
*)
# Failed to stop
log_end_msg 1
;;
esac
;;
*)
#echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
exit 3
;;
esac
exit 0
I edited this script starting from the one included in the Debian’s package.
Please note the DATA_BASE_DIR and the DAEMON_ARGS variables. The first contains the base destination directory for our NetFlow data. The second represents arguments we want to use when launching nfcapd; you can add as many -n options as you want, one for each NetFlow source you want to collect data from. Remember: man nfcapd is your friend!
Finally:
chmod a+x /etc/init.d/nfdump update-rc.d nfdump defaults 20 /etc/init.d/nfdump start
References
Cisco.com: NetFlow version 9
NFDUMP: http://nfdump.sourceforge.net/
