NetFlow is a very useful tool/protocol to monitor network traffic’s patterns. Many tools have been developed to collect and analyze NetFlow data, here I chose flow-tools and FlowViewer packages, and I would like to show how to get them work on a fresh Debian 5.0 (Lenny) setup.
Components used in the solution
There are two main packages involved in the solution: flow-tools, used to acquire and collect NetFlow data generated by our routers, and FlowViewer, used to process, view and paint them on nice graphs.
The flow-tools package is built up by a lot of components, many of them are transparently used by FlowViewer; here I’ll focus on the flow-capture program, the one which acquires and collect the data.
The FlowViewer package is splitted up on 3 CGIs: FlowViewer, FlowGrapher and FlowTracker. They can be used through a web-server and they let us analyze data collected by flow-capture. The package contains 2 programs also, FlowTracker_Collector and FlowTracker_Grapher, which run periodically and build MRTG-like graphs, storing data in RRD databases.
Acquiring data
First off, we need to acquire NetFlow data generated by our routers; flow-tools is the package we need:
apt-get install flow-tools
Let’s edit the main configuration file, flow-capture.conf, where we tell flow-capture what we want to acquire and how we want to store it:
cd /etc/flow-tools
nano flow-capture.conf
Its configuration file is pretty simple; it’s built up by many lines containing the command line arguments of flow-capture: each line is used to run an instance of the program.
You can execute man flow-capture to view all the arguments it accepts; here I used the following example:
# MYROUTER
-V 5 -E 5G -N 3 -w /var/flows/MYROUTER 0.0.0.0/192.168.0.1/3001
# MYSECONDROUTER
-V 5 -E 5G -N 3 -w /var/flows/MYSECONDROUTER 0.0.0.0/192.168.0.2/3002
So, I acquire data from MYROUTER, which sends NetFlow version 5 data from 192.168.0.1; this flow-capture instance will be listening on port 3001 of every local IP address (0.0.0.0) and it will store data on the /var/flows/MYROUTER directory, with a nesting level of type 3, that is directories like /var/flows/MYROUTER/YYYY/YYYY-MM/YYYY-MM-DD/. It will keep files up to a maximum of 5 GB.
This may be a sample configuration for MYROUTER (Cisco), where 192.168.0.9 is the IP address of our NetFlow box:
ip flow-export source FastEthernet0/0.1
ip flow-export version 5
ip flow-export destination 192.168.0.9 3001
interface FastEthernet0/0
description LAN facing
no ip address
interface FastEthernet0/0.1
encapsulation dot1Q 1 native
ip address 192.168.0.1 255.255.255.0
interface FastEthernet0/1
description WAN facing
ip address 10.0.0.1 255.0.0.0
ip route-cache flow
So on for MYSECONDROUTER…
We just have to build the destination directories and then run the program:
mkdir -p /var/flows/MYROUTER
mkdir -p /var/flows/MYSECONDROUTER
/etc/init.d/flow-capture start
The tool is now working:
ls -l -R /var/flows/MYROUTER/
/var/flows/MYROUTER/:
totale 4
drwxr-xr-x 3 root root 4096 5 mar 10:11 2010
/var/flows/MYROUTER/2010:
totale 4
drwxr-xr-x 3 root root 4096 5 mar 10:11 2010-03
/var/flows/MYROUTER/2010/2010-03:
totale 4
drwxr-xr-x 2 root root 4096 5 mar 10:11 2010-03-05
/var/flows/MYROUTER/2010/2010-03/2010-03-05:
totale 4
-rw-r--r-- 1 root root 92 5 mar 10:11 ft-v05.2010-03-05.101125+0100
FlowViewer
Once we are collecting data using flow-tools we want to analyze them, so we need to get FlowViewer up and running!
Requirements
As we can see from the FlowViewer web site we have to satisfy some requirements in order to run it:
- a web server with CGI support;
- Perl 5.0 or later;
- FlowTools;
- GD and GD:Graph;
- RRDTool.
Let’s start installing them:
apt-get install apache2
apt-get install libgd-graph-perl
apt-get install rrdtool
As we’ll see later in the FlowViewer configuration, it uses also another utility to resolve IP addresses in host names: dig. We have to install the dnsutils Debian package in order to have it:
apt-get install dnsutils
Installation and configuration
Download and untar the package:
cd /usr/local/src
wget http://ensight.eos.nasa.gov/FlowViewer/FlowViewer_3.3.1.tar
tar -xf FlowViewer_3.3.1.tar
Now that all requiremets are met and FlowViewer is on the disk, let’s start configuring it!
In order to get FlowViewer up and running we have to edit its configuration file and build some directories it needs. The user guide provided by the author is very complete, you can find there any information you need. The web site’s FAQ section is very useful too. Here I’ll just provide a basic configuration and layout.
In this sample configuration I use the Apache’s default web site as starting point, so I have:
- / (the root) on /var/www/
- /cgi-bin/ on /usr/lib/cgi-bin/
Let’s move FlowViewer into the cgi-bin directory…
mv FlowViewer_3.3.1 /usr/lib/cgi-bin/
… and edit the configuration file:
cd /usr/lib/cgi-bin/FlowViewer_3.3.1
nano FlowViewer_Configuration.pm
There are a bit of parameters to change, but remember: the user guide is your friend.
Here is the diff of my file against the original:
diff -y --suppress-common-lines -W 250 FlowViewer_Configuration.pm FlowViewer_Configuration.pm.ORIG
$FlowViewer_server = "192.168.0.9"; | $FlowViewer_server = "www.yourcompany.com"; # (IP address or hostname)
$FlowViewer_service = "http"; | $FlowViewer_service = "https"; # (http, or https)
$reports_directory = "/var/www/FlowViewer"; | $reports_directory = "/htp/htdocs/FlowViewer_3.3.1";
$reports_short = "/FlowViewer"; | $reports_short = "/FlowViewer_3.3.1";
$graphs_directory = "/var/www/FlowGrapher"; | $graphs_directory = "/htp/htdocs/FlowGrapher_3.3.1";
$graphs_short = "/FlowGrapher"; | $graphs_short = "/FlowGrapher_3.3.1";
$tracker_directory = "/var/www/FlowTracker"; | $tracker_directory = "/htp/htdocs/FlowTracker_3.3.1";
$tracker_short = "/FlowTracker"; | $tracker_short = "/FlowTracker_3.3.1";
$cgi_bin_directory = "/usr/lib/cgi-bin/FlowViewer_3.3.1"; | $cgi_bin_directory = "/htp/cgi-bin/FlowViewer_3.3.1";
$work_directory = "/usr/lib/cgi-bin/FlowViewer_3.3.1/Flow_Working"; | $work_directory = "/htp/cgi-bin/FlowViewer_3.3.1/Flow_Working";
$save_directory = "/var/www/FlowViewer_Saves"; | $save_directory = "/htp/htdocs/FlowViewer_Saves";
$names_directory = "/usr/lib/cgi-bin/FlowViewer_3.3.1"; | $names_directory = "/htp/cgi-bin/FlowViewer_3.3.1";
$filter_directory = "/usr/lib/cgi-bin/FlowViewer_3.3.1/FlowTracker_Files/FlowTracker_Filters"; | $filter_directory = "/htp/cgi-bin/FlowTracker_Files/FlowTracker_Filters";
$rrdtool_directory = "/usr/lib/cgi-bin/FlowViewer_3.3.1/FlowTracker_Files/FlowTracker_RRDtool"; | $rrdtool_directory = "/htp/cgi-bin/FlowTracker_Files/FlowTracker_RRDtool";
$flow_data_directory = "/var/flows"; | $flow_data_directory = "/htp/flows";
$exporter_directory = "/var/flows/all_routers"; | $exporter_directory = "/htp/flows/all_routers";
$rrdtool_bin_directory = "/usr/bin"; | $rrdtool_bin_directory = "/usr/local/rrdtool-1.2.12/bin";
$trackings_title = "FlowViewer Saves"; | $trackings_title = "Your Company Name";
$user_hyperlink = "/FlowViewer_Saves"; | $user_hyperlink = "http://www.yourcompany.com/";
@devices = ("MYROUTER","MYSECONDROUTER"); | @devices = ("router_1","router_2","router_3");
$log_directory = "/usr/lib/cgi-bin/FlowViewer_3.3.1"; | $log_directory = "/htp/cgi-bin/FlowViewer_3.3.1";
And here is every “directory” or “short” parameter:
cat FlowViewer_Configuration.pm | grep "directory\|short"
$reports_directory = "/var/www/FlowViewer";
$reports_short = "/FlowViewer";
$graphs_directory = "/var/www/FlowGrapher";
$graphs_short = "/FlowGrapher";
$tracker_directory = "/var/www/FlowTracker";
$tracker_short = "/FlowTracker";
$cgi_bin_directory = "/usr/lib/cgi-bin/FlowViewer_3.3.1";
$cgi_bin_short = "/cgi-bin/FlowViewer_3.3.1";
$work_directory = "/usr/lib/cgi-bin/FlowViewer_3.3.1/Flow_Working";
$work_short = "/cgi-bin/FlowViewer_3.3.1/Flow_Working";
$save_directory = "/var/www/FlowViewer_Saves";
$save_short = "/FlowViewer_Saves";
$names_directory = "/usr/lib/cgi-bin/FlowViewer_3.3.1";
$filter_directory = "/usr/lib/cgi-bin/FlowViewer_3.3.1/FlowTracker_Files/FlowTracker_Filters";
$rrdtool_directory = "/usr/lib/cgi-bin/FlowViewer_3.3.1/FlowTracker_Files/FlowTracker_RRDtool";
$flow_data_directory = "/var/flows";
$exporter_directory = "/htp/flows/all_routers";
$flow_bin_directory = "/usr/bin";
$rrdtool_bin_directory = "/usr/bin";
$log_directory = "/usr/lib/cgi-bin/FlowViewer_3.3.1";
$log_collector_short= "Y";
$log_grapher_short = "Y";
Apart from the directories, please note the $FlowViewer_server and $FlowViewer_service parameters, and the @devices array, containing the comma-separated list of routers we already configured in flow-captures.
Now, we have to build the directories used by FlowViewer and, of course, we have to set the needed permissions on them.
Here they are:
mkdir -p /var/www/FlowViewer
chmod -R a=rwx /var/www/FlowViewer
...
do for every directory which needs to be created and set the rights permissions
It just remains to copy some images into the proper directories…
cp Generic_Logo.jpg /var/www/FlowViewer/
cp FlowViewer_Save.png /var/www/FlowViewer/
cp FlowViewer.png /var/www/FlowViewer_Saves/
… and we are ready to use our NetFlow solution!! Simply point your browser at http://your_server_IP_address/cgi-bin/FlowViewer_3.3.1/FlowViewer.cgi
Is that all? No, it isn’t! 
FlowTracker Grapher and Collector
FlowViewer is not just a CGIs collection, it also includes two programs intended to be ran continously on background: they are FlowTracker_Grapher and FlowTracker_Collector.
Once you defined some trackings in FlowTracker they constantly grab data from flow-capture files, store them in RRD databases and build graphs, like MRTG does with SNMP data. They also send you a warning when traffic goes over the thresholds you defined.
To start them up automatically I edited the flowcap script included within FlowViewer package (/usr/lib/cgi-bin/FlowViewer_3.3.1/flowcap); here it is:
#!/bin/sh
#
# FlowTracker: Starts all processes concerning FlowTracker
#
# description: This script starts up the the FlowTracker tools (Collector and Grapher)
#
# processname: There is not a single process associated with these
# actions, rather there are multiple processes. This
# script takes care of all of them.
#
# can be restarted by using the following command:
#
# sudo /etc/init.d/FlowTracker restart
RETVAL=0
start() {
echo -n $"Starting FlowTracker processes: "
echo -n "FlowTracker_Collector "
cd /usr/lib/cgi-bin/FlowViewer_3.3.1
./FlowTracker_Collector &> /dev/null &
RETVAL=$?
echo -n "FlowTracker_Grapher "
cd /usr/lib/cgi-bin/FlowViewer_3.3.1
./FlowTracker_Grapher &> /dev/null &
RETVAL=$?
echo ""
}
stop() {
echo -n $"Stopping FlowTracker processes: "
RETVAL=0
for p in `pidof perl`
do
ps $p | grep FlowTracker_Collector > /dev/null
if [ $? -eq 0 ]; then
echo -n "FlowTracker_Collector "
kill $p
fi
ps $p | grep FlowTracker_Grapher > /dev/null
if [ $? -eq 0 ]; then
echo -n "FlowTracker_Grapher "
kill $p
fi
done
echo ""
}
# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
*)
echo $"Usage: $0 {start|stop|restart}"
;;
esac
exit $RETVAL
Then I moved it in the /etc/init.d directory as FlowTracker, made it executable and scheduled it to be executed at startup:
mv flowcap /etc/init.d/FlowTracker
chmod a+x /etc/init.d/FlowTracker
update-rc.d FlowTracker defaults 30
Finally:
/etc/init.d/FlowTracker
Housekeeping
Just to clean some files now and then, add an entry in your crontab file pointing to a cleanup script, like the following one:
cd /usr/lib/cgi-bin/FlowViewer_3.3.1/
/usr/lib/cgi-bin/FlowViewer_3.3.1/FlowViewer_CleanFiles &> /usr/lib/cgi-bin/FlowViewer_3.3.1/cleanup.log
Give it execution permissions:
chmod a+x /usr/lib/cgi-bin/FlowViewer_3.3.1/cleanup
Then put it in your crontab:
# m h dom mon dow user command
5 0 * * * /usr/lib/cgi-bin/FlowViewer_3.3.1/cleanup
Everything is done! Enjoy exploring your network traffic and stay tuned for more NetFlow posts! 
References
Cisco.com: Cisco IOS NetFlow
Wikipedia: Netflow
Flow-tools: http://www.splintered.net/sw/flow-tools/
FlowViewer: FlowViewer Web Site and F.A.Q.
Switch.ch: List of NetFlow related software
