GNS-3 Lab: 4-byte Autonomous System Number – interactions between 32 and 16 bit speaking BGP routers

Such as IPv4 addresses, 2-byte pool of Autonomous System (AS) Numbers is going to be exhausted soon. By the time I’m writing, it seems Internet will run out of 16-bit AS Number on 26-Sep-2012. To prevent this situation, IANA extended the AS Number field to 32 bits and, on 2007, RIRs started assigning them… or, at least, to offer them!
Of course, in order to let 4-byte ASN to work properly, a little adjustement was needed in the BGP protocol. Enter RFC 4893, BGP Support for Four-octet AS Number Space.

In this post I don’t want to cover the mechanisms of this protocol extension, if you want to understand them I please you to follow some links in the References section; I prefer to show some interactions between OLD 2-byte ASN and NEW 4-byte ASN BGP speakers. I built a little GNS3/Dynamips lab using a couple of 7200s with IOS 12.2(33)SRE as NEW speakers, and a couple of 3640s as OLD peers.

In the topology every router announces any subnet to the others; BGP peering sessions follow the physical topology. Green ASs support 4-byte AS, while gray do not.
For the sake of readibility I’ll use asdot notation in this post, with the exception of some configuration blocks where I’ll use asplain just to show both notation usage.

I also uploaded two packet captures on PacketLife.net: they show BGP UPDATES with and without the NEW_AS_PATH attributes. They are 4-byte_AS_numbers_Full_Support.cap and 4-byte_AS_numbers_Mixed_Scenario.cap.

Partial 4-byte ASN support: A-B routers

Router A is on AS 10.1 / 655361 (asdot / asplain notation):

A#sh run | sec bgp
router bgp 655361
 no synchronization
 bgp log-neighbor-changes
 network 10.0.0.0
 neighbor 172.16.3.2 remote-as 2
 no auto-summary
B#sh run | sec bgp
router bgp 2
 no synchronization
 bgp log-neighbor-changes
 network 20.0.0.0
 neighbor 172.16.3.1 remote-as 23456
 no auto-summary

As you can see, on router B neighbor statement we can’t use the real ASN to peer with A, so we have to use the AS_TRANS (23456). Let’s wait until our sessions come up, then show BGP neighbors and tables:

A#sh ip bgp neighbors 172.16.3.2
BGP neighbor is 172.16.3.2,  remote AS 2, external link
  BGP version 4, remote router ID 20.0.0.1
  BGP state = Established, up for 00:00:27
  Last read 00:00:27, last write 00:00:27, hold time is 180, keepalive interval is 60 seconds
  Neighbor sessions:
    1 active, is not multisession capable
  Neighbor capabilities:
    Route refresh: advertised and received(new)
    Four-octets ASN Capability: advertised
    Address family IPv4 Unicast: advertised and received
    Multisession Capability: advertised
...

Please note the Four-octets ASN Capability: advertised line: A advertised this capability but did not receive it back from B.

A#sh ip bgp | beg Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 10.0.0.0         0.0.0.0                  0         32768 i
*> 20.0.0.0         172.16.3.2               0             0 2 i

As expected, A receives the B‘s route but…

B#sh ip bgp | beg Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 10.0.0.0         172.16.3.1               0             0 23456 i
*> 20.0.0.0         0.0.0.0                  0         32768 i

… B only sees A‘s subnet as from AS_TRANS (23456).

This is because B does not support 4-byte ASN, and router A knows this (capabilities exchange during session setup), so it just sends AS_PATH attributes containing the 16-bit AS_TRANS in place of the real 32-bit AS number.

You can find a similar UPDATE on the first capture I sent on PacketLife.net (4-byte_AS_numbers_Mixed_Scenario.cap), packet number 2.

Full 4-byte ASN support: A-D routers

Let’s setup the BGP session between routers A and D; they both support 4-byte ASN.

A#sh run | sec bgp
router bgp 655361
 no synchronization
 bgp log-neighbor-changes
 network 10.0.0.0
 neighbor 172.16.1.2 remote-as 2621441
 neighbor 172.16.3.2 remote-as 2
 no auto-summary
D#sh run | sec bgp
router bgp 2621441
 no synchronization
 bgp log-neighbor-changes
 network 40.0.0.0
 neighbor 172.16.1.1 remote-as 655361
 no auto-summary

When sessions come up…

A#sh ip bgp neighbors 172.16.1.2
BGP neighbor is 172.16.1.2,  remote AS 40.1, external link
  BGP version 4, remote router ID 40.0.0.1
  BGP state = Established, up for 00:04:04
  Last read 00:00:46, last write 00:00:53, hold time is 180, keepalive interval is 60 seconds
  Neighbor sessions:
    1 active, is multisession capable
  Neighbor capabilities:
    Route refresh: advertised and received(new)
    Four-octets ASN Capability: advertised and received
    Address family IPv4 Unicast: advertised and received
    Multisession Capability: advertised and received
...

Here A both sent and received the 4-byte ASN capability to/by B, because it supports this feature.

A#sh ip bgp | beg Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 10.0.0.0         0.0.0.0                  0         32768 i
*> 20.0.0.0         172.16.3.2               0             0 2 i
*> 40.0.0.0         172.16.1.2               0             0 2621441 i
D#sh ip bgp | beg Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 10.0.0.0         172.16.1.1               0             0 655361 i
*> 20.0.0.0         172.16.1.1                             0 655361 2 i
*> 40.0.0.0         0.0.0.0                  0         32768 i

Both A and D have subnets with the full 32-bit AS number.

You can see the capture on the file 4-byte_AS_numbers_Full_Support.cap on PacketLife.net.

As I already said, let’s switch to asdot notation now, just to improve readibility:

D#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
D(config)#router bgp 40.1
D(config-router)#bgp asnotation dot
D(config-router)#do show ip bgp | beg Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 10.0.0.0         172.16.1.1               0             0 10.1 i
*> 20.0.0.0         172.16.1.1                             0 10.1 2 i
*> 40.0.0.0         0.0.0.0                  0         32768 i
A#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
A(config)#router bgp 10.1
A(config-router)#bgp asnotation dot

Ok, 10.1 is better than 655361, and 40.1 than 2621441!!

Mixed scenario: D-B routers

Let’s have more fun now and bring D-B session up!

D(config-router)#neighbor 172.16.2.1 remote-as 2
B(config-router)#neighbor 172.16.2.2 remote-as 23456

As we already saw, B doesn’t support 4-byte ASN, so we have to use AS_TRANS to peer with D.

D#show ip bgp | beg Network
   Network          Next Hop            Metric LocPrf Weight Path
*  10.0.0.0         172.16.2.1                             0 2 10.1 i
*>                  172.16.1.1               0             0 10.1 i
*> 20.0.0.0         172.16.2.1               0             0 2 i
*                   172.16.1.1                             0 10.1 2 i
*> 40.0.0.0         0.0.0.0                  0         32768 i

The interesting thing we can see now on D is the presence of A‘s subnet (10.0.0.0) advertised by B: even if B does not support 4-octect ASN, D receives it with the real 32-bit path. When A advertises the subnet to B it makes the AS_PATH attribute up using AS_TRANS, but it also adds the transitive attribute NEW_AS_PATH, containing the full 32-bit AS numbers; when B advertises the subnet to D it adds this attribute and so D can rebuild the real 32-bit path.

As before, you can find a similar UPDATE on the capture I sent on PacketLife.net; the file is 4-byte_AS_numbers_Mixed_Scenario.cap, packet number 3.

Another interesting aspect of 4-byte AS lack of support can be seen in the following table:

B#sh ip bgp | beg Network
   Network          Next Hop            Metric LocPrf Weight Path
*  10.0.0.0         172.16.2.2                             0 23456 23456 i
*>                  172.16.3.1               0             0 23456 i
*> 20.0.0.0         0.0.0.0                  0         32768 i
*> 40.0.0.0         172.16.2.2               0             0 23456 i
*                   172.16.3.1                             0 23456 23456 i

Here B has two entries for the 10.0.0.0 subnet: they both seem to be originated from the same AS, but that’s not true: the first comes from D (AS 40.1), while the second (the selected one) comes directly from A.

Similar behaviour is for the 40.0.0.0 subnet. Of course, this is not a really big problem, but may lead to false assumptions and compromise traffic patterns analysis.

16-bit stub router

Just to complete all the possible scenarios, here is C configuration:

B#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
B(config)#router bgp 2
B(config-router)#neighbor 172.16.4.2 remote-as 3
C#sh run | sec bgp
router bgp 3
 no synchronization
 bgp log-neighbor-changes
 network 30.0.0.0
 neighbor 172.16.4.1 remote-as 2
 no auto-summary
C#sh ip bgp | beg Network
   Network          Next Hop            Metric LocPrf Weight Path
*> 10.0.0.0         172.16.4.1                             0 2 23456 i
*> 20.0.0.0         172.16.4.1               0             0 2 i
*> 30.0.0.0         0.0.0.0                  0         32768 i
*> 40.0.0.0         172.16.4.1                             0 2 23456 i

It has no idea of what 4-byte ASN are, so it sees both 10.0.0.0 and 40.0.0.0 subnets as originated from the same AS.

Thanks

I have to say thanks to Marco Rizzi for his kind support on helping me to build this lab! And, of course, I suggest you to visit his blog: Marco Rizzi Blog – networking with passion!!!

Download the lab

To download the lab and configuration files click here.

References

Report on consumption of AS Numbers: http://www.potaroo.net/tools/asns/

BGP Support for Four-octet AS Number Space: RFC 4893

APRICOT 2007: 4-Byte AS Numbers (PDF)

Cisco.com: Migration Guide for Explaining 4-Byte Autonomous System

Cisco.com: Cisco IOS BGP 4-Byte ASN Support

Cisco.com: Explaining 4-Byte Autonomous System (AS) ASPLAIN and ASDOT Notation for Cisco IOS

NetFlow: installation and configuration of NFDUMP and NfSen on Debian

After the brief overview about the installation of flow-tools and FlowViewer, in this post I’d like to share my experience about the setup of a basic solution based on another pair of tools: NFDUMP and NfSen. As always on my posts, the starting point is a fresh Debian 5.0 setup.

Components used in the solution

As it was for the previous solution, two main components are involved, one to acquire NetFlow data, the other to analyze them in an easy way.

NFDUMP is a suite of tools composed by many programs; nfcapd is one of them and it is the daemon which listens for incoming NetFlow data. It is launched by NfSen.

NfSen is the web based front end we use to analyze NFDUMP NetFlow data. It is splitted up in two parts: a Perl program which runs in the background and launches nfcapd, and some PHP web pages.

The biggest difference between a NFDUMP/NfSen based solution and a flow-tools/FlowViewer solution is that the former does support NetFlow v9, while the latter may result in a simpler and easier tool to work with. Anyway, there is not one perfect solution for all the needs!

In this post I’ll use NFDUMP 1.6.1 and NfSen 1.3.2, the latest stable releases I can find on SourceForge while I’m writing this blog entry; you can find their web pages in the References section of this post. I’ll build them from source code.

NFDUMP

Let’s add some tools we’ll need to build the package:

apt-get install gcc flex librrd-dev make

We are ready to start! The steps are very simple: download source code and extract the tar:

cd /usr/local/src/
wget http://sourceforge.net/projects/nfdump/files/stable/nfdump-1.6.1/nfdump-1.6.1.tar.gz/download
gzip -dc nfdump-1.6.1.tar.gz | tar -xf -
cd nfdump-1.6.1

In order to use NFDUMP with NfSen we need to enable nfprofile in the configure step; then we have to make && make install it:

./configure --enable-nfprofile 
make
make install

Now it is ready to be ran.

If we want to use it without NfSen we can stop here and create a simple init.d script to launch NFDUMP at startup; it will acquire NetFlow data and we can use command line tools (such as nfdump) to analyze them. You can find a basic init.d script at the end of this post (“Annex A: stand-alone NFDUMP”).

But that’s not our goal, we want NfSen!

NfSen

As said, NfSen has a PHP front-end so it needs a web server with PHP support in order to be used. It also uses RRD to store data and paint nice graphs and some Perl modules to produce alerts: let’s satisfy its prerequisites:

apt-get install apache2 libapache2-mod-php5 php5-common libmailtools-perl rrdtool librrds-perl

Now it’s time to download and configure the program:

cd /usr/local/src/
wget http://sourceforge.net/projects/nfsen/files/stable/nfsen-1.3.2/nfsen-1.3.2.tar.gz/download
gzip -dc nfsen-1.3.2.tar.gz | tar -xf -
cd nfsen-1.3.2

Once extracted, we have to copy the config file into /etc and modify it:

cp etc/nfsen-dist.conf /etc/nfsen.conf
nano /etc/nfsen.conf

As you can see from the support web page it presents a lot of parameters, but many of them may be left unchanged; in this example I chose to leave as many parameters as possible at their default settings, so we have all the files in a single directory: /data/nfsen. Here the parameters I changed:

$USER    = "www-data";
$WWWUSER  = "www-data";
$WWWGROUP = "www-data";

%sources = (
    'MYROUTER'    => { 'port'    => '9995', 'col' => '#0000ff', 'type' => 'netflow' },
);

$MAIL_FROM   = 'MYEMAIL@MYDOMAIN.COM';
$SMTP_SERVER = 'MY.SMTPSERVER.COM';

For a better setup I suggest you to read the configuration guide on the project’s web site.

Let’s make the destination directory and complete the installation by running the install.pl script:

mkdir -p /data/nfsen
./install.pl /etc/nfsen.conf

We are ready to run the nfsen program and let it executes nfcapd daemon:

cd /data/nfsen/bin
./nfsen start

We can also schedule it to be ran at startup:

ln -s /data/nfsen/bin/nfsen /etc/init.d/nfsen
update-rc.d nfsen defaults 20

It’s ready: point your browser at http://YOUR_IP_ADDRESS/nfsen/nfsen.php and enjoy! Of course, this is just a very basic setup, both NFDUMP and NfSen have many options you can set and use to have better performances and to improve stability and scalability. If you want to go deep into their configuration I suggest you to use the links you can find at the end of this post.

P.S.: Don’t forget to configure your routers to export NetFlow data!

Annex A: stand-alone NFDUMP

As said, NFDUMP may be used in a stand-alone way, without the help of NfSen.

We just have to build a destination directory for NetFlow data…

mkdir -p /var/flows/MYROUTER
mkdir -p /var/flows/MYSECONDROUTER

and add an init.d script like the following one:

#! /bin/sh
### BEGIN INIT INFO
# Provides:          nfcapd
# Required-Start:    $network
# Required-Stop:
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: netflow capture daemon
# Description:       nfcapd is the netflow capture daemon of the nfdump tools.
### END INIT INFO

# Author: Erik Wenzel <erik@debian.org>
# Edited by Pierky for the blog post "NetFlow: installation and configuration of 
# NFDUMP and NfSen on Debian" on http://pierky.wordpress.com

# Do NOT "set -e"

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/bin
DESC="netflow capture daemon"
NAME=nfcapd
DAEMON=/usr/local/bin/$NAME
DATA_BASE_DIR="/var/flows"
PIDFILE=/var/run/$NAME.pid
DAEMON_ARGS="-D -w -S 1 -P $PIDFILE -n MYROUTER,192.168.0.1,$DATA_BASE_DIR/MYROUTER"
SCRIPTNAME=/etc/init.d/nfdump

# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service
#
do_start()
{
        # Return
        #   0 if daemon has been started
        #   1 if daemon was already running
        #   2 if daemon could not be started
        start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
                || return 1
        start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
                $DAEMON_ARGS \
                || return 2
        # Add code here, if necessary, that waits for the process to be ready
        # to handle requests from services started subsequently which depend
        # on this one.  As a last resort, sleep for some time.
}

#
# Function that stops the daemon/service
#
do_stop()
{
        # Return
        #   0 if daemon has been stopped
        #   1 if daemon was already stopped
        #   2 if daemon could not be stopped
        #   other if a failure occurred
        start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
        RETVAL="$?"
        [ "$RETVAL" = 2 ] && return 2
        # Wait for children to finish too if this is a daemon that forks
        # and if the daemon is only ever run from this initscript.
        # If the above conditions are not satisfied then add some other code
        # that waits for the process to drop all resources that could be
        # needed by services started subsequently.  A last resort is to
        # sleep for some time.
        #
        # Disabled second call, because is kills nfsen controlled nfcapd
        #start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
        #[ "$?" = 2 ] && return 2
        # Many daemons don't delete their pidfiles when they exit.
        rm -f $PIDFILE
        return "$RETVAL"
}

#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
        #
        # If the daemon can reload its configuration without
        # restarting (for example, when it is sent a SIGHUP),
        # then implement that here.
        #
        start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME
        return 0
}

case "$1" in
  start)
        [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
        do_start
        case "$?" in
                0|1) [ q"$VERBOSE" != qno ] && log_end_msg 0 ;;
                2)   [ p"$VERBOSE" != pno ] && log_end_msg 1 ;;
        esac
        ;;
  stop)
        [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
        do_stop
        case "$?" in
                0|1) [ "r$VERBOSE" != rno ] && log_end_msg 0 ;;
                2) [ "s$VERBOSE" != sno ] && log_end_msg 1 ;;
        esac
        ;;
  #reload|force-reload)
        #
        # If do_reload() is not implemented then leave this commented out
        # and leave 'force-reload' as an alias for 'restart'.
        #
        #log_daemon_msg "Reloading $DESC" "$NAME"
        #do_reload
        #log_end_msg $?
        #;;
  restart|force-reload)
        #
        # If the "reload" option is implemented then remove the
        # 'force-reload' alias
        #
        log_daemon_msg "Restarting $DESC" "$NAME"
        do_stop
        case "$?" in
          0|1)
                do_start
                case "$?" in
                        0|1) log_end_msg 0 ;;
                        *) log_end_msg 1 ;; # Failed to start
                esac
                ;;
          *)
                # Failed to stop
                log_end_msg 1
                ;;
        esac
        ;;
  *)
        #echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
        echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
        exit 3
        ;;
esac
exit 0

I edited this script starting from the one included in the Debian’s package.

Please note the DATA_BASE_DIR and the DAEMON_ARGS variables. The first contains the base destination directory for our NetFlow data. The second represents arguments we want to use when launching nfcapd; you can add as many -n options as you want, one for each NetFlow source you want to collect data from. Remember: man nfcapd is your friend!

Finally:

chmod a+x /etc/init.d/nfdump
update-rc.d nfdump defaults 20
/etc/init.d/nfdump start

References

Cisco.com: NetFlow version 9

NFDUMP: http://nfdump.sourceforge.net/

NfSen: http://nfsen.sourceforge.net/

NetFlow: how to install and configure flow-tools and FlowViewer on a fresh Debian setup

NetFlow is a very useful tool/protocol to monitor network traffic’s patterns. Many tools have been developed to collect and analyze NetFlow data, here I chose flow-tools and FlowViewer packages, and I would like to show how to get them work on a fresh Debian 5.0 (Lenny) setup.

Components used in the solution

There are two main packages involved in the solution: flow-tools, used to acquire and collect NetFlow data generated by our routers, and FlowViewer, used to process, view and paint them on nice graphs.

The flow-tools package is built up by a lot of components, many of them are transparently used by FlowViewer; here I’ll focus on the flow-capture program, the one which acquires and collect the data.

The FlowViewer package is splitted up on 3 CGIs: FlowViewer, FlowGrapher and FlowTracker. They can be used through a web-server and they let us analyze data collected by flow-capture. The package contains 2 programs also, FlowTracker_Collector and FlowTracker_Grapher, which run periodically and build MRTG-like graphs, storing data in RRD databases.

Acquiring data

First off, we need to acquire NetFlow data generated by our routers; flow-tools is the package we need:

apt-get install flow-tools

Let’s edit the main configuration file, flow-capture.conf, where we tell flow-capture what we want to acquire and how we want to store it:

cd /etc/flow-tools
nano flow-capture.conf

Its configuration file is pretty simple; it’s built up by many lines containing the command line arguments of flow-capture: each line is used to run an instance of the program.

You can execute man flow-capture to view all the arguments it accepts; here I used the following example:

# MYROUTER
-V 5 -E 5G -N 3 -w /var/flows/MYROUTER 0.0.0.0/192.168.0.1/3001

# MYSECONDROUTER
-V 5 -E 5G -N 3 -w /var/flows/MYSECONDROUTER 0.0.0.0/192.168.0.2/3002

So, I acquire data from MYROUTER, which sends NetFlow version 5 data from 192.168.0.1; this flow-capture instance will be listening on port 3001 of every local IP address (0.0.0.0) and it will store data on the /var/flows/MYROUTER directory, with a nesting level of type 3, that is directories like /var/flows/MYROUTER/YYYY/YYYY-MM/YYYY-MM-DD/. It will keep files up to a maximum of 5 GB.

This may be a sample configuration for MYROUTER (Cisco), where 192.168.0.9 is the IP address of our NetFlow box:

ip flow-export source FastEthernet0/0.1
ip flow-export version 5
ip flow-export destination 192.168.0.9 3001

interface FastEthernet0/0
 description LAN facing
 no ip address

interface FastEthernet0/0.1
 encapsulation dot1Q 1 native
 ip address 192.168.0.1 255.255.255.0

interface FastEthernet0/1
 description WAN facing
 ip address 10.0.0.1 255.0.0.0
 ip route-cache flow

So on for MYSECONDROUTER…

We just have to build the destination directories and then run the program:

mkdir -p /var/flows/MYROUTER
mkdir -p /var/flows/MYSECONDROUTER

/etc/init.d/flow-capture start

The tool is now working:

ls -l -R /var/flows/MYROUTER/
/var/flows/MYROUTER/:
totale 4
drwxr-xr-x 3 root root 4096  5 mar 10:11 2010

/var/flows/MYROUTER/2010:
totale 4
drwxr-xr-x 3 root root 4096  5 mar 10:11 2010-03

/var/flows/MYROUTER/2010/2010-03:
totale 4
drwxr-xr-x 2 root root 4096  5 mar 10:11 2010-03-05

/var/flows/MYROUTER/2010/2010-03/2010-03-05:
totale 4
-rw-r--r-- 1 root root 92  5 mar 10:11 ft-v05.2010-03-05.101125+0100

FlowViewer

Once we are collecting data using flow-tools we want to analyze them, so we need to get FlowViewer up and running!

Requirements

As we can see from the FlowViewer web site we have to satisfy some requirements in order to run it:

- a web server with CGI support;

- Perl 5.0 or later;

- FlowTools;

- GD and GD:Graph;

- RRDTool.

Let’s start installing them:

apt-get install apache2

apt-get install libgd-graph-perl

apt-get install rrdtool

As we’ll see later in the FlowViewer configuration, it uses also another utility to resolve IP addresses in host names: dig. We have to install the dnsutils Debian package in order to have it:

apt-get install dnsutils

Installation and configuration

Download and untar the package:

cd /usr/local/src
wget http://ensight.eos.nasa.gov/FlowViewer/FlowViewer_3.3.1.tar
tar -xf FlowViewer_3.3.1.tar

Now that all requiremets are met and FlowViewer is on the disk, let’s start configuring it!

In order to get FlowViewer up and running we have to edit its configuration file and build some directories it needs. The user guide provided by the author is very complete, you can find there any information you need. The web site’s FAQ section is very useful too. Here I’ll just provide a basic configuration and layout.

In this sample configuration I use the Apache’s default web site as starting point, so I have:

- / (the root) on /var/www/

- /cgi-bin/ on /usr/lib/cgi-bin/

Let’s move FlowViewer into the cgi-bin directory…

mv FlowViewer_3.3.1 /usr/lib/cgi-bin/

… and edit the configuration file:

cd /usr/lib/cgi-bin/FlowViewer_3.3.1
nano FlowViewer_Configuration.pm

There are a bit of parameters to change, but remember: the user guide is your friend.

Here is the diff of my file against the original:

diff -y --suppress-common-lines -W 250 FlowViewer_Configuration.pm FlowViewer_Configuration.pm.ORIG
$FlowViewer_server       = "192.168.0.9";                                                             |   $FlowViewer_server       = "www.yourcompany.com";     # (IP address or hostname)
$FlowViewer_service      = "http";                                                                    |   $FlowViewer_service      = "https";           # (http, or https)
$reports_directory       = "/var/www/FlowViewer";                                                     |   $reports_directory       = "/htp/htdocs/FlowViewer_3.3.1";
$reports_short           = "/FlowViewer";                                                             |   $reports_short           = "/FlowViewer_3.3.1";
$graphs_directory        = "/var/www/FlowGrapher";                                                    |   $graphs_directory        = "/htp/htdocs/FlowGrapher_3.3.1";
$graphs_short            = "/FlowGrapher";                                                            |   $graphs_short            = "/FlowGrapher_3.3.1";
$tracker_directory       = "/var/www/FlowTracker";                                                    |   $tracker_directory       = "/htp/htdocs/FlowTracker_3.3.1";
$tracker_short           = "/FlowTracker";                                                            |   $tracker_short           = "/FlowTracker_3.3.1";
$cgi_bin_directory       = "/usr/lib/cgi-bin/FlowViewer_3.3.1";                                       |   $cgi_bin_directory       = "/htp/cgi-bin/FlowViewer_3.3.1";
$work_directory          = "/usr/lib/cgi-bin/FlowViewer_3.3.1/Flow_Working";                          |   $work_directory          = "/htp/cgi-bin/FlowViewer_3.3.1/Flow_Working";
$save_directory          = "/var/www/FlowViewer_Saves";                                               |   $save_directory          = "/htp/htdocs/FlowViewer_Saves";
$names_directory         = "/usr/lib/cgi-bin/FlowViewer_3.3.1";                                       |   $names_directory         = "/htp/cgi-bin/FlowViewer_3.3.1";
$filter_directory        = "/usr/lib/cgi-bin/FlowViewer_3.3.1/FlowTracker_Files/FlowTracker_Filters"; |   $filter_directory        = "/htp/cgi-bin/FlowTracker_Files/FlowTracker_Filters";
$rrdtool_directory       = "/usr/lib/cgi-bin/FlowViewer_3.3.1/FlowTracker_Files/FlowTracker_RRDtool"; |   $rrdtool_directory       = "/htp/cgi-bin/FlowTracker_Files/FlowTracker_RRDtool";
$flow_data_directory     = "/var/flows";                                                              |   $flow_data_directory     = "/htp/flows";
$exporter_directory      = "/var/flows/all_routers";                                                  |   $exporter_directory      = "/htp/flows/all_routers";
$rrdtool_bin_directory   = "/usr/bin";                                                                |   $rrdtool_bin_directory   = "/usr/local/rrdtool-1.2.12/bin";
$trackings_title         = "FlowViewer Saves";                                                        |   $trackings_title         = "Your Company Name";
$user_hyperlink          = "/FlowViewer_Saves";                                                       |   $user_hyperlink          = "http://www.yourcompany.com/";
@devices                 = ("MYROUTER","MYSECONDROUTER");                                             |   @devices                 = ("router_1","router_2","router_3");
$log_directory      = "/usr/lib/cgi-bin/FlowViewer_3.3.1";                                            |   $log_directory      = "/htp/cgi-bin/FlowViewer_3.3.1";

And here is every “directory” or “short” parameter:

cat FlowViewer_Configuration.pm | grep "directory\|short"
$reports_directory       = "/var/www/FlowViewer";
$reports_short           = "/FlowViewer";
$graphs_directory        = "/var/www/FlowGrapher";
$graphs_short            = "/FlowGrapher";
$tracker_directory       = "/var/www/FlowTracker";
$tracker_short           = "/FlowTracker";
$cgi_bin_directory       = "/usr/lib/cgi-bin/FlowViewer_3.3.1";
$cgi_bin_short           = "/cgi-bin/FlowViewer_3.3.1";
$work_directory          = "/usr/lib/cgi-bin/FlowViewer_3.3.1/Flow_Working";
$work_short              = "/cgi-bin/FlowViewer_3.3.1/Flow_Working";
$save_directory          = "/var/www/FlowViewer_Saves";
$save_short              = "/FlowViewer_Saves";
$names_directory         = "/usr/lib/cgi-bin/FlowViewer_3.3.1";
$filter_directory        = "/usr/lib/cgi-bin/FlowViewer_3.3.1/FlowTracker_Files/FlowTracker_Filters";
$rrdtool_directory       = "/usr/lib/cgi-bin/FlowViewer_3.3.1/FlowTracker_Files/FlowTracker_RRDtool";
$flow_data_directory     = "/var/flows";
$exporter_directory      = "/htp/flows/all_routers";
$flow_bin_directory      = "/usr/bin";
$rrdtool_bin_directory   = "/usr/bin";
$log_directory      = "/usr/lib/cgi-bin/FlowViewer_3.3.1";
$log_collector_short= "Y";
$log_grapher_short  = "Y";

Apart from the directories, please note the $FlowViewer_server and $FlowViewer_service parameters, and the @devices array, containing the comma-separated list of routers we already configured in flow-captures.

Now, we have to build the directories used by FlowViewer and, of course, we have to set the needed permissions on them.

Here they are:

for d in `cat FlowViewer_Configuration.pm | grep directory | awk -F \" '{print $2}'`; do mkdir -p $d/; done
for d in `cat FlowViewer_Configuration.pm | grep directory | awk -F \" '{print $2}'`; do chmod -R a=rwx $d/; done

It just remains to copy some images into the proper directories…

cp Generic_Logo.jpg /var/www/FlowViewer/
cp FlowViewer_Save.png /var/www/FlowViewer/
cp FlowViewer.png /var/www/FlowViewer_Saves/

… and we are ready to use our NetFlow solution!! Simply point your browser at http://your_server_IP_address/cgi-bin/FlowViewer_3.3.1/FlowViewer.cgi

Is that all? No, it isn’t!

FlowTracker Grapher and Collector

FlowViewer is not just a CGIs collection, it also includes two programs intended to be ran continously on background: they are FlowTracker_Grapher and FlowTracker_Collector.

Once you defined some trackings in FlowTracker they constantly grab data from flow-capture files, store them in RRD databases and build graphs, like MRTG does with SNMP data. They also send you a warning when traffic goes over the thresholds you defined.

To start them up automatically I edited the flowcap script included within FlowViewer package (/usr/lib/cgi-bin/FlowViewer_3.3.1/flowcap); here it is:

#!/bin/sh
#
# FlowTracker:      Starts all processes concerning FlowTracker
#
# description:  This script starts up the the FlowTracker tools (Collector and Grapher)
#
# processname:  There is not a single process associated with these
#               actions, rather there are multiple processes. This
#               script takes care of all of them.
#
# can be restarted by using the following command:
#
# sudo /etc/init.d/FlowTracker restart

RETVAL=0

start() {
    echo -n $"Starting FlowTracker processes: "

    echo -n "FlowTracker_Collector "

    cd /usr/lib/cgi-bin/FlowViewer_3.3.1
    ./FlowTracker_Collector &> /dev/null &
    RETVAL=$?

    echo -n "FlowTracker_Grapher "

    cd /usr/lib/cgi-bin/FlowViewer_3.3.1
    ./FlowTracker_Grapher &> /dev/null &
    RETVAL=$?

    echo ""
}
stop() {
    echo -n $"Stopping FlowTracker processes: "
    RETVAL=0

    for p in `pidof perl`
    do
        ps $p | grep FlowTracker_Collector > /dev/null
        if [ $? -eq 0 ]; then
            echo -n "FlowTracker_Collector "
            kill $p
        fi

        ps $p | grep FlowTracker_Grapher > /dev/null
        if [ $? -eq 0 ]; then
            echo -n "FlowTracker_Grapher "
            kill $p
        fi
    done

    echo ""
}

# See how we were called.

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        start
        ;;
    *)
        echo $"Usage: $0 {start|stop|restart}"
        ;;
esac
exit $RETVAL

Then I moved it in the /etc/init.d directory as FlowTracker, made it executable and scheduled it to be executed at startup:

mv flowcap /etc/init.d/FlowTracker
chmod a+x /etc/init.d/FlowTracker
update-rc.d FlowTracker defaults 30

Finally:

/etc/init.d/FlowTracker

Housekeeping

Just to clean some files now and then, add an entry in your crontab file pointing to a cleanup script, like the following one:

cd /usr/lib/cgi-bin/FlowViewer_3.3.1/
/usr/lib/cgi-bin/FlowViewer_3.3.1/FlowViewer_CleanFiles &> /usr/lib/cgi-bin/FlowViewer_3.3.1/cleanup.log

Give it execution permissions:

chmod a+x /usr/lib/cgi-bin/FlowViewer_3.3.1/cleanup

Then put it in your crontab:

# m h dom mon dow user  command
5   0    *    *    *     /usr/lib/cgi-bin/FlowViewer_3.3.1/cleanup

Everything is done! Enjoy exploring your network traffic and stay tuned for more NetFlow posts!

References

Cisco.com: Cisco IOS NetFlow

Wikipedia: Netflow

Flow-tools: http://www.splintered.net/sw/flow-tools/

FlowViewer: FlowViewer Web Site and F.A.Q.

Switch.ch: List of NetFlow related software

GNS3 Labs: Source-based Remote Triggered Black Hole (RTBH) Filtering with Unicast Reverse Path Forwarding (uRPF)

This post is part of a series about “ISP Security Tools and Techniques“; in this series I talk about some (I think) useful practices:

1. Remote Triggered Black Holing

2. BGP Customer triggered black holing

3. BGP triggered rate limiting and less-than-best-effort (LBE) with QPPB

4. Source-based RTBH with Unicast Reverse Path Forwarding (uRPF)

Stay tuned!

Today I drew inspiration from a brand new RFC to add a post to this little series: RFC-5635, Remote Triggered Black Hole Filtering with Unicast Reverse Path Forwarding (uRPF).

Especially, I would like to focus on section 4 of this RFC, Source Address RTBH Filtering.

To fully understand this post I would suggest to read my previous post Remote Triggered Black Holing.

What is source-based RTBH? and what are the differences with destination-based RTBH?

Until now, in my previous posts, I always talked about destination-based RTBH. But a source-based RTBH filtering exists too.

This mitigation technique allows an ISP to stop malicious traffic (let’s think to DDOS) on the basis of the source address it comes from. Indeed, destination-based RTBH can just be used to stop traffic on the basis of the attacked hosts addresses, or in the best case, on the couple attacked-hosts/incoming-upstream-provider (you can use different BGP communities to black-hole a prefix only on specific edge routers).

As mentioned in the RFC, if a DDOS software is instructed to attack an host name rather than an IP address, even if you change the IP address resolved by that host name, the attack won’t stop. In this scenario you just have to disrupt the whole service by filtering out the attacked prefix. Source-based RTBH can help you!

How does it work?

Source-based RTBH uses Unicast Reverse Path Forwarding (uRPF) as background mechanism to work.
As RFC says…

uRPF performs a route lookup of the source address of the packet and checks to see if the ingress interface of the packet is a valid egress interface for the packet source address (strict mode) or if any route to the source address of the packet exists (loose mode). If the check fails, the packet is typically dropped.

So, in order to stop incoming traffic from hosts A, B and C toward host Z through router R we just need to add discard routes for A, B and C on router R. And, as seen for destination-based RTBH, we can do it using BGP and communities.

Is that all?

Well, no! Some policy enforcements are due!

As first, in the same way we did for destination-based RTBH, we must use the no-export community to be sure our black-holed prefix doesn’t leave our AS.

Our policy must accept prefixes outside our network (while destination-based RTBH only accepts prefixes within our network) and, as a rule of thumb, we should not accept source-based RTBH prefixes from our customers, but we should just use this tool from management workstations under our control.

Configuration

The starting configuration is the one we left on the post BGP triggered rate limiting and less-than-best-effort (LBE) with QPPB.

In our scenario source-based RTBH is triggered from the Core router, using tagged static routes redistribution in BGP; nothing different from destination-based RTBH. So, let’s define the tags and communities:

300:300 - tag 300 - global source-based black-hole
300:301 - tag 301 - ISP1 source-based black-hole
300:302 - tag 302 - ISP2 source-based black-hole

On the Core router, here used as RTBH trigger, we have to enable BGP redistribution of static routes tagged with the news tags; indeed, we will trigger RTBH by adding tagged static routes:

route-map RTBH permit 50
 match tag 300
 set local-preference 200
 set origin igp
 set community 300:300 no-export
route-map RTBH permit 60
 match tag 301
 set local-preference 200
 set origin igp
 set community 300:301 no-export
route-map RTBH permit 70
 match tag 302
 set local-preference 200
 set origin igp
 set community 300:302 no-export

On edge routers facing upstream providers we have to implement our new communities in the route-map:

Edge1:

ip community-list 5 permit 19661100     ! 300:300
ip community-list 5 permit 19661101     ! 300:301
ip community-list 6 permit 19661102     ! 300:302
!
ip as-path access-list 2 permit ^$
!
ip access-list extended InternalNetworks
 permit ip 192.168.0.0 0.0.255.255 255.255.0.0 0.0.255.255
 deny   ip any any
!
route-map FROM_RR deny 30
 match community 6
!
route-map FROM_RR deny 35
 match ip address InternalNetworks
 match community 5
!
route-map FROM_RR permit 40
 match as-path 2
 match community 5
 set community no-export no-advertise
 set ip next-hop 192.0.2.1

Here, 300:300 and 300:301 communities (community-list 5) are welcome, while 300:302 community is not accepted – it’s only for ISP2 facing router. Moreover, we just accept prefixes external to our network (192.168.0.0/16).

Similar config is on Edge2:

ip community-list 5 permit 19661100     ! 300:300
ip community-list 5 permit 19661102     ! 300:302
ip community-list 6 permit 19661101     ! 300:301
!
ip as-path access-list 2 permit ^$
!
ip access-list extended InternalNetworks
 permit ip 192.168.0.0 0.0.255.255 255.255.0.0 0.0.255.255
 deny   ip any any
!
route-map FROM_RR deny 30
 match community 6
!
route-map FROM_RR deny 35
 match ip address InternalNetworks
 match community 5
!
route-map FROM_RR permit 40
 match as-path 2
 match community 5
 set community no-export no-advertise
 set ip next-hop 192.0.2.1

Finally, we have to enable uRPF on ISP-facing interfaces; we use the loose mode here:

Edge1:

interface Serial1/0
 ip address 172.16.1.1 255.255.255.252
 ip verify unicast source reachable-via any

Edge2:

interface Serial1/0
 ip address 172.16.2.1 255.255.255.252
 ip verify unicast source reachable-via any

Tests

To test the solution we have to add another loopback interface to ISPs routers, in order to have two different source IP addresses for each ISP:

ISP1:

interface Loopback1
 ip address 10.0.1.2 255.255.255.255
!
router bgp 100
 network 10.0.1.2 mask 255.255.255.255

ISP2:

interface Loopback1
 ip address 10.0.2.2 255.255.255.255
!
router bgp 200
 network 10.0.2.2 mask 255.255.255.255

Now, suppose we have an attack from ISP1 Loopback1 toward Cust10 (so, from 10.0.1.2 to 192.168.10.1).

We can stop the attack using destination-based RTBH, but traffic from Loopback0 will be disrupted too:

Core(config)#ip route 192.168.10.1 255.255.255.255 null0 tag 101

ISP1#ping 192.168.10.1 so lo1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.10.1, timeout is 2 seconds:
Packet sent with a source address of 10.0.1.2
.....
Success rate is 0 percent (0/5)


ISP1#ping 192.168.10.1 so lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.10.1, timeout is 2 seconds:
Packet sent with a source address of 10.0.1.1
.....
Success rate is 0 percent (0/5)

As you can see, both source addresses can’t reach our host.

So, let’s try our new solution; as first remove the previous RTBH filter:

Core(config)#no ip route 192.168.10.1 255.255.255.255 null0 tag 101

then trigger the source-based filtering for the attacking IP address:

Core(config)#ip route 10.0.1.2 255.255.255.255 null0 tag 301

Edge1 receives the new BGP announcement and it adds the discard interface route:

Edge1#sh ip route bgp | i 10.0.1.2
B       10.0.1.2 [200/0] via 192.0.2.1, 00:01:03

Edge1#sh ip cef 10.0.1.2
10.0.1.2/32, version 32, epoch 0
0 packets, 0 bytes
  via 192.0.2.1, 0 dependencies, recursive
    next hop 192.0.2.1, Null0 via 192.0.2.1/32
    valid null adjacency

As we can see, packets from ISP1 Loopback1 are dropped at Edge1, while packets from Loopback0 pass:

ISP1#ping 192.168.10.1 so lo1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.10.1, timeout is 2 seconds:
Packet sent with a source address of 10.0.1.2
.....
Success rate is 0 percent (0/5)


ISP1#ping 192.168.10.1 so lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.10.1, timeout is 2 seconds:
Packet sent with a source address of 10.0.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 476/608/724 ms

Download

You can download the updated GNS3 file and configs here. The ZIP file contains multiple config subdirectories, one for each step covered on previous posts, and the new “6. Source-based RTBH filtering”.

References

IETF: RFC-5635, Remote Triggered Black Hole Filtering with Unicast Reverse Path Forwarding (uRPF)

GNS3 Labs: BGP triggered rate limiting and less-than-best-effort (LBE) with QPPB

This post is part of a series about “ISP Security Tools and Techniques“; in this series I talk about some (I think) useful practices:

1. Remote Triggered Black Holing

2. BGP Customer triggered black holing

3. BGP triggered rate limiting and less-than-best-effort (LBE) with QPPB

4. Source-based RTBH with Unicast Reverse Path Forwarding (uRPF)

Stay tuned!

As I already wrote in my previous posts, an ISP can provide their customers some useful tools to mitigate (D)DoS attacks: Remote Triggered Black Holing and its NOC-independent version, Customer triggered black holing are tools that, once identified attacked hosts or networks, let us to stop malicious traffic at the provider’s edge.

Anyway, when we drop traffic toward attacked hosts, we can’t investigate the attack anymore; we would need a tool which allowed us to analyze traffic and, in the meantime, that would avoid wasting network resources. Our (dear) provider could provide it by implementing rate limiting and less-than-best-effort services using QoS Policy Propagation via BGP (QPPB).

Scenario and goals

The scenario I will use in this post is the same I used for previous posts I already mentioned before. We are the AS 300 provider, we have 2 customers and 2 upstream providers connected with BGP sessions. Edge1 and Edge2 are routers toward the upstream providers, Edge3 is the router our customers are connected to.

Startup configurations are the same I left on my last post, “Customer triggered black holing”.

As said, the goal is to provide a tool customers can use to rate-limit traffic toward attacked hosts, in order to let them to investigate attacks. This tool should be used by customers avoiding NOC intervention. Scalability is a must.

The solution and how it works

The solution proposed here is based on QoS Policy Propagation via BGP (QPPB).

Cisco defines it as a feature that “allows you to classify packets by IP precedence based on BGP community lists, BGP autonomous system paths, and access control lists” (see Cisco 10000 Series Router Quality of Service Configuration Guide). And, of course, that is all! There’s not much more to say!

How does it work? A BGP router running QPPB receives a prefix and matches it using a route-map, then it sets that prefix’s IP precedence or QoS-group accordingly; router’s interfaces are configured to match traffic’s source or destination address against the prefix and to classify packets; once classified, packets can be matched against normal QoS class-matches and policies. Take a look at the picture for a diagram.

Be aware, no attributes or other infos are added to BGP UPDATEs; it’s just a local mechanism to mark routes and classify packets.

Implementation

Now that we know how QPPB works we can use it to implement rate-limiting QoS policies and to trigger them via BGP.

As first, we have to define on each edge router a QoS policy with two class-maps: one used to rate-limit traffic, and the other used to mark the traffic as less-than-best-effort (LBE): lets say we’ll use QoS-group 1 to rate-limit traffic at 8Kbps and QoS-group 2 to mark traffic as LBE.

Edge1 and Edge2:

class-map match-all QPPB-QoSGroup-1
 match qos-group 1
class-map match-all QPPB-QoSGroup-2
 match qos-group 2
!
policy-map QPPB
 class QPPB-QoSGroup-1
   police cir 8000
     conform-action transmit
     exceed-action drop
 class QPPB-QoSGroup-2
  set dscp cs1

Then we can apply our policy to the core facing interfaces, for outgoing traffic (traffic from upstream providers toward our core/customers):

Edge1 and Edge2:

interface FastEthernet0/0
 service-policy output QPPB

Now, we have to define BGP communities to map QoS groups: we will use the following mapping:

300:201 - QoS group 1 (rate-limit to 8 Kbps)
300:202 - QoS group 2 (LBE marking)

Once defined, we have to implement communities in BGP:

Edge1 and Edge2:

ip community-list 3 permit 300:201
ip community-list 4 permit 300:202
!
route-map QPPB permit 10
 match community 3
 set ip qos-group 1
route-map QPPB permit 20
 match community 4
 set ip qos-group 2
route-map QPPB permit 1000
!
router bgp 300
 table-map QPPB

As you can see, the route-map used for QPPB can’t be the same used for the inbound BGP UPDATEs. We must add another route-map to the BGP process using the table-map subcommand: as Cisco says in the Command Lookup Tool this command is used “to modify metric and tag values when the IP routing table is updated with BGP learned routes”.

Now we can enable QPPB on upstream ISPs facing interfaces to classify incoming traffic based on its destination IP address:

Edge1 and Edge2:

interface Serial1/0
 bgp-policy destination ip-qos-map

For the sake of completion, I say you can use this command to match source or destination address of a packet, and to use IP precedence or QoS-groups for classification. Here we use destination based matching and QoS-groups.

Now, let our customer routers to trigger the services: as usual, we will use a tagged static route redistributed in BGP. We already configured the route-map and redistribution in previous posts, so we just need to add some entries to it:

Cust10 and Cust20:

route-map RTBH permit 100
 match tag 201
 set community 300:201
!
route-map RTBH permit 110
 match tag 202
 set community 300:202

Tests

To test the solution we just have to add a static route toward the prefix we want to rate-limit (or to mark as LBE) on the customer router:

Cust10(config)#ip route 192.168.10.20 255.255.255.255 fa1/0 tag 201

On the edge router we have the new /32 prefix with the expected community:

Edge1#sh ip bgp 192.168.10.20
BGP routing table entry for 192.168.10.20/32, version 12
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Flag: 0x820
  Advertised to update-groups:
     2
  65310
    192.168.0.2 (metric 66) from 192.168.255.0 (192.168.255.0)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Community: 19661001
      Originator: 192.168.3.2, Cluster list: 192.168.255.0

and we also have a route and a CEF entry tagged with QoS-group 1:

Edge1#sh ip route 192.168.10.20
Routing entry for 192.168.10.20/32
  Known via "bgp 300", distance 200, metric 0
  Tag 65310, qos-group 1, type internal
  Last update from 192.168.0.2 00:00:55 ago
  Routing Descriptor Blocks:
  * 192.168.0.2, from 192.168.255.0, 00:00:55 ago
      Route metric is 0, traffic share count is 1
      AS Hops 1
      Route tag 65310

Edge1#sh ip cef 192.168.10.20
192.168.10.20/32, version 34, epoch 0, cached adjacency 192.168.1.1
0 packets, 0 bytes, qos-group 1
  via 192.168.0.2, 0 dependencies, recursive
    next hop 192.168.1.1, FastEthernet0/0 via 192.168.0.0/30
    valid cached adjacency

Now, we ping 192.168.10.20 from ISP1 (don’t expect echo replies, there is not a host at that address, but we just need traffic going toward it):

ISP1#ping 192.168.10.20 source lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.10.20, timeout is 2 seconds:
Packet sent with a source address of 10.0.1.1
.....
Success rate is 0 percent (0/5)

On Edge1, fa0/0 output policy counters show 5 packets classified as QPPB-QoSGroup-1 class-match:

Edge1#sh policy-map interface fa0/0 output
 FastEthernet0/0

  Service-policy output: QPPB

    Class-map: QPPB-QoSGroup-1 (match-all)
      5 packets, 570 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: qos-group 1
      police:
          cir 8000 bps, bc 1500 bytes
        conformed 5 packets, 570 bytes; actions:
          transmit
        exceeded 0 packets, 0 bytes; actions:
          drop
        conformed 0 bps, exceed 0 bps

[cut]

Let’s try with Cust20 and LBE service:

Cust20(config)#ip route 192.168.20.40 255.255.255.255 fa1/0 tag 202

The prefix is on Edge2, right community, right route tag:

Edge2#sh ip bgp 192.168.20.40
BGP routing table entry for 192.168.20.40/32, version 15
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Flag: 0x820
  Advertised to update-groups:
     2
  65320
    192.168.0.6 (metric 66) from 192.168.255.0 (192.168.255.0)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Community: 19661002
      Originator: 192.168.3.2, Cluster list: 192.168.255.0

Edge2#sh ip route 192.168.20.40
Routing entry for 192.168.20.40/32
  Known via "bgp 300", distance 200, metric 0
  Tag 65320, qos-group 2, type internal
  Last update from 192.168.0.6 00:01:15 ago
  Routing Descriptor Blocks:
  * 192.168.0.6, from 192.168.255.0, 00:01:16 ago
      Route metric is 0, traffic share count is 1
      AS Hops 1
      Route tag 65320

Edge2#sh ip cef 192.168.20.40
192.168.20.40/32, version 32, epoch 0, cached adjacency 192.168.2.1
0 packets, 0 bytes, qos-group 2
  via 192.168.0.6, 0 dependencies, recursive
    next hop 192.168.2.1, FastEthernet0/0 via 192.168.0.4/30
    valid cached adjacency

We try to ping the host from ISP2…

ISP2#ping 192.168.20.40 so lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.20.40, timeout is 2 seconds:
Packet sent with a source address of 10.0.2.1
.....
Success rate is 0 percent (0/5)

… and Edge2 policy counters go up for QPPB-QoSGroup-2:

Edge2#sh policy-map interface fa0/0 output
 FastEthernet0/0

  Service-policy output: QPPB

    Class-map: QPPB-QoSGroup-1 (match-all)
      0 packets, 0 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: qos-group 1
      police:
          cir 8000 bps, bc 1500 bytes
        conformed 0 packets, 0 bytes; actions:
          transmit
        exceeded 0 packets, 0 bytes; actions:
          drop
        conformed 0 bps, exceed 0 bps

    Class-map: QPPB-QoSGroup-2 (match-all)
      5 packets, 570 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: qos-group 2
      QoS Set
        dscp cs1
          Packets marked 5

[cut]

Download

In the RateLimit_LBE.zip file you can find the GNS3 Lab with previous and current configuration files. On the RTBH_Configs directory you will find the “5. Rate-limit and LBE” subdirectory with the configuration discussed in this post.

References

QoS Policy Propagation with BGP (QPPB) on Informit: http://www.informit.com/content/images/9781587201240/appendix/QPPBSection.pdf

Cisco 10000 Series Router Quality of Service Configuration Guide: http://www.cisco.com/en/US/docs/routers/10000/10008/configuration/guides/qos/10qqppb.html