Cisco HSRP monitoring using SNMP

Cisco HSRP MIB is defined in CISCO-HSRP-MIB and CISCO-HSRP-EXT-MIB; for a basic SNMP monitoring the first MIB is more than enough.

The most important table in order to get HSRP status information is cHsrpGrpTable, where we can find as many cHsrpGrpEntry objects as HSRP groups configured in the router. Each cHsrpGrpEntry object represents the HSRP configuration and status for a given HSRP group number on a given interface; it has, so, a double index: SNMP interface ID and HSRP group number.

Here is an example of a snmpwalk over a router:

root@NMS:~# snmpwalk -v 2c -c public 10.0.0.1 .1.3.6.1.4.1.9.9.106.1.2.1.1
iso.3.6.1.4.1.9.9.106.1.2.1.1.2.2.10 = STRING: "cisco"
iso.3.6.1.4.1.9.9.106.1.2.1.1.3.2.10 = Gauge32: 255
iso.3.6.1.4.1.9.9.106.1.2.1.1.4.2.10 = INTEGER: 1
iso.3.6.1.4.1.9.9.106.1.2.1.1.5.2.10 = Gauge32: 0
iso.3.6.1.4.1.9.9.106.1.2.1.1.6.2.10 = INTEGER: 2
iso.3.6.1.4.1.9.9.106.1.2.1.1.7.2.10 = Gauge32: 0
iso.3.6.1.4.1.9.9.106.1.2.1.1.8.2.10 = Gauge32: 0
iso.3.6.1.4.1.9.9.106.1.2.1.1.9.2.10 = Gauge32: 3000
iso.3.6.1.4.1.9.9.106.1.2.1.1.10.2.10 = Gauge32: 10000
iso.3.6.1.4.1.9.9.106.1.2.1.1.11.2.10 = IpAddress: 10.0.0.254
iso.3.6.1.4.1.9.9.106.1.2.1.1.12.2.10 = INTEGER: 1
iso.3.6.1.4.1.9.9.106.1.2.1.1.13.2.10 = IpAddress: 10.0.0.1
iso.3.6.1.4.1.9.9.106.1.2.1.1.14.2.10 = IpAddress: 10.0.0.2
iso.3.6.1.4.1.9.9.106.1.2.1.1.15.2.10 = INTEGER: 6
iso.3.6.1.4.1.9.9.106.1.2.1.1.16.2.10 = Hex-STRING: 00 00 0C 07 AC 0A
iso.3.6.1.4.1.9.9.106.1.2.1.1.17.2.10 = INTEGER: 1

The first highlighted value is the SNMP interface ID: you can get the SNMP ID for a given interface using the show snmp mib ifmib ifindex command:

CiscoRouter#show snmp mib ifmib ifindex FastEthernet 0/1
Interface = GigabitEthernet0/1, Ifindex = 2.

The second highlighted value is the HSRP group, the one you use while configuring HSRP:

interface FastEthernet0/1
 standby 10 ip 10.0.0.254
 standby 10 priority 255
 ...

In order to monitor the HSRP group state you just have to grab the cHsrpGrpStandbyState parameter (OID iso.3.6.1.4.1.9.9.106.1.2.1.1.15.2.10), which can have one of the following values:

1: initial
2: learn
3: listen
4: speak
5: standby
6: active

In my previous example the router was in the active state.

References

Cisco.com: Hot Standby Router Protocol Features and Functionality

Cisco.com: CISCO-HSRP-MIB

Cisco.com: CISCO-HSRP-EXT-MIB

Wikipedia: HSRP

Remember the “vlan dot1q tag native” command: untagged ingress frames are dropped!

Today I got crazy with a pair of switches dropping traffic on a 802.1q trunk. Finally, I realized the real problem was a leak in my brain, which led me to forgot how things work!

The scenario I worked on had two switches, a 3560 and a 2960, with a 802.1q (etherchannel) trunk between them; the 3560 was the gateway for the VLAN 100 while on the 2960 I only had some access ports and the management interface.

3560:

! Port-channel toward 2960, 802.1q trunk carrying VLAN 100
interface Port-channel1
 description 3560-to-2960
 switchport trunk encapsulation dot1q
 switchport trunk native vlan 100
 switchport trunk allowed vlan 100
 switchport mode trunk
 switchport nonegotiate
end
!
! Native VLAN tagging
vlan dot1q tag native
!
! VLAN 100 declaration
vlan 100
!
! Layer3 interface for VLAN 100
interface Vlan100
 description SVI100
 ip address 10.0.100.1 255.255.255.0
end

2960:

! Port-channel toward 3560, 802.1q trunk carrying VLAN 100
interface Port-channel1
 description 2960-to-3560
 switchport trunk native vlan 100
 switchport trunk allowed vlan 200
 switchport mode trunk
 switchport nonegotiate
end
!
! VLAN 100 declaration
vlan 100
!
! Default management interface is shutdown
interface Vlan1
 no ip address
 no ip route-cache
 shutdown
end
!
! Management interface
interface Vlan100
 ip address 10.0.100.2 255.255.255.0
 no ip route-cache
end

A ping from the 3560 toward the 2960 (where I ran a debug ip icmp) showed that ICMP echo requests was coming to the switch, replies were crafted by 2960 but they never arrived to 3560.

When I focused on the native VLANs topic, I found they were aligned on both switches: I thought that frames leaving 2960 toward 3560 were untagged (because of the switchport trunk native vlan 100 command) but on 3560 side they should be accepted thanks to the same command. Here I was wrong! I missed the vlan dot1q tag native full behaviour, which means that every untagged ingress frame is dropped, even if it matches the configured native VLAN.

In order to get this configuration to work properly, I had to ensure that every 2960 egress frame was tagged, but it seems 2960s don’t support native VLAN tagging: here I had not the vlan dot1q tag native global configuration capability, nor the switchport trunk native vlan tag interface command, so I removed the switchport trunk native vlan 100 command and everything worked.

References

Cisco.com: Command Lookup Tool

Cisco Support Community: cat2960 native vlan tagged on trunk discussion

One single route-map for both IPv4 and IPv6 BGP prefixes

I just want to share here a note-to-myself about route-maps and IPv4/IPv6 BGP prefixes matching.

R1 and R2 were BGP speakers, R1 was announcing both IPv4 and IPv6 prefixes to R2 and I wanted to set, let’s say, different weights on those prefixes: weight = 4 to the IPv4 prefix and weight = 6 to the IPv6 prefix. It’s stupid, of course, but it’s just to simplify the real task I had to face.

My goal was to have a single route-map for both IPv6 and IPv4 routes manipulation: a single route-map applied to the two neighbor statements in ipv4 and ipv6 address-family:

So, on R2 I wrote the route-map and applied it to the neighbor statement:

R2:

ip prefix-list IPv4 seq 5 permit 192.168.1.1/32
!
ipv6 prefix-list IPv6 seq 5 permit 2001:DB8:1::/64
!
route-map RouteMapIN permit 10
 match ip address prefix-list IPv4
 set weight 4
!
route-map RouteMapIN permit 20
 match ipv6 address prefix-list IPv6
 set weight 6
!
route-map RouteMapIN permit 100
!
router bgp 65535
 address-family ipv4
  neighbor 192.168.0.1 route-map RouteMapIN in
 exit-address-family
 !
 address-family ipv6
  neighbor 2001:DB8::1 route-map RouteMapIN in
 exit-address-family
!

This is what I expected:

I thought it was fine to match only the IPv4 prefix on sequence number 10, and only the IPv6 prefix on sequence number 20, but that was incorrect:

R2#clear bgp all 65535
R2#show bgp all
For address family: IPv4 Unicast
BGP table version is 6, local router ID is 192.168.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*>i192.168.1.1/32   192.168.0.1              0    100      4 i

For address family: IPv6 Unicast
BGP table version is 6, local router ID is 192.168.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*>i2001:DB8:1::/64  2001:DB8::1              0    100      4 i

Both prefixes had weight = 4!

So I tried to exchange the route-map entries’ position to confirm my doubt:

R2:

no route-map RouteMapIN
!
route-map RouteMapIN permit 10
 match ipv6 address prefix-list IPv6
 set weight 6
!
route-map RouteMapIN permit 20
 match ip address prefix-list IPv4
 set weight 4
!
route-map RouteMapIN permit 100

R2#clear bgp all 65535
R2#show bgp all
For address family: IPv4 Unicast
BGP table version is 8, local router ID is 192.168.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*>i192.168.1.1/32   192.168.0.1              0    100      6 i

For address family: IPv6 Unicast
BGP table version is 8, local router ID is 192.168.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*>i2001:DB8:1::/64  2001:DB8::1              0    100      6 i

As supposed, both prefixes had weight = 6!

Finally, I decided to use more restrictive statements, by matching the specific prefix I wanted to match and explicitly denying the other…

R2:

! Here I deny every IPv4 prefix
ip prefix-list NoIPv4 seq 5 deny 0.0.0.0/0 le 32
!
! Here I deny every IPv6 prefix
ipv6 prefix-list NoIPv6 seq 5 deny ::/0 le 128
!
no route-map RouteMapIN
!
route-map RouteMapIN permit 10
 ! Match the IPv4 prefix...
 match ip address prefix-list IPv4
 ! ... but not IPv6 prefixes.
 match ipv6 address prefix-list NoIPv6
 set weight 4
!
route-map RouteMapIN permit 20
 ! Do not match IPv4 prefixes...
 match ip address prefix-list NoIPv4
 ! ... but match only the wanted IPv6 prefix.
 match ipv6 address prefix-list IPv6
 set weight 6
!
route-map RouteMapIN permit 100

… and I got the right result:

R2#clear bgp all 65535
R2#show bgp all
For address family: IPv4 Unicast
BGP table version is 10, local router ID is 192.168.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*>i192.168.1.1/32   192.168.0.1              0    100      4 i

For address family: IPv6 Unicast
BGP table version is 10, local router ID is 192.168.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*>i2001:DB8:1::/64  2001:DB8::1              0    100      6 i

The IPv4 prefix had weight = 4, and the IPv6 one weight = 6!

What did I miss? Any suggestion is strongly appreciated!

Multiple GRE Tunnels with the same source address

Post off-line. Please see comments for more details.

Hosts Subnets Monitor (HSM): get notified when hosts’ subnets change

In the recent past I have been tasked to block traffic from a LAN segment toward some public websites by blackholing their subnets.

While this approach may be not fully convincing, it is easy to implement and with few impacts on the infrastructure. The real problem is the management overhead it introduces, since websites may change the IP subnet they are run on.

How does it work?

In order to ease this task I wrote a little script which, given a hosts list, resolves IP addresses and gets the most specific subnet they fall on. Public whois databases are used to acquire subnet information: RIPE RIS, TeamCymru IP to ASN Mapping, RIRs databases. Some of them (RIPE RIS and TeamCymru IP-to-ASN) are based on BGP feeds collected around the world, others (RIRs databases) are based on LIRs allocations and assignments; the configuration section of the script allows to set which databases have to be used and how.

Requirements

The script is written in Linux Bash and uses some basic programs and an optional (but really recommended) Perl addition (implemented in the hsm-utils.pl file).

The programs used by the script are dig (to resolve hostnames in IP addresses), whois (the improved Whois client by Marco D’Itri, to get data from whois databases) and sendmail, alias of exim4 (to send email notifications when subnets change).

The hsm-utils.pl script needs the NetAddr::IP::Lite module, which can be installed using the CPAN installer:

LINUX:~#perl -MCPAN -e shell
cpan>install NetAddr::IP::Lite

How to use it

To be used it just needs the “hosts” file in the working directory (/var/local/hsm by default) containing a list of hosts to be monitored:

LINUX:/var/local/hsm#cat hosts
www.facebook.com
www.blogspot.com
www.youtube.com
www.twitter.com

In order to receive email notifications the EMAIL_TO parameter has to be set with a working email address.

The script

Following is an excerpt from the script, I suggest you to read it since it contains some (I think) useful notes and configuration options; in the bottom of this post you may find the link to download it.

##################################################################################
# HOW DOES IT WORK?
# --------------------------------------------------------------------------------
#
# Everything is done in the DATA_DIR directory (default to /var/local/hsm); HSM
# reads a list of hosts to monitor from the 'hosts' file and, for each of them, it
# resolves the IP address and gets its subnet.
#
# In order to identify the subnet, it queries public databases using the whois
# client: it may be configured to use the following sources (see CONFIGURATION):
#
# - RIPE RIS database: http://www.ripe.net/ris/
# - TeamCymru IP to ASN Mapping: http://www.team-cymru.org/Services/ip-to-asn.html
# - general RIRs databases
#
# Subnets are stored in the 'subnets' file; at the end of the execution, if there
# are new subnets HSM notifies them in the output. It also notifies expired
# subnets, that is subnets appeared in the past which seem to be not binded to
# hosts anymore.
#
# Output is written to the 'output' file; you may let HSM to send the output by
# email too.
#
# The script may be scheduled to be run periodically through the crontab file.

##################################################################################
# REQUIREMENTS AND DEPENDENCIES
# --------------------------------------------------------------------------------
#
# dig           Used to resolve hostnames
#
# whois         I used the improved Whois client by Marco D'Itri:
#               http://www.linux.it/~md/software/
#
# sendmail      Used as alias of exim4; optional, only if EMAIL_TO is set
#
# awk, grep,    Some basic utilities
# tail, sed
#
# Highly Recommended:
#
# Perl with NetAddr::IP::Lite module, in order to execute the hsm-utils.pl script.
# Please see CHANGE LOG & KNOWN ISSUES and USEHSMUTILS in the CONFIGURATION
# section for more details about it.
#
# Developed and tested under Debian GNU/Linux 4.0 (Etch).

##################################################################################
# CHANGE LOG & KNOWN ISSUES
# --------------------------------------------------------------------------------
#
# Date          Ver.    Note
# 2010-11-15    0.1     First release
#
# If you configure HSM to not use the hsm-utils.pl script (see CONFIGURATION for
# more details) you have to consider the following issues:
#
# - subnets are stored as they appear on the whois output; that is
# "192.168.0.0/24" is different from "192.168.0.0 - 192.168.0.255". This may lead
# to a wrong behaviour when the same IP address is queried against a whois
# database which returns information in a format different from the previous one.
#
# - when GENERAL whois is used, or when GETLONGESTMATCH = 1, there are no
# guarantees that the more specific subnet is choosen among those returned.

##################################################################################
# CONFIGURATION
# --------------------------------------------------------------------------------

# USEHSMUTILS
# ------------------------------------
# If USEHSMUTILS = 1 then HSM uses the hsm-utils.pl file.
# It is a Perl script which implements some functions
# performing subnets normalization and selection. It is
# needed to solve some issues reported in the CHANGE LOG
# & KNOWN ISSUES section. If you can't run a Perl script
# or you prefer to avoid it you may set USEHSMUTILS to 0.

USEHSMUTILS=1
HSMUTILSPATH=`dirname $0`/hsm-utils.pl

# DATA_DIR
# ------------------------------------
# Where files are stored:
# - the file containing the input hosts list ('hosts')
# - one file for each IP address resolved by hostnames
# - the subnets file ('subnets'), where subnets information are stored
# - the output file ('output')
# - temporary files
# No trailing slash.

DATA_DIR=/var/local/hsm

# USECACHE and CACHE_TIME
# ------------------------------------
# If USECACHE = 1 then IP addresses resolved by hostnames
# are checked against whois databases only if they were
# checked before CACHE_TIME days ago.

USECACHE=1
CACHE_TIME=3

# DNS_QUERIES
# ------------------------------------
# HSM sends this number of DNS queries in order to resolve
# hostnames IP addresses. It may be useful to discover IP
# address of hostnames with round-robin records.

DNS_QUERIES=3

# SUBNET_EXPIRY
# ------------------------------------
# When a subnet is not seen for more than SUBNET_EXPIRY days
# it is removed from the subnets file and a notification
# is written in the output.

SUBNET_EXPIRY=15

# WHOIS_LIST and GETLONGESTMATCH
# ------------------------------------
# WHOIS_LIST contains a list of sources to be used to get
# subnets information from IP addresses.
# The following sources are allowed:
# TEAMCYMRU, RIPERIS, GENERAL.
# Sources are used in the order they appear in the list.
# If GETLONGESTMATCH = 0, as soon as HSM succeds to obtain
# the subnet it stops searching.
# If a source is not working properly it uses the next one.
# If GETLONGESTMATCH = 1 HSM grabs results from all the
# listed sources, then uses the most specific one.

WHOIS_LIST="RIPERIS TEAMCYMRU GENERAL"
GETLONGESTMATCH=1

# EMAIL
# ------------------------------------
# Set EMAIL_TO with your email address if you want the output
# to be sent by email. The other parameters are optional.

EMAIL_TO=
EMAIL_FROM=
EMAIL_SUBJECT=

Download and installation

Here you can download the script – Licensed under the terms of the GNU General Public License.

To run it:

LINUX:/usr/local/bin#tar -xf hsm.tar # extract it
LINUX:/usr/local/bin#nano hsm # do your configuration
LINUX:/usr/local/bin#nano /var/local/hsm/hosts # edit hosts you want to monitor
LINUX:/usr/local/bin#./hsm # run it

You may also schedule it using crontab:

LINUX:/usr/local/bin#cat /etc/crontab
# /etc/crontab: system-wide crontab

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user  command
53 22   * * *   root    /usr/local/bin/hsm
#

Any feedback, comment or suggestion is appreciated!