This article provides in-depth analysis of DHCP Option 82 (DHCP Relay Agent) which is one of the +180 DHCP Options available to the DHCP protocol and used by the Bootstrap Protocol (BOOTP) used for allowing  diskless client machines to discover and obtain their IP address. We’ll show you how DHCP Option 82 is used when implementing DHCP Snooping, the structure and content of DHCP Option 82, how and where it’s injected and removed from DHCP messages plus much more. You’ll can also download our DHCP/BOOTP Options Excel file and Wireshark packet captures of DHCP packets with Option 82 used in this article to help further understand all topics covered.

Key Topics:

• The DHCP Options field within a DHCP Packet
• DHCP Option 82 (Agent Relay) Message Format, Structure & Fields
• Detailed Analysis of DHCP Option 82 – SubOption 1 & SubOption2
• Purpose & Usage Examples of DHCP Option 82 (Agent Relay)
• DHCP Snooping & Option 82 (Agent Relay) Considerations. Switches & Trusted – Untrusted Ports
• Summary

It’s highly recommend to read through our DHCP Snooping – DHCP Attack Mitigation article which is a foundation article.

The ‘DHCP Options’ Field within a DHCP Packet

The DHCP Options field is included inside every DHCP packet and is critical for the correct operation of the DHCP/BOOTP protocol.  You’d be surprised to know that there are almost 200 different DHCP Options available and there are more added as new features are introduced in the protocol.

The material used in this article such as wireshark DHCP Options 82 packet captures, are freely available to download from our Article Attachments section.

The diagram below shows the structure of a DHCP packet and highlights the position of the DHCP Options field.

It is important to understand that the above DHCP packet is the data payload within an Ethernet frame using UDP as the transport protocol.

The below screenshot was taken from a packet analyzer and shows an Ethernet frame with the DHCP data payload expanded:

We’ve highlighted sections of the DHCP protocol using the same colours as our previous diagram to help the correlation process. Every field shown in our diagram maps directly to the fields of the captured DHCP packet.

The area marked in green is the section where the DHCP Options field is located. In our captured packet there are a total of 8 DHCP Options used, among them is also Option 82 (Agent Information Option).

DHCP Option 82 (Agent Relay) Message Format, Structure & Fields

The DHCP Option 82, aka Agent Relay Information Option or Agent Information Option, was originally created by RFC 3046 to allow the DHCP relay agent (e.g switch, router, firewall or server) to identify itself and the DHCP client that sent the original DHCP message.

The DHCP Option 82 is inserted and removed by the DHCP Agent Relay (e.g switch) as shown in the diagram below:

While some DHCP servers might not support the Option 82 they are still required to copy the Option 82 value received from the DHCP client and include it in all replies back to the client. We’ll discuss the Option 82 insertion and removal process in the next section.

As we saw earlier, the DHCP Options field is positioned at the end of the DHCP packet and always contains multiple DHCP options. This of course means the DHCP Option field varies in length according to the number of options used:

Let’s now take a closer look into the DHCP Options field at the end of the packet. This can contain multiple options as shown below in our packet analyzer screenshot:

 Each option expands to include its own parameters however we will focus on Option 82 shown below:

Due to space restrictions we are only depicting the first (Message Type), second last (Option 82) and last (End) option.

Remember there are over 200 different DHCP options (Code options) available and multiple used in just a single DHCP packet so it can get very challenging analyzing only one DHCP packet!

Looking at the above diagram we can appreciate that the structure of each DHCP Option varies depending on its purpose and information contained however there is a common set of fields used by all except the last (Option 255 – End):

• Code (light green box). Identifies the DHCP Option type. Examples are Code=82 (DHCP Agent Information), Code=53 (DHCP Message Type: Discover, Offer, Request or Ack), etc.
• Length (green box). This is the DHCP option type length in bytes. For DHCP Option 82, this includes the combined the length of SubOption1 + SubOption2.
• Value (blue box). This contains value or data related to the DHCP Option type. DHCP Option 82 contains two SubOptions, each with its own unique value as shown above.

It’s probably worth mentioning at this point that RFC 3046 states that DHCP Option 82 should always be the last DHCP Option before the END option (Code 255).

The material used in this article such as wireshark DHCP Options 82 packet captures, are freely available to download from our Article Attachments section.

Detailed Analysis of DHCP Option 82 – SubOption 1 & SubOption 2

Before we begin analyzing the two SubOptions we need to understand that DHCP Option 82 is inserted by the Agent Relay (switch) as the client’s DHCP packets traverse it.

In this scenario the switch has DHCP Snooping enabled and the SubOption parameters configured accordingly. In the example below, switch DC-SW1 has DHCP Snooping plus DHCP Options 82 enabled and configured:

As the client’s DHCP Discover packet enters switch DC-SW1 via port Gi0/5 the switch will automatically add the DHCP Option 82 and continue forwarding the packet to the DHCP server.

Below is the breakdown of DHCP Option 82 added inside the DHCP Options field:

The DHCP Option 82 in this example has the following configured:

• SubOption 1 (Agent Circuit ID) = Gi0/5. Used to identify the individual switchport.
• SubOption 2 (Agent Remote ID) = DC-SW1. The Hostname or description of the DHCP Relay Agent

Here is what a DHCP Option 82 packet capture looks like in network protocol analyzer:

The top section highlights the two SubOptions along with their parameters and values which are all in HEX while the lower right section shows these values in ASCII – making them easy to decipher.

Before we complete this section let’s take a closer look at the fields each SubOption consists of:

• SubOption Number. This identifies the first (Agent Circuit ID) or second (Agent Remote ID) SubOption.
• Length. The length of the specific SubOption in bytes.
• Value. The specific SubOption value.

This completed the protocol analysis of DHCP Option 82.  Next up, we’ll take a look at examples where DHCP Option 82 plays a significant role in the operation of the network infrastructure.

Purpose & Usage Examples of DHCP Option 82 (Agent Relay)

Most modern DHCP Servers, e.g Windows Server 2012 & Windows Server 2016, support DHCP Option 82 therefore allowing organizations to create DHCP policies according to the information contained inside the DHCP Option 82 field. For example DHCP Pools or IP address ranges can be reserved and assigned to DHCP clients connecting to specific switches within the network or specific ports on those switches.

Large metropolitan networks, for example ISPs or university campuses make extensive use of the DHCP Option 82 as it provides them with the capability of managing and maintaining DHCP network services from a centralized location without the need of dispersed DHCP servers at each site or campus.

DHCP client requests are directed to the main datacenter with the help of local DHCP relay agents (switches, routers, etc) configured to inject the DHCP Option 82 inside the client’s original DHCP packet. This packet is then forwarded to the DHCP Servers with all the necessary information that will allow them to identify the site, network switch and port to which the client is connected to. DHCP server policies then come into effect and ensure each site is served from the correct DHCP pool and clients are assigned the correct IP address.

The diagram above shows how a client’s DHCP Discover packet is modified by the local DHCP Relay Agent (DC-SW1) to include the DHCP Option 82 message allowing the DHCP server at the Core Network identify the campus, switch and port to which the client sending the request is connected to.

As previously noted, the DHCP server is required to maintain the DHCP Option 82 information when replying to the client. This is also shown in the diagram below:

The DHCP Relay Agent (DC-SW1) will receive the DHCP server’s reply and remove the Option 82 information before forwarding it out to the DHCP client.

Many sources on the internet incorrectly mention that the DHCP Relay Agent Option (Option 82) is automatically inserted by a DHCP Snooping enabled switch. RFC 3046 (Section 2.1 – Agent Operation) specifically notes that this function should be disabled by default.

DHCP Snooping & Option 82 (Relay Agent) Considerations. Switches & Trusted - Untrusted Ports

We already know that with DHCP Snooping enabled and Option 82 configured a Cisco Catalyst or Nexus switch, it will insert the Option 82 field into the client’s DHCP message as shown in the below diagram:

As shown in the example above, the DHCP client’s DHCP Discover packet is received by the switch on interface Gi0/5 which is by default an untrusted port. The switch, which acts as a DHCP relay agent, immediately inserts the DHCP Option 82 in the original DHCP Discover packet, updates the frame as needed (MAC addresses, destination IP, CRC) then sends it out Gi0/1, a trusted port, to the DHCP Server.

As a general rule of thumb, any switch interface expected to receive DHCP packets containing DHCP Option 82 must be configured as a Trusted interface otherwise the DHCP packet will be discarded by the switch:

In the case where there are multiple switches with involved in the path to reach the DHCP server the same rule applies to ensure DHCP packets with Option 82 can traverse each hop:

Interfaces Gi0/1 from SW1 and Gi0/4, Gi0/2 from SW2 will always receive DHCP packets with Option 82 therefore these ports must be configured as trusted ports.

Related Articles

• Complete Guide to DHCP Snooping, Snooping Database & mitigating DHCP Attacks
• Basic & Advanced Catalyst Layer 3 Switch Configuration
• Understanding & Designing VLAN Networks
• Ethernet II Frame Formats
• MAC Address

Summary

This article provided in-depth analysis of the DHCP Options field and more specifically the DHCP Option 82. We examined the DHCP Option 82 message format, structure and fields while also taking a close look at SubOptions 1 & 2 and explaining their usage. Finally we talked about the purpose and real-usage examples of DHCP Option 82 and showed how switchports should be configured on DHCP Snooping enabled switches with DHCP Option 82 configured.

Topics covered include:

• DHCP Starvation Attack, Man-in-the-Middle Attack, DHCP Hijacking & Reconnaissance Attacks
• Rogue DHCP Servers – A Major Security Threat & Source of Network Disruptions
• DHCP Snooping Support for Cisco Catalyst and Nexus Switches. Licensing & Features
• How DHCP Snooping Works – DHCP Snooping Concepts - Trusted, Untrusted Ports/Interfaces
• Traffic Dropped by DHCP Snooping, DHCP Snooping Violations – Syslog Messages
• The IP DHCP Snooping Binding Database – Dynamic ARP Inspection
• DHCP Snooping Option-82 Data Insertion
• Summary

Related Articles

• DHCP Option 82 Message Format, Analysis. DHCP Snooping Option 82 Injection & Removal Method, Trusted – Untrusted Switch Ports
• Basic & Advanced Catalyst Layer 3 Switch Configuration
• Understanding & Designing VLAN Networks
• Ethernet II Frame Formats
• MAC Address

DHCP Starvation Attack, Man-in-the-Middle Attack, DHCP Hijacking & Reconnaissance Attacks

DHCP Starvation attack is a common network attack that targets network DHCP servers. Its primary objective is to flood the organization’s DHCP server with DHCP REQUEST messages using spoofed source MAC addresses. The DHCP server will respond to all requests, not knowing this is a DHCP Starvation attack, and assign available IP addresses until its DHCP pool is depleted.

At this point the attacker has rendered the organization’s DHCP server useless and can now enable his own rogue DHCP server to serve network clients. DHCP Starvation is often accompanied by a Man-in-the-Middle attack as the rogue DHCP server distributes fake IP address parameters, including Gateway & DNS IP address, so that all client traffic passes through the attacker for inspection.

Typical Man-in-the-Middle attack. Client data streams flow through the attacker

Using packet capture and protocol analysis tools the attacker is able to fully reconstruct any data stream captured and export files from it. In fact the process so simple it only requires a basic level of understanding of these type of network tools.

In other cases the Man-in-the-Middle attack can be used as a reconnaissance attack with the objective to obtain information about the network infrastructure, services but also identify hosts of high interest such as financial or database servers.

It should be by now evident how a simple attack can become a major security threat for any organization. The above attacks are examples on how easy hackers can infiltrate the network and get access to valuable information by simply connecting an unauthorized/untrusted device to an available network port effectively bypassing firewalls and other levels of security.

Rogue DHCP Servers – A Major Security Threat & Source of Network Disruptions

Rogue DHCP servers are a common problem within enterprise organizations and are not always directly related with an attack. Rogue DHCP Servers tend to appear out of nowhere thanks to users who connect consumer-grade network devices to the network infrastructure unaware that they have connected an unauthorized device with a rogue DHCP server enabled.

The Rogue DHCP server then begins assigning IP addresses to hosts within the network therefore causing network connectivity problems and in many cases – major service disruptions. In a best case scenario DHCP clients are served with an invalid IP address disconnecting them from the rest of the network. Worst case scenario would be the clients been assigned an IP address used by network infrastructure devices e.g the VLAN interface on the Core switch or a firewall interface, causing serious network disruptions and conflicts.

While many organizations enforce security policies that do not allow 3^rd party or unauthorized devices to be connected to their network, there are still incidents where users who do not understand (or care about) the security implications continue to connect these devices to the network infrastructure without consulting their IT Department.

Educating users and enforcing security policies can be extremely challenging which is why security mechanisms need to be in place to help mitigate these incidents and is where DHCP Snooping comes into the picture.

DHCP Snooping Support for Cisco Catalyst and Nexus Switches. Licensing & Features

DHCP Snooping is available on both the Cisco Catalyst and Cisco Nexus platform switches. Both platforms are classified as enterprise-grade switches and fully support all DHCP Snooping functions.

DHCP Snooping is considered a standard security feature and does not require any additional licensing for the older Catalyst IOS, newer Catalyst IOS XE and Nexus NS-OS operating systems, therefore the feature is available and readily configurable on all switches.

Examples of Cisco Catalyst switches that support DHCP Snooping are: Cisco Catalyst 2960S, 2960-X, 3560, 3750, 3750-X, 3850, 4500, 6500, 9300, 9400 and 9500 series.

Examples of Cisco Nexus switches that support DHCP Snooping are: Nexus 2000, 3000, 5000, 7000 and 9000 series.

DHCP Snooping can be enabled globally and on a per-VLAN basis. This means you can enable it for all VLANs (globally) or only for specific including VLAN ranges e.g VLANs 1-20 & VLANs 45-50.

How DHCP Snooping Works – DHCP Snooping Concepts – Trusted, Untrusted Ports/Interfaces

DHCP Snooping is a Layer 2 security switch feature which blocks unauthorized (rogue) DHCP servers from distributing IP addresses to DHCP clients. In fact Cisco was the first vendor to implement DHCP Snooping as a security feature in its network switches and other vendors have since then followed with similar features.

It is important to note that DHCP SNOOPING is an access layer protection service – it does not belong in the core network.

The way DHCP Snooping works is fairly straight forward. DHCP Snooping categorizes all switchports into two simple categories:

• Trusted Ports
• Untrusted Ports

A Trusted Port, also known as a Trusted Source or Trusted Interface, is a port or source whose DHCP server messages are trusted because it is under the organization’s administrative control. For example, the port to which your organization’s DHCP server connects to is considered a Trusted Port. This is also shown in the diagram below:

 

An Untrusted Port, also known as an Untrusted Source or Untrusted Interface, is a port from which DHCP server messages are not trusted. An example on an untrusted port is one where hosts or PCs connect to from which DHCP OFFER, DHCP ACK or DHCPNAK messages should never be seen as these are sent only by DHCP Servers.

Traffic Dropped by DHCP Snooping, DHCP Snooping Violations - Syslog Messages

When enabling DHCP Snooping the switch will begin to drop specific type of DHCP traffic in order to protect the network from rogue DHCP servers. Here is a list of the type of traffic DHCP Snooping will drop:

• DHCP Snooping will drop DHCP messages DHCPACK, DHCPNAK, DHCPOFFER originating from a DHCP server that is not trusted – that is, connected to an untrusted port.
• DHCP Snooping will drop DHCP messages that release or decline an offer if these messages are not originating from the port where the original DHCP conversation was held. This stops attackers from trying to terminate or decline a DHCP offer on behalf of the actual DHCP client.
• A  DHCP relay agent forwards a DHCP packet that includes a relay-agent IP address that is not 0.0.0.0, or the relay agent forwards a packet that includes Option 82 information to an untrusted port. For an in-depth analysis please refer to our DHCP Option 82 article.
• DHCP Snooping will drop DHCP messages where the Source MAC address and client MAC address are not identical (see DHCP_SNOOPING-5-DHCP_SNOOPING_MATCH_MAC_FAIL below).

When DHCP Snooping detects a violation the DHCP packet(s) triggering the event is dropped and a message is logged in the switch’s log. The message can contain one of the following entries:

• %DHCP_SNOOPING-5-DHCP_SNOOPING_UNTRUSTED_PORT: DHCP Snooping has detected DHCP server messages from an untrusted port. This is a serious violation and usually points to a rogue DHCP server operating on an untrusted port.

• %DHCP_SNOOPING-5-DHCP_SNOOPING_MATCH_MAC_FAIL: DHCP Snooping has detected the Source MAC address of the Ethernet frame and Client MAC address in the DHCP message are not identical (see image below).

 

The IP DHCP Snooping Binding Database – Dynamic ARP Inspection

When DHCP Snooping is enabled it will begin to build a dynamic database containing an entry for each untrusted host with a leased IP address if the host is associated with a VLAN that has DHCP Snooping enabled. No entries are created for hosts connected to trusted interfaces.

Each entry in the binding database contains the following information:

• MAC address of the untrusted host
• Leased IP address of the untrusted host
• Lease time
• Binding type
• VLAN number & interface the untrusted host is associated with

As untrusted hosts are assigned IP addresses from the trusted DHCP server the switch will automatically create new entries, update and cleanup the DHCP Snooping Binding Database.

For example, when an IP address lease expires or the switch receives a DHCPRELEASE message from the untrusted host, it will remove the specific entry from the database. On the other hand an entry will be created in the database if the switch sees a DHCPACK message from the trusted DHCP server acknowledging the assignment of an IP address to an untrusted host.

The show ip dhcp snooping binding command displays all entries inside the DHCP Snooping Binding Database:

Cat3560-Firewall.cx# show ip dhcp snooping binding

MacAddress         IpAddress      Lease(sec)   Type           VLAN   Interface
------------       ------------   ----------   -------------  ----   -----------------
D0:76:58:0C:BB:80  192.168.4.50   85228        dhcp-snooping   4     GigabitEthernet0/5

Total number of bindings: 1

The DHCP Snooping Binding Database is also used by other Layer2/3 security features such as Dynamic ARP Inspection which help protect the network against ARP Poisoning & ARP Spoofing attacks.

IP DHCP Snooping configuration for Cisco Catalyst and Cisco Nexus switching platforms will be covered extensively in an upcoming technical article.

Dynamic ARP Inspection, ARP Poisoning, ARP Spoofing attacks will be covered in an upcoming security article.

DHCP Snooping Option 82 – Relay Agent Information

The DHCP Option 82, aka Relay Agent Information Option, was originally created by RFC 3046 to allow the DHCP relay agent (e.g switch or router) to identify itself and the DHCP client that sent the DHCP messages. DHCP Option 82 is used in large metropolitan Ethernet-access deployments where DHCP is required to centrally manage the IP addresses for a large number of subscribers.

When DHCP Snooping is enabled on a Cisco Catalyst or Nexus switch, it will insert the Option 82 field field into the client’s DHCP message:

 

DHCP Option 82 is not often used within organizations but it does provide an additional layer of protection if the DHCP server supports it.  For example the DHCP Server on Windows Server 2012 or 2016 supports Option 82 allowing administrators to create DHCP Policies that control the assignment of IP addresses to specific switches within the network.

Analyzing the structure of DHCP Option 82 is out of this article’s scope but will be covered in great depth in an upcoming article.

Read our article "DHCP Option 82 Message Format, Analysis. DHCP Snooping Option 82 Injection & Removal Method, Trusted – Untrusted Switch Ports" for in-depth analysis of DHCP Option 82.

Summary

Man-in-the-Middle attacks and network disruptions from rogue DHCP servers is a serious network security threat organizations are faced to deal with on a daily basis. In this article we explained how Man-in-the-Middle attacks allow attackers to gain visibility of your network and can potentially lead exposing sensitive data flowing between servers and clients. We explained what DHCP Snooping is, examined how DHCP Snooping works and how it can effectively protect the network from these attacks. We looked at the type of traffic dropped by DHCP snooping, violation warnings and also explained the purpose and operation of the DHCP Snooping Binding Database. Finally we touched on the DHCP Snooping Option 82.

The Nexus 7010 is one of the larger data center switches in the Nexus portfolio found in most enterprise-class data centers. Even though the Nexus 7000 series switches have been in the market since 2008 there are still a lot of data centers powering their core infrastructure using the well-known Cisco Catalyst series.

The Nexus 7000 series switches are designed for continuous operation, which means all parts are hot-swappable thereby eliminating downtime for upgrades or parts replacement.

The process covered in this installation guide can be used with all Nexus 7000 series modules including:

• 48-port 10/100/1000 Ethernet module (N7K-M148GT-11)
• 48-port 10/100/1000 Ethernet module with XL option (N7K-M148GT-11L)
• 48-port 1-Gigabit Ethernet I/O module (N7K-M148GS-11)
• 48-port 1-Gigabit Ethernet I/O module with XL option (N7K-M148GS-11L)
• 48-port 1-/10-Gigabit Ethernet I/O modules with XL (N7K-F248XP-25 and N7K-F248XP-25E)
• 32-port 10-Gigabit Ethernet I/O module (N7K-M132XP-12)
• 32-port 10-Gigabit Ethernet I/O module with XL option (N7K-M132XP-12L)
• 32-port 1- and 10-Gigabit Ethernet I/O module (N7K-F132XP-15)
• 8-port 10-Gigabit Ethernet I/O module with XL option (N7K-M108X2-12L)

Step 1. Nexus 7000 Module Shutdown - Poweroff

The Nexus 7000 series modules are hot swappable and support automatic shutdown when ejected, however, it is always advisable to poweroff the module before removing it. If the module is to be removed or swapped with a different module type it is advisable to also ensure all configuration associated with the old module’s ports is cleared and ports are shutdown before the module is removed.

Locate the slot number of the module to be uninstalled and remove all attached cables. It is very important no cables are attached to the module and there is enough space on both sides of the module. In our example we’ll be removing the module located in slot No.9:

Click on the images to enlarge

Figure 1. Nexus 7010 with module No.9 to be removed.

Issuing the show module 9 command will reveal the module’s model, status, capabilities, serial number and diagnostic status:

The output of the show module is also reflected on the module’s status LED. A green Status LED, as shown in the photo on the left, tells us that the module is currently online (powered on) and operating.

The orange interface LEDs confirm that the interfaces are in a shutdown state.

The specific card we are about to remove is a 48-port 10/100/1000 Ethernet card (N7K-M148GT-11L):

Figure 2. Nexus 7000 Module Status and Interface LEDs

Now proceed to power off the module using the poweroff module 9 command:

Once the poweroff command has been executed the Nexus will begin powering off the module and removing all associated interfaces from the configuration:

Notice the final message confirms that module 9 is now powered down.

When looking at the module notice the change in its LEDs. The Status LED will be now flashing red, while all interface LEDs are off:

Figure 3. Nexus 7000 module powered down. Status LED flashing red

Using the show module 9 command we can verify the module’s powered-down status:

Step 2. Removing A Nexus 7000 Module

With the module powered down and the Status LED flashing red we are now able to remove the module from its slot.

Note: Ensure you’re using an antistatic wrist band that has been appropriately grounded.

It’s now time to leave the keyboard, grab a screw driver and get our hands dirty:)

First unscrew the two captive screws located at each end of the module as shown below. A Philips or straight head screw driver will be adequate for the job:

Figure 4. Unscrewing Nexus 7000 module captive screws

Next, using your fingers from both hands press simultaneously the ejector buttons located at each end of the module, right next to the captive screws, as shown below:

Figure 5. Pressing the ejector buttons on a Nexus 7000 module

When pressing the ejector buttons (No.1 below), two things will occur:

• The ejector levers will swing into the OPEN position (No.2 below)
• The Nexus 7000 switch will generate a message notifying that the ejectors are in the OPEN position

 Figure 6. Nexus 7000 Module ejector lever in OPEN position after pressing the ejector button

Following is the message generated by the Nexus 7000 operating system after pressing the ejector buttons:

2016 Oct 15 11:48:39 FCX_NEXUS_7010 %PLATFORM-3-EJECTOR_STAT_CHANGED: Ejectors' status in slot 9 has changed, Top Ejector is OPEN, Bottom Ejector is OPEN

Now simultaneously swing open both ejector levers to unseat the module (No. 3 in the photo below) and with a hand on each ejector, pull the module part way out of its slot in the chassis (No.4 in the photo below):

Figure 7. Nexus 7000 module – Swinging the ejector levers and pulling the module out

As soon as the module disconnects from the Nexus backplane the system will produce a notification message:

2016 Oct 15 11:50:07 FCX_NEXUS_7010 %PLATFORM-2-MOD_REMOVE: Module 9 removed (Serial number JAF1327BFHA)

When the module is almost out grasp the front edge with one hand and place your other hand under the module to support its weight without touching the module’s circuitry:

Figure 8. Proper handling during the removal of the Nexus 7000 module

Once the module has been removed from the slot place it inside an antistatic bag.

If using a disposable anti-static wrist strap take extra caution as the fans inside the Nexus 7000 series chassis create strong airflow capable of pulling in lightweight items as shown in the photo below.

Figure 9. The Nexus 7000 airflow is strong enough to pull light items inside the empty slot

Cisco advises not to leave the slot open for a long as it can disrupt the system airflow causing the system to overheat and shut down. This, however, should not be a problem in a properly cooled data center.

Conclusion

Removing a module from a Nexus switch is a very delicate process. Special care must be taken to ensure the slot’s configuration is cleared, the module is powered down and safely removed using an anti-static wrist strap. Future articles will cover the installation of modules on Nexus 7000 data center switches.

The key to a successful STP deployment is understanding how each STP feature should be used and implemented.

Understanding & Configuring BPDU Guard

BPDU Guard is an STP enhancement which, when enabled, will place a port in the errdisable mode when it receives any BPDU packet from that port.

BPDU Guard is usually configured on access layer ports where we are not expecting to see any BPDU packets arriving from devices connected to these ports e.g computers, printers, IP phones or other user-end devices.

Ports used as uplinks or downlinks to other switches should not have BPDU Guard enabled as these are more likely to have BPDU packets transmitted and received as switches actively monitor for network loops.

BPDU Guard can be configured either in Global mode or Interface mode.

When configured in Global mode the feature is enabled globally for all switch ports configured with port-fast configuration. Port-Fast is an STP feature configured at each individual port that forces the port to go directly into a forwarding state rather than through the normal STP states (Listening, Learning, Forwarding).

While port-fast is a very handy feature that forces a network port to transition immediately to the forwarding state (similar to an unmanaged switch), it must be used with caution as STP won’t be able to immediately detect a network loop through a Port-Fast enabled port.

To configure BPDU Guard in Global mode use the spanning-tree portfast bpduguard default command in Global Configuration Mode:

SW2(config)# spanning-tree portfast bpduguard default

To configure BPDU Guard in Interface mode use the spanning-tree bpduguard enable command under the interface:

SW2(config-if)# spanning-tree bpduguard enable

Note: It is important to keep in mind that if the interface is configured as an access port, with port-fast enabled, and receives a BPDU packet it will automatically be disabled and placed in an errdisabled state.

To help illustrate how BPDU Guard works, we’ve configured port G1/0/1 on our 3750-X as an access link with port-fast and BPDU Guard enabled:

Figure 1. Spanning Tree BPDU Guard configuration and example

Next, we connect another switch (rogue switch) running spanning tree protocol to port G1/0/1 on SW2. As soon as a BPDU packet is received on G1/0/1, here’s how SW2 reacted:

As expected, the port was automatically disabled by the spanning tree protocol.

Note: If interface G1/0/1 was configured as a Trunk Port and received a BPDU packet, it would not be disabled.

Recovery From Disabled Port Due To BPDU Guard

There are many ways to recover from a port that has been placed in the errdisabled state and the most typical way used by engineers today is to issue the shutdown command followed by the no shutdown command directly under the interface’s configuration.

Automatic recovery of the errdisabled port can also be configured on the Cisco Catalyst switch. This handy feature will prevent the port from being permanently disabled, however, it means that if the unwanted condition of the port receiving a BPDU packet is continuously met, the port will flap between the up and errdisabled state.

For this reason, it is advised to use the automatic errdisable recovery feature only when absolutely necessary e.g for a remote switch that is difficult to visit.

The above global configuration will recover an errdisabled port, due to bpduguard, automatically every 30 seconds.

Summary

A number of enhancements have been introduced into the Spanning Tree Protocol to make it flexible for today’s increasingly complex and demanding networks. Using enhanced STP features like BPDUGuard helps protect the network from rogue switches and avoid unwanted STP topology changes, however, it is important to understand the correct usage of BPDUGuard to ensure it’s correctly configured and placed in the STP topology.

The password recovery mechanism is enabled by default which means anyone with physical access to the switch is able to initiate the process and gain access to the switch or stack’s configuration. In some environments this might be a major security concern which is why Cisco provides the option to disable the password recovery mechanism.

In cases where the mechanism is disabled the only option available to gain access to the switch is to delete its startup configuration.

How To Disable or Enable The Password Recovery Service On Cisco Catalyst Switches

Disabling the password recovery mechanism is achieved by using the no service password-recovery command in global configuration mode as shown below:

3750-X-Stack1 (config) # no service password-recovery

Note: When applying the no service password-recovery command on the stack master, the command is propagated to all stack members, making it impossible to perform a password recovery on any switch part of a stack.

When trying to initiate the password recovery process on a switch or stack that has the mechanism disabled, the user will receive the following message:

The password-recovery mechanism has been triggered, but is currently disabled. Access to the boot loader prompt through the password-recovery mechanism is disallowed at this point. However, if you agree to let the system be reset back to the default system configuration, access to the boot loader prompt can still be allowed.

Would you like to reset the system back to the default configuration (y/n)?

Answering “y” at the prompt will wipe the current startup configuration from the switch.

To enable the password recovery mechanism, simply enter service password-recovery in global configuration mode:

3750-X-Stack1 (config) # service password-recovery

Once all configuration changes are complete, don’t forget to save the configuration.

Summary

This article explained the usage of the Cisco password recovery mechanism on Cisco Catalyst switches. We showed how network engineers and administrators can disable the recovery mechanism to increase their security and stop unauthorized people from gaining access to their configuration files and even user account credentials. More technical articles on Cisco Catalyst switches can be found in our Cisco Catalyst Switches section.

Once the switch has loaded its operating system we can enter privileged-exec mode, rename back the flash:config.text.old to flash:config.text (startup-config), copy the startup-config file to memory (DRAM), make the necessary password changes and save the configuration.

Password Recovery – Reset Procedure

The procedure described below assumes the password recovery mechanism is enabled (by default, it is) and there is physical access to the switch or stack (3750-X only).

Note: If this procedure is being performed on a 3750-X stack, it is important to understand that all switches participating in the stack should be powered off and only the Master switch is powered on when initiating the password recovery procedure. The Master switch can be easily identified by searching for the switch with the green “Master” LED on.

Step 1

On a 3750-X switch, Power off the entire stack or standalone switch. On a Catalyst 3560-X switch, power off the switch. Connect your console cable to the switch – 3750-X Master or the standalone switch.

Step 2

Reconnect the power to the switch (standalone 3750-X or 3750-X) or stack master (3750-X stack only). Within 10 seconds, press and hold the Mode button while the System LED is flashing green. After the System LED turns amber and then solid green, release the Mode button.

Step 3

Now initialize the flash file system, rename the startup configuration file (config.text) and boot the IOS:

Now search for the startup configuration file (config.text) and rename it:

We can now boot the switch IOS:

Step 4

At this point, the switch has booted bypassing its configuration file. At the prompt, type enable to enter privileged exec mode and rename back the config.text.old file:

3750-X Note: At this point, power on any 3750-X stack members and wait until they are loaded. This is a very important step to ensure no configuration is lost.

Step 5

Finally, load the startup configuration of the master or standalone switch to memory and make the necessary changes to the enable secret / password or user account in question:

If you require to change the password to an account e.g admin, use the following command:

Step 6

Depending on the switch model and configuration, it is possible that after executing the password recovery procedure VLAN interfaces might be in a shutdown state. Issue the show running-config command and search for any shutdown command under the vlan interfaces. If found, enter the interface and issue the no shutdown command to ensure the interface is enabled.

When done, save your configuration and reload the switch or stack:

Summary

This article showed in detailed steps the password recovery process for Cisco Catalyst 3560-X and 3750-X switches including standalone or stacked 3750-Xs. We explained how to safely gain access to the switch configuration and change the enable/secret password and/or administrator user accounts passwords. More technical and security articles on Catalyst switch can be found at our Cisco Catalyst Switches Section.

Small-sized networks usually have DHCP services configured on their Cisco router, while large-sized networks (with multiple VLANs) assign DHCP services to their backbone layer-3 switch (Catalyst 6500, 4500, 3750 etc). The good news is that configuration and debugging commands are identical for both Cisco Catalyst switches and Cisco routers.

Debugging DHCP Server On Cisco Catalyst Switch & Cisco Router

The first symptoms of DHCP server issues are users nagging that they cannot connect to the network because they haven’t got an IP address, and that’s where the fun begins.

Assuming no configuration changes have been made to the Cisco DHCP server, the best way to troubleshoot the problem is to enable debugging on the dhcp server.  The debug ip dhcp events & debug ip dhcp server packets are useful debugging commands that will help us identify what is happening:

The key information provided by our debugging is highlighted in bold. This information tells us that our address pool named Guest-WiFi-VLAN is the DHCP Pool where we have a problem because the pool is empty, which means the DHCP server has no more free IP addresses to assign to new clients.

The next step is to understand why there are no more free IP addresses. A common reason is that there are more clients in the specific VLAN requesting IP addresses, than what the DHCP server can assign. Let’s see if this is the case.

First, we take a look at the configured IP address range for that VLAN/Pool. Note that our Guest VLAN is assigned to VLAN7:

Let’s check and see how many clients have been assigned an IP address for VLAN7:

The output surprisingly shows us that we have only 3 clients to which IP addresses have been allocated. So the question now is where did all the rest of the 231 (234-3) IP addresses go?

Another useful command to check the DHCP pool usage is the show ip dhcp pool. It provides the overall usage of the pool alongside with the total addresses, leased and excluded addresses:

Here we can again confirm that from the 254 total IP addresses, 251 are excluded and 3 are leased. Note that the Excluded addresses includes the manually excluded and conflicted IP addresses.

The Current index column shows the next IP address that will be assigned by the DHCP server. Under normal operation, we would expect to see and IP address within the 192.168.7.0 network, however the value of 0.0.0.0 means that there are no more available IP addresses to lease.

The next step is to check the DHCP server for possible conflicts using the show ip dhcp conflict command – we are sure to find something here:

To save space, we had to remove the rest of the command’s output. Surprisingly enough, we found that all 231 IP addresses were listed in the dhcp conflict table.  As shown above, the IP addresses are listed by date of conflict with the older entries shown first.

Understanding The DHCP Conflict Table & Cisco DHCP Server Functionality

Before a Cisco DHCP server hands out an IP address to a client, it always ARPs and then pings the address to be handed out to make sure no one is using it.

When a Cisco DHCP server discovers a conflict, it will place the IP address into the conflict table stating the address was conflicting and how it came to that conclusion, as noted under the Detection method column.

If for any reason the client who is already using the IP address that is about to be handed out by the Cisco DHCP server, does not respond to the ping from the DHCP server, the DHCP server will lease out the IP address since it cannot identify any conflict issues.

The first thing the client will do once the IP address assignment is complete, is to send out a gratuitous ARP message with its new IP address. If not reply is received, then it is safe to assume no one else is using it. However if it does receive a gratuitous ARP reply, then it will indicate that another device on the network is already using that address.

Assuming a gratuitous ARP reply is received, the client will send a DECLINE message to the DHCP server, rejecting the IP address it was just assigned. Since Cisco DHCP server has seen two gratuitous ARP messages and discovered there is a conflict, it will move the IP address into its conflict table and assign the next available IP address to the client.

Clearing The IP DHCP Conflict Table

When the DHCP server detects there is a conflict of an IP address before or right after it is assigned to a client, it will automatically remove the IP address from the DHCP pool and move it to the DHCP conflict table.  The IP address in question will remain there until an administrator sees and clears the DHCP conflict table.

If DHCP conflicts are occurring frequently, it is only a matter of time until all available IP addresses are moved to the DHCP conflict table and the DHCP Pool is left empty with no IP addresses to hand out.

We can clear the DHCP conflict table by using the clear ip dhcp conflict * command. This will instruct the DHCP server to clear the conflict table and return all IP addresses to the DHCP Pool.  In case we have multiple VLANs and Pools, the command will affect them as well:

Issuing the show ip dhcp conflict command confirms that there are no more IP addresses in the table.

The show ip dhcp pool command will now show all previously conflicted IP addresses, available to be handed out to our clients.

In many cases, these Catalyst Layer 3 switches are purchase and installed with basic configuration or features enabled, without leveraging their layer 3 capabilities.

After observing many installations that fell into this category (almost out of the box configurations), we decided it was a great idea to begin covering configuration best-practices that will help engineers understand the capabilities of this equipment and better adapt configurations to their company needs.

By correctly leveraging the capabilities offered by any Cisco Catalyst Layer 3 switch we can create a solid network backbone with high security standards that will have the necessary flexibility to ensure the smooth operation of our network.

Article Key Topics::

• Layer 2 Switching Limitations
• Introduction to Layer 3 Switches
• IOS License Requirements for InterVLAN Routing/ SVI IP Routing
• Creating, Configuring and Verifying VLANs
• Enable InterVLAN Routing (SVI - ip routing) and Configuring Default Gateway
• VLAN Security: Moving Ports (interfaces) off the Management VLAN (VLAN1)
• Configuring & Securing Access & Trunk Links Against VLAN Hopping
• Configuring Virtual Trunk Protocol (VTP) Server
• Configuring Network Time Protocol (NTP) - Understanding Why NTP is Essential

Layer 2 Switching Limitations

Cisco's portfolio of catalyst switches is designed to meet any network requirement (in speed, capacity, expandability and security) from small companies up to very large enterprises. Take for example the popular Catalyst Cisco 2960 series switches – these Layer 2 switches provide a healthy amount of features in speed, functionality and security, leaving very little to be desired even in very demanding network environments.

The biggest problem with Layer 2 switches is that while they can support the creation of multiple VLAN, Layer 2 switches are unable to route packets between VLANs – a process known as InterVLAN routing. Making InterVLAN routing possible requires a switch (or any other device) that operates at the 3^rd layer of the OSI model, this translates to a router (Router-on-a-stick method), layer 3 switch or server.

Note: VLAN concepts and InterVLAN routing theory is covered in great depth at our popular Firewall.cx VLAN section.

Introducing Cisco Catalyst Layer-3 Switches

Cisco produces a number of high-end switches that are capable of delivering Layer 3 routing functionality within an enterprise network of any size. The most popular models found on the market are as follows:

• Cisco Catalyst 3560, 3560G, and newer 3560-E & 3560-X
• Cisco Catalyst 3750, 3750G and newer 3750-E & 3750-X
• Cisco Catalyst 3850
• Cisco Catalyst 4500 series. Includes 4503-E, 4506-E, 4507R+E, 4510R+E etc.
• Cisco Catalyst 6500 series. Includes 6503-E, 6504-E, 6506-E, 6506-E, 6509(E,V, NEB) & 6513 etc.
• Cisco Catalyst 9300 series
• Cisco Catalyst 9400 series

These Layer 3 switches are usually found at the Core Network Layer, interconnecting all other Layer 2 switches, providing secure access to all VLAN networks according to the company’s security policy. VLANs are created at the Core Layer (on these switches) and InterVLAN routing is configured according to the company’s requirements.

While some might consider Layer 3 switching unnecessary, the truth is that if there is more than one VLAN on the network, chances are that a Layer 3 switch is necessary as it will allow control of VLAN communications and integrate security, flexibility and network performance into one (or more) physical device(s).

IOS License Requirements For InterVLAN Routing – SVI IP Routing

Due to Cisco’s new licensing model, purchasing a Layer 3 switch doesn’t really mean that essential features such as InterVLAN routing can be used.

In order to enable features such as InterVLAN routing, the switch must have the appropriate license. This is applicable for the newer 3560-E, 3560-X, 3750-E, 3750-X models & all 4500/6500 Supervisor engines using the new Universal Cisco IOS software image (IOS 15.x onwards).

Thankfully Catalyst switch models 3560, 3560G, 3750, 3750G, Catalyst 4500/4000 Series with Sup II+ or later, or Catalyst 6500/6000 Series that run Cisco IOS system software, support basic InterVLAN routing features in all their software versions.

So if purchasing a Layer 3 switch is on the company's plans, special attention must be given to ensure it has the correct license that will allow InterVLAN routing or other desired features.  In the event an incorrect model number/license is purchased, the only way to enable the feature required is to purchase an additional license – an exercise that will increase costs significantly and should be avoided. For this reason it is very important the correct license is selected with the initial purchase.

The table below outlines the Feature Set Characteristics and Differences of all available licensing models:

As a rule of thumb, any newer 3560-E, 3560-X, 3750-E or 3750-X switches must be purchased with the IP Base license in order to support InterVLAN routing. The LAN Base license will allow Switched Virtual Interfaces (VLANs) to be created, however, it does not support InterVLAN routing or IP routing.

Now that we’ve got the licensing covered, it’s time to begin creating our VLAN interfaces.

Creating & Configuring VLANs

First step on any Layer 3 switch is to create the necessary VLANs.  

By default, VLAN1 exists on every switch. VLAN1 is also known as the Management VLAN and it's highly advisable VLAN1 is not used to carry user data/traffic, as VLAN1 is used only for the management of the network’s switches. Company traffic (Servers, workstations etc) should be placed on a different VLAN, for example, VLAN2.  Voice traffic e.g IP Phones, CallManager, CallManager Express or Voice Gateways, should also be placed on a VLAN of their own – also known as the Voice VLAN.

As part of the design and implementation phase, we strongly advise to create a list of the VLANs that will be created along with their name and any additional information to help identify their purpose and of course the IP address that will be assigned to every VLAN interface on the core Layer 3 switch. This will ensure all VLANs are created and everything is documented for future reference.

Below is an example of a VLAN list we created during the installation of our Cisco Catalyst 3560G:

Before we begin creating our VLANs, let’s take a look and see the default VLANs that exist on Catalyst Layer 3 switches using the show vlan brief command:

First step is to create and name the new VLANs in the switch’s VLAN database. This is accomplished by using the vlan command, followed by the name command. Depending on the switch model, these commands might or might-not appear in the configuration:

We can verify the new VLANs have been created in the VLAN database by issuing the show vlan brief command:

Note that created VLANs are stored in the switch’s VLAN database. The VLAN database is a file named vlan.dat and is located in the switch’s FLASH memory:

Looking carefuly at the creation/modified date of the files, it seems like we are off by a bit more than 10 years, so it is evident the correct date and time have not yet been configured. We’ll take care of this later.

 
Next, we create our VLAN interfaces and assign IP addresses and descriptions:

Note: When configuring the new VLAN interfaces, the switch will show the following message on the console for each VLAN interface configured: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan2, changed state to down. This message can safely be ignored as the VLAN Line protocol will come up as soon as ports on the switch are assigned to the VLAN.
 
There is a possibility that Interface VLAN1 might have the shutdown command configured. This can be checked by issuing the show run command. In the case the shutdown command is present under VLAN1 interface, it is imperative to issue the no shutdown command so that the Management VLAN interface comes up.
 
The show ip interface brief command will verify all VLANs are up (Status), but with a protocol down status as explained earlier:

Enable SVI InterVLAN Routing – IP Routing & Configuring Default Gateway

A Switch Virtual Interface (SVI) is a VLAN of switch ports represented by one interface to a routing or bridging system. Since there is no physical interface for the VLAN, the SVI provides the Layer 3 processing for packets from all switch ports associated with the VLAN. Once VLANs have been created and VLAN interfaces are configured with their IP addresses, we can enable ip routing on our switch, effectively switching ‘on’ the InterVLAN routing capabilities of the switch and enabling the supported routing protocols.

Let’s take a look at the routing capabilities before enabling ip routing. This can be done using the show ip route command:

Notice the ip routing protocol is not enabled, and therefore no entries exist in the switch’s routing table. 

It is important to mention that hosts belonging to different VLANs are not able to communicate or ‘ping’ each other at this point. IP routing must be enabled to allow the switch to pass (route) packets from one VLAN to another.

IP routing is easily enabled using the ip routing command:

At this point, ip routing has been enabled. When issuing the show ip route command, the switch shows all networks learnt from the supported routing protocols (none active at the moment).  Hosts between different VLANs are now able to communicate with each other as long as they have their IP gateway set to the 3560’s VLAN IP interface of their network.

Next, we need to configure a default gateway for the Layer 3 switch. This will instruct the switch to send any packets not belonging to the local network(s), to the next hop, which is usually a router: 

With the default gateway set, we can now issue the show ip route command again and check the results:

VLAN Security: Moving All Ports (interfaces) Off The Management-VLAN

When configuring a switch, it is very important not to leave any ports (interfaces) assigned to the Management VLAN. Doing so will expose the backbone network to anyone who connects to a port assigned to VLAN1. All ports should be configured for another VLAN or, even better, shutdown any unused ports.

The 3560G in our lab has 48 Gigabit Ethernet ports plus 4 SFP ports, but we will use the interface range command to apply the configuration changes to all 48 Gigabit Ethernet ports at the same time, saving us valuable time:

Issuing the show vlan brief command will verify our new configuration changes:

As shown, only the last four ports (0/49 – 0/52) are still on VLAN1. These ports represent the four SFP ports on the switch, normally used to connect to other switches via fiber optic links.  We’ll talk about them soon.

Configuring & Securing Access & Trunk Links Against VLAN Hopping

VLAN Hopping is a well-known attack method by which an attacker uses insecure switch ports to help him jump from one VLAN to another, gaining access to multiple networks. Preventing attacks such as VLAN hopping can be easily achieved by following some basic steps.

The previous section explained why it is important all access ports are configured with a VLAN other than the management VLAN. In addition to the switchport mode access command, the switchport nonegotiate command will ensure that the specific port configured to never negotiate trunks automatically:

On the other hand, Trunk links are essential on any VLAN network as they carry traffic from all VLANs, allowing switches to connect between each other and move traffic between them as necessary. A port is usually configured as a Trunk when it connects to one of the following:

• Switch
• IP Phone
• Access Point configured with multiple SSIDs

Trunk links are also configured with a native VLAN. A Trunk link’s native VLAN carries untagged traffic, just like any port configured as an access link. So if we were to connect an access device (e.g computer) to a port configured as a Trunk link, then the computer will have access to the Trunk's native VLAN. The problem here is that, by default, all Trunk links have their native VLAN configured to VLAN1 - the Management VLAN!

The workaround here is to ensure the native VLAN of any Trunk link is set to an unused VLAN. For this reason we normally create a ‘dummy’ VLAN and configure it as the native VLAN for our Trunk Links. Assuming we have created VLAN 20 as our 'dummy' VLAN, we configure the Trunk Links native VLAN to that.

By using a few additional commands under our switch port interfaces, we can secure our switch interfaces and effectively avoid these attack methods.

Configuring Virtual Trunk Protocol Server (VTP)

Configuring VTP is essential in any Cisco powered network infrastructure. The VTP protocol ensures all VLAN information is propagated to all Cisco switches that are part of the configured VTP Domain.  VTP is analysed extensively at Firewall.cx’s Virtual Trunk Protocol section. The section provides in-depth analysis of the VTP protocol structure with 3d protocol structure frames, how VTP works, VTP modes and advantages offered, plus a lot more. For more information on VTP, please visit the Virtual Trunk Protocol section.

VTP configuration consists of a few basic parameters and a number of optional parameters.  We’ve selected a number of these parameters that are necessary to get VTP working in any working environment:

• Configuration of VTP Mode
• Configuration of VTP Domain
• Configuration of VTP Password
• Verification of VTP Status

Configuring the VTP mode is essential for VTP’s correct functionality.  In any given network of switches only one switch is assigned with the VTP Server role, while all other switches are set to VTP Client or VTP Transparent mode.

Configuration of VTP Mode

By default, all switches are configured in VTP Server mode. We can view the mode status by using the show vtp status command:

Notice the VTP Operating Mode is set to Server and the amount of VLANs (12) already contained in the database. The Configuration Revision number is automatically incremented every time a change is made to the existing VLAN database.

Configuring the VTP Mode to server is accomplished with the vtp mode server command:

If the switch is already in Server Mode, expect to see the Device mode already VTP SERVER message.

Configuration of VTP Domain

Configuring the VTP Domain is accomplished with the use of the vtp domain command. Network switches configured as VTP Clients and the same VTP Domain will ‘listen’ to the VTP Server. VLANs created at the VTP server will automatically be made available on all other switches configured as VTP Clients:

As shown, the initial VTP Domain is by default set to NULL. This changes as soon as we set the VTP Domain.

Configuration of VTP Password

Setting the VTP password is very important as it ensures only VTP clients with the correct password are able to exchange VTP information for the specified VTP Domain.

Verification of VTP Status

Finally, we can verify our VTP setup and status using the show vtp status command:

Of course VTP offers a number of additional parameters that can be tweaked to reduce broadcast traffic (VTP Pruning), increased security (VTP v2 mode) and more,  but these commands are out of this article's scope.

NTP Configuration – Understanding Why NTP Service is Essential

The NTP service is used by the switch to synchronize its internal clock and date with an NTP server. This ensures that the switch is correctly synced at all times. Many engineers wonder why it is so important to configure a switch’s clock, considering the device is simply a switch and not a file server or domain controller. The truth is that NTP configuration is extremely important and we will explain why.

A switch with a properly configure NTP service allows us to easily analyse the switch log messages to help determine when specific events might have happened. If for example a file server’s network card was disconnected briefly from the network, we could then view the switch’s log messages and see when exactly this occurred. Another frequently observed example is when switch ports in violation are automatically shutdown by the switch. Examining the logs, we can view when a specific violation occurred and why it caused the switch to shutdown its port. As a last example, if a syslog server is configured to track all switch messages, errors and events or if we are using a Radius server to restrict network access for specific days of the week and times, then NTP configuration is mandatory.

In every case, it’s good practice to ensure NTP is configured on the network equipment we are responsible for. Along with NTP, it’s always important to remember to configure the correct timezone and daylight saving, so that the switch automatically adjusts the time when necessary.

When configuring NTP, we can select either an internal or public NTP server from which the switch will synchronize. In the event a public ntp server is the preferred choice, we advise visiting the NTP Pool Project homepage http://www.pool.ntp.org/en/  and selecting a server from the available lists. Currently there are over 4000 servers all over the world to choose from.

Let’s take a look at the current date and time our lab switch has:

First step is to configure a preferred NTP server. We’ve selected 0.europe.pool.ntp.org from the European NTP Pool. Note that in order for our switch to resolve 0.europe.pool.ntp.org to an IP address, we must have configured a valid name server. We used Google’s public DNS server 8.8.8.8:

Notice how the switch will automatically attempt to translate the FQDN into an IP address. The IP address of the NTP server is then stored into the configuration. Viewing the date and clock after executing the above commands will show that the switch has correctly synchronized with the selected NTP server:

Notice that the time is synched for UTC or GMT 0. Since we are located in Greece, we need to instruct the switch to use the correct timezone (GMT +2) and daylight saving:

As soon as the above commands are entered, the switch will automatically adjust the time:

Finally, issuing the show clock command will confirm everything is configured correctly:

As a final note, do not forget to save the configuration!

Cisco makes an enormous effort to bring new features to its products through new versions of its popular IOS software, especially for the enterprise-class series switches such as the 4500 and 6500 Catalyst switches. Unlike other vendor operating systems, it is always advisable to keep your Cisco Catalyst operating system up to date by fitting it with the latest IOS image.

To be eligible to download an IOS image from Cisco’s website, one must have a valid contract support with Cisco Systems. This contract not only makes you eligible for all software upgrades for the duration of your contract, but also binds Cisco in delivering Top-Class support for your covered device(s).

Note: Information on installing the Cisco 4507R-E/4507R+E Catalyst Switch, Supervisor Engines, Line cards and power supplies can be found in our following articles which, combined, contain over 30 pictures of all switch components, including the passive backplane, fantray and more:

• Cisco 4507R+E Layer 3 Installation: Redundant WS-X45-SUP7L-E Supervisor Engines & WS-X4648-RJ45V+E Line Card

• Installation of a Cisco Catalyst 4507R-E Layer 3 Switch
  

Are Network Services Affected During Or After The Catalyst Supervisor Engine(s) Upgrade?

This is perhaps one of the most frequently asked question that troubles engineers, administrators and IT Managers who have not dealt with the Supervisor Engine upgrade process before.  Who wouldn’t be? This is, after all, the network backbone switch and when it comes to large campus, Enterprise networks or networks that operate around the clock, downtime is not an option.

At this point, we should note that when upgrading a Supervisor Engine the engine must be reloaded (rebooted) in order for the system to load the new IOS image. For the Cisco Catalyst 4503 and 4506 series, which can only accept a single Supervisor Engine, this means network service interruption is unavoidable as the single Supervisor Engine must reboot. On the other hand, the Cisco Catalyst 4507R and 4510R series are capable of accepting up to two Supervisor Engines (hence the ‘R’ – Redundancy), therefore in case of a 4507R/4510R with two Supervisor Engines installed, IOS upgrades can be performed with guaranteed Zero-Service-Interruption. 

Note: The process we are about to describe was performed on a live 4507R+E with two Supervisor Engines 7L-E, on a network of 120+ users and 11 servers connected to our 4507R+E via dual 10Gbps fiber optic links (one 10Gbps link on each Supervisor Engine).

Quick Overview Of The Supervisor Engine Upgrade Process

Before we dive into the upgrade process, let’s take a quick look at the steps to be followed. This will help understand the process and caveats of each step.

When upgrading a system with redundant Supervisor Engines the upgrade process has to be performed in a specific way as each Supervisor Engine is upgraded in turn.  

Following is a brief overview of the upgrade steps:

• Load the new IOS image on to the Active Supervisor Engine (SE1)
• Copy IOS image to Standby Supervisor Engine (SE2)
• Configure Supervisor Engines to load the new image upon reboot
• Set Configuration-Register variable to ensure newest image is loaded upon bootup
• Force reload of Standby Supervisor Engine (SE2) & Switchover to Standby Supervisor Engine (SE2). This now becomes the new Active Supervisor Engine
• Force reload of previously Active Supervisor Engine (SE1).

Step 1: Loading The New IOS Image On To The Active Supervisor Engine (SE1)

The first step is to copy the new IOS image on to the active Supervisor Engine (SE1). For this, a TFTP server is required for the file transfer. Users can download a selection of Free TFTP Servers from our FTP/TFTP Servers & Clients download section.

Once the TFTP server is ready, we issue the necessary command to initiate the file transfer:

Note that the new IOS image is saved to the bootflash: file system.  Cisco Supervisor Engines use the bootflash: file system rather than the flash: file system which most of us are used to.

If in doubt, simply make use of the show file system command that will reveal the file systems on your Catalyst switch:

Step 2: Copy IOS Image To Standby Supervisor Engine (SE2)

Once the IOS image is loaded on to the active Supervisor Engine (SE1), it must be copied to the standby Supervisor Engine (SE2). For this, we use the copy bootflash: slavebootflash: command. Note that the slavebootflash: file system refers to the bootflash: of the standby Supervisor Engine, regardless of which physical engine is on standby mode.

To verify the image is correctly loaded on both Supervisor Engine bootflash systems, use the show bootflash & show slavebootflash commands:

Note:  Notice that the first IOS image listed is the previous version (03.03.00.SG.151-1.SG). We’ll need to keep this information in mind for our next step.

Step 3: Configure Supervisor Engines To Load The New Image Upon Reboot

In our next step, we configure the active Supervisor Engine to load the new IOS image when it reboots. This is easily done using the boot system flash command as shown below:

It is highly likely the boot system flash bootflash command already exists in the system’s configuration for the previous IOS image, so we’ll need to remove the command from the configuration to ensure the newest IOS image we just uploaded (03.04.00.SG.151-2.SG) is the only one referenced:

Next, we must ensure the configuration is saved to the startup-configuration:

Readers wondering about the % VRF table-id 0 not activeCompressed message when saving the running-configuration should not be alarmed as this is a cosmetic bug and can be safely ignored. If no such message was reported when saving the configuration, it means that the IOS currently running does not have this cosmetic bug.

On another note, every time we save our configuration to startup-config the system will immediately synchronize the configuration changes to the standby supervisor. When this happens, we’ll receive a message, similar to the one show above, confirming the synchronization has been successful.

Step 4: Set Configuration-Register Variable To Ensure Newest Image Is Loaded Upon Bootup

Cisco Supervisor Engines, by factory default, have their configuration register set to 0x2101. While this value is a combination of settings, we will focus on two specific values: 0x2101 & 0x2102.  The value 0x2101 instructs the system to boot the first system image in the onboard flash memory (bootflash). This is usually the oldest image in the flash. The value of 0x2102 instructs the system to use the image specified in the BOOT environment variable, which is essentially whatever was specified in the previous step (No.3) using the boot system flash bootflash: command.

To view the environment variables on both Supervisor Engines, use the show bootvar command:

Notice how the newly uploaded IOS image cat4500e-universal.SPA.03.04.00.SG.151-2.SG.bin is set for the BOOT variable, however the Configuration register value is 0x2101, which means it will force the Supervisor Engine to boot the first image it is going to find on the bootflash. Which image is that?  Let’s refresh our memory:

As we can see, despite the boot system flash command being correctly set to load the second IOS image, the configuration register setting of 0x2101 will force the Supervisor Engine to boot the first image found, that is: cat4500e-universal.SPA.03.03.00.SG.151-1.SG.bin.

To overcome this problem, we have two options: 

1) Delete all older IOS images from the Bootflash 

2) Set the configuration register to 0x2102. 

We decided to set the configuration register just to be on the safe side:

Take note that the system is confirming that both configuration register and startup-config have been successfully synchronized with the standby supervisor.

…and there’s that cosmetic bug again :)

Step 5: Force Reload Of Standby Supervisor (SE2) & Switchover To Standby Supervisor Engine (SE2)

At this point we are ready to force the standby Supervisor Engine(SE2) to reload. Once this happens, the Supervisor Engine (SE2) will load the new IOS. Once we confirm the new IOS is loaded, we can then make the standby Supervisor Engine (SE2) the active Supervisor Engine.

To force the reload of the standby Supervisor Engine (SE2), use the redundancy reload peer command:

The second (standby) Supervisor Engine is now restarting. This process will take a couple of minutes and will have no negative impact on the 4507R switch.

Once the IOS has loaded and the restart process is complete we will receive a message similar to the following:

*Mar 21 01:24:54.312: %C4K_REDUNDANCY-6-DUPLEX_MODE: The peer Supervisor has been detected
*Mar 21 01:25:36.066: %C4K_IOSMODPORTMAN-6-MODULEONLINE: Module 4 (WS-X45-SUP7L-E S/N: CAT1714L4T4 Hw: 1.1) is online
*Mar 21 01:25:36.094: %C4K_REDUNDANCY-6-MODE: ACTIVE supervisor initializing for sso mode
*Mar 21 01:25:36.344: %C4K_REDUNDANCY-3-COMMUNICATION: Communication with the peer Supervisor has been established
*Mar 21 01:25:37.098: %C4K_REDUNDANCY-6-MODE: ACTIVE supervisor initializing for sso mode

The above messages indicate that the standby Supervisor Engine has successfully completed its reboot and is fully synchronized with the active Supervisor Engine. The system shows the product ID (WS-X45-SUP7L-E), serial number and hardware revision of the Supervisor Engine that has just established communication with the active Supervisor Engine.

To verify that the standby Supervisor Engine is running the latest and greatest IOS image we just loaded, use the show module command:

The show module command provides a generous amount of information including all line cards installed, Supervisor Engine models, IOS software versions, serial numbers, operational status and redundancy mode (SSO by default).

We’ve highlighted our standby Supervisor Engine which happens to be in slot No.4. Notice the software image the system is reporting for both Supervisor Engines. Our current active Supervisor Engine is running version 03.03.00.SG, while our standby Supervisor Engine is running version 03.04.00.SG – the new IOS loaded!

We’ve now confirmed that the standby Supervisor Engine has loaded the new IOS and is fully operational, waiting eagerly to manage our 4507R switch and all its data!!

Step 6: Force Reload Of Previously Active Supervisor Engine (SE1)

On our final step, we will need to force the reload of the active Supervisor Engine (SE1) so it too can boot using the new IOS image. Forcing a Supervisor Engine switch over is an experience when you consider what’s really happening inside the switch at that moment. During the process of the switchover we don’t get LEDs lighting up, flashing like crazy. The SUP ACTIVE LED will simply switch off from the current active Supervisor Engine and switch on on our previously standby Supervisor Engine, indicating it is now the new active engine.

To initiate the Supervisor Engine switchover, use the redundancy force-switchover command as shown below:

Once the command is entered and we’ve confirmed by hitting ENTER, we will lose our telnet session to the 4507R. This is normal expected behaviour – do not be alarmed!  Apart from the switch cutting our telnet session, users will not notice any service disruption – the switchover will be completely transparent to them no matter the network load during the switchover.

To reconnect to the switch, simply telnet back into the same IP address. If connected via console cable it will be necessary to connect it to the new active supervisor engine in order to continue controlling the switch.

Engineers who would like to monitor, via telnet or direct console cable connection (to the new active engine), the reload progress of the Supervisor Engine can use the show module command. By typing the command we will see the 4507R identifying a Supervisor Engine in slot 3 (that’s SE1 that is reloading), however, further down we will see that no information about the engine’s MAC address or IOS software is provided due to the fact that it has not fully booted into its new IOS. In addition the redundancy status of the first engine is Disabled – an expected result since the Supervisor Engine (SE1) has not booted yet.

At this point, we can continue to issue the show module command and monitor the output changes. At some point the system will show the Supervisor Engine (SE1) loaded with the new IOS, however, the redundancy status will go through the following phases until it is ready (Standby hot):

Standby Supervisor Redundancy Status Cycle:

1. Disabled
2. In progress to Standby cold
3. Standby cold
4. In progress to Issu negotiation la
5. In progress to Standby config
6. In progress to Standby bulk  
7. Standby hot

When the Supervisor Engine reaches Standby hot status, it is ready to take over in the event the active Supervisor Engine fails.

Following is the expected output when the Supervisor Engine IOS is loaded and fully synced with the active Supervisor Engine:

Summary

This article explained the importance of upgrading the Cisco Supervisor Engine IOS to the latest version, the process that needs to be followed for the upgrade and how the Catalyst switch’s operations are affected during or after the upgrade. We also covered the IOS upgrade process for redundant Supervisor Engine configurations, how to force a Supervisor Engine switchover and monitoring the Supervisor Engine bootup and sync process.

More articles covering Cisco Catalyst switches and LAN switching technologies can be found in our Cisco Knowledgebase Switching category.

Many might be aware of our first 4507R article that covered the installation of a Cisco Catalyst 4507R-E switch.  Since then, Cisco has replaced the 4507R-E with the newer 4507R+E chassis and introduced new Supervisor Engines. The difference between the two chassis is that the 4507R-E supports up to 24Gbps bandwidth per slot, whereas the newer 4507R+E supports up to 48Gbps per slot, bringing the chassis up to date with the new market trends and high-connectivity speed requirements of enterprise companies.

To make things more interesting, we ensured we captured as many pictures as possible from our 4507R+E switch installation so that our readers can familiarise themselves with it as much as possible.

After unpacking and looking at the back of the switch chassis we noticed that not much has changed except that its label now mentions 4500+E Series, indicating that it is the newer +E series. On the front side, the fan tray manages to give away that this is the newer series as it too is labelled Catalyst 4507R+E.  Apart from these minor cosmetic changes the switch looks exactly the same as its predecessor.

The picture below shows the back of the Cisco 4507R+E chassis. The dual power supplies are positioned at the top part of the switch and the specially designed grid allows adequate air to be pumped through the power supplies and out the back, with the help of the massive power supply fans.  The fans used are extremely high quality with very little friction – when during our test run, we switched off the power supplies, the fans continued to spin for at least another 20 seconds before coming to a complete stop:

Mounting a Catalyst 4500 into a rack can be a daunting experience, mainly due to its weight. When fully populated, the switch can weigh up to 55 Kgs and requires at least two people to safely pick up and place the switch into the rack, then you’d need one more person to tighten the necessary screws to keep it inside the rack!

We also found it necessary for the rack to have adequate spacing above and below the area where the switch is to be placed, because it’s very difficult to keep the switch steady during installation because of its weight. In addition, it is imperative the rack’s side covers can be removed so the two handles on the switch (one on each side) are accessible.

To overcome the problem of installing the heavy switch, we decided to remove both power supplies and all cards from the chassis. The empty chassis made things much easier.

Revealing The Magnificent Cisco 4507R+E Backplane

As we begun to slowly remove the switch’s power supplies, line cards and Supervisor Engines, we had a clear view of the spectacular 4507R backplane! The backplane is the switch’s ‘spine’, responsible of interconnecting all components together. Naturally, we had to capture this moment and here it is in all its glory:

We should note that extreme caution must be given when removing and inserting the 4507 cards. The engineer performing the procedure must be properly grounded and the cards must be placed on antistatic mats or, even better, inside antistatic bags. This will help avoid electrostatic discharge that can possibly damage the line cards or supervisor engines.

After removing all five cards we had a clear view of the 4507R+E’s backplane – something you don’t get to see every day. On the left side of the backplane the white and black sockets caught our attention – these are the connectors for the line cards and supervisor engines (marked in red). The Supervisor Engine sockets extend to the far right to a total of three sockets per Supervisor Engine:

Looking to the left, the strategically placed fan tray is visible, ready to blow cool air into the chassis and directly on all cards. As shown in our picture the fan tray consists of two larger fans on the same level as the Supervisor Engines, and six smaller fans taking care of the cooling for the rest of the line cards:

Obviously the most critical cards are the Supervisor Engines, generating most of the heat under intensive workload.

Introducing The WS-X45-SUP7L-E Supervisor Engine

The Supervisor Engine is the heart of every modular Cisco 4500 and 6500 series switch. The Supervisor Engine 7L-E provides a number of enhancements over its predecessor Supervisor Engine 6L-E,making it a primary choice for Enterprise-class networks seeking unprecedented performance.  

Following are key innovations offered by the Supervisor Engine 7L-E:

• 520Gbps system performance with 48Gpbs per slot to every line-card slot and 225mpps throughput
• Dual 10Gigabit Ethernet uplinks (via SFP+ optics) or four Gigabit Ethernet uplinks (via SFP optics)
• Application visibility through Flexible NetFlow (FNF) supporting Layer 2/3/4 information and synthetic traffic monitoring with IP SLA
• Cisco IOS XE Software which provides the ability to host third-party applications
• Support of 802.3az Energy Efficient Ethernet (EEE) capable line cards
• First and only modular switch with 8 bidirectional line-rate SPAN/RSPAN sessions
• Supports up to 244 ports 10/100/1000 in a 7-slot chassis
• External USB & SD card support for flexible storage options
• Maximum resiliency with redundant components, Nonstop Forwarding/Stateful Switchover (NSF/SSO), and In-Service Software Upgrade (ISSU) support
• Full backward compatibility with 6 G, 24 G, and 48 Gbps slot line cards with no performance degradation

Product Datasheet for the Supervisor Engine 7L-E can be found in our Cisco Catalyst 4500-6500 Supervisor Engine download section

Our users can refer to our popular Cisco Catalyst 4500 Series Zero-Downtime IOS Upgrade Process for Supervisor Engine 7-E, 7L-E, 6L-E and V-10GE Redundant Configurations article to learn how to upgrade their Supervisor Engine without network service interruption.

Looking to the front left side of the Supervisor Engine, we can see the Status LED, Active SUP LED (on when the Sup. Engine is in active state) and Utilization LEDs. Visible behind the faceplate is the circuitry board and heatsinks attached to the dual-CPUs and other processors.:

On the right we can see the Console port, Management port and 4 SFP/SFP+ slots providing two 10Gbps or four 1Gbps links! The LEDs below each SFP/SFP+ port will turn on according to the negotiated speed with the other end:

Looking at the back of the Supervisor Engine 7L-E, we can see the three connectors that plug directly into the 4507R+E chassis backplane. The heat sinks fins, covering the dual core CPU and other processors, are placed in a direction where the air from the fan tray can provide maximum cooling and heat dissipation:

The Supervisor Engine 7L-E board is indeed impressive. The picture above shows the Supervisor Engine almost fully inserted into the 4507R+E chassis. We took this picture to show how the Supervisor Engine’s connectors perfectly line up with the 4507R’s backplane sockets:

Note how the Supervisor Engine’s processors carrying the large heatsinks are strategically placed next to the fan tray’s largest fans.

The WS-X4648-RJ45V+E Line Card

The WS-X4648-RJ45V+E line card provides a number of enhanced features designed to bring maximum flexibility and expandability some of which are:

• 48 ports
• 10/100/1000 module (RJ-45)
• Supported from Cisco IOS Software Release 12.2(40)SG or later
• IEEE 802.3af/at and Cisco prestandard PoE, IEEE 802.3x flow control
• Bandwidth is allocated across eight 6-port groups, providing 3 Gbps per port group (2:1)
• L2-4 Jumbo Frame support (up to 9216 bytes)
• Capable of up to 30 Watts of inline power per port on up to 24 ports simultaneously
• Enterprise and commercial: designed to power next-generation IP phones, wireless base stations, video cameras, and other PoE devices
• Campus and branch applications requiring enhanced performance for large file transfers and network backups

In the photo below, we can see the WS-X4648 main board with its chipsets, two of which are equipped with large heatsinks. The white connector at the back provides the connection with the 4507R+E backplane through which the card and its PoE ports are powered, and data traffic is carried to the rest of the system:

We should note that all 4500 & 6500 series line cards, including Supervisor Engines, have a metal carrier on the bottom of the board that covers almost all of the circuit board, making it safe to hold the line card without touching any circuits:

Network Administrator Savvas Filippidis gets ready to install a line card

After a careful inspection of the card it was time to place it into its slot and complete the physical installation of the 4507R+E switch.  After ensuring the two module ejector levers (one on each side) were out and away from the faceplate, we gently lined it up with the two chassis slot guides of our slot and slowly pushed the line card inwards:

As soon as the line card’s connector ‘touched’ the backplane we continued to push it inwards, with a little more strength this time, and the two module ejector levers automatically started to fold inwards until the line card was fully in place:

Cisco 4507R+E 4200Watt Power Supplies

The Cisco 4507R+E supports a number of different power supply configurations. Depending on how the switch is intended to be used, it can be populated fully with 48 Gigabit PoE ports, providing full PoE to all 5 line cards – a total of 240 ports where each line card can draw a maximum of 820 Watts.

Of course, the switch needs the appropriate power supplies to be able to undertake the load and this is why it’s always a great idea to purchase large power supplies as you’ll never need to upgrade them in the future when the switch is fully populated with PoE line cards.

The whopping 4200Watt power supplies were a considerable weight. To remove them, we had to loosen the captive installation screws (two for each power supply) and pull the power supply slowly outwards. Pulling out the power supply did not require much effort as it came out smoothly with the little force we used.

The 4200Watt power supply is the second largest power supply available for the 4500 series and has two IEC60320-C20 connectors (per power supply) to ensure the power drawn from the UPS or power circuit  is evenly split.

We took both power supplies out and took a picture of them to show both front and back side.  At the back of each power supply, as expected, we found one long connector, split into three groups, the middle group containing pins which are barely visible in the picture:

Going back to the chassis, all we found was the power supply connector where the middle pin sockets are easily seen. In total we counted an impressive 46 connection points between the power supply and chassis:

When we initially powered the Cisco 4507R+E switch, after it booted up we found both Supervisor Engine Status LEDs  orange. After checking Cisco's site to find out more information, the explanation for the orange LEDs was that the Supervisor Engine was performing a 'System boot or a diagnostic test is in progress', however it was not true as the system was fully booted and working.

A careful inspection showed that we had forgotten to switch on the second 4200Watt power supply and the system was running off only one power supply.  At the time, the power requirements were only around 500Watts and the system was reporting the second power supply as "bad/off".

As soon as we switched on the second power supply, the Supervisor Engine LEDs magically turned Green! While we found no documentation to explain this behavior, we thought it would be worth mentioning for our readers and engineers preparing to install a 4500 series switch!

Summary

Those who have had the luck to physically install and work on a Cisco Catalyst 4507R or any Catalyst 4500 series switch would surely agree that it is a wonderful experience and impressive piece of equipment. Examining the design and construction of the chassis, line cards and Supervisor Engines shows how much thought and work have gone into the product.

Our next article covers the IOS upgrade of the Supervisor engine, health checkup, using the zero downtime IOS upgrade procedure – a necessary and extremely handy procedure that ensures the upgrade of the IOS Supervisor engines without any service interruption!

With 480Gbps stacking bandwidth, support for 802.11ac (at least 500Mbit/sec single link throughput) wireless, Power over Ethernet Plus (30Watt/ port), StackPower and Flexible NetFlow support on all ports, this is one of the most comprehensive list of features ever produced in a Catalyst switch.

Cisco 3850 Integrated Wireless Controller

Perhaps one of the features most engineers and managers would have asked for is now available with the new 3850 series. The integrated wireless controller allows organizations to invest in one switch that will cover their wired and wireless needs, and we are not talking about a simple wireless LAN controller.

The wireless capabilities provided by the new Catalyst 3850 are nothing less than impressive. With support for up to 50 access points, 2000 wireless clients per 3850 switch/stack, support of the new upcoming 802.11ac standard that allows throughput of at least 500Mbps per link and, get ready for it, a total of 40Gbps wireless throughput (20Gbps on 24port models) - makes this switch blazingly fast with enough room to cover today’s and tomorrow’s wireless needs.

Each Cisco Catalyst 3850 switch/stack can operate the Wireless Controller in two modes:  Mobility Agent (MA) or Mobility Controller (MC).

Mobility Agent is the default mode where the switch is capable of terminating CAPWAP tunnels from access points and provide wireless connectivity to wireless clients. Mobility Agent mode requires the IP Base license installed.

Mobility Controller mode allows the Catalyst 3850 to perform additional tasks such as radio resource management (RRM), Cisco Clean Air coordination inside a mobility subdomain. Mobility Controller mode is supported in the IP Base license and can be enabled via CLI.

Catalyst 3850 Features

The Catalyst 3850 comes in 3 different configurations: Non-PoE, PoE+ and Full PoE+ support.

Each configuration supports 24 or 48 ports, except for Full PoE+ which is only supported in the 48 port version as shown in the table below:

Full PoE+ configurations guarantees 30watts of PoE power on all ports.

More features include:

• Integrated Wireless Controller.  Supports up to 50 access points and 2000 wireless clients on each individual switch or stack.
• Dual Power Supply. Combine up to two power supplies in each switch.
• Cisco StackPower. Allows power stacking between stack members for power redundancy.
• StackWise Technology. Stack your 3850’s together and create one single manageable switch with 480Gbps of bandwidth – enough to satisfy the most demanding network environments
• Full PoE+ support (IEEE 802.3at)
• Cisco IOS software support. Engineers with experience in IOS will have no trouble learning to operate and configure the new features offered.
• Enhanced limited lifetime warranty (E-LLW) with next business day (NBD) advanced hardware replacement and 90-days Cisco Technical Assistance Center Support (Cisco TAC).

Network Modules

The Catalyst 3850 supports three optional network modules for optical uplinks to other switches or central servers. As shown in the table below, the network modules allow support for the following configurations:

• 4 x 1 Gigabit Ethernet via SFP modules
• 2 x 10 Gigabit Ethernet via SFP+ modules
• 4 x 40 Gigabit Ethernet via SFP+ modules (Only on 48-port models)

For a comprehensive list of features, services and configurations, readers can download the Cisco Catalyst 3850 Datasheet, now available in our Cisco Product Datashets and Guides download section.

Back in the old days whenever there was a need to monitor or capture network traffic, a hub would be introduced somewhere in the network link and, thanks to the hub’s inefficient design, it would copy all packets incoming from one port out to all the rest of the ports, making it very easy to monitor network traffic. Those interested in hub fundamentals can read our Hubs & Repeaters article.

Of course switches work on an entirely different principle and do not replicate unicast packets out of every port on the switch, but keep them isolated unless it’s a broadcast or multicast.

Thankfully, monitoring network traffic on Cisco Catalyst switches is a straightforward process and does not require the presence of a hub. The Cisco method is called Switched Port Analyser also known as  SPAN.

Understanding SPAN Terminology

• Ingress Traffic: Traffic that enters the switch
• Egress Traffic: Traffic that leaves the switch
• Source (SPAN) port: A port that is monitored
• Source (SPAN) VLAN: A VLAN whose traffic is monitored
• Destination (SPAN) port: A port that monitors source ports. This is usually the point to which a network analyser is connected.
• Remote SPAN (RSPAN): When Source ports are not located on the same switch as the Destination port. RSPAN is an advanced feature that requires a special VLAN to carry the monitored traffic and is not supported by all switches. RSPAN explanation and configuration will be covered in another article.

Figure 1. The network diagram above helps us understand the terminology and implementation of SPAN.

Source SPAN ports are monitored for received (RX - Ingress), transmitted (TX - Egress) or bidirectional (both) traffic.  Traffic entering or exiting the Source SPAN ports is mirrored to the Destination SPAN port. Typically, you would connect a PC with a network analyser  on the Destination SPAN port, and configure it to capture and analyse the traffic.

The amount of information you can obtain from a SPAN session really depends on how well the captured data can be interpreted and understood. A reliable Network Analyser will not only show the captured packets but automatically diagnose problems such as TCP retransmissions, DNS failures, slow TCP responses, ICMP redirect messages and much more. These capabilities help any engineer to quickly locate network problems which otherwise could not be easily found.

Basic Characteristics & Limitations Of Source Port

A source port has the following characteristics:

• It can be any port type such as EtherChannel, Fast Ethernet, Gigabit Ethernet and so forth.
• It can be monitored in multiple SPAN sessions.
• It cannot be a destination port (that’s where the packet analyser is connected)
• Each source port can be configured with a direction (ingress, egress, or both) to monitor. For EtherChannel sources, the monitored direction applies to all physical ports in the group.
• Source ports can be in the same or different VLANs.
• For VLAN SPAN sources, all active ports in the source VLAN are included as source ports.

Basic Characteristics & Limitations Of Destination Port

Each SPAN session must have a destination port that receives a copy of the traffic from the source ports and VLANs.

A destination port has these characteristics:

• A destination port must reside on the same switch as the source port (for a local SPAN session).
• A destination port can be any Ethernet physical port.
• A destination port can participate in only one SPAN session at a time.
• A destination port in one SPAN session cannot be a destination port for a second SPAN session.
• A destination port cannot be a source port.
• A destination port cannot be an EtherChannel group.

Limitations Of SPAN On Cisco Catalyst Models

Following are the limitations of SPAN on various Cisco Catalyst switches:

• Cisco Catalyst 2950 switches are only able to have one SPAN session active at a time and can monitor source ports. These switches cannot monitor VLAN source.
• Cisco Catalyst switches can forward traffic on a destination SPAN port in Cisco IOS 12.1(13)EA1 and later
• Cisco Catalyst 3550, 3560 and 3750 switches can support up to two SPAN sessions at a time and can monitor source ports as well as VLANs
• The Catalyst 2970, 3560, and 3750 switches do not require the configuration of a reflector port when you configure an RSPAN session.
• The Catalyst 3750 switches support session configuration with the use of source and destination ports that reside on any of the switch stack members.
• Only one destination port is allowed per SPAN session and the same port cannot be a destination port for multiple SPAN sessions. Therefore, you cannot have two SPAN sessions that use the same destination port.

Configuring SPAN On Cisco Catalyst Switches

Our test-bed was a Cisco Catalyst 3550 Layer 3 switch, however, the commands used are fully supported on all Cisco Catalyst 2940, 2950, 2955, 2960, 2970, 3550, 3560, 3560−E, 3750, 3750−E and 4507R Series Switches.

The diagram below represents a typical network setup where there is a need to monitor traffic entering (Ingress) and exiting (Egress) the port to which the router connects (FE0/1). This strategically selected port essentially monitors all traffic entering and exiting our network.

Since router R1 connects to the 3550 Catalyst switch on port FE0/1, this port is configured as the Source SPAN port. Traffic copied from FE0/1 is to be mirrored out FE0/24 where our monitoring workstation is waiting to capture the traffic.

Once we have our network analyser setup and running, the first step is to configure FastEthernet 0/1 as a source SPAN port:

Next, configure FastEthernet 0/24 as the destination SPAN port:

After entering both commands, we noticed our destination’s SPAN port LED (FE0/24) began flashing in synchronisation with that of FE0/1’s LED – an expected behaviour considering all FE0/1 packets were being copied to FE0/24.

Confirming the monitoring session and operation requires one simple command, show monitor session 1:

To display the detailed information from a saved version of the monitor configuration for a specific session, issue the show monitor session 1 detail command:

Notice how the Source Ports section shows Fa0/1 for the row named Both. This means that we are monitoring both RX & TX packets for Fa0/1, while the Destination Port is set to Fa0/24.

Turning to our network analyser, thanks to its predefined filters we were able to catch packets to and from the worksation monitored:

This completes our discussion on SPAN configuration and how to monitor/capture packets on a Cisco Catalyst switch. Upcoming articles will cover RSPAN and more advanced packet capturing techniques using dedicated VLANs for captured traffic and other complex scenarios.

Table 1 describes the end-of-life milestones, definitions, and dates for the affected product(s). Table 2 lists the product part numbers affected by this announcement. For customers with active and paid service and support contracts, support will be available until the termination date of the contract, even if this date exceeds the Last Date of Support shown in Table 1.

Table 1. End-of-Life Milestones and Dates for the Cisco Catalyst 3750G, 3560G, 3750-E, and 3560-E Series Switches

Table 2. Product Part Numbers Affected by This Announcement

Cisco saw the massive gap between its entry level Catalyst switches (2960S & 3560) and the competition, and decided to hit them as much as it could with the SG series switches.

The specifications on the newer SG 500 series switches are impressive: switching capacity starting from 28.8Gbps for the smallest 24-port SG500, up to 176Gbps for the largest models that include 10Gpbs uplinks, all with layer-3 switching, stacking and power efficiency capabilities!

Here are some highlights of the SG 500 Series:

• High-Power Power over Ethernet Plus (PoE+), providing up to 30 watts per port
• Full IPv6 Support
• Advanced Layer 3 Traffic Management (InterVLAN-Routing)
• Strong Security. Access Control Lists (ACLs), Voice VLAN, Guest VLAN and many more security features.
• Power Efficiency. Ability to automatic power shutoff ports not used, adjusting signal strength based on length of connecting cable etc.
• Expandability. Offering 1G and 1G/5G Ethernet expansion slots. 10G expansion slots for the 500X series.
• Limited lifetime warranty with next-business-day advanced replacement.

For our readers' convenience, we've made the following downloads available directly from Firewall.cx:

• Cisco SG500 Series Overview
• Cisco SG500 Series Datasheet
• Cisco SG500 Quick Start Guide

 These are easily accessible in our SG500 Datasheet Download section.

Initial Setup of SG500-52P

Just like the SG200 & SG300 series, the SG500 can be configured by both Web and CLI interface. The Web interface is highly intuitive but does require some time to understand its layout and where to configure different aspects of the switch.

The CLI mode is similar to that of the Cisco IOS Catalyst switches but it has its own logic, something we believe Cisco did deliberately to ensure the SG series configuration experience is not ‘identical’ to the much more expensive and well known Catalyst switches.

We found that the best way to configure the switch was to use the CLI interface for specific functions such as setting up IP Addresses, creating and naming VLANs, setting default gateway, then using the web interface for configuring the trunk and access links, allowed VLANs etc. When we completed the configuration and performed a ‘show running-config’ in CLI mode, we then understood that some of the web configuration could have easily been done through CLI.

As with the SG200 & SG300 models, it is advised to always keep the firmware updated to the latest available version. Earlier SG300 firmware suffered from plenty of problems that could cause the switch to stop processing packets and required a reboot to restore functionality.

Hopefully, we won’t be experiencing the same problems with the SG500 firmware.

Before You Begin

Both SG300 and SG500 series switches are layer-3 capable, which means you can create multiple VLANs and route between them - a function called InterVLAN routing. For more information about InterVLAN routing, you can read our InterVLAN  routing article.

Most people are not aware that when an SG300 or SG500 switch is powered up for the first time, it defaults to Layer-2 mode!  In order to create multiple VLANs, assign IP Addresses and enable Layer-3 Switching, you must switch the SG300 & SG500 to router mode! When this is done all configuration is erased and the device is reset, losing any configuration performed.

It is therefore highly advisable to always switch to router mode before any configuration is performed on the switch!

Switching to ‘Router’ Mode – Enabling Layer-3 Switching

To switch to router mode, connect to the serial port using the provided DB9 serial cable (read up on our serial cable articles for info on DB9 connectors) and set the com port thus:

• 115200 Baud Rate
• 8 Data Bits
• No Parity
• 1 Stop Bit
• No Flow Control

When presented with the login prompt, use ‘cisco’ as the username and password. You will be requested to change the password before you perform any configuration:

User Name:cisco  Password:*****

Please change your password from the default settings. Please change the password for better protection of your network.

Do you want to change the password (Y/N)[Y] ?Y

Enter old password  : *****
Enter new password  : ************
Confirm new password: ************

When complete, the CLI prompt will be presented along with the familiar hash symbol. At the prompt, enter show system mode to view the current mode:

switch# show system mode

Feature    State
------   ---------
Mode:      Switch

Without delay, let’s switch to router mode:

switch# set system mode router

Changing the switch working mode will *delete* the startup configuration file and reset the device right after that. It is highly recommended that you will backup it before changing the mode, continue ? (Y/N)[N] Y

As the reset process begins, a number of messages will be displayed on the console and the switch will finally reboot:

switch# 02-Feb-2012 10:48:36 %AAA-I-CONNECT: User CLI session for user cisco over console , source 0.0.0.0 destination  0.0.0.0 ACCEPTED, aggregated (1)


02-Feb-2012 10:48:55 %FILE-I-DELETE: File Delete - file URL flash://startup-config
Resetting local unit

**************************************************
*****************  SYSTEM RESET  *****************
**************************************************
Boot1 Checksum Test...............................PASS

Boot2 Checksum Test...............................PASS

Flash Image Validation Test.......................PASS

BOOT Software Version 1.2.0.12 Built  23-Nov-2011  08:31:59

Networking device with Marvell ARM CPU core. 256 MByte SDRAM.
I-Cache 16 KB. D-Cache 16 KB. L2 Cache 256 KB. Cache Enabled.

MAC Address   :  88:43:e1:ad:52:53.

Autoboot in 2 seconds - press RETURN or Esc. to abort and enter prom.
Preparing to decompress...
 100%
Decompressing SW from image-1
 100%

OK
Running from RAM...
About to erase CDB. Terminal baud rate will now be set to default!
Board ID is 27
Device ID 0xdc7411ab

*********************************************************************
*** Running  SW  Ver. 1.2.0.97  Date  02-Feb-2012  Time  10:12:46 ***
*********************************************************************
HW version is V01
Base Mac address is: 88:43:e1:ad:52:53
Dram size is  : 256M bytes
Dram first block size is  : 208896K bytes
Dram first PTR is  : 0x3000000
Dram second block size is  : 4096K bytes
Dram second PTR is  : 0xFC00000
Flash size is: 32M
02-Feb-2012 10:13:09 %CDB-I-LOADCONFIG: Loading running configuration.
02-Feb-2012 10:13:09 %CDB-I-LOADCONFIG: Loading startup configuration.
Device configuration:
Slot 1 - SG500-52P
Device 0: GT_98DX3124 (TomCat)
Device 1: GT_98DX3124 (TomCat)

------------------------------------
-- Unit Number 1                  --
------------------------------------
02-Feb-2012 10:13:21 %Entity-I-SEND-ENT-CONF-CHANGE-TRAP: entity configuration change trap.
02-Feb-2012 10:13:32 %INIT-I-InitCompleted: Initialization task is completed

>
-----------------------------------
-- Unit Number 1  Master Enabled --
-----------------------------------

Tapi Version: v1.9.5
Core Version: v1.9.5
02-Feb-2012 10:13:39 %Stack-I-STCK-CFG-CHNG: Configuration changed: chain
02-Feb-2012 10:13:39 %MLDP-I-MASTER: Switching to the Master Mode.
02-Feb-2012 10:13:43 %Environment-I-FAN-STAT-CHNG: FAN# 1 status changed - operational.
02-Feb-2012 10:13:43 %Environment-I-FAN-STAT-CHNG: FAN# 2 status changed - operational.
02-Feb-2012 10:13:43 %Environment-I-FAN-STAT-CHNG: FAN# 3 status changed - operational.
02-Feb-2012 10:13:43 %Environment-I-FAN-STAT-CHNG: FAN# 4 status changed - operational.
02-Feb-2012 10:13:43 %Environment-I-FAN-STAT-CHNG: FAN# 1 status changed - operational.
02-Feb-2012 10:13:43 %Environment-I-FAN-STAT-CHNG: FAN# 2 status changed - operational.
02-Feb-2012 10:13:43 %SNMP-I-CDBITEMSNUM: Number of running configuration items loaded: 0
02-Feb-2012 10:13:43 %Environment-I-FAN-STAT-CHNG: FAN# 3 status changed - operational.
02-Feb-2012 10:13:43 %Environment-I-FAN-STAT-CHNG: FAN# 4 status changed - operational.
02-Feb-2012 10:13:43 %SNMP-I-CDBITEMSNUM: Number of startup configuration items loaded: 0
02-Feb-2012 10:13:43 %Entity-I-SEND-ENT-CONF-CHANGE-TRAP: entity configuration change trap.

>lcli

At this point, it is necessary to login using the cisco username & password, then change the password as prompted.

Issuing the show system mode command will then confirm the switch is in router mode, which means we are in business:

Creating VLANs, Assigning IP Addresses, Default Gateway, DNS Name-Server & Enabling IP Routing

The process of creating VLANs on the SG500 is similar to that of the Catalyst switches. First create your VLANs and then VLAN interfaces to configure IP addresses.  Since VLAN 1, the Default VLAN is already created, we only require that we change its IP address to match our network.  Keep in mind that the switch has VLAN 1 preconfigured with IP address 192.168.1.254, but also has DHCP enabled, so if the switch finds a DHCP server during startup it will automatically obtain an IP address. When the system uses its default IP address (192.168.1.254), the System LED shown below will flash continuously:

We’ve now set VLAN 1’s IP address to 192.168.1.2. Next step is to create VLAN2 & 5, our Voice VLAN & Guest VLAN, name them and configure an IP address for each:

switch(config)# vlan 2
switch(config)# interface vlan 2
switch(config-if)# name Voice-VLAN
switch(config-if)# ip address 192.168.10.2 255.255.255.0
switch(config-if)# exit
switch(config)# vlan 5
switch(config)# interface vlan 5
switch(config-if)# name Guest-VLAN
switch(config-if)# ip address 192.168.50.2 255.255.255.0
switch(config-if)# exit

The vlan 2 & vlan 5 command creates VLAN 2 and VLAN 5, however the switch’s prompt will not change, so do not be alarmed.

Finally, we set the switch’s hostname, configure the default gateway, name-server for dns resolution and enable ip routing:

Saving Our Configuration

Saving the configuration is easily performed using the classical command:

Web Configuration

For those wishing to use the web interface to configure the switch, don’t despair as there are still plenty of features that can be configured through the web interface.  VLAN creation and IP address configuration are certainly a lot faster and easier through the CLI interface, especially if you make a mistake and need to make corrections.

To access the web interface, enter the switch’s VLAN 1 IP address as configured previously. In our example, this is 192.168.1.2.  You’ll be greeted with the login screen and prompted to enter a valid username and password. Once entered, the Getting Started screen is shown:

Assigning Ports To VLANs

The web interface provides two different ways to assign VLANs to the switch’s ports. Under the VLAN Management menu, you’ll find the Port-to-VLAN and Port-VLAN-Membership options.  The first, Port-to-VLAN option, presents all available ports and by selecting the appropriate VLAN from the top you can assign it, exclude it or make it the native VLAN for any of the selected ports:

In our example, we’ve selected VLAN 2 (VLAN ID equals 2), our Voice VLAN, and configured all but one port to carry VLAN 2 traffic as Tagged. When configuring a VLAN as Tagged traffic, the port automatically becomes a trunk port and the Trunk option above is greyed out as you cannot disable it – a logical restriction.  When configuring a VLAN to Untagged it then becomes the Native VLAN for that port. If these concepts are new, we would highly recommend you read through ourVLAN section.

We’ve configured VLAN2 traffic as Tagged, which means we plan to connect an IP Phone to these ports and from there on a PC. VLAN 1 traffic is set as the Untagged traffic, or Native VLAN for all ports.

Finally, Port GE1 is forbidden to carry VLAN2 traffic. The reason for this is that we plan to connect our Internet router on port GE1 and there is no reason for our Voice VLAN traffic to exist on that port, for security reasons of course.

When done, click on Apply to save the changes and continue with the rest of the VLAN port configuration.

The Port-VLAN-Membership menu provides an overview of all port configuration, however, changes can only be made for one port at a time:

Our screenshot shows no configuration has been made as VLAN2 and 5 are not configured for any port. Select the port of interest and click on Join VLAN at the bottom of the page:

The small pop-up window will appear in which we can select a VLAN from the area on the right (under Select VLAN:), then choose the tagging method for the selected VLAN and finally assign it to the port by clicking on the right arrow ‘>’:

Once complete, click on Apply to save the changes followed by Close to return to the menu, or select the next port to be configured from the upper area of the page.

We should note that the Port-to-VLAN is the fastest way to configure multiple switch ports simultaneously.

Configuring Voice VLAN Settings

Configuration of the Voice VLAN settings is necessary to ensure the switch understands which VLAN will carry the traffic.  Experience shows its best to specify the Voice VLAN port under these settings, rather than leave it to the switch's discretion to figure it out.

In our example, VLAN ID 2 is our Voice VLAN, so we've changed the default VLAN ID from 1, to 2 and disabled the Dynamic Auto Voice VLAN feature for security purposes.

Finally, click on Apply to save the changes.

Setting System Time

The system’s time can be configured under the Administration > Time Settings > System Time:

To configure synchronization with an NTP server, enable the Main Clock Source SNTP Servers and set the correct Time Zone Offset. Next, configure the Daylight Saving Settings and click on Apply. 

Move to SNTP Unicast menu option, enable the SNTP Client Unicast and add your preferred NTP server as shown below:

Click on Apply to save your changes.

Creating User Accounts

To create the desired user accounts to access the switch, select the Administration > User Accounts menu.

Currently there is only the default account "cisco". Click on Add and enter the username and  password:

Note the different User Levels available for the user being created. For full access, select level 15.

Once created, the cisco user can be deleted, however it is imperative the configuration is saved by clicking on the flashing Save button at the top of the page.

Defining Management Method

Our final step for the basic setup of the switch is to define the management method. The SG500 and previous models support a variety of management methods which include Telnet, SSH, HTTP, HTTPS and SNMP.

To setup your access profile, visit the Security > Mgmt Access Method > Access Profiles:

 Next, click on the Add button to add a new profile:

The pop-up window will allow you to define the access profile name, rule priority (the highest rule number takes priority over other access profiles), management method, interface to which it applies and IP Addresses to which access is allowed or denied:

In our example we named our access profile All-Access, set the rule priority to No.1 which takes precedence over other rules, management method of All, permit action, interface VLAN 1 only and source IP of 192.168.1.0 / 24, which of course is the VLAN1 network.

When complete click on Apply and Close.

This action will return you to the Access Profiles section. We now select the Active Access Profile we just created (All-Access) and click on Apply. A pop-up window will request us to confirm this action. Click on OK:

If the session is disconnected, simply reconnect to the switch using VLAN1’s IP address.

Note: If configuring the switch for Telnet or SSH remote access, it is important not to forget to enable these services from the Security > TCP/UDP Services menu option as shown below:

Again, do not forget to click on the flashing Save button at the top of the page:

Clicking on the Save button will take us to the Copy/Save Configuration page. Select Running-Configuration as the Source File Name and Startup-Configuration as the Destination File Name and hit Apply:

Configuration Backup

To download the switch's configuration to a workstation for backup purposes, select File Management > Download/Backup Configuration/Log from the main menu. Here, select HTTP method and Backup action. Finally select Running or Startup Configuration depending on your requirements and Apply.  You'll soon be prompted with the option to save the configuration file to your hard disk drive:

This completes our introduction to the SG500-52p PoE Switch and its basic configuration.

Errdisable

The error disabled  feature is supported on most Catalyst switches running the Cisco IOS software. Including all the following models:

• Catalyst 2940 / 2950 / 2960 / 2960S
• Catalyst 3550 / 3560 / 3560-E / 3750 / 3750-E
• Catalyst 4000 / 4500 / 4507R
• Catalyst 6000 / 6500

 The Errdisable error disable feature was designed to inform the administrator when there is a port problem or error.  The reasons a catalyst switch can go into Errdisable mode and shutdown a port are many and include:

• Duplex Mismatch
• Loopback Error
• Link Flapping (up/down)
• Port Security Violation
• Unicast Flodding
• UDLD Failure
• Broadcast Storms
• BPDU Guard

When a port is in error-disabled state, it is effectively shut down and no traffic is sent or received on that port. The port LED is set to the orange color and, when you issue the show interfaces command, the port status shows as Errdisabled.

Following is an example of what an error-disabled port looks like:

To recover a port that is in an Errdisable state, manual intervention is required, and the administrator must access the switch and configure the specific port with 'shutdown' followed by the 'no shutdown' command. This command sequence will enable the port again, however, if the problem persists expect to find the port in Errdisable state again soon.

Understanding And Configuring Errdisable AutoRecovery

As outlined above, there are a number of reasons a port can enter the Errdisable state.  One common reason is the Port Security error, also used in our example below.

Of all the errors, Port Security is more a feature rather than an error. Port Security allows the restriction of MAC Addresses on an interface configured as a layer 2 port. This effectively prevents others connecting unwanted hubs or switches on the network. Port Security allows us to specify a single MAC Address to be connected to a specific port, thus restricting access to a specific computer.

In the case of a violation, Port Security will automatically disable the port. This is the behaviour of the default port security policy when enabling Port Security. Following is a configuration example of port security:

Once a host is connected to the port, we can get more information on its port-security status and actions that will be taken when a violation occurs:

Note that the Violation Mode is set to Shutdown. This means that when a violation is detected, the switch will place gigabitethernet 0/48 in the err-disable shutdown state as shown below:

While it's almost always necessary to know when a port security violation occurs there are some circumstances where autorecovery is a desirable feature, especially durng accidental violations.

The following commands enable the autorecovery feature 30 seconds after a port security violation:

Determine The Reason For The Errdisabled State

To view the Errdisabled reasons, and see for which reason the autorecovery feature has been enabled, use the show Errdisable recovery command:

We have now confirmed that autorecovery is enabled for port-security violations. If it is required to enable the Errdisable autorecovery feature for all supported reasons, use the following command:

To test our configuration we forced a port security violation, causing the switch to place the offending port in the shutdown state. Notice we've enabled autorecovery for all Errdisable reasons and the time left to enable the interfaces placed in shutdown state by the port security violation:

Seventeen seconds later, the switch automatically recovered from the port security violation and re-enabled the interface:

Disabling The Errdisable Feature

There are cases where it might be necessary to disable the Errdisable mechanism for specific supported features in order to overcome constant interface shutdowns and auto recoveries.  While the Catalyst IOS does not allow disabling all features we can still fine-tune the mechanism and selectively disable a few.

To view the Errdisable reasons monitored by the switch, use the show Errdisable detect command:

2960G# show errdisable detect

ErrDisable Reason      Detection    Mode
-----------------      ---------    ----
bpduguard               Enabled      port
channel-misconfig       Enabled      port
community-limit         Enabled      port
dhcp-rate-limit         Enabled      port
dtp-flap                Enabled      port
gbic-invalid            Enabled      port
inline-power            Enabled      port
invalid-policy          Enabled      port
link-flap               Enabled      port
loopback                Enabled      port
lsgroup                 Enabled      port
mac-limit               Enabled      port
pagp-flap               Enabled      port
port-mode-failure       Enabled      port
secure-violation        Enabled      port/vlan
security-violation      Enabled      port
sfp-config-mismatch     Enabled      port
small-frame             Enabled      port
storm-control           Enabled      port
udld                    Enabled      port
vmps                    Enabled      port

As shown, the command lists all supported Errdisable reasons.  For our example, let's assume we want to disable the inline-power Errdisable feature.

To achieve this, we simply use the following command:

And verify that Errdisable has been disabled for the feature:

Overall, the Errdisable feature is an extremely useful tool if configured and monitored correctly. Take the necessary time to play around with the supported options of your Cisco Catalyst switch and fine-tune it to suit your network needs.

A frequent customer problem with Cisco's new line of Catalyst switches is that they do not support 3rd party (non-Cisco) SFPs - or at least they do not seem to...

If you've just replaced your network switches and tried using any 3rd party SFPs to connect your network backbone, you'll quickly stumble across an error similar to the following:

Congratulations!  The Catalyst switch has just disabled the GBIC port! This happens because Cisco Catalyst switches are configured by default not to work with non-Cisco SFPs.

When a SFP is inserted into a switch's GBIC port, the switch immediately reads a number of values from the SFP and if it doesn't like what it sees, it throws the above error message and disables the port.

All SFP modules contain a number of recorded values in their EEPROM and include:

• Vendor Name
• Vendor ID
• Serial Number
• Security Code
• CRC

How To Force Your Cisco Switch To Use 3rd Party SFPs

Despite the error displayed, which leaves no hope for a solution, keep smiling as you're about to be given one.

There are two undocumented commands which can be used to force the Cisco Catalyst switch to enable the GBIC port and use the 3rd party SFP:

When entering the service unsupported-transceiver command, the switch will automatically throw a warning message as a last hope to prevent the usage of a 3rd party SFP.

The no errdisable detect cause gbic-invalid command will help ensure the GBIC port is not disabled when inserting an invalid GIBC.

Since the service unsupported-transceiver  is undocumented, if you try searching for the command with the usual method (?), you won't find it:

The same applies for the no errdisable detect cause gbic-invalid command.

We tried both service unsupported-transceiver & no errdisable detect cause gbic-invalid commands on 2960G, 3560G, 3750G, 4507R and 4507R-E Catalyst switches and all accepted the commands without a problem. In fact if the Catalyst switch is running IOS 12.2(25)SE and above, the undocumented commands are available.

Should 3rd Party SFPs Be Used?

There are mixed feelings about this. We certainly do not recommend using non-Cisco SFP's in production environments, however in a lab environment, its most probably a cheap way out.

When using 3rd party GBICs, one must keep in mind that Cisco TAC will not provide any support for problems related to the SFPs as they are totally unsupported. Here is a small portion from the Cisco Catalyst 3750G Q&A that refers to the usage of 3rd party SFP modules on the switch:

Even though many Administrators and IT Managers are aware of VLAN technologies and concepts, unfortunately, it has been proven that the same does not apply when it comes to VLAN Security. While this section mainly focuses on security implemented on Cisco switches, many of the concepts can be applied on other vendor switches.

The first principle in securing a VLAN network is physical security. If you do not want your devices to be tampered with, physical access to the device must be strictly controlled.  Core switches are usually safely located in a datacenter with restricted access, however edge switches are not that lucky and are usually placed in areas where they are left exposed.

Just as physical security guidelines require equipment to be in a controlled space, VLAN-based security requires the usage of special tools and following a few ‘best security practices’ to give the desired result.

Let’s take a look at a few important steps an Administrator or IT Manager can take, to strip their network from the security problems most networks suffer today.

Removal of Console-port Cables, Introduction of Password-Protected Console/Vty Access with Specified Timeouts and Restricted Access

Console ports on the back side of Cisco switches provide direct access to the system. If no care is taken to secure this access method, then the switch might remain fully exposed to anyone with the popular ‘blue console cable’. Configuration of complex user credentials on the console and telnet/ssh ports will ensure any unwanted visitor will remain in the dark when trying to access the device.  Using special commands such as the ‘exec-timeout’ commands,  when the Administrator accidently forgets to logout of the session, it will automatically timeout after the programmed timeout value.

Following is a set of commands that will help you accomplish the above measures to help restrict access to the swich:

We also apply the same commands to our VTY (telnet/ssh) section and create an access-list 115 to restrict telnet/ssh access from specific networks & hosts:

Always ensure the use of the ‘secret’ parameter rather than the ‘password’ parameter in your username syntax, when defining usernames and their passwords. The classic ‘password’ parameter uses a much weaker encryption algorithm that is easily unencrypted.

To demonstrate this, you can use the 'password' parameter and then copy past the encrypted password into our popular Cisco Type 7 Password decrypt page and see what happens!

Avoid Using VLAN1 (Default VLAN)  for your Network Data

VLAN 1 is a special VLAN selected by design to carry specific information such as CDP (Cisco Discovery Protocol), VTP, PAgP and more.  VLAN 1 was never intended to be used as standard VLAN to carry network data.

By default configuration, any Access Link on a Cisco switch is set to VLAN 1, causing a major security issue as direct access to the network backbone is given.  As a consequence, VLAN 1 can end up unwisely spanning the entire network if not appropriately pruned.

The practice of using a potentially omnipresent VLAN for management purposes puts trusted devices to higher risk of security attacks from untrusted devices that by misconfiguration or pure accident gain access to VLAN 1 and try to exploit this unexpected security hole.

As a general rule of thumb, the network Administrator should prune any VLAN, and in particular VLAN 1 from all ports where that VLAN is not needed.

The following example prunes VLANs 1  to 5 and 7 to 8, allowing access only to VLAN 6 when in trunking mode. Furthermore, we assign the port to VLAN 6 only:

Disable High-Risk Protocols on Switchports

If a port is connected to a ‘foreign’ device, don’t try to speak its language – it could be turned to someone else’s advantage and used against your network. Ensure you disable protocols such as CDP, DTP, PAgP, UDLD (Unidirectional Link Detection Protocol)  and always enable spanning-tree  portfast & bpduguard on the port.

Here is an example on how to disable the above mentioned protocols and enable spanning-tree portfast bpduguard:

Finally, if the port is not to be used, issue the ‘shutdown’ command to ensure it won’t be accessed by anyone without the proper authorization.

VTP Domain, VTP Pruning and Password Protection

Two choices exists here – either configure the VTP domain appropriately or turn off VTP altogether!  VTP is a great tool that ensures all VLAN information is carried to your network switches. If necessary security measures are not taken, wiping your network-wide VLAN configuration is as easy as connecting a switch with the ‘proper’ devastating configuration.

 A switch configured with the same ‘VTP domain’, a role type of ‘Server’ and a higher ‘VTP revision’ number of the real VTP server (usually the core switch), is all that’s required to cause major disruption and panic across any network size. All other switches will automatically ‘listen’ to the new ‘VTP Server’ and wipe all existing VLAN information. You can then start looking for a new job.

A few simple self-explanatory commands on your core switch will help ensure the above scenario is avoided:

Edge switches will require the ‘vtp mode client’ and ‘vtp password’ command, after which they will automatically receive all necessary VLAN information from your core switch.

You can verify the configuration using the ‘show vtp status’ command:

Control Inter-VLAN Routing Using IP Access Lists

Inter-VLAN routing is a great and necessary feature. Because in many cases there is the need to isolate VLANs or restrict access between them, the usage of IP Access lists is mandatory.

IP Access lists should be created in such a way, that they allow the normal flow of traffic between VLANs, but do not expose the networks that need to be protected. Once the Access Lists are created, they are applied directly on the VLAN interface of the core layer-3 switch.   All traffic from the designated VLAN trying to pass to other VLANs will be denied according to the Access Lists, making sure the core network is not exposed. 

Let’s take a common example to make this tip more practical.

You’ve created a new guest VLAN (VLAN 6 – Network 192.168.141.0/24) to provide free Internet access to your company visitors. The requirement is to allow full Internet access, but restrict access to other VLANs.  In addition, configuration of a DHCP server is also deemed necessary, to make your life easier and less troublesome.

Here’s the configuration used for the DHCP server serving this VLAN:

Note that 192.168.141.1 is our core switch VLAN 6 IP Address, and 192.168.130.5 is our DNS server located on a different VLAN.

Next, we create our necessary Access Lists.

Notice that we permit DNS and DHCP requests initially, and then deny access to all VLANs. Finally we permit access everywhere else. This logical structure of our Access List is built to comply with the Top-Down Access List examination performed by the Core switch.

If we were to place the DNS or Bootp last in the Access List, it would clearly fail as the deny statements would prevail.  Finally, the ‘log’ parameter seen on our deny statements would trigger a log entry on our core switch, allowing us to catch any guests trying persistently to access our other VLANs

Last step would be to apply the access-list to the newly created VLAN interface, in the ‘incoming’ direction:

Summary

VLAN Technology is wonderful – it offers great enhancements to the network and provides paths to run multiple services in isolated environments without sacrificing speed, quality and network availability. If the necessary basic security guidelines are taken in consideration during its initial implementation and ongoing administration, it can perform wonders and dramatically reduce the administrative overhead from your IT Administrators or Managers. On other hand, if these security guidelines are ignored, the imminent exposure of the whole network is at risk and simply a matter of time.

Perhaps the most serious mistake that an IT Administrator or Manager can make, is to underestimate the importance of the DataLink layer, and of VLANs in particular, in the architecture of switched networks. It should not be forgotten that any network is only as robust as its weakest link, and that therefore an equal amount of attention should be given to any of its layers, to make sure that its entire structure is sound.

We recently had the chance to unpack and install a Cisco Catalyst 4507R-E Layer 3 switch, which we must admit was extremely impressive. The Cisco Catalyst series is world-known for its superior network performance and modularity that allows it to 'adapt' to any demands your network might have.

For those who haven't seen or worked with a 4507R/4507R-E switch, it's a very big and heavy switch in a metal cabinet (chassis) supporting up to two large power supplies and a total of 7 cards (modules), two of which are the supervisor engines that do all the switching and management work.

The new 4507R-E series is a mammoth switch that allows a maximum of 320Gbps (full duplex) switching capacity by utilising all 7 slots, in other words 5 modules alongside with two Supervisor Engine 6-E cards (with two full line rate 10Gb Uplinks).

The 4507R-E switch is shipped in a fairly large box 50(H)x44(W)x32(D) cm and weights around 21 Kgrs with its shipping box. The practical height of the unit for a rack is 11U which means you need quite a bit of room to make sure it's comfortably placed.

Unboxing the Cisco Catalyst 4507R

Like most Cisco engineers, we couldn't wait to open the heavy box and smell the freshly packaged item that came directly from Cisco's manufacturing line. We carefully moved the 4507R-E box to the datacenter and opened the top side of the box.....

The upper area of the picture is where you'll find the two large cube slots for the power supplies. Below them, you can identify 6 out of the 7 slots waiting to be populated and give this monster unbelievable functionality!

After opening the package and removing the plastic wrapping, we placed the switch on the floor so we could take a better look at it.

Because we couldn't wait any longer, we quickly opened one of two power supplies and inserted it into the designated slot. The power supplies installed were rated at 2800Watts each - providing more than enough juice to power a significant number of IP Phones via the PoE cards installed later on.

The picture below shows both power supplies, one inserted into its slot, while the other was placed on top of the chassis with its connectors facing frontwards so you can get a glimpse of them. When inserted into its slot, the power supply's bottom connectors plug firmly into the chassis connectors and power up the Catalyst switch:

Turning on the power supplies for the first time made the datacenter's light dim instantantly as they began to draw power for the first time! Interestingly enough, if you take a look at the power supply on top of the chassis, you'll notice three long white strips inside the power supply. These are actually three very large electrolytic capacitors - quite impressive!

For those interested, the power supplies were made by Sony (yes, they had a Sony sticker on them!).

Supervisor Engine Line Card Installation

As we mentioned in the beginning of this article, the powering engine of any 4500 series Catalyst switch is the Supervisor Engine. The Supervisor engines occupy up to two slots on the 4507R chassis, one of them used for redundancy in case the other fails. When working with two supervisor engines, the 4507R is usually configured to automatically switch from one engine to the other without network interruptions, even for a VoIP network with active calls between ends.

Cisco currently has around 7 different Supervisor Engines, each with unique characteristics, designed for various levels of density and bandwidth requirements.

Currently, the Supervisor Engine 6-E is the best performing engine available, providing 320Gbps bandwidth (full duplex) and 250 million packets per second forwarding rate!

Our users can refer to our popular Cisco Catalyst 4500 Series Zero-Downtime IOS Upgrade Process for Supervisor Engine 7-E, 7L-E, 6L-E and V-10GE Redundant Configurations article to learn how to upgrade their Supervisor Engine without network service interruption.

For our installation, we worked with the Supervisor Engine II-Plus, also known as Cisco part WS-X4013+. Here's one of the supervisor engines in its original antistatic bag:

After placing on my wrist the antistatic wrist-strap contained in the package and carefully unwrapping the supervisor engine, the green circuit-board with its black towers (heatsinks) is revealed. You can easily see the 5 heatsinks, two of which are quite large and do an excellent job in keeping the processors cool:

At the back left side of the board, you can see the supervisor engine's connector which is equally impressive with 450 pin connectors - 50 on each row!

We took a picture from the back of the board to make sure the connector was clearly visible:

Just looking at the connector makes you imagine the number of signals that pass through it to give the 4507R-E the performance rating it has! On the left of the board's connector is the engine's RAM (256MB), while right behind it is the main CPU with the large heatsink, running at 266Mhz.

Here is a close up of the engine's RAM module. The existing 256MB memory module can be removed and upgraded according to your requirements:

Moving to the front side of the Supervisor Engine, you can see the part number and description:

The uplink ports visible on the front are GBIC (GigaBit Interface Converter) that can be used as normal Gigabit interfaces. By using different GBIC's you can connect multimode, singlemode fiber optic cable or standard CAT5e/CAT6 Ethernet cabling. These ports can come in handy when you're approaching your switch's full capacity.

The impressive Supervisor Engine fits right into one of the two dedicated slots available on the 4507R-E chassis. These are slots 3 & 4 as shown in the picture below. Also visible is the switch's backplane and black connectors awaiting the Supervisor Engine boards (marked with red):

We took another picture inside the chassis to make things as clear as possible:

Here you can see the backplane with the two Supervisor Engine connectors. The white coloured connectors just above and below the Supervisor Engines are used by the rest of the boards available to the 4507R.

After inserting one of the Supervisor Engines and two power supplies, here is the result:

One detail well worth noticing is the colour coded bars on the left and right side of the Supervisor card. These colour codes exist to ensure engineers don't accidently try to insert a Supervisor card into an inappropriate slot. The 4507R-E can accept upto two supervisor engines, therefore you have two slots dedicated to them, leaving 5 slots available.

Cisco engineers have thought of everything on the 4507R-E. The cooling mechanisim is a good example of smart-thinking and intelligent engineering. With 7 cards installed on the system, pumping a generous amount of heat, the cooling had to be as effective as possible. Any heat captured between the cards could inadvertably lower the components' reliability and cause damage in the long term.

This challenge was dealt with by placing a fan-tray right next to the cards in a vertical direction. The fan-tray is not easily noticed when taking a quick glance, but the available handle on the front gives away that something is hidden in there. Unscrew the top & bottom bolts, place your hand firmly around the handle and pulling outwards will suprise you:

 The picture taken on the left shows the eight fans placed on the fan-tray. These fans work full speed the moment you power the switch on, consuming 140Watts alone!

Once they start spinning, you really can't argue that the cooling is inadequate, as the air flow produced is so great that when we powered the 4507R-E, the antistatic bags accidently forgotten on the right hand side of the chassis were sucked almost immediately against the chassis grip, just at it happens when you leave a plastic bag behind a powerful fan!

Of course, anything on the left side of the chassis (vieweable in our picture) would be immediately blown away.

After inserting the fan-tray back in place, it was time to take a look around and see what else what left to play with.

Our eyes caught another Cisco box and we approached it, picked it up and checked out the label:

The product number WS-X4548-GB-RJ45V and size of the package made it clear we were looking at a card designated for the 4507R-E. Opening the package confirmed our thoughts - this was a 48port Gigabit card with PoE support:

We carefully unwrapped the contents always using our antistatic wrist-strap so that we don't damage the card, and then placed it on top of its box:

The card has an impressive quantity of heatsinks, two of which are quite large and therefore must generate a lot of heat. The backplane connector is visible with its white colour (back left corner), and right behind the 48 ports is an area covered with a metallic housing. This attracted our attention as we thought something very senstive must be in that area for Cisco to protect it in such a way.

Taking a look under the protective shield we found a PCB board that ran along the length of the board:

Our understanding is that this rail of PCB with transistors and other electrical circuits mounted on it seemed to be regulators for the PoE support. Taking into consideration that we didn't see the same protection in other similar non-PoE boards, we couldn't image it being something else.

When we completed our checkup, we decided it was time to install the card and finally power the 4507R-E switch.

The picture on the left shows our 4507R-E installed with two Supervisor Engine II-Plus engines in active-standby redundancy mode and one 48 port Gigabit Ethernet card with PoE support.

On top is the editor's (Chris Partsenidis) laptop with a familair website loaded, Firewall.cx!

Configuring the Supervisor engines was a simple task. When the 4507R-E is powered on, both engines will boot by first performing a POST test on their modules, memory buffers etc. When this internal POST phase is successfully complete without errors, the engines begin to boot the IOS.

The screenshot below shows us the described procedure from one Supervisor engine since you can't monitor both engines unless you have one serial port connected to each supervisor's console port:

As shown above, the Supervisor engine passed all tests and then proceeded to boot the IOS.

Once loaded, the IOS will check for the existence of a second Supervisor engine, establish connection with it and, depending on which slot it is located in, it will automatically initialise the second engine in standby mode as shown below:

 Once the Supervisor engine bootup process is complete, you are able to configure any aspect of the switch according to your needs, just as you would with any other Cisco Catalyst switch. The interesting part is when you try to save your configuration:

In the above screenshot, we've configured the switch to boot using a specific IOS located in the bootflash, as soon as we saved the configuration using the wr command, the Supervisor engine automatically synchronised the two engines' nvram without any additional commands. This excellent functionality makes sure that whatever configuration is applied to the active Supervisor engine will be available to the standby engine should the first one fail.

The great part of this switch is that you can obtain any type of information you require from it. For example, we switched off one of the two power supplies and executed the show modules command. This command gives a report of the installed modules (cards) in the catalyst switch along with a few more details:

The command reveals that the backplane power consumption is approximately 40 Watts followed by a detailed report of the installed modules. In our example, you can see the two Supervisor engines in slot 3 & 4, followed by the 48 Gigabit Ethernet module in slot 5. The command also shows the Supervisor engines' configured redundany operating mode and status. Lastly, any system failures are reported at the end - this output shows that we've got a problem with one of the power supplies, but rest assured, we had simply switched it off to see if it was going to show up in the report!

Summary

This article covered the initial installation and setup of a new Cisco Catalyst 4507R-E switch, populated with two Supervisor Engines II-Plus and a 48 port Gigabit module with PoE support. We saw areas of the switch which you won't easily find elsewhere and our generous amount of pictures made sure you understood what the 4507R-E looks like, inside and out! Lastly, we saw the switch bootup procedure and Supervisor engine POST test and syncronization process.