Cisco Data Center (Nexus/UCS)

Nexus 7000/7700 Software Upgrade via ISSU. Complete Upgrade Guide, Configuration Check, Verifying ISSU Capability

2018-09-13T18:50:23+10:00

This article shows how to perform an ISSU (In-Service Software Upgrade) on a Nexus Data Center switch (7000 and 7700 models) and avoid service and network disruption. We explain the importance of keeping your NX-OS software updated, how the upgrade process is executed, explain the purpose of the Kickstart and System images, provide methods on how to transfer the NX-OS images to the switch bootflash on both supervisor engines, verify ISSU capability and test/simulate the upgrade process.

In addition we cover useful commands to discover issues that might occur during the upgrade process, configuration backup methods, upgrading a Nexus 7000 and Nexus 7700 series with single or dual Supervisor Engines (SUP1 and SUP2 models).

Key Topics:

Why Upgrade Your Nexus 7000/7700 NX-OS Software

Upgrading your NX-OS can be a daunting task as there is always the risk something might go wrong. Despite this, it is very important to ensure your core Nexus switch is running one of the latest and supported images.

If you’re looking for reasons why to take the risk and upgrade, here are a few that might help convince:

Old NX-OS images might be stable but usually contain a number of bugs and security vulnerabilities that can put your core network and organization in risk.
Your NX-OS version might not be supported any more. This means that in an event of a failure or problem, Cisco Technical Assistance Center (TAC) might require you to upgrade to a supported NX-OS version before providing any support.
Support of new features, services and technologies. By upgrading to a newer NX-OS you’ll be able to take advantage of newer features that will now be supported.
Support of new Modules and Supervisor Engines. When considering upgrading your Nexus Supervisor Engines or adding new modules it’s likely an upgrade will be required to support them.
Peace of Mind. Knowing you’re on a supported, tested and patched up version always helps sleeping better at night!

It’s always recommended to perform a thorough research of the NX-OS version under consideration to identify caveats or issues that might affect your production environment. This information can be found on Cisco’s website or by opening a Cisco TAC Service Request.

What is an ISSU Upgrade?

The ISSU upgrade process provides us with the ability to upgrade a Nexus 7000/7700 switch without network or service disruption. During the ISSU process all Nexus modules and Supervisor Engines are fully upgraded without requiring a switch reboot.

Prerequisite for the ISSU upgrade is to have Dual Supervisor Engines and have an ISSU supported release loaded on your Nexus switch. The Dual Supervisor Engines are necessary as the ISSU process upgrades one Supervisor Engine at a time to keep the system up and running.

Cisco publishes a list of ISSU supported releases for every new NX-OS release. This means engineers should check the release notes of the candidate release they wish to upgrade to and see if their current version is amongst the ISSU supported releases.

Finally, an ISSU upgrade might be disruptive if there are configured features that are not supported on the new software image. We’ll show how you can test the ISSU upgrade process before initiating it.

How The ISSU Upgrade Works

Below is the process an ISSU upgrade follows on a Nexus 7000 with dual supervisor engines:

Installation begins with the install all command.
The installation process will verify the location and integrity of the new software image files.
System verifies the operational status and the current software version of both supervisor engines and all switching modules to ensure that the system is capable of an ISSU.
System initially upgrades all module cards bios/loader/bootrom.
System loads the new software images to the standby supervisor engine and brings it up to the HA ready state.
A supervisor switchover is then forced.
The new software image is loaded on the formerly active (now standby) supervisor and brings it up to the HA ready state.
A non-disruptive upgrade is performed on each of the switching modules starting from module 1.
Finally, on a Nexus 7000 with SUP-1 supervisor engines, each Connectivity Management Processor (CMP) is upgraded one at a time.

During the ISSU upgrade the switch provides continuous update of its progress and no command input is possible until the upgrade has been completed.

The ISSU upgrade can be initialled via a SSH or Telnet session to the Nexus switch or directly from the active supervisor engine console port.

When a supervisor switchover occurs, it’s possible the SSH/Telnet session will be lost but you can re-connect immediately and continue to monitor the upgrade process by issuing the show install all status command. Alternatively connect to both supervisor engine console ports simultaneously.

Understanding Nexus Kickstart and System Images

The Nexus 7000 requires two images in order to run. First is a Kickstart image while the second is a System image.

Here’s what they look like:

NEXUS_7000# dir bootflash://sup-1/
392990621 Sep 08 11:14:54 2018 n7000-s1-dk9.6.2.20a.bin
31178240 Sep 08 11:01:10 2018 n7000-s1-kickstart.6.2.20a.bin

The Kickstart image is around 31Mb-70Mb in size depending on the NX-OS version and contains the Linux kernel, basic drivers and initial file system. The System image is much larger, around 400Mb-650Mb in size depending on the NX-OS version and contains the system software and infrastructure code.

Transferring Images to Nexus 7000/7700 Switch

FTP is the recommended method of transfer. Thanks to the TCP transport protocol utilized by the FTP protocol, it is highly unlikely the image integrity will be compromised during the transfer.

Following are the possible locations where the image files can be stored on the Nexus:

bootflash: or bootflash://sup-1/ (essentially the same location on supervisor engine 1)
bootflash://sup-2/ This is the supervisor engine 2 bootflash

Image transfer can be initiated using the ftp command as shown in the example below:

NEXUS_7000# copy ftp://192.168.1.1/n7000-s1-kickstart.6.2.20a.bin bootflash:
Enter vrf (If no input, current vrf 'default' is considered):
Enter username: cisco
Password:
[################# ] 29.55MB ***** Transfer of file Completed Successfully *****
Copy complete, now saving to disk (please wait)...

At this point we can transfer the image to the supervisor engine 2 bootflash using the command:

NEXUS_7000# copy ftp://192.168.1.1/n7000-s1-kickstart.6.2.20a.bin bootflash://sup-2/

or copy the image directly from supervisor engine 1 with the following command:

NEXUS_7000# copy bootflash://sup-1/n7000-s1-kickstart.6.2.20a.bin bootflash://sup-2/

The second method is faster and preferred.

Configuration Backup

Creating a configuration backup should be mandatory in any upgrade process. This can be achieved via a simple show running-configuration and copy-pasting the output to a text file, or using the Nexus CheckPoint feature.

Incompatible Configuration Check, Verifying ISSU Capability, Testing The Upgrade Process

Verifying the upgrade and ISSU process is an extremely important step and should never be skipped. Cisco’s release notes clearly state the supported ISSU Paths however executing the test commands shown in this section will reveal any incompatible configuration and provide a complete insight of what will happen during the upgrade.

Here are the two commands highly recommended to be executed before the upgrade. Both are non-disruptive:

show incompatibility system bootflash:<image> . Performs a configuration compatibility check that will highlight any configuration or features that might impact the upgrade process.
show install all impact kickstart bootflash:<image> system bootflash:<image> . Performs a simulated upgrade that verifies the new firmware integrity, ISSU upgrade process, provides detailed report of which module images will be upgraded and more.

Below is the output of each command. In this particular environment we are checking an upgrade from NX-OS 6.2(16) to 6.2(20a):

NEXUS_7000# show incompatibility system bootflash:n7000-s1-dk9.6.2.20a.bin
Checking incompatible configuration(s)
No incompatible configurations
Checking dynamic incompatibilities:
-----------------------------------
No incompatible configurations

The system has reported there are no issues with our configuration. Next, we execute a test/simulation of the upgrade process:

The show install all impact command will take a long time to complete as it simulates the upgrade process.

PH_NEXUS_7000# show install all impact kickstart bootflash:n7000-s1-kickstart.6.2.20a.bin system bootflash:n7000-s1-dk9.6.2.20a.bin

Installer will perform impact only check. Please wait.

Verifying image bootflash:/n7000-s1-kickstart.6.2.20a.bin for boot variable "kickstart".

[####################] 100% -- SUCCESS

Verifying image bootflash:/n7000-s1-dk9.6.2.20a.bin for boot variable "system".

[####################] 100% -- SUCCESS

Performing module support checks.

[####################] 100% -- SUCCESS

Verifying image type.

[# [####################] 100% -- SUCCESS

Extracting "lc1n7k" version from image bootflash:/n7000-s1-dk9.6.2.20a.bin.

[####################] 100% -- SUCCESS

Extracting "bios" version from image bootflash:/n7000-s1-dk9.6.2.20a.bin.

[####################] 100% -- SUCCESS

Extracting "system" version from image bootflash:/n7000-s1-dk9.6.2.20a.bin.

[####################] 100% -- SUCCESS

Extracting "kickstart" version from image bootflash:/n7000-s1-kickstart.6.2.20a.bin.

[####################] 100% -- SUCCESS

Extracting "cmp" version from image bootflash:/n7000-s1-dk9.6.2.20a.bin.

[[####################] 100% -- SUCCESS

Extracting "cmp-bios" version from image bootflash:/n7000-s1-dk9.6.2.20a.bin.

[[####################] 100% -- SUCCESS

Notifying services about system upgrade.

[####################] 100% -- SUCCESS

Compatibility check is done:

Module bootable Impact Install-type Reason

------ -------- -------------- ------------ ------

1 yes non-disruptive rolling

2 yes non-disruptive rolling

3 yes non-disruptive rolling

4 yes non-disruptive rolling

5 yes non-disruptive reset

6 yes non-disruptive reset

7 yes non-disruptive rolling

8 yes non-disruptive rolling

9 yes non-disruptive rolling

10 yes non-disruptive rolling

The key column here is the Impact column at the end. This confirms that our upgrade to the new image will be non-disruptive for every module including the supervisor engines (Modules 5 & 6). Supervisor engines will be reset (one at a time), but an automatic switchover between them will ensure there is no service disruption.

Performing the Nexus ISSU Upgrade

Once we are confident and ready we can proceed with the upgrade using the install all command. Keep in mind that if you’re connected via SSH, you’ll be disconnected from the SSH session as soon as the active supervisor engine is rebooted.

To reconnect and continue monitoring the installation process, SSH back in and issue the show install all status command. Alternatively you’ll need to be connected to both supervisor engines console ports.

Once the initial checks are complete, you’ll be promoted to confirm with a "y" (yes) to continue with the installation

NEXUS_7000# install all kickstart bootflash:n7000-s1-kickstart.6.2.20a.bin system bootflash:n7000-s1-dk9.6.2.20a.bin

Installer will perform compatibility check first. Please wait.

Verifying image bootflash:/n7000-s1-kickstart.6.2.20a.bin for boot variable "kickstart".

[####################] 100% -- SUCCESS

Verifying image bootflash:/n7000-s1-dk9.6.2.20a.bin for boot variable "system".

[####################] 100% -- SUCCESS

Performing module support checks.

[####################] 100% -- SUCCESS

Verifying image type.

[# [####################] 100% -- SUCCESS

Extracting "lc1n7k" version from image bootflash:/n7000-s1-dk9.6.2.20a.bin.

[####################] 100% -- SUCCESS

Extracting "bios" version from image bootflash:/n7000-s1-dk9.6.2.20a.bin.

[####################] 100% -- SUCCESS

Extracting "system" version from image bootflash:/n7000-s1-dk9.6.2.20a.bin.

[####################] 100% -- SUCCESS

Extracting "kickstart" version from image bootflash:/n7000-s1-kickstart.6.2.20a.bin.

[####################] 100% -- SUCCESS

Extracting "cmp" version from image bootflash:/n7000-s1-dk9.6.2.20a.bin.

[[####################] 100% -- SUCCESS

Extracting "cmp-bios" version from image bootflash:/n7000-s1-dk9.6.2.20a.bin.

[[####################] 100% -- SUCCESS

Notifying services about system upgrade.

[####################] 100% -- SUCCESS

Compatibility check is done:

Module bootable Impact Install-type Reason

------ -------- -------------- ------------ ------

1 yes non-disruptive rolling

2 yes non-disruptive rolling

3 yes non-disruptive rolling

4 yes non-disruptive rolling

5 yes non-disruptive reset

6 yes non-disruptive reset

7 yes non-disruptive rolling

8 yes non-disruptive rolling

9 yes non-disruptive rolling

10 yes non-disruptive rolling

Images will be upgraded according to following table:

Module Image Running-Version(pri:alt) New-Version Upg-Required

------ ---------- ---------------------------------------- -------------------- ------------

1 lc1n7k 6.2(16) 6.2(20a) yes

1 bios v1.10.13(03/15/10):v1.10.13(03/15/10) v1.10.21(11/26/12) yes

2 lc1n7k 6.2(16) 6.2(20a) yes

2 bios v1.10.13(03/15/10):v1.10.13(03/15/10) v1.10.21(11/26/12) yes

3 lc1n7k 6.2(16) 6.2(20a) yes

3 bios v1.10.13(03/15/10):v1.10.13(03/15/10) v1.10.21(11/26/12) yes

4 lc1n7k 6.2(16) 6.2(20a) yes

4 bios v1.10.11(11/24/09):v1.10.11(11/24/09) v1.10.21(11/26/12) yes

5 system 6.2(16) 6.2(20a) yes

5 kickstart 6.2(16) 6.2(20a) yes

5 bios v3.22.0(02/20/10): v3.22.0(02/20/10) v3.22.0(02/20/10) no

5 cmp 6.2(16) 6.2(20a) yes

5 cmp-bios 02.01.05 02.01.05 no

6 system 6.2(16) 6.2(20a) yes

6 kickstart 6.2(16) 6.2(20a) yes

6 bios v3.22.0(02/20/10): v3.22.0(02/20/10) v3.22.0(02/20/10) no

6 cmp 6.2(16) 6.2(20a) yes

6 cmp-bios 02.01.05 02.01.05 no

7 lc1n7k 6.2(16) 6.2(20a) yes

7 bios v1.10.13(03/15/10):v1.10.13(03/15/10) v1.10.21(11/26/12) yes

8 lc1n7k 6.2(16) 6.2(20a) yes

8 bios v1.10.13(03/15/10):v1.10.13(03/15/10) v1.10.21(11/26/12) yes

9 lc1n7k 6.2(16) 6.2(20a) yes

9 bios v1.10.13(03/15/10):v1.10.13(03/15/10) v1.10.21(11/26/12) yes

10 lc1n7k 6.2(16) 6.2(20a) yes

10 bios v1.10.11(11/24/09):v1.10.11(11/24/09) v1.10.21(11/26/12) yes

Do you want to continue with the installation (y/n)? [n] y

Install is in progress, please wait.

Performing runtime checks.

[####################] 100% -- SUCCESS

Syncing image bootflash:/n7000-s1-kickstart.6.2.20a.bin to standby.

[####################] 100% -- SUCCESS

Syncing image bootflash:/n7000-s1-dk9.6.2.20a.bin to standby.

[####################] 100% -- SUCCESS

Setting boot variables.

[####################] 100% -- SUCCESS

Performing configuration copy.

[[####################] 100% -- SUCCESS

Module 1: Upgrading bios/loader/bootrom.

Warning: please do not remove or power off the module at this time.

[####################] 100% -- SUCCESS

Module 2: Upgrading bios/loader/bootrom.

Warning: please do not remove or power off the module at this time.

[[####################] 100% -- SUCCESS

Module 3: Upgrading bios/loader/bootrom.

Warning: please do not remove or power off the module at this time.

[####################] 100% -- SUCCESS

Module 4: Upgrading bios/loader/bootrom.

Warning: please do not remove or power off the module at this time.

[[####################] 100% -- SUCCESS

Module 5: Upgrading bios/loader/bootrom.

Warning: please do not remove or power off the module at this time.

[####################] 100% -- SUCCESS

Module 6: Upgrading bios/loader/bootrom.

Warning: please do not remove or power off the module at this time.

[####################] 100% -- SUCCESS

Module 7: Upgrading bios/loader/bootrom.

Warning: please do not remove or power off the module at this time.

[####################] 100% -- SUCCESS

Module 8: Upgrading bios/loader/bootrom.

Warning: please do not remove or power off the module at this time.

[[####################] 100% -- SUCCESS

Module 9: Upgrading bios/loader/bootrom.

Warning: please do not remove or power off the module at this time.

[####################] 100% -- SUCCESS

Module 10: Upgrading bios/loader/bootrom.

Warning: please do not remove or power off the module at this time.

[[####################] 100% -- SUCCESS

,,,,

Module 6: Waiting for module online.

-- SUCCESS

Notifying services about the switchover.

[####################] 100% -- SUCCESS

Module 6: <Sat Sep 8 12:59:40>

Waiting for module online.

-- SUCCESS

Module 1: <Sat Sep 8 13:08:54>

Non-disruptive upgrading.

-- SUCCESS

Module 2: <Sat Sep 8 13:10:48>

Non-disruptive upgrading.

-- SUCCESS

Module 3: <Sat Sep 8 13:12:43>

Non-disruptive upgrading.

-- SUCCESS

Module 4: <Sat Sep 8 13:14:38>

Non-disruptive upgrading.

-- SUCCESS

Module 7: <Sat Sep 8 13:16:36>

Non-disruptive upgrading.

-- SUCCESS

Module 8: <Sat Sep 8 13:18:34>

Non-disruptive upgrading.

-- SUCCESS

Module 9: <Sat Sep 8 13:20:30>

Non-disruptive upgrading.

-- SUCCESS

Module 10: <Sat Sep 8 13:22:29>

Non-disruptive upgrading.

-- SUCCESS

Module 6: <Sat Sep 8 13:24:25>

Upgrading CMP image.

Warning: please do not reload or power cycle CMP module at this time.

-- SUCCESS

Module 5: <Sat Sep 8 13:29:26>

Upgrading CMP image.

Warning: please do not reload or power cycle CMP module at this time.

-- SUCCESS

Recommended action::

"Please reload CMP(s) manually to have it run in the newer version.".

Install has been successful.

Nexus Upgrade Verification – Supervisor Engines and Modules

Once the installation is complete, a simple show version will verify the Nexus switch operating system has been upgraded:

NEXUS_7000# show version

Cisco Nexus Operating System (NX-OS) Software

TAC support: http://www.cisco.com/tac

Documents: http://www.cisco.com/en/US/products/ps9372/tsd_products_support_series_home.html

The copyrights to certain works contained in this software are

owned by other third parties and used and distributed under

license. Certain components of this software are licensed under

the GNU General Public License (GPL) version 2.0 or the GNU

Lesser General Public License (LGPL) Version 2.1. A copy of each

such license is available at

http://www.opensource.org/licenses/gpl-2.0.php and

http://www.opensource.org/licenses/lgpl-2.1.php

Software

BIOS: version 3.22.0

kickstart: version 6.2(20a)

system: version 6.2(20a)

BIOS compile time: 02/20/10

kickstart image file is: bootflash:///n7000-s1-kickstart.6.2.20a.bin

kickstart compile time: 8/10/2018 12:00:00 [07/16/2018 16:23:37]

system image file is: bootflash:///n7000-s1-dk9.6.2.20a.bin

system compile time: 8/10/2018 12:00:00 [07/16/2018 17:36:43]

In addition, the show module command will provide detailed information on what modules are installed and their software version which should match the newly installed software:

NEXUS_7000# show module

Mod Ports Module-Type Model Status

--- ----- ----------------------------------- ------------------ ----------

1 48 10/100/1000 Mbps Ethernet XL Module N7K-M148GT-11L ok

2 48 10/100/1000 Mbps Ethernet XL Module N7K-M148GT-11L ok

3 48 10/100/1000 Mbps Ethernet XL Module N7K-M148GT-11L ok

4 48 1000 Mbps Optical Ethernet XL Modul N7K-M148GS-11L ok

5 0 Supervisor Module-1X N7K-SUP1 active *

6 0 Supervisor Module-1X N7K-SUP1 ha-standby

7 32 10 Gbps Ethernet XL Module N7K-M132XP-12L ok

8 48 10/100/1000 Mbps Ethernet XL Module N7K-M148GT-11L ok

9 32 10 Gbps Ethernet XL Module N7K-M132XP-12L ok

10 48 1000 Mbps Optical Ethernet XL Modul N7K-M148GS-11L ok

Mod Sw Hw

--- -------------- ------

1 6.2(20a) 1.0

2 6.2(20a) 1.0

3 6.2(20a) 1.0

4 6.2(20a) 1.1

5 6.2(20a) 2.0

6 6.2(20a) 2.0

7 6.2(20a) 1.3

8 6.2(20a) 1.0

9 6.2(20a) 1.5

10 6.2(20a) 1.5

Finally, you can view the installation log using the show install all status command.

Summary

This article explained the importance of upgrading your Cisco Nexus 7000/7700 NX-OS operating system, what an ISSU (In-Service Software Upgrade) is, how ISSU works and the steps involved during the process. We also talked about the purpose and role of the Nexus Kickstart image and Nexus System image and showed how to transfer an image to your Nexus 7000/7700 switch.

In addition, we touched on different methods to backup your Nexus configuration and perform the important configuration compatibility check, ISSU capability verification and test the upgrade process. We finally saw how the Nexus ISSU upgrade is performed along with the upgrade verification process which includes supervisor engines and modules installed.

The Complete Cisco Nexus vPC Guide. Features & Advantages, Design Guidelines, Configuration, Failure Scenarios, Troubleshooting, VSS vs vPC

2017-08-09T09:03:49+10:00

Cisco virtual Port Channel (vPC) is a virtualization technology, launched in 2009, which allows links that are physically connected to two different Cisco Nexus Series devices to appear as a single port channel to a third endpoint. The endpoint can be a switch, server, router or any other device such as Firewall or Load Balancers that support the link aggregation technology (EtherChannel).

To correctly design and configure vPC one must have sound knowledge of the vPC architecture components (vPC Domain, vPC Peer, vPC Peer-Link, vPC Peer Keepalive Link, vPC Member Port, vPC Orphan Port etc) but also follow the recommended design guidelines for the vPC Peer Keepalive Link and vPC Peer-Link. Furthermore, understanding vPC failure scenarios such as vPC Peer-Link failure, vPC Peer Keepalive Link failure, vPC Peer Switch failure, vPC Dual Active or Split Brain failure will help plan ahead to minimise network service disruption in the event of a link or device failure.

All the above including verifying & troubleshooting vPC operation are covered extensively in this article making it the most comprehensive and complete Cisco Nexus vPC guide.

The diagram below clearly illustrates the differences in both logical and physical topology between a non-vPC deployment and a vPC deployment:

vPC Deployment Concept

The Cisco Nexus vPC technology has been widely deployed and in particular by almost 95% of Cisco Data Centers based on information provided by the Cisco Live Berlin 2016. In addition, virtual Port Channel was introduced in NX-OS version 4.1(4) and is included in the base NX-OS software license. This technology is supported on the Nexus 9000, 7000, 5000 and 3000 Series.

Let's take a look at the vPC topics covered:

We must point out that basic knowledge of the Cisco NX-OS is recommended for this article. You can also refer to our Introduction to Nexus Family – Nexus OS vs Catalyst IOS for an introduction study on the Nexus Series switches family. Finally, a Quiz is included at the last section and we are waiting for your comments and answers!

Additional related articles:

vPC Feature Overview & Guidelines

The Nexus 9000, 7000, 5000 and 3000 series switches take port-channel functionality to the next level by enabling links connected to different devices to aggregate into a single, logical link. The peer switches run a control protocol that synchronizes the state of the port channel and maintains it. In particular, the vPC belongs to the Multichassis EtherChannel (MEC) family of technology and provides the following main technical benefits:

Eliminates Spanning Tree Protocol (STP) blocked ports
Uses all available uplink bandwidth
Allows dual-homed servers (dual uplinks) to operate in active-active mode
Provides fast convergence upon link or device failure
Offers dual active/active default gateways for servers
Maintains independent control planes
Simplifies Network Design

The following general guidelines and recommendations should be taken into account when deploying vPC technology at a Cisco Nexus Data Center:

The same type of Cisco Nexus switches must be used for vPC pairing. It is not possible to configure vPC on a pair of switches consisting of a Nexus 7000 series and a Nexus 5000 series switch. vPC is not possible between a Nexus 5000 and Nexus 5500 switches.
The vPC peers must run the same NX-OS version except during the non-disruptive upgrade, that is, In-Service Software Upgrade (ISSU).
The vPC Peer-Link must consist of at least two 10G Ethernet ports in dedicated mode. Utilizing Ethernet ports from two different modules will improve the availability and redundancy should a module fail. Finally the use of 40G or 100G interfaces for vPC links will increase the bandwidth of the vPC Peer-Link.
vPC keepalive link must be separate from the vPC Peer-Link.
vPC can be configured in multiple VDCs, but the configuration is entirely independent. In particular, each VDC for the Nexus 7000 Series switches requires its own vPC peer and keepalive links and cannot be shared among the VDCs.
The maximum number of switches in a vPC domain is two.
The maximum number of vPC peers per switch or VDC is one.
When Static routing from a device to vPC peer switches with next hop, FHRP virtual IP is supported.
Dynamic routing adjacency from vPC peer switches to any Layer3 device connected on a vPC is not supported. It is recommended that routing adjacencies are established on separate routed links.
vPC member ports must be on the same line card type e.g. M2 type cards at each end.

vPC Architecture Components – vPC Peer, Peer-Link, Keepalive Link, Domain, Member Port, Orphan Port & Member

vPC architecture consists of the following components:

vPC Peer

This is the adjacent device, which is connected via the vPC Peer-link. A vPC setup consists of two Nexus devices in a pair. One acts as the Primary and the other as a Secondary, which allows other devices to connect to the two chassis using Multi-Channel Ethernet (MEC).

vPC Architecture Components

vPC Peer-link

The vPC peer-link is the most important connectivity element in the vPC setup. This link is used to synchronize the state between vPC peer devices via vPC control packets which creates the illusion of a single control plane. In addition the vPC peer-link provides the necessary transport for multicast, broadcast, unknown unicast traffic and for the traffic of orphaned ports. Finally, in the case of a vPC device that is also a Layer 3 switch, the peer-link carries Hot Standby Router Protocol (HSRP) packets.

vPC Peer Keepalive Link

The Peer Keepalive Link provides a Layer 3 communications path that is used as a secondary test in order to determine whether the remote peer is operating properly. In particular, it helps the vPC switch to determine whether the peer link itself has failed or whether the vPC peer is down. No data or synchronization traffic is sent over the vPC Peer Keepalive Link—only IP/UDP packets on port 3200 to indicate that the originating switch is operating and running vPC. The default timers are an interval of 1 second with a timeout of 5 seconds.

vPC Domain

This is the common domain configured across two vPC peer devices and this value identifies the vPC. A vPC domain id per device is permitted.

vPC Member Port

This is the interface that is a member of one of the vPCs configured on the vPC peers.

Cisco Fabric Services (CFS)

This protocol is used for stateful synchronization and configuration. It utilizes the peer link and does not require any configuration by the administrators. The Cisco Fabric Services over Ethernet protocol is used to perform compatibility checks in order to validate the compatibility of vPC member ports to form the channel, to synchronize the IGMP snooping status, to monitor the status of the vPC member ports, and to synchronize the Address Resolution Protocol (ARP) table.

Orphan Device

This is a device that is on a VPC VLAN but only connected to one VPC peer and not to both.

Orphan Port

An orphan port is an interface that connects to an orphan device vPC VLAN.

non-vPC VLAN

Any of the STP VLANs not carried over the peer-link.

Virtual Port Channel (vPC - Nexus) vs Virtual Switching System (VSS - Catalyst)

Virtual Switching System (VSS) is a virtualization technology that pools multiple Cisco Catalyst Switches into one virtual switch, increasing operational efficiency, boosting nonstop communications, and scaling system bandwidth capacity. VSS was first available in the Cisco 6500 series and was later introduced to the Cisco 4500, the newer 4500X, 6800 Series switches and the Catalyst 3850 (April 2017 onwards).

The vPC feature is currently not supported by any Cisco Catalyst Series Switches and is available only on the Nexus switches family.

While VSS makes use of Multi Ether Channel (MEC) to bond Cisco Catalyst Series switches together, vPC is used on Cisco Nexus Series switches for the same purpose. Both technologies are similar from the perspective of the downstream switch but there are differences, mainly in that the control plane works on the upstream devices. The next table summarizes the main characteristics and features of the VSS and vPC technologies:

Feature	VSS	vPC
Multi-Chassis Port Channel	Yes	Yes
Loop Free Topology	Yes	Yes
Spanning Tree as failsafe protocol	Yes	Yes
Maximum physical Nodes	2	2
No Disruptive ISSU support	No	Yes
Control Plane	Single logical node	Two independents active nodes
Layer 3 port channel	YES	Limited
Configuration	Common configuration	Two different configurations
Etherchannel	Static, PAgP, PAgP, LACP	Static, LACP

Table 1. Comparing Catalyst VSS with Nexus vPC

Deploying MEC or vPC could require minimal changes to an existing switching infrastructure. Catalyst Switches may need a supervisor engine upgrade to form a VSS. Then, the primary loop avoidance mechanism is provided by MEC or vPC control protocols. STP is still in operation but is running only as a failsafe mechanism. Finally, the devices e.g. access switches, servers, etc., should be connected with multiple links to Data Center Distribution or Core switches. Link Aggregation Control Protocol (LACP) is the protocol that allows for dynamic portchannel negotiation and allows up to 16 physical interfaces to become members of a single port channel.

vPC Peer Keepalive Link Design Guidelines

Taking into account the importance and impact of the Peer Keepalive link and vPC Peer-Link, Cisco recommends the following type of interconnections for the vPC Keepalive link:

Recommendations in order of preference for the vPC Keepalive link interconnection
Nexus 7000 & 9000 Series Switches	Nexus 5000 & 3000 Series Switches
1. Dedicated link(s) (1GE LC)	1. mgmt0 interface (along with management traffic)
2. mgmt0 interface (along with management traffic)	2. Dedicated link(s) (1/10GE front panel ports)
3. As last resort, can be routed in-band over the L3 infrastructure

Table 2. vPC Keepalive Link Interconnection methods

Special attention is needed where the mgmt interfaces of a Nexus are used to route the vPC keepalive packets via an Out of Band (OOB) Management switch. Turning off the OOB Management switch, or removing by accident the keepalive links from this switch in parallel with vPC Peer-Link failure, could lead to split brain scenario and network outage.

Using a dedicated interface for vPC keepalive link has the advantage that there’s no other network device that could affect the vPC keepalive link. Using point to point links makes it easier to control the path and minimizes the risk of failure. However, an interface for each vPC peer switch should be used to host the keepalive link. This could be a problem where there’s a limited number of available interfaces or SFPs.

Layer 3 connectivity for the Keepalive Link can be accomplished either with the SVI or with L3 (no switchport) configuration of the interfaces involved. The SVI configuration is the only option where the Nexus vPC Peer switches do not support L3 features. In any case, it is recommended to set the Keepalive Link to a separate VRF in order to isolate it from the default VRF. If the SVI is configured to route the keepalive packets, then this vlan should not be routed over vPC link. This is why the Keepalive VLAN should be removed from the trunk allowed list of the vPC Peer-Link or the vPC member ports. Allowing the Keepalive VLAN over the vPC peer trunk could lead to split brain scenario (analyzed below) and network outage if the vPC Peer-Link fails!

vPC Peer-Link Design Guidelines

The following design guidelines are recommended for the vPC Peer-Links:

Member ports must be at least 10GE interfaces.
Use only point-to-point without other devices between the vPC peers (Nexus switches). E.g. transceivers, microwave bridge link, etc.
Use at least two 10Gbps links spread between two separate I/O module cards at each switch for best resiliency.
The ports should be in dedicated mode for the oversubscribed modules.
vPC Peer-Link ports should be located on a different I/O module than that used by the Peer Keepalive Link.

The next section describes how the vPC Nexus switches interact with events triggered by failure of links (vPC Peer Keepalive Link, Peer-Link etc) or vPC Peer switch.

vPC Failure Scenario: vPC Peer-Link Failure

In the scenario the vPC Peer-Links on the Secondary Nexus fail the status of the peer vPC is examined using the Peer Keepalive Link:

vPC Peer-Link Failure Scenario

If both vPC peers are active, the secondary vPC (i.e. the switch with the higher priority) disables all the vPC member ports to avoid uncertain traffic behavior and network loops which can result in service disruption.

At this point traffic continues flowing through the Primary vPC without any disruptions.

In the unfortunate event there is an orphan device connected to the secondary peer, then its traffic will be black-holed.

vPC Failure Scenario: vPC Peer Keepalive Link Failure

In the event the Peer Keepalive Link fails it will not have a negative effect on the operation of the vPC, which will continue forwarding traffic. The Keepalive Link is used as a secondary test mechanism to confirm the vPC peer is live in case the Peer-Link goes down:

vPC Peer Keepalive Link Failure Scenario

During a Keepalive Link failure there is no change of roles between the vPC (primary/secondary) and no down time in the network.

As soon as the Keepalive Link is restored the vPC will continue to operate.

vPC Failure Scenario: vPC Peer Switch Failure

In the case of a vPC peer switch total failure, the remote switch learns from the failure via the Peer Keepalive link since no keepalive messages are received. The data traffic is forwarded by utilizing the remaining links til the failed switch recovers. It should be noticed that the Keepalive messages are used only when all the links in the Peer-Link fail:

vPC Peer Switch Failure Scenario

Spanning Tree Protocol is used as a loop prevention mechanism in the case of a Peer Keepalive Link and vPC Peer-Link simultaneous failure.

vPC Failure Scenario: Dual Active or Split Brain

The Dual-Active or Split Brain vPC failure scenario occurs when the Peer Keepalive Link fails followed by the Peer-Link. Under this condition both switches undertake the vPC primary roles.

If this happens, the vPC primary switch will remain as the primary and the vPC secondary switch will become operational primary causing severe network instability and outage:

vPC Dual-Active or Split Brain Failure Scenario

Nexus vPC Configuration & Troubleshooting Guide

The vPC is configured and normal operation is verified by following the nine steps defined below. It should be noted that the order of the vPC configuration is important and that a basic vPC setup is established by using the first 4 steps:

vPC Configuration Steps

Step 1: Enable the vPC feature and configure the vPC domain ID on both Nexus switches.

Step 2: Select a Peer Keepalive deployment option.

Step 3: Establish the vPC peer keepalive link.

Step 4: Configure the vPC Peer-Link.

Step 4 completes the global vPC configuration on both vPC peer switches.

Step 5: Configure individual vPCs to downstream switches or devices.

Step 6: Optionally, enable the peer gateway feature to modify the First Hop Redundancy Protocol (FHRP) operation.

Step 7: Optionally, enable the peer switch feature to optimize the STP behaviour with vPCs.

Step 8: Optionally, enable the additional features to optimize the vPCs setup.

Step 9: Optionally, verify operation of the vPC and vPC consistency parameters.

Cisco Nexus vPC Configuration Example

To help illustrate the setup of the vPC technology we used two Nexus 5548 data center switches. Typically, a similar process would be followed for any other type of Nexus switches.

Our two Nexus 5548 were given hostnames N5k-Primary & N5k-Secondary and the order outlined above was followed for the vPC setup and configuration:

Step 1: Enable the vPC Feature and Configure the vPC Domain ID on Both Switches

Following are the commands used to enable vPC and configure the vPC domain ID on the first switch:

N5k-Primary(config)# feature vpc

N5k-Primary(config)# vpc domain 1

N5k-Primary(config-vpc-domain)# show vpc role

vPC Role status

----------------------------------------------------

vPC role : none established

Dual Active Detection Status : 0

vPC system-mac : 00:23:04:ee:be:01

vPC system-priority : 32667

vPC local system-mac : 8c:60:4f:2c:b3:01

vPC local role-priority : 0

Now we configure the Nexus Secondary switch using the same commands:

N5k-Secondary(config)# feature vpc

N5k-Secondary(config)# vpc domain 1

N5k-Secondary(config-vpc-domain)# show vpc role

vPC Role status

----------------------------------------------------

vPC role : none established

Dual Active Detection Status : 0

vPC system-mac : 00:23:04:ee:be:01

vPC system-priority : 32667

vPC local system-mac : 8c:60:4f:aa:c2:3c

vPC local role-priority : 0

The same domain ID (ID 1 in our example) must be used on both vPC peer switches in the vPC domain. The output of the show vpc role command shows that the system MAC address is derived from the vPC domain ID, which is equal to 01.

Step 2: Choose a Peer Keepalive Deployment Option

Our setup below utilizes the SVI technology and the second option (dedicated 1G link) proposed for the N5k series switches keepalive link setup (table 2). This deployment option involves a dedicated VLAN with a configured SVI used for the keepalive link within an isolated VRF (named keepalive) for complete isolation from the rest of the network. Interface Ethernet 1/32 is used by both switches as a dedicated interface for the keepalive link.

On the first switch we create VLAN 23 with an SVI (assign an IP address to the VLAN interface) and make it a member of the VRF instance created for this purpose. We complete the configuration by assigning Ethernet 1/32 to VLAN 23:

N5k-Primary(config)# vlan 23

N5k-Primary(config-vlan)# name keepalive

N5k-Primary(config)# vrf context keepalive

interface Vlan23

vrf member keepalive

ip address 192.168.1.1/24

interface Ethernet1/32

switchport access vlan 23

speed 1000

duplex full

We follow the same configuration steps on our Secondary Nexus switch:

N5k-Secondary (config)# vlan 23

N5k-Secondary(config-vlan)# name keepalive

N5k-Secondary(config)# vrf context keepalive

interface Vlan23

vrf member keepalive

ip address 192.168.1.2/24

interface Ethernet1/32

switchport access vlan 23

speed 1000

duplex full

The ping connectivity test between the Peer Keepalive Links is successful:

N5k-Secondary# ping 192.168.1.1 vrf keepalive

PING 192.168.1.1 (192.168.1.1): 56 data bytes

36 bytes from 192.168.1.2: Destination Host Unreachable

Request 0 timed out

64 bytes from 192.168.1.1: icmp_seq=1 ttl=254 time=3.91 ms

64 bytes from 192.168.1.1: icmp_seq=2 ttl=254 time=3.05 ms

64 bytes from 192.168.1.1: icmp_seq=3 ttl=254 time=1.523 ms

64 bytes from 192.168.1.1: icmp_seq=4 ttl=254 time=1.501 ms

Note: The initial ICMP timeout is normal behavior as the switch needs to initially send out an ARP request to obtain 192.168.1.1’s MAC address and then send the ICMP (ping) packet.

Step 3: Establish the vPC Peer Keepalive Link

By default, the vPC Peer Keepalive packets are routed in the management VRF and use the Out-Of-Band (OOB) mgmt interface.

It is, however, highly recommended to configure the vPC Peer Keepalive link to use a separate VRF instance to ensure that the peer keepalive traffic is always carried on that link and never on the Peer-Link. In addition, the keepalive vlan should be removed from the trunk allowed list of the vPC Peer-Link or the vPC Member Ports.

N5k-Primary(config)# vpc domain 1

N5k-Primary (config-vpc-domain)# peer-keepalive destination 192.168.1.2 source 192.168.1.1 vrf keepalive

Configuration of the Secondary vPC follows:

N5k-Secondary(config)# vpc domain 1

N5k-Secondary(config-vpc-domain)# peer-keepalive destination 192.168.1.1 source 192.168.1.2 vrf keepalive

We can verify the status of the vPC Peer Keepalive Link using the show vpc peer-keepalive command on both switches:

N5k-Primary# show vpc peer-keepalive

vPC keep-alive status : peer is alive

--Peer is alive for : (95) seconds, (201) msec

--Send status : Success

--Last send at : 2017.06.22 23:03:50 720 ms

--Sent on interface : Vlan23

--Receive status : Success

--Last receive at : 2017.06.22 23:03:50 828 ms

--Received on interface : Vlan23

--Last update from peer : (0) seconds, (201) msec

vPC Keep-alive parameters

--Destination : 192.168.1.2

--Keepalive interval : 1000 msec

--Keepalive timeout : 5 seconds

--Keepalive hold timeout : 3 seconds

--Keepalive vrf : keepalive

--Keepalive udp port : 3200

--Keepalive tos : 192

Verifying the status of the vPC Peer Keepalive Link on our Secondary switch:

N5k-Secondary# show vpc peer-keepalive

vPC keep-alive status : peer is alive

--Peer is alive for : (106) seconds, (385) msec

--Send status : Success

--Last send at : 2017.06.22 22:46:32 106 ms

--Sent on interface : Vlan23

--Receive status : Success

--Last receive at : 2017.06.22 22:46:32 5 ms

--Received on interface : Vlan23

--Last update from peer : (0) seconds, (333) msec

vPC Keep-alive parameters

--Destination : 192.168.1.1

--Keepalive interval : 1000 msec

--Keepalive timeout : 5 seconds

--Keepalive hold timeout : 3 seconds

--Keepalive vrf : keepalive

--Keepalive udp port : 3200

--Keepalive tos : 192

Step 4: Configure the vPC Peer-Link

This step completes the global vPC configuration on both vPC peer switches and involves the creation of the Port-Channel to be used as the vPC Peer-Link.

First we need to enable the lacp feature then create our high-capacity port channel between the two switches to carry all necessary traffic.

The interfaces Eth1/2 and Eth1/3 are selected to become members of the vPC Peer-Link in LACP mode. In addition, the vPC is configured as a trunk. The allowed VLAN list for the trunk should be configured in such a way that only vPC VLANs (VLANs that are present on any vPCs) are allowed on the trunk. VLAN 10 has been created and allowed on the vPC Peer-Link:

N5k-Primary (config)# feature lacp

N5k-Primary(config)# interface ethernet 1/2-3

N5k-Primary(config-if-range)# description *** VPC PEER LINKS ***

N5k-Primary(config-if-range)# channel-group 23 mode active

N5k-Primary(config)# vlan 10

N5k-Primary(config)# interface port-channel 23

N5k-Primary(config-if)# description *** VPC PEER LINKS ***

N5k-Primary(config-if)# switchport mode trunk

N5k-Primary(config-if)# switchport trunk allowed vlan 10

N5k-Primary(config-if)# vpc peer-link

Please note that spanning tree port type is changed to "network" port type on vPC peer-link. This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge Assurance(which is enabled by default) is not disabled.

N5k-Primary(config-if)# spanning-tree port type network

An identical configuration follows for our Secondary switch:

N5k-Secondary(config)# feature lacp

N5k-Seondary(config)# interface ethernet 1/2-3

N5k-Secondary(config-if-range)# description *** VPC PEER LINKS ***

N5k-Secondary(config-if-range)# channel-group 23 mode active

N5k-Seondary(config)# vlan 10

N5k-Secondary(config)# interface port-channel 23

N5k-Secondary(config-if)# description *** VPC PEER LINKS ***

N5k-Secondary(config-if)# switchport mode trunk

N5k-Secondary(config-if)# switchport trunk allowed vlan 10

N5k-Secondary(config-if)# vpc peer-link

N5k-Secondary(config-if)# spanning-tree port type network

It is not recommended to carry non-vPC VLANs on the vPC Peer-Link, because this configuration could cause severe traffic disruption for the non-vPC VLANs if the vPC Peer-Link fails. Finally, the vPC Peer Keepalive messages should not be routed over the vPC Peer-Link, which is why the VLAN associated with the Peer Keepalive connection (VLAN 23) is not allowed on the vPC Peer-Link.

We can perform a final check on our vPC using the show vpc command:

N5k-Primary# show vpc

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1

Peer status : peer adjacency formed ok

vPC keep-alive status : peer is alive

Configuration consistency status : success

Per-vlan consistency status : success

Type-2 consistency status : success

vPC role : primary

Number of vPCs configured : 0

Peer Gateway : Disabled

Dual-active excluded VLANs : -

Graceful Consistency Check : Enabled

Auto-recovery status : Enabled (timeout = 240 seconds)

vPC Peer-link status

---------------------------------------------------------------------

id Port Status Active vlans

-- ---- ------ --------------------------------------------------

1 Po23 up 10

Verifying the vPC on the Secondary peer:

N5k-Secondary# show vpc

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1

Peer status : peer adjacency formed ok

vPC keep-alive status : peer is alive

Configuration consistency status : success

Per-vlan consistency status : success

Type-2 consistency status : success

vPC role : secondary, operational primary

Number of vPCs configured : 0

Peer Gateway : Disabled

Dual-active excluded VLANs : -

Graceful Consistency Check : Enabled

Auto-recovery status : Enabled (timeout = 240 seconds)

vPC Peer-link status

---------------------------------------------------------------------

id Port Status Active vlans

-- ---- ------ --------------------------------------------------

1 Po23 up 10

The show vpc output shows that the vPC Peer-Link has been successfully established between the Nexus 5548 switches.

Step 5: Configure Individual vPCs to Downstream Devices

Individual vPCs can now be configured since the vPC domain has been properly established in the previous step.

Individual vPCs are used to connect network devices to both data center switches. For example, a router or server can connect with two or more network interfaces to both switches simultaneously for increased redundancy and bandwidth availability.

For each individual vPC, a port channel is configured on both vPC peer switches. The two port channels are then associated with each other by assigning a vPC number to the port channel interfaces:

interface Ethernet1/1

description *** Connected to ISR Gig0/2/4 ***

switchport access vlan 10

speed 1000

channel-group 10

interface port-channel10

switchport access vlan 10

vpc 10

In our setup, vpc index 10 has been assigned to port-channel 10. It is generally good practice to keep the port-channel (e.g. port-channel 10) and vpc index (e.g. vpc 10) the same to make tracking easier and avoid configuration mistakes.

Finally the vPC port number (e.g. port-channel 10) to the downstream device (e.g router) is unique for each individual vPC within the vPC domain and must be identical between the two peer switches as shown in the diagram below:

Nexus vPC port-channel configuration to downstream devices

Finally the vPC member ports should have a compatible and consistent configuration for all the ports to both switches. Here is the configuration on the Primary Nexus switch:

interface Ethernet1/1

description *** Connected to ISR Gig0/2/0 ***

switchport access vlan 10

speed 1000

channel-group 10

interface port-channel10

switchport access vlan 10

vpc 10

Verifiying our vPC to the downstream device from the Primary vPC:

N5k-Primary# show vpc | begin "vPC status"

vPC status

----------------------------------------------------------------------------

id Port Status Consistency Reason Active vlans

------ ----------- ------ ----------- -------------------------- -----------

10 Po10 up success success 10

Verifiying our vPC to the downstream device from the Secondary vPC:

N5k-Secondary# show vpc | begin "vPC status"

vPC status

----------------------------------------------------------------------------

id Port Status Consistency Reason Active vlans

------ ----------- ------ ----------- -------------------------- -----------

10 Po10 up success success 10

Step 6: (Optional) Enable the Peer-Gateway Feature to Modify the FHRP Operation

The vPC Peer-Gateway feature causes a vPC peer to act as a gateway for packets that are destined for the peer device’s MAC address. So, it enables local forwarding of such packets without the need to cross the vPC Peer-Link. This feature optimizes the use of the peer link and avoids potential traffic loss in FHRP scenarios.

When enabled, the peer gateway feature must be configured on both primary and secondary vPC peers:

N5k-Primary(config)# vpc domain 1

N5k-Primary(config-vpc-domain)# peer-gateway

Configuring the secondary vPC peer:

N5k-Secondary(config)# vpc domain 1

N5k-Secondary(config-vpc-domain)# peer-gateway

Step 7: (Optional) Enable the Peer-Switch Feature to Optimize the STP Behaviour with the vPCs

This feature allows a pair of Cisco Nexus switches to appear as a single spanning tree root in the Layer 2 topology. It eliminates the need to pin the spanning tree root to the vPC primary switch and improves vPC convergence if the vPC primary switch fails:

N5k-Primary(config)# vpc domain 1

N5k-Primary(config-vpc-domain)# peer-switch

Configuring the peer-switch command on the Secondary vPC:

N5k-Secondary(config)# vpc domain 1

N5k-Secondary(config-vpc-domain)# peer-switch

Step 8: (Optional) Optimize vPC performance: ‘ip arp synchronize’, ‘delay restore’, ‘auto-recovery’, ‘graceful consistency-check’ & ‘role priority’ commands

Configure the following vPC commands in the vPC domain configuration mode, this will increase resiliency, optimize performance, and reduce disruptions in vPC operations.

The ip arp synchronize feature allows the synchronization of the ARP table when the peer-link comes up. The vPC offers the option to delay the restoration of the vPC ports for a configurable time by using the delay restore command, which is useful to avoid traffic blackholing after a reboot of the switch. The auto-recovery command has a default timer of 240 seconds.

In addition, it is recommended to use the configuration synchronization graceful consistency-check feature to minimize disruption when a Type 1 mismatch occurs. Examples of Type 1 mismatches could be the STP mode or the STP port type between the vPC peer switches. The show vpc consistency-parameters global output illustrates the Type 1 and Type 2 parameters of a vPC.

The commands below enable and configure all the above mentioned features:

N5k-Primary(config)# vpc domain 1

N5k-Primary(config-vpc-domain)# delay restore 360

N5k-Primary(config-vpc-domain)# auto-recovery

Warning:

Enables restoring of vPCs in a peer-detached state after reload, will wait for 240 seconds to determine if peer is un-reachable

N5k-Primary(config-vpc-domain)# graceful consistency-check

N5k-Primary(config-vpc-domain)# ip arp synchronize

Once the Primary switch is configured we apply the same configuration to the Secondary switch:

N5k-Secondary(config)# vpc domain 1

N5k-Secondary(config-vpc-domain)# delay restore 360

N5k-Secondary(config-vpc-domain)# auto-recovery

Warning:

Enables restoring of vPCs in a peer-detached state after reload, will wait for 240 seconds to determine if peer is un-reachable

N5k-Secondary(config-vpc-domain)# graceful consistency-check

N5k-Secondary(config-vpc-domain)# ip arp synchronize

Finally, it should be noted that it is feasible to set the role priority under vpc domain configuration with the command role priority to affect the election of the primary vPC switch.

The default role priority value is 32,667 and the switch with lowest priority is elected as the vPC primary switch.

If the vPC primary switch is alive and the vPC Peer-Link goes down, the vPC secondary switch suspends its vPC member ports to prevent dual active scenario, while the vPC primary switch keeps all of its vPC member ports active. It is recommended for this reason the orphan ports (ports connecting to only one switch) be connected to the vPC primary switch.

Verifying Operation and Troubleshooting the Status of the vPC

The show vpc brief command displays the vPC domain ID, the Peer-Link status, the Keepalive message status, whether the configuration consistency is successful, and whether a peer link has formed. It also states the status of the vPC Port Channel (Po10 in our setup).

N5k-Primary# show vpc brief

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1

Peer status : peer adjacency formed ok

vPC keep-alive status : peer is alive

Configuration consistency status : success

Per-vlan consistency status : success

Type-2 consistency status : success

vPC role : primary, operational secondary

Number of vPCs configured : 1

Peer Gateway : Enabled

Peer gateway excluded VLANs : -

Dual-active excluded VLANs : -

Graceful Consistency Check : Enabled

Auto-recovery status : Enabled (timeout = 240 seconds)

vPC Peer-link status

---------------------------------------------------------------------

id Port Status Active vlans

-- ---- ------ --------------------------------------------------

1 Po23 up 10

vPC status

----------------------------------------------------------------------------

id Port Status Consistency Reason Active vlans

------ ----------- ------ ----------- -------------------------- -----------

10 Po10 up success success 10

The show vpc consistency-parameters command is useful for troubleshooting and identifying specific parameters that might have caused the consistency check to fail either on the vPC Peer-Link or to the vPC enabled Portchannels:

N5k-Primary# show vpc consistency-parameters global

Legend:

Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value

------------- ---- ---------------------- -----------------------

QoS 2 ([ ], [ ], [ ], [ ], [ ], ([ ], [ ], [ ], [ ], [ ],

[ ]) [ ])

Network QoS (MTU) 2 (1538, 0, 0, 0, 0, 0) (1538, 0, 0, 0, 0, 0)

Network Qos (Pause) 2 (F, F, F, F, F, F) (F, F, F, F, F, F)

Input Queuing (Bandwidth) 2 (100, 0, 0, 0, 0, 0) (100, 0, 0, 0, 0, 0)

Input Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F)

Priority)

Output Queuing (Bandwidth) 2 (100, 0, 0, 0, 0, 0) (100, 0, 0, 0, 0, 0)

Output Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F)

Priority)

STP Mode 1 Rapid-PVST Rapid-PVST

STP Disabled 1 None None

STP MST Region Name 1 "" ""

STP MST Region Revision 1 0 0

STP MST Region Instance to 1

VLAN Mapping

STP Loopguard 1 Disabled Disabled

STP Bridge Assurance 1 Enabled Enabled

STP Port Type, Edge 1 Normal, Disabled, Normal, Disabled,

BPDUFilter, Edge BPDUGuard Disabled Disabled

STP MST Simulate PVST 1 Enabled Enabled

IGMP Snooping Group-Limit 2 4000 4000

Interface-vlan admin up 2 10 10

Interface-vlan routing 2 10 10

capability

Allowed VLANs - 10 10

Local suspended VLANs - - -

N5k-Primary# show vpc consistency-parameters vpc 10

Legend:

Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value

------------- ---- ---------------------- -----------------------

Shut Lan 1 No No

STP Port Type 1 Default Default

STP Port Guard 1 None None

STP MST Simulate PVST 1 Default Default

mode 1 on on

Speed 1 1000 Mb/s 1000 Mb/s

Duplex 1 full full

Port Mode 1 access access

MTU 1 1500 1500

Admin port mode 1 access access

vPC card type 1 Empty Empty

Allowed VLANs - 10 10

Local suspended VLANs - - -

vPC Quiz

Our Nexus 5500 switches used the management interface to establish the vPC keepalive link between them. The management interfaces on both switches are connected to a 2960 Catalyst management switch which was accidently switched off due to an unplanned power disruption, causing the management interface and vPC keepalive link to go down. What is the impact of this failure on the Nexus vPC setup?

Answer:

There will be no service impact to the Nexus infrastructure! Read the vPC failure scenarios section in this article for a thorough explanation.

Summary

In this article we reviewed the Nexus vPC features and vPC design guidelines. In addition we discussed the vPC architecture components and explained the importance of each component.

Next we analyzed different vPC failure scenarios including vPC Peer-Link Failure and Peer Keepalive link failure. We compared vPC with VSS technology developed for the Catalyst Switches in order to provide MEC feature capabilities. Finally, the vPC configuration guide and best practices section showed how to configure vPC and apply optional configuration commands to increase resiliency and reduce disruptions in vPC operations. We also provided useful show commands needed to validate and troubleshoot the status of the vPC.

NEXUS NX-OS: Useful Commands, CLI Scripting, Hints & Tips, Python Scripting and more

2017-07-04T13:50:33+10:00

Whether you’re new to Cisco Nexus switches or have been working with them for years this article will show how to get around the Nexus NX-OS using smart CLI commands and parameters, create your own commands and more. Learn how to filter show command outputs, efficiently use include | begin | exclude search operators, Turn pagination on/off, redirect output to files, run multiple commands in one CLI line, capture specific keywords from show command output, create custom CLI commands using alias, execute scripts, introduction of Python environment in the Nexus NX-OS, executing Python scripts and much more!

By the end of this article we’ll agree there’s no doubt the Cisco NX-OS has several interesting commands and powerful scripting capabilities that can improve and facilitate the day-to-day administration of CISCO Nexus network devices.

While basic knowledge on the Cisco NX-OS, Linux and Python scripting is recommended, it is not mandatory in order to understand the topics covered.

Key Topics:

Additional related articles:

NX-OS Command Output Filtering – Search Operators

The output from NX-OS show commands can be lengthy and that makes it difficult to find the information we are looking for. The Cisco NX-OS software provides the means to search and filter the output to assist in locating the information we are after.

Experienced Cisco users will surely be familiar with the IOS (Catalyst) include | begin | exclude search operators which are also offered in the Nexus NX-OS (see below). The NX-OS offers additional searching and filtering options, which follow a pipe character (|) at the end of the show command. This provides amazing flexibility and helps make administration of any Nexus device a great experience. The most “Linux-like” useful options are displayed below:

N5k-UP# show interface brief | ?
<…>
diff Show difference between current and previous invocation (creates temp files: remove them with 'diff-clean' command and dont use it on commands with big outputs, like 'show tech'!)
egrep Egrep - print lines matching a pattern
grep Grep - print lines matching a pattern
less Filter for paging
no-more Turn-off pagination for command output
section Show lines that include the pattern as well as the subsequent lines that are more indented than matching line
sort Stream Sorter
source Run a script (python, tcl,...) from bootflash:scripts
vsh The shell that understands cli command
wc Count words, lines, characters
xml Output in xml format (according to .xsd definitions)
begin Begin with the line that matches
count Count number of lines
exclude Exclude lines that match
include Include lines that match

Filtering Output From The ‘Show’ Command - ‘Show <command> | grep’ & ‘Show <command>egrep’ Parameters

The grep & egrep parameters can be used to filter the show command output for easier to read results.

The example below shows how to filter the show running-config output by specifying the number of lines to display before and after a matched line. The matching variable in our example is the keyword Firewall:

N5k-UP# show running-config | grep prev 1 next 2 Firewall
interface Ethernet1/1
description Firewall – LAN
interface Ethernet1/2
--
interface Ethernet1/4
description Firewall - WAN
interface Ethernet1/5

You can use the less operator to display the contents of the show command output in one page at a time. There are various command options at the ‘:’ prompt. To display all support less command options you enter ‘h’ at the ‘:’ prompt.

An interesting output and useful option is using the show log | less command which Unix/Linux users will welcome as it has the same effect as the tail –f <filename> Linux command. This command will display the last entries of the system’s log and automatically update the display with any new content/log entries inserted. Engineers and admins can now easily troubleshoot problems while continually keeping an eye on the Nexus syslog without the need to use the show log command every minute to get any new updates written to the system’s log:

N5k-UP# show log | less
:F
<…>
2023 May 15 11:59:20 N5k-UP %EEM_ACTION-2-CRIT: SLA-PYTHON-SCRIPT-FOR-8.8.8.8/32-EXECUTED
2023 May 15 11:59:20 N5k-UP %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configured from vty by admin on vsh.1115
Waiting for data... (interrupt to abort)

To exit this mode simply type Ctrl-C and then q to abort.

Another useful operator is the sort parameter used to filter the show command output in an order as shown below with the Ethernet interfaces. Keep in mind that the command will sort the output based on the character order which means any interface with an Eth1 will show first, then Eth2 and so on.

N5k-UP# show interface brief | sort
Eth1/1 1 eth access down SFP validation failed 10G(D) --
Eth1/10 1 eth access down SFP not inserted 10G(D) --
Eth1/11 1 eth access down SFP not inserted 10G(D) --
Eth1/12 1 eth access down SFP not inserted 10G(D) –
<…>

Turning Off Pagination For Lengthy ‘Show’ Command Outputs. ‘Show <option> | no-more’ Parameter

The | no-more parameter is particularly useful when there is a need to display all output without stopping at the end of the page. A good example is to show the running-configuration or obtain the output of any command in one hit. By default, the Nexus OS will pause the output once it hits the end of the user’s terminal page. This feature can be easily bypassed by appending the |no-more parameter at the end of the command:

N5k-UP# show interface brief | no-more

Searching & Filtering Output from ‘Show’ Command: ‘Show <option>’–More-

You can search and filter output from the --More– prompt displayed in the show command output. When the --More— prompt appears (as shown below), simply type h to view all possible options. An interesting feature is to filter the output by typing / (forward-slash) and then to search for the pattern that you are looking for:

N5k-UP# show running-config
!Command: show running-config
!Time: Mon May 15 12:30:09 2023
<…>
--More—
<…>
/<regular expression>

Displaying Last Lines From The ‘Show’ Command Output – ‘Show <option> | last ’

When working with lengthy outputs from commands such as show logging it’s often desirable to display the last lines of the command output. The show <option> | last command will display the last 10 lines by default. Appending a number after the keyword last will adjust the lines displayed. The example below shows the last 5 log entries in our Nexus system:

N5k-UP# show logging | last 5
2023 May 15 12:34:30 N5k-UP %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configured from vty by admin on vsh.5669
2023 May 15 12:34:40 N5k-UP %EEM_ACTION-2-CRIT: SLA-PYTHON-SCRIPT-FOR-8.8.8.8/32-EXECUTED
2023 May 15 12:34:40 N5k-UP %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configured from vty by admin on vsh.5691
2023 May 15 12:34:50 N5k-UP %EEM_ACTION-2-CRIT: SLA-PYTHON-SCRIPT-FOR-8.8.8.8/32-EXECUTED
2023 May 15 12:34:50 N5k-UP %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configured from vty by admin on vsh.5709

Redirecting ‘Show Command’ Output to File with or without Timestamp. ‘Show running-config>backupcfg.$(TIMESTAMP) ‘

The ability to redirect the output of a command to a file is a feature most Linux users/administrators will welcome. Capturing lengthy outputs from commands such as show tech-support can become quite challenging as these can sometime be over 10,000 lines. Redirecting the output of a command to a file is a very simple process making it easy to execute even by less experienced users.

The command below shows how to redirect the output of the show running-config command to a file on the system’s flash:

N5k-UP# show running-config > backupcfg
N5k-UP# dir | include backup
4352 May 15 13:30:00 2023 backupcfg

When adding the System-Defined Timestamp Variable into the command line the Nexus OS will automatically append the time and date to the filename making it easier to store and track files. The next example redirects the show running-config output to a file that includes the system’s timestamp:

N5k-UP# show running-config> backupcfg.$(TIMESTAMP)
N5k-UP# dir | include backup
4352 May 15 13:30:00 2023 backupcfg
4352 May 15 13:46:17 2023 backupcfg.2023-05-15-13.46.17

Combining Multiple Search Strings – ‘| Include’ Parameter

Sometimes, it is necessary to combine search strings from the show command to filter the output and quickly obtain the information we need. The | include parameter is frequently used to filter the output and display lines containing specific keywords.

The next command will show the configured descriptions from all interfaces and include the interface utilization which is captured by the rate keyword:

N5k-UP# show interface | include description | rate
PH_NEXUS_7000# show interface | incdescription|rate
1 minute input rate 0 bits/sec, 0 packets/sec
1 minute output rate 0 bits/sec, 0 packets/sec
30 seconds input rate 56 bits/sec, 0 packets/sec
30 seconds output rate 896 bits/sec, 1 packets/sec
input rate 56 bps, 0 pps; output rate 896 bps, 1 pps
300 seconds input rate 112 bits/sec, 0 packets/sec
300 seconds output rate 488 bits/sec, 1 packets/sec
input rate 112 bps, 0 pps; output rate 488 bps, 1 pps
30 seconds input rate 120 bits/sec, 0 packets/sec
30 seconds output rate 1072 bits/sec, 1 packets/sec
..<output omitted>

All search strings are case sensitive and there is no space between the last pipe and keywords (description|rate).

Next example command is equivalent to the OR option for a filter.

N5k-UP# show run | include 'interface Vlan|ip address'
iptacacs source-interface Vlan1
source-interface Vlan1
ip address 10.1.1.101/24
interface Vlan1
ip address 192.168.250.247/24
interface Vlan4
ip address 192.168.4.247/24
interface Vlan7
ip address 172.20.199.247/24
interface Vlan11
interface Vlan25
ip address 192.168.25.247/24
interface Vlan26
ip address 172.26.1.251/24
interface Vlan27
ip address 172.27.1.250/24
interface Vlan60
interface Vlan100
ip address 172.20.100.247/24
interface Vlan105
ip address 172.20.105.247/24
interface Vlan109
ip address 172.20.109.247/24
interface Vlan110
ip address 172.20.110.247/24

Finally the last example is used when there is a need to use more than one word to filter specific keywords/patterns. The search strings should be included between apostrophes as shown below:

N5k-UP# show running-config | include 'ip route'
ip route 0.0.0.0/0 192.168.245.245
ip route 0.0.0.0/0 192.168.231.4
ip route 0.0.0.0/0 172.26.1.250
ip route 0.0.0.0/0 10.1.1.1

Scripting in NX-OS – Executing Multiple Commands within a File

Automating time-consuming tasks such as configuring multiple interfaces or changing large portions of a configuration is easily achieved thanks to the flexibility the NX-OS provides.

Here are a few examples where automated scripts can be used to help speed up troubleshooting or even resolve problems:

Temporarily change the running configuration, obtain debugs and then roll back the change
Have a series of commands ready to be executed when specific events occur e.g. link failure or switch becomes unresponsive
Execute commands on the Nexus after the switch is deployed at a remote location
Periodically obtain information from the Nexus switch using show commands.

The possibilities and combinations are really limitless.

Unfortunately the script containing the commands cannot be created within NX-OS. The script needs to be created on a workstation using a standard text editor and then uploaded to the Nexus switch bootflash.

Uploading the file to the Nexus bootflash is achieved using the copy tftp: bootflash: command. This assumes there is already a tftp server configured, operating and serving the script:

N5k-UP# copy tftp: bootflash:
Enter source filename: nexus-script.txt
Enter vrf (If no input, current vrf 'default' is considered): management
Enter hostname for the tftp server: 10.10.8.176
Trying to connect to tftp server......
Connection to Server Established.
TFTP get operation was successful
Copy complete, now saving to disk (please wait)...

N5k-UP# dir | include nexus
179 May 15 00:03:12 2023 nexus-script.txt
N5k-UP#

The script’s content can be viewed using the show file command as displayed below:

N5k-UP# show file bootflash:///nexus-script.txt
Configure terminal
interface Ethernet1/6
description *** TEST 1 ***
no shutdown
interface Ethernet1/7
description *** TEST 2 ***
no shutdown
interface Ethernet1/8
description *** TEST 3 ***
no shutdown
end

As we can see, the script contains commands that will configure the description on 3 Ethernet interfaces and place them in an administratively up status (no shutdown). Currently these interfaces do not have any configuration:

N5k-UP# show run interface
!Command: show running-config interface
!Time: Mon May 15 00:00:41 2023
version 7.0(2)N1(1)
<…>
interface Ethernet1/6
interface Ethernet1/7
interface Ethernet1/8

Executing our script can be easily done using the run-script command. The run-script command is used to execute the commands specified in a file:

N5k-UP# run-script bootflash:///nexus-script.txt
`configure terminal
`interface Ethernet1/6
`description *** TEST 1 ***
`no shutdown
`interface Ethernet1/7
`description *** TEST 2 ***
`no shutdown

`interface Ethernet1/8
`description *** TEST 3 ***
`no shutdown
`end

Once the script is executed we can check the running-configuration and verify interfaces have been successfully configured:

N5k-UP# show running-config interface
!Command: show running-config interface
!Time: Mon May 15 00:04:04 2023
version 7.0(2)N1(1)
<output omitted>..
interface Ethernet1/6
description *** TEST 1 ***
no shutdown
interface Ethernet1/7
description *** TEST 2 ***
no shutdown
interface Ethernet1/8
description *** TEST 3 ***
no shutdown

Another option is to use the vsh command and run commands directly. The vsh stands for virtual shell and is mainly used to run NX-OS cli commands from Bash Shell however, we can still run the same script but this time by taking advantage of the vsh command.

The vsh command is executed on a clean configuration and the interface descriptions are successfully configured:

N5k-UP# show file script-description2.txt | vsh

N5k-UP# show run interface
!Command: show running-config interface
!Time: Mon May 15 10:49:16 2023
<…>
interface Ethernet1/6
description *** TEST 1 ***
interface Ethernet1/7
description *** TEST 2 ***
interface Ethernet1/8
description *** TEST 3 ***

There is also the option to execute commands directly as illustrated in the following example.

N5k-UP# echo "hostname TEST-VSH" | vsh
TEST-VSH#

Introducing Python in the Nexus NX-OS – Uploading and Executing Python Scripts

Nexus switches offer powerful scripting capabilities since integrating Python into NX-OS and can simplify network operations through the ability to run Python scripts directly on the switch. Python is a powerful programming language with a simple approach to object-oriented programming. The Cisco Nexus 5000 series switches with Releases 5.2(1)N1(1) and later and the Cisco Nexus 6000 series switches with Releases 6.0(2)N1(1) and later, support all the features available in Python v2.7.2. The Cisco Nexus 7000 series also support Python v2.7.2 and the Cisco Nexus 9000 Series devices support Python v2.7.5. The python scripts can be used to execute configuration commands, show commands, parse CLI output, call other scripts etc.

To enter the python environment on the Nexus NX-OS use the python command. Once in the python environment the hash (#) command prompt will be replaced by three greater-than signs (>>>). At this point we are able to directly execute python commands and scripts.

The integrated python in NX-OS supports both interactive and non-interactive modes. Python commands in interactive mode can be executed after switching to the python environment. The next interactive mode example illustrates how to print the old time classic in the programming world, “hello world” output, directly from the NX-OS python environment.

Note: The following commands can be used to exit the python environment and switch back to NX-OS privileged mode (#): quit(), exit (), Ctrl-C or Ctrl-D (i.e. EOF)

Pythonin NX-OS can run also in non-interactive (script) mode by running the Python script name as an argument to the Python CLI command.

For demonstration purposes we’ve created a simple python script named helloPython.py. This script has been created using a standard text editor, e.g. Notepad from the Windows OS, and has been uploaded to a TFTP server. The script has been downloaded to the Nexus switch and stored in the bootflash:scripts location which is where pythons scripts must be stored and executed. The content of our Python script is shown below:

N5k-UP# show file bootflash:scripts/helloPython.py
#!/usr/bin/env python
import sys
argvList = sys.argv[1:]
forargv in argvList:
print 'Hello ' + argv

Executing Python Scripts

The python script is executed using the source <filename> command followed by the necessary parameters the script is expecting. A variable (argv) is expected to be received as input to the script. This variable (firewall.cx) is printed along with the word “Hello” at the beginning of the output.

N5k-UP# source helloPython.py firewall.cx
Hello firewall.cx

NOTE: Before the NX-OS 7.0 version, python scripts are executed with the ‘python’ instead of the ‘source’ command.

Finally, you can create your own NX-OS commands by taking advantage of the python scripting. A new command named hello has been configured, using the cli alias command, which executes our Python script along with the necessary parameters:

N5k-UP(config)# cli alias name hello source helloPython.py
N5k-UP(config)# hello Vasilis
Hello Vasilis

The cli alias command above instructs the NX-OS to create a new command named hello which, when executed, will run in its turn the command “source helloPython.py” but also accept any parameters given (for our Python script). The cli alias command is covered extensively later in this article.

Nexus NX-OS Hints & Tips

Working with the Nexus NX-OS is a pleasant experience considering the similarities with the Linux operating system. Unix/Linux users will surely feel right at home. To further enhance user experience we’ve written the top 5 NX-OS handy commands section below that could be useful for the day to day operation and administration of Nexus switches. Let’s start the countdown...

Nexus NX-OS Tip No.5 – Executing Multiple Commands in One Line

The Nexus NX-OS allows the execution of multiple show or configuration commands in one go using the semi-colon ; character between them:

N5k-UP# show clock ; show checkpoint summary ; show hostname ;
12:56:57.370 UTC Mon May 15 2023
User Checkpoint Summary
--------------------------------------------------------------------------------
1) FIRST-Checkpoint:
Created by admin
Created at Wed, 16:13:19 10 May 2023
Size is 15,831 bytes
Description: None
2) SLA:
Created by admin
Created at Sun, 14:21:06 14 May 2023
Size is 16,183 bytes
Description: PYTHON-SCRIPT
N5k-UP

N5k-UP# configure terminal ; interface eth1/6 ; description *** test multiple commands *** ;
Enter configuration commands, one per line. End with CNTL/Z.
N5k-UP(config-if)# show run interface ethernet 1/6
!Command: show running-config interface Ethernet1/6
!Time: Mon May 15 12:58:46 2023
version 7.0(2)N1(1)
interface Ethernet1/6
description *** test multiple commands *** ;

Nexus NX-OS Tip No.4 – Tracking Recent User Configuration Changes

All commands executed within the Nexus NX-OS are logged by default. You can easily find who modified the configuration and when, as well as the exact commands that have been applied using the show accounting log command:

N5k-UP(config-if)# show accounting log | last 3
Mon May 15 13:05:12 2023:type=update:id=10.10.8.174@pts/2:user=admin:cmd=configure terminal ; interface Ethernet1/6 ; description test (REDIRECT))
Mon May 15 13:05:12 2023:type=update:id=10.10.8.174@pts/2:user=admin:cmd=configure terminal ; interface Ethernet1/6 ; description test (SUCCESS)
Mon May 15 13:05:16 2023:type=start:id=vsh.9446:user=admin:cmd=

The | last 3 parameter will display the last 3 entries.

Nexus NX-OS Tip No.3 – Creating Your Own NX-OS Alias Commands

Creating your own NX-OS alias commands is a great feature which helps simplify long and tedious commands. Cisco IOS users can also use the cli alias command to create IOS equivalent commands. For example we can create an alias copy running-config startup-config command and save it as wr to help other users with more experience on Cisco IOS devices work more easily around the Nexus switch.

Several useful examples are provided below:

N5k-UP(config)# cli alias name ipb show ip interface brief
N5k-UP(config)# cli alias name is show interface status
N5k-UP(config)# cli alias name hb show hsrp brief
N5k-UP(config)# cli alias name ps show port-channel summary
N5k-UP(config)# cli alias name wr copy running-config startup-config

N5k-UP(config)# wr
[########################################] 100%
Copy complete, now saving to disk (please wait)...

Nexus NX-OS Tip No.2 – Quickly Viewing and Executing Past Commands

The Nexus NX-OS allows users to easily view and recall past commands with the use of the show cli history command. When entered, the switch will list commands entered from the oldest to the most recent (indicated by the number on the left) along with the date/time executed. The switch will execute the selected command by using the exclamation mark (!) and the number of the command line.

In the example below we selected command No.9 from the history list by entering !9:

N5k-UP# show cli history
0 13:18:30 conf
<output omitted>
8 13:20:59 cli alias name id show interface description
9 13:21:04 show run | include alias
10 13:23:06 show cli alias

N5k-UP# !9
N5k-UP# show run | include alias
cli alias name sla source routetrack-1.3.py 8.8.8.8/32 management 10.10.8.176
cli alias name hello source helloPython.py
cli alias name ipb show ip interface brief
cli alias name is show interface status
cli alias name hb show hsrp brief
cli alias name ps show port-channel summary
cli alias name wr copy running-config startup-config
N5k-UP#

Nexus NX-OS Tip No.1 – Comparing Differences in Running & Startup Configuration

You can compare the output from a show command with the output from the previous snapshot of the same command. In particular, the Cisco NX-OS software creates temporary files for the most current output “show command” for all current and previous users’ sessions.

The “show run diff” command can be used to display the difference between running and startup configuration. The section starting with *** (stars) refers to the Startup-config while the section under --- (dashes) refers to the Running-config.

Note: The switch will not show any differences to the configuration after it is saved

N5k-UP# show run diff
*** Startup-config
--- Running-config
***************

*** 46,56 ****
interface Ethernet1/2
interface Ethernet1/3
interface Ethernet1/4
! description GREP2
interface Ethernet1/5
interface Ethernet1/6
description test

--- 45,55 ----
interface Ethernet1/2
interface Ethernet1/3
interface Ethernet1/4
! description *** TEST DIFF ***
interface Ethernet1/5
interface Ethernet1/6

N5k-UP# wr
[########################################] 100%
Copy complete, now saving to disk (please wait)...
N5k-UP# show run diff
N5k-UP#

The diff-clean command can be used to remove the temporary files for the current user's active session or for all past and present sessions for all users.

N5k-UP# diff-clean all-sessions
N5k-UP#

Summary

The Cisco NX-OS Software is a data center-class operating system with powerful scripting capabilities. This article showed how to make use of various Nexus NX-OS command options and operators, how to combine multiple Nexus commands, filter Show commands output, create and execute NX-OS scripts, introduced Python scripting and environment and covered a number of Nexus NX-OS hints and tips to help administrators and engineers make their day-to-day operation and administration of the Nexus Data Center switches faster, easier and safer!

Complete Guide to Nexus Checkpoint & Rollback Feature. Fast Recovery from Nexus Misconfiguration. Nexus 9000, 7000, 5000, 3000

2017-05-27T09:26:52+10:00

The Cisco NX-OS checkpoint feature provides the capability to capture at any time a snapshot (backup) of the Cisco Nexus configuration before making any changes. The captured configuration (checkpoint) can then be used to roll back and restore the original configuration.

The NX-OS checkpoint and rollback feature is extremely useful, and a life saver in some cases, when a new configuration change to a production system has caused unwanted effects or was incorrectly made/planned and we need to immediately return to an original/stable configuration.

With Catalyst IOS switches we would manually remove or restore IOS commands but with the Nexus NS-OS checkpoint-rollback feature this is a much faster and safer process that can be executed even by an authorized user with basic experience/knowledge on Nexus switches.

Key Topics:

Finally, we must point out that basic knowledge of the Cisco NX-OS is required for this article.

Additional Reading:

NX-OS Checkpoint & Rollback Limitations - Guidelines

The Checkpoint and Rollback feature has the following main configuration guidelines and limitations:

The maximum number of checkpoints supported is equal to ten (10).
Checkpoints are stored in an internal repository not accessible by the common user.
Checkpoints are persistent and synced between redundant supervisors.
It is not possible to apply or import the checkpoint file from another Nexus switch. Checkpoints can only be used on the device they were created on.
Only one user at a time can perform a checkpoint, rollback, or copy of the running configuration to the startup configuration.
Checkpoints are cleared from the system’s database after executing the write erase or reload command (switch reload).
Checkpoints can be manually cleared by running the clear checkpoint database command. The checkpoints saved to the bootflash are not affected by the aforementioned commands.
Checkpoints are only local to the NX-OS switch.
Rollback using files stored in bootflash is supported only if it has been created using the checkpoint command.
Checkpoint names must be unique. You cannot overwrite previously saved checkpoints. If attempting to overwrite existing checkpoints the user will receive the following error: ERROR: ascii-cfg: Checkpoint Name already exists (err id 0x405F002B)
Checkpoints are local to a virtual device context (VDC) for the Nexus 7000.
Rollback is not supported in the storage VDC for the Nexus 7000
Rollback is not supported on the Nexus 5000 after enabling the FCoE feature. System will generate the following error after enabling the FCoE feature: ERROR: FCOE is enabled. Disabling rollback module.

Understanding and Configuring the NX-OS Checkpoint Feature

Working with the Nexus Checkpoint feature is a very easy process and the commands used follow a logical order allowing the easy usage of this fantastic feature.

It’s important to always create a checkpoint before you begin making changes to the existing configuration of your Nexus switch. The command below shows the creation of our first checkpoint named Checkpoint-1 along with the optional description parameter (max 80 characters) allowing the easy identification of the checkpoint:

N5k-UP(config)# checkpoint Checkpoint-1 description *** Testing the checkpoint feature ***
.....Done

Once the checkpoint has been created, we can easily confirm its creation and details by issuing the show checkpoint summary command as shown below:

N5k-UP(config)# show checkpoint summary
User Checkpoint Summary
--------------------------------------------------------------------------------
1) Checkpoint-1:
Created by admin
Created at Mon, 08:10:29 22 May2023
Size is 15,568 bytes
Description: *** Testing the checkpoint feature ***

Note how the system not only provides all the necessary details about the recently created checkpoint but also shows which system user created it (admin). This detail is particularly important if the system is managed by multiple admins or engineers.

The NX-OS checkpoint feature doesn’t stop surprising thanks to the system’s intelligent features. Users can actually configure their Nexus switch to automatically generate checkpoints when specific changes occur in its running configuration. This awesome capability minimizes the risk for network downtime due to NX-OS key features misconfigurations and helps ensure there is always a valid snapshot that we can rollback to in case someone forgot to create a checkpoint before applying their changes!

Further capabilities built into this great feature allow the system to automatically create a checkpoint when aconfigured expiration period, e.g. the 120 days trial grace period of a license, has been exceeded. Reasons that could trigger automated system checkpoints are highlighted below:

License expiration of a feature
Disabling a feature with the no feature command
Removing an instance of a Layer 3 protocol

The system-generated checkpoint name convention has the format system-fm-feature. To help illustrate this automated feature we attempted to disable the VRRP feature on our Nexus 5000 therefore triggering the system to create a checkpoint. First we confirm the VRRP feature is enabled by issuing the show feature | include vrrp command then disable it and then verify it has been disabled:

N5k-UP(config)# show feature | include vrrp
vrrp 1 enabled
N5k-UP(config)# no feature vrrp

vrrp 1 disabled

N5k-UP# show checkpoint summary
User Checkpoint Summary
--------------------------------------------------------------------------------
1) Checkpoint-1:
Created by admin
Created at Mon, 08:10:29 22 May2023
Size is 15,568 bytes
Description: *** Testing the checkpoint feature ***
System Checkpoint Summary
--------------------------------------------------------------------------------
2) system-fm-vrrp:
Created by admin
Created at Mon, 11:31:41 22 May2023
Size is 15,581 bytes
Description: Created by Feature Manager.

Notice that the system now shows a second checkpoint system-fm-vrrp which did not previously exist. This second checkpoint was created automatically by the Nexus as soon as we disabled the vrrp feature.

Multiple checkpoints can be created to save different versions of the running configuration, however as previously noted, there is a limit of ten (10) checkpoints a user can create. When this limit is reached a warning message to overwrite the oldest checkpoint is shown:

N5k-UP# checkpoint Checkpoint-11
Checkpoints limit reached, this will overwrite the oldest checkpoint,
Continue? (y/n) [n]

How to Backup Nexus Checkpoints – Exceeding The 10 Checkpoint Limit

Those who require the ability to store more than 10 checkpoints can store checkpoint files in the bootflash (internal compact flash memory). This is a useful methodology to safely store checkpoint files as they won’t be erased with the write erase command or reboot of the Nexus switch.

To store the checkpoint to the bootflash simple use the checkpoint file bootflash: command and append the name to be used for the checkpoint file:

N5k-UP# checkpoint file bootflash:Checkpoint-11
.Done
N5k-UP# dir | grep 11
15568 May 22 11:54:53 2023 Checkpoint-11

The user and system checkpoint database can be manually cleared using the clear checkpoint database command. However, the checkpoint files stored at the bootflash are not affected by the clear checkpoint database command as displayed below:

N5k-UP# clear checkpoint database
...Done
N5k-UP# show checkpoint summary
N5k-UP#
N5k-UP# dir | grep 11
15568 May22 11:54:53 2023 Checkpoint-11

Rollback Configuration – Checking Differences Between ‘Running Config’ & ‘CheckPoint’

The rollback feature allows us to apply a checkpoint backup configuration of the Cisco NX-OS switch at any point without having to reload the switch. When executed, rollback will compare the running-configuration with the checkpoint and make the necessary changes to the running configuration so that they become identical. Network-admin user privileges are required to configure rollback.

To test the rollback feature on our Nexus 5000 we created a checkpoint (Checkpoint-1) and configured a description for interfaces E1/10-E1/14. We then will attempt to rollback to the initial checkpoint created (Checkpoint-1) which should remove the descriptions from interfaces E1/10-E1/14:

N5k-UP(config)# interface ethernet 1/10-14
N5k-UP(config-if-range)# description *** TESTING CHECKPOINT FEATURE ***

We can review the differences between the running-config and a checkpoint before applying the rollback command by executing the show diff command. It is always recommended to use the show diff command and review the configuration changes before applying the checkpoint configuration file:

N5k-UP(config)# show diff rollback-patch running-config checkpoint Checkpoint-1
Collecting Running-Config
#Generating Rollback Patch
!!
interface Ethernet1/14
no description *** TESTING CHECKPOINT FEATURE ***
exit
interface Ethernet1/13
no description *** TESTING CHECKPOINT FEATURE ***
exit
interface Ethernet1/12
no description *** TESTING CHECKPOINT FEATURE ***
exit
interface Ethernet1/11
no description *** TESTING CHECKPOINT FEATURE ***
exit
interface Ethernet1/10
no description *** TESTING CHECKPOINT FEATURE ***
exit

The next rollback command options are provided for the Nexus 5000, Nexus 7000 and Nexus 9000 Series:

Atomic: This is the default rollback type and applies the rollback file only if no errors occur
Verbose: This option displays the execution log and allows the user to see the applied configuration

N5k-UP(config)# rollback running-config checkpoint Checkpoint-1 ?
<CR>
atomic Stop rollback and revert to original configuration (default)
verbose Show the execution log

In addition, the Nexus 7000 and Nexus 9000 Series support the following extra rollback options:

Best-effort: Implement a rollback and skip any errors
Stop-at-first-failure: Implement a rollback that stops if an error occurs

The Nexus 3000 supports only the atomic rollback option.

Finally the rollback mechanism has been successfully applied and…

N5k-UP(config)# rollback running-config checkpoint Checkpoint-1 verbose
Collecting Running-Config
Generating Rollback patch for switch profile
Rollback Patch is Empty
Note: Applying config parallelly may fail Rollback verification
Collecting Running-Config
#Generating Rollback Patch
Executing Rollback Patch
========================================================
`config t `
`interface Ethernet1/14 `
`no description *** TESTING CHECKPOINT FEATURE *** `
`exit`
`interface Ethernet1/13 `
`no description *** TESTING CHECKPOINT FEATURE *** `
`exit`
`interface Ethernet1/12 `
`no description *** TESTING CHECKPOINT FEATURE *** `
`exit`
`interface Ethernet1/11 `
`no description *** TESTING CHECKPOINT FEATURE *** `
`exit`
`interface Ethernet1/10 `
`no description *** TESTING CHECKPOINT FEATURE *** `
`exit`
========================================================
Generating Running-config for verification
Generating Patch for verification

At this point it seems like the rollback to the selected checkpoint was successful. We can verify this by checking to see if there is any description on Ethernet interfaces 1/10-14:

N5k-UP# sh interface ethernet 1/10-14 | include description
N5k-UP#

The rollback configuration test has been completed successfully.

Finally, to rollback using a checkpoint file located in the system’s bootflash we simply specify its location as shown below:

N5k-UP# rollback running-config file bootflash:///Checkpoint-11

Summary

This article serves as a configuration guide for the Nexus NX-OS checkpoint and rollback feature. We have covered the main limitations and guidelines of this powerful Nexus NX-OS feature, demonstrated how to use the checkpoint & rollback feature and showed how to save checkpoints so they are not lost during a switch reboot or write erase. Finally, it is recommended that the configuration rollback procedure be used for managing change controls and notas a long term configuration management solution.

Introduction to Cisco Nexus Switches – Nexus Product Family. Differences Between Nexus NX-OS & Catalyst IOS. Comparing High-End Nexus & Catalyst Switches

2017-05-19T07:04:07+10:00

This article introduces the Cisco Nexus product family (Nexus 9000, Nexus 7000, Nexus 5000, Nexus 3000, Nexus 2000, Nexus 1000V and MDS 9000). We explain the differences between Nexus and Catalyst switches but also compare commands, naming conventions, hardware capabilities etc. between Nexus NX-OS and Catalyst IOS operating systems. To provide a comprehensive overview we explain where each Nexus model is best positioned in the Data Center and directly compare high-end Nexus switches (Nexus 9000/7000) with high-end Catalyst switches (Catalyst 6800 / 6500) examining specifications, bandwidth – capacity, modules and features (High-Availability, Port Scalability, VDC, vPC – VSS, OTV, VXLAN, etc).

For our readers convenience we have made available for free download over 90 different datasheets in our Cisco Data Center download section.

Following are the topics covered in this article:

Additional Reading:

Cisco Nexus Product Family

The Cisco Nexus Family of products has become extremely popular in small and large data centers thanks to their capability for unifying storage, data and networking services. Thanks to the Cisco Fabric Interconnect they are able not only to support all these services but also provide a rock-solid programmable platform that fully supports any virtualized environment. The Cisco Nexus family includes a generous number of different Nexus models to meet the demands of any Data Center environment. Let’s take a look at what the Nexus Family has to offer!

The Nexus Product Family

Cisco Nexus 9000 Series Switches

These data center switches can operate in Cisco NX-OS Software or Application Centric Infrastructure (ACI) modes. The main features of the new Cisco Nexus 9000 Series are: support of Fabric Extender Technology (FEX), virtual Port Channel (vPC), and Virtual Extensible LAN (VXLAN). There are a few key differences between the Cisco Nexus 7000 Series and Nexus 9000 DC switches. The Nexus 9000 supports Application Centric Infrastructure (ACI) in contrast to the Nexus 7000 switches. However, the Cisco Nexus 9000 switches do not support the VDCs (Virtual Device Context) technology like the Nexus 7000 and the Nexus 9000 Series doesn't support storage protocols, in contrast to the Nexus 7000. Finally, it is foreseen that the Nexus 9000 will complement the Nexus 7000 as data centers transition to ACI.

The Nexus 9000 Series Data Center Switches

The Nexus 9000 switches are available in a variety of models and configurations starting from the Nexus 9200 series (1 RU) Cloud Scale - standalone, Nexus 9300 series (1RU), Nexus 9300-EX (1RU) Cloud Scale standalone/ACI, Nexus 9500-EX (1RU) Cloud Scale Modules to the Nexus 9500 Cloud Scale switches (4, 8, 16 slots).

You can compare all available modes at the following URL:

https://www.cisco.com/c/en/us/products/switches/nexus-9000-series-switches/models-comparison.html

Download complete data sheets: Nexus 9500 series, Nexus 9300-EX series, Nexus 9300 series and Nexus 9200 series

Cisco Nexus 7000 Series Switches

They can provide an end-to-end data center architecture on a single platform, including data center core, aggregation, and access layer. The N7k series provides high-density 10, 40, and 100 Gigabit Ethernetinterfaces. The main features of the Cisco Nexus 7000 Series are: support for FEX, virtual Port Channel (vPC), VDC, MPLS and Fabricpath. In addition, the N7K supports fairly robust and established technologies for multi-DC interconnect (DCI) such as OTV and LISP. The N9K does not support these well-established DCI technologies, but a newer DCI technology, VXLAN, BGP, EVPN, that can be deployed for site-to-site DCI.

The Nexus 7000 Series Data Center Switches

The Nexus 7000 series consists of the 7000 and 7700 series switches, the latter being an updated series to the original 7000 series. The Nexus 7700 series offers higher bandwidth per slot (1.3Tbps compared to 550Gbps), greater performance and ability to support up to an impressive 192 100GE ports (7700 – 18 slot) compared to 96 100GE ports (7000 – 18 slot).

The Nexus 7000 is offered in 4, 9, 10 and 18 slot models while the 7700 comes in 2, 6, 10 and 18 slot models.

You can compare all available models at the following URL:

https://www.cisco.com/c/en/us/products/switches/nexus-7000-series-switches/models-comparison.html

Download complete data sheets: Nexus 7700 series or Nexus 7000 series

Cisco Nexus 5000 Series Switches

This product line is ideal for the DC access layer (End of Row), providing architectural support for virtualization and Unified Fabric environments. Cisco Nexus 5000 Series (N5k) can support VXLAN and comprehensive Layer 2 and 3 features for scaling data center networking. It supports Native Fibre Channel, Ethernet, and FCoE interfaces. The default system software includes most Cisco Nexus 5000 Platform features, such as Layer 2 security and management features. Licensed features include: Layer 3 routing, IP multicast, and enhanced Layer 2 (Cisco Fabric Path).

The Nexus 5000 Series Data Center Switches

The Nexus 5000 series switches are available in two platforms: 10 Gbps and 40 Gbps. The 5600 Series 10 Gbps platform is capable of delivering up to 2.56 Tbps switching capacity while the 5600 Series 40 Gbps platform can squeeze up to an impressive 7.68 Tbps.

All units except the Nexus 5696Q (40 Gbps) occupy between 1 and 2 RUs space whereas the Nexus 5696Q requires a generous 4 RU of rack space.

Full comparison of all available models can be found here: https://www.cisco.com/c/en/us/products/switches/nexus-5000-series-switches/index.html - Note the 5000 series Nexus switches are now End-of-Sales / End-of-Support.

Download complete data sheets: Nexus 5500 series or Nexus 5600 series

Cisco Nexus 3000 Series Switches

The product family offers features such as latency of less than a microsecond, line-rate at Layer 2 & 3 unicast, multicast switching, and the support of 40 Gigabit Ethernet interfaces. The Cisco Nexus 3000 Series switches are positioned for use in environments with ultra-low latency requirements such as financial High-Frequency Trading (HFT), High-Performance Computing (HPC) and automotive crash-test simulation Applications.

The Nexus 3000 Series Data Center Switches

The Cisco Nexus 3000 platform offers more than 15 models to satisfy all the switching needs an organization might have. The Nexus 3000 series offers switches starting with 1GE ports (Nexus 3000) and scales all the way up to 32 port 100GE ports with the Nexus 3232C model. Environments sensitive to delays will surely benefit from this series as they have been designed to practically eliminate any switching latency while at the same time offering large buffer spaces per port. Some models also have the ability to monitor their latency.

Full comparison can be found here: https://www.cisco.com/c/en/us/products/switches/nexus-3000-series-switches/models-comparison.html#~nexus3500

Download complete data sheets: Nexus 3000 series

Cisco Nexus 2000 Series Switches

These integrate into existing data center networking infrastructures as well as the Cisco ACI setup. The Cisco Nexus 2000 Series (N2k) utilizes FEX technology to provide flexible data center deployment models and to meet the growing server demands. This series is a flexible and low cost solution to add access and server ports to a data center. The parent switch of an N2k switch can be a Nexus 5000, Nexus 7000 or Nexus 9000 series switch. With FEX technology deployed, all the configuration and management is performed on the parent switch. In particular the N2k, with FEX technology, acts as a remote line card of the parent switches.

The Cisco Nexus 2000 platform offers over 10 models starting with a 24-port 1GE (Nexus 2224TP) all the way up to 48-port 1/10GE SFP/SFP+ (Nexus 2300).

Download complete data sheets: Nexus 2000 series

Cisco Nexus 1000v Series Switches

The Cisco Nexus 1000V Series (N1KV) is a software-based switch. It operates inside the VMware ESX hypervisor and utilizes the NX-OS Software. The Nexus 1000v architecture has two components: the Virtual Ethernet Module (VEM) and the Virtual Supervisor Module (VSM). These two components together make up the Cisco Nexus 1000V Series Switch, with the VSM providing the management plane and the VEM providing the data plane.

It should be noted that the Nexus 1000V Essential license is available at no cost and can provide various Layer 2 networking features.

Download complete data sheets: Nexus 1000v series

Cisco MDS 9000 SAN Switches

Cisco MDS 9000 Series Multilayer Switches are used to support Data Center SAN infrastructure. This series offers director-class platforms and Fabric switches. It utilizes the Cisco NX-OS software. Finally, the MDS 9000 can offer native fibre channel, storage services, and FCoE.

Download complete data sheets: MDS 9000 series

The Nexus Operating System – NX-OS Software

The Cisco NX-OS Software is a data center-class operating system that is built with modularity, resilience, and serviceability as its foundation. It is ideal for implementation within mission-critical data center environments where reliability and fault tolerance are very important.

The NX-OS architecture can perform three different main functions of a Data Center by being able to process Layer 2, Layer 3, and storage protocols. Each service (feature) in NX-OS runs as a separate independent protected process. In particular, each non-kernel process runs in its own protected memory space, providing fault tolerance while isolating any issues that arise with that process. For instance, if a Layer 2 service such as RSTP (Rapid Spanning-Tree Protocol) fails, it will not affect any other services running at that time such as the Layer 3 EIGRP service. In addition, NX-OS is based on the Linux kernel taking advantage of the characteristics offered by the most reliable OS.

Most NX-OS features are not enabled by default to achieve optimum processing and memory utilization, so if it is needed to deploy a technology like UDLD, this feature should be enabled manually. It should be mentioned that NX-OS offers feature testing for a 120-day grace period. Using the grace period enables customers to test a feature prior to purchasing a license.

A network engineer who is familiar with the traditional Cisco IOS command-line interface (CLI) will not face difficulties in using the NX-OS CLI for basic operations. The official Cisco Tool, Cisco IOS to NXOS Configuration Converter (required Cisco CCO account), can be helpful for the translation between CISCO IOS and NX-OS. This online tool is free and supports Catalyst 4900-6500 IOS configurations, which can be translated to NS-OS configuration for the Nexus 3000, Nexus 5000, Nexus 6000, Nexus 7000 and Nexus 9000 series.

Nexus NX-OS – Catalyst IOS Key Differences

There are key differences that should be understood prior to getting involved with the Cisco Nexus Operating System (NX-OS), these are highlighted below:

NX-OS uses a feature-based license model. Features such as Unidirectional Link Detection (UDLD) and Fibre Channel over Ethernet (FCoE) can be enabled via the feature configuration command. Configuration and verification commands for a specific feature are not available until that feature has been enabled.
NX-OS supports VDCs for Nexus 7000 platforms, which enables a physical device to be partitioned into logical devices. The default VDC is used when you log in for the first time.
By default, Secure Shell version 2 (SSHv2) is enabled and Telnet is disabled.
The default login administrator user is admin. It is no longer possible to login with just a password.
NX-OS uses a kickstart image and a system image. The kickstart image provides the Linux kernel and the system image provides the Layer 2/3 functionalities and features such as OTV, DHCP etc.
NX-OS supports Checkpoint & Rollback feature that allows the creation of configuration snapshots with the ability to rollback changes at any point without interrupting system functionality.
All Ethernet interfaces are called Ethernet. The FastEthernet, GigabitEthernet, TenGigabitEthernet interface naming conventions no longer exist.
The EtherChannel(IOS) naming convention has been replaced by Port-Channel (NX-OS).
The Write memory command is no longer available and has been replaced with the copy running-config startup-config.

Show commands can be executed identically from both the exec and config mode. e.g:

N7K (config)# show version

Show commands have parser help even in configuration mode.

Slash (forward-slash) notation supported for all IPv4/IPv6 masks. For instance:

N5K (config)# int e1/1
N5K (config-if)# ip address 10.1.1.1/24
N5K (config-if)# ipv6 address ::1/120

Two configuration models exist for the routing protocols:

- IGPs follow interface-centric model

- BGP follows neighbor-centric model

In any case the NX-OS alias command syntax can be used to create an alias for a shortcut. For instance, to use the write IOS command in NX-OS to save the running configuration, the next alias can be used:

N5K (config)#cli alias name write copy running-config startup-config

This alias executing the write command will run the command copy running-config startup-config

High-End Switches: Nexus vs Catalyst

The Nexus product family is tailored mainly for Data Center environments and offers the following advantages over Catalyst Core switches:

Interfaces: Only the Nexus 7000 series has 100GbE line cards available. Catalyst 6500 & 6800 Core switches offer interfaces up to 40GbE.
Capacity: Nexus 7000 series (Nexus 7700) has a maximum system capacity ~42Tbps and the Nexus 9000 (9500 model) 60Tbps. In contrast, the maximum system capacity of the Catalyst 6800 is much lower ~6Tbps.
Port Scalability: The Nexus family is much more scalable than the Catalyst 6500/6800 regarding maximum port density of 1G, 10G & 40G ports.
High Availability (HA): Nexus products can utilize vPC technology, which is one of the most commonly used Nexus HA features and is similar to the Catalyst VSS mode. It is used to provide multi-chassis link aggregation. The key difference is that vPC does not rely on a unified control plane as the VSS setup, so both Nexus switches can operate independently.
The Nexus 7000 VDC feature offers the capability to partition the Nexus switch into multiple independent logical switches. There is no possible way for VDCs to communicate with each other, aside from physically connecting a physical port in one VDC to a port in another VDC. A maximum number of four VDC for a Supervisor 1 (SUP1) - or Supervisor 2 (SUP2) based system, and up to eight for a Supervisor 2 Enhanced (SUP2E) based system is supported. The VDC feature actually implements a separate control plane for each context. The VDC virtual technology feature offers the advantage of consolidating several network physical devices.
The Nexus 5000, 7000 & 9000 series family support the use of the Nexus 2000 Series Fabric Extenders to additionally expand the system and provide a large-scale virtual chassis in the data center. This unique feature of the Nexus switches can greatly simplify the management and operation of a data center network.
The Nexus 7000 series can support several DC interconnection technologies which are not applicable to the Catalyst 6500 & 6800 Core Switches. In particular, the Nexus 7000 Series supports the well-established technologies OTV, VXLAN and Fabric path.
The NX-OS is much more robust operating system than IOS. NX-OS is built with modularity, resilience, and service ability as its foundation.
The Nexus 7000 and 5000 series switches can implement Converge LAN/SAN Network setup by supporting storage protocols (FC, FCoE) which are not supported by the Catalyst 6500 & 6800 switches.
The Nexus switches cannot accept service module line cards such as Firewall (FWSM) or Wireless (WISM) Service modules like the Catalyst 6500 & 6800 switches.
Finally, the Nexus switches do not support the NAT feature in contrast to the Catalyst 6500 & 6800.

Nexus Basic Design Aspects – Where The Nexus and MDS Switches Fit In a Data Center

This section identifies the typical placement of the Cisco Nexus and MDS Families series switches in a Cisco Data Center.

Single-Tier Nexus Data Center Design

The Cisco Nexus 7000 Series can be used for both access and core layer connectivity in the single-tier data center architecture. The access layer connectivity for the servers can be provided with low cost 48-port Gigabit Ethernet linecards or with the 32-port 10 Gigabit Ethernet linecards if 10GE interfaces are required.

Single-Tier Nexus Data Center Topology

The single-tier data center architecture (shown above) can be expanded by connecting Cisco Nexus 2000 fabric extenders to Cisco Nexus 7000 Series switches to provide connectivity for the servers. It should be mentioned that the Nexus 2000 can be used only to provide connectivity to servers or end hosts and should not be connected with switches. This setup would provide a Top-of-Rack (ToR) solution for the servers with a Cisco Nexus 7000 Series switch acting as the management point, and collapsing the access, aggregation, & core layers. It should be highlighted that if the budget is limited then the Nexus 9000 is the best alternative to the Nexus 7000. A pair of Nexus 5000 switches in a single-tier setup is a common low cost solution for small Data Centers.

Two-Tier Nexus Data Center Design

The two-tier data center option connects the Cisco Nexus 2000 Fabric Extenders to an upstream Cisco Nexus 5000 Series switch. The Cisco Nexus 5000 functions as an End-of-Row (EOR) access switch and is connected via multiple links to a pair of Cisco Nexus 7000 switches. This topology provides an access layer and a collapsed core and aggregation layer.

Two-Tier Nexus Data Center Topology

Three-Tier Nexus Data Center Design

The three-tier data center is similar to the two-tier data center architecture regarding the access layer and the placement of the Nexus 5000 and Nexus 2000 switches. However, multiple Nexus 7000 switches are used to the aggregation layer. The core layer is provided by a pair of Cisco Nexus 7000 Series switches:

Three-Tier Nexus Data Center Topology

The Nexus 9000 switches, due to their exceptional performance and comprehensive feature set, are versatile platforms that can be deployed in multiple scenarios such as layered access-aggregation-core designs, Leaf-and-spine architecture and Compact aggregation-layer solutions.

The Cisco MDS 9000 Series Multilayer Switches can provide the SAN connectivity at the accesslayer and the storage core layer. Connectivity between the SAN and LAN infrastructures to support FCoE would be supported through the Cisco Nexus 7000or 5000 seriesswitches and the Cisco MDS 9000 Series core layer.

Summary

This article introduced the Cisco Nexus product family. We explained how the Nexus platform differentiates from the well-known Catalyst switches and examined key-differences in the two platforms and operating systems (NX-OS – IOS). We analyzed each Nexus series including the well-known MDS 9000 and showed how Single-Tier, Two-Tier and Three-Tier Data Center topologies make use of the Nexus platform. For more information including technical articles visit our Cisco Data Center section.

Join Australia’s First Official Cisco Data Center User Group (DCUG) & Become Part of a Friendly Fast-Growing Professional Community That Meets Once a Month in Melbourne!

2016-05-31T19:52:33+10:00

It’s a reality – Australia now has its own Official Cisco Data Center User Group (DCUG) and it’s growing fast! Originally inspired by Cisco Champions Chris Partsenidis and Derek Hennessy, the idea was fully backed by Cisco Systems as they happened to be looking to start up something similar on a global scale.

The idea was born in the morning hours of the 18^th of March 2016 over a hot cup of coffee when Chris Partsenidis and Derek Hennessy met for the first time, after Cisco’s Live! in Melbourne Australia. Both Chris and Derek agreed that it was time to create a friendly professional Cisco community group that would gather Cisco professionals and encourage users to share knowledge and experience.

The proposal was sent to Lauren Friedman at Cisco Systems, who just happened to be working on a similar concept on a global scale. Lauren loved the idea and, with her help, Australia got its first official Cisco Data Center User Group!

Becoming part of the Melbourne Cisco Data Center User Group is absolutely free and, by joining, you’ll be part of Australia’s first official Cisco user group, which is currently the largest in the world!

Where are the Meetings Held and What’s Included?

The user group will catch up on the first Tuesday of every month at the The Crafty Squire at 127 Russell Street in Melbourne CBD. We’ll be located upstairs in Porter Place. Our first meeting will be on Tuesday June 7th 2016 and all meetings will take place between 17:30 and 19:30.

For the duration of the meeting, we’ll have free beer for all registered members, food and if we are lucky – free Cisco beer mugs! The mugs are actually on their way from the USA and we are hoping to have them in time before the meeting otherwise we’ll be handing them out during the following meeting.

Figure 1. The Porter Place - Crafty Squire

For more details about our regular meet ups and join our community, head over to the Cisco Data Center User Group page on Meetup.com.

We're really excited to start building a Data Center community in Melbourne so come along and join us!

Agenda – 7^th of June 2016

Vendor Session: Infrastructure as Code and DevOps

Speaker: Chris Gascoigne - Technical Solutions Architect, Cisco Systems Melbourne, Australia

Chris Gascoigne is a Technical Solutions Architect with Cisco Systems working in the Australia/New Zealand Data Centre team. Chris has been with Cisco for nine years and specialises in Application Centric Infrastructure.

Community Session: GNS3 Connectivity

Speaker: Will Robinson - Senior Systems Engineer

Will Robinson is a Senior Systems Engineer and has extensive networking and data center experience. Will is an active community member and is the only Australian member of the NetAppATeam group.

Cisco Data Center (Nexus/UCS)

Nexus 7000/7700 Software Upgrade via ISSU. Complete Upgrade Guide, Configuration Check, Verifying ISSU Capability

Related Articles

The Complete Cisco Nexus vPC Guide. Features & Advantages, Design Guidelines, Configuration, Failure Scenarios, Troubleshooting, VSS vs vPC

vPC Peer

vPC Peer-link

vPC Peer Keepalive Link

vPC Domain

vPC Member Port

Cisco Fabric Services (CFS)

Orphan Device

Orphan Port

non-vPC VLAN

Step 1: Enable the vPC Feature and Configure the vPC Domain ID on Both Switches

Step 2: Choose a Peer Keepalive Deployment Option

Step 3: Establish the vPC Peer Keepalive Link

Step 4: Configure the vPC Peer-Link

Step 5: Configure Individual vPCs to Downstream Devices

Step 6: (Optional) Enable the Peer-Gateway Feature to Modify the FHRP Operation

Step 7: (Optional) Enable the Peer-Switch Feature to Optimize the STP Behaviour with the vPCs

Step 8: (Optional) Optimize vPC performance: ‘ip arp synchronize’, ‘delay restore’, ‘auto-recovery’, ‘graceful consistency-check’ & ‘role priority’ commands

Answer:

NEXUS NX-OS: Useful Commands, CLI Scripting, Hints & Tips, Python Scripting and more

Executing Python Scripts

Summary

Complete Guide to Nexus Checkpoint & Rollback Feature. Fast Recovery from Nexus Misconfiguration. Nexus 9000, 7000, 5000, 3000

Additional Reading:

Introduction to Cisco Nexus Switches – Nexus Product Family. Differences Between Nexus NX-OS & Catalyst IOS. Comparing High-End Nexus & Catalyst Switches

Additional Reading:

Join Australia’s First Official Cisco Data Center User Group (DCUG) & Become Part of a Friendly Fast-Growing Professional Community That Meets Once a Month in Melbourne!

Where are the Meetings Held and What’s Included?

Agenda – 7th of June 2016

Agenda – 7^th of June 2016