Note: NPV is not supported on all switches, but only on selected models of the Cisco MDS 9000 and Cisco Nexus 5000 series. I conducted tests on the Cisco MDS 9148S with NX-OS version 6.2(9).
Fibre Channel (FC) switch can operate in two modes. Either the classic Fabric mode (sometimes referred to as Switch mode), where communication is processed and switched, zoning is performed, etc. Or in NPV mode, where communication is only forwarded, the switch acts as a proxy or forwarder. It becomes merely an extension of ports (some extender) for another FC (core) switch that already operates in the classic Fabric mode.
When we want to use NPV, we must have one switch that we switch to NPV mode. And another that supports NPIV and will process the communication of devices connected to the first switch. We connect these switches together, but not using E ports - ISL links (we described port types last time), because the switch in NPV mode does not behave like a switch, but rather like an end device. So on the superior switch, we use F port. And on the NPV switch, a special type of NP port.
- N port - Node port - this is a port on the end device side (it can be an HBA port of a server or a target port on a disk array)
- NP port - Proxy Node port - a port on the switch in NPV mode, which connects to the superior (core) switch, behaves like an N port, but allows multiple physical N ports to be proxied through it
NPIV (N Port Identifier Virtualization)
N Port (Node Port) is a Fibre Channel (FC) port on the end device (this port connects to the F Port on the switch). It is typically assigned a single 3-byte (24-bit) N Port Identifier (N Port ID), more commonly referred to by Cisco as Fibre Channel ID (FCID). This FCID is assigned by the switch during the FLOGI process. Each N Port should therefore be assigned exactly one FCID and typically has one WWPN. Simplified, we see on the switch at each F Port (where the end N Port is connected) one FCID and one WWPN.
NPIV allows one physical N Port to have multiple FCID and WWPN. After the normal FLOGI process, subsequent registrations of additional WWPN and FCID can occur. A switch with NPIV enabled allows an F Port to see multiple WWPN and FCID and handle them.
We use this, for example, in virtualization (if we want to bring Fibre Channel to the virtual machine), where we would normally see only one WWPN of the physical port of the node. This way, we can see the WWPN of individual virtual servers and perform detailed zoning on the FC switch.
NPV (N Port Virtualization) mode
NPV is used in large SAN deployments to reduce the number of Fibre Channel domain IDs. Each switch that connects to the Fabric (SAN or VSAN) is assigned an 8-bit Domain ID (this is used as the first part of the assigned FCID) and a maximum of 239 Domain IDs (i.e., the number of switches) is supported. A switch in NPV mode does not connect to the fabric but transfers traffic between the Core switch and end devices. Therefore, it does not need a Domain ID (FCIDs are assigned by the Core switch and use its Domain ID). In some cases, it can also simplify management.
Another option is that NPV allows us to connect a Cisco MDS switch with a Fibre Channel switch from other manufacturers (e.g., Brocade). NPV solves interoperability issues because it does not run the Fabric service (it does not terminate the Fabric) but forwards requests further. Standard client communication thus arrives at the Core switch. We connect to a regular F port, so we do not need to establish an InterSwitch Link (ISL) with E Ports. Different manufacturers use different addressing methods or send different data over ISL.
If we have a setup where there is a central Core switch and Edge switches connected to it, to which end devices are connected. We can enable NPIV on the Core switch and set the Edge switches to NPV mode. The port on the Edge switch that connects to the Core is set as an NP Port, on the other side an F Port. NPV aggregates multiple locally connected N Ports into one or more external NP links. All end devices connected to the Edge switch will be seen on the Core switch port. The NP Port acts as a proxy for all N Ports on the NPV switch. Zoning and other logic will be set on the Core switch.
NP link (uplink, also referred to as External Interface) is a connection from the NP port of the Edge switch to the F port of the Core switch. When established, the switch first logs itself in using FLOGI and registers itself on the Name Server of the Core switch. Subsequent FLOGIs from end devices are converted to Fabric Discovery Messages (FDISC).
Configuration in NX-OS
Enabling NPIV
Enabling is done by allowing the NPIV feature. At least on some devices, it is enabled by default.
SWITCH(config)#feature npiv
Checking the NPIV status
SWITCH#show npiv status
NPV Configuration
Enabling NPV mode is done for the entire switch, resulting in its restart, configuration reset (everything related to FC is erased), and a significant change in its behavior. In NPV mode, only a limited set of CLI commands and features is available. Therefore, before converting the switch to NPV mode, it is necessary to carefully consider and back up the current configuration.
For example, information about Fabric Login and FCNS is not available because this occurs on the Core switch. Therefore, commands like show flogi database and show fcns database must be used on the Core switch.
The official documentation states that switching to NPV mode is done with the following command, but I do not have such a command on my switch.
SWITCH(config)#npv enable
In practice, enabling the NPV feature directly results in the switch restarting and switching modes. We receive a warning (and can cancel the action) that a restart will occur and the configuration will be removed.
SWITCH(config)#feature npv Verify that boot variables are set and the changes are saved. Changing to npv mode erases the current configuration and reboots the switch in npv mode. Do you want to continue? (y/n):y
As we have already mentioned, the configuration is not completely erased. Basic configuration such as management IP and GW, switch name, users (AAA settings), things like SNMP, Syslog, NTP, time settings remain. So we can still connect to the switch remotely. FC, VSAN, interface, zone, etc. configurations are removed.
After restarting in NPV mode, we configure the uplink port on the Core switch (and of course the port on the other side). The main thing is to set it to NP mode and enable it.
SWITCH(config)#interface fc1/1 SWITCH(config-if)#switchport mode NP SWITCH(config-if)#no shutdown
If we want to use VSAN, we need to define it anew and assign the port.
SWITCH(config)#vsan database SWITCH(config-vsan-db)#vsan 10 interface fc1/1
Other ports, where end devices are connected, are set to F mode and assigned to the appropriate VSAN.
SWITCH(config)#interface fc1/2 - 10 SWITCH(config-if)#switchport mode F SWITCH(config-if)#no shutdown SWITCH(config)#vsan database SWITCH(config-vsan-db)#vsan 10 interface fc1/2 - 10
One final note regarding the Port Channel and VSAN Trunk features. Last time we mentioned that both features can only be used on E ports (or TE ports). However, Cisco also supports other options, mainly in cooperation with the NPV switch. Therefore, we can set a Trunk on an F port (TF) connected to an N port (TN) on the end device (using the Exchange Virtual Fabrics Protocol - EVFP) or an NP port (TNP) on the NPV switch (using the Cisco Port Trunking Protocol - PTP). Multiple F ports connected to NP ports can also be combined into a PortChannel and use a Trunk (again using PTP, which supports trunking PortChannels). It should also work to connect an F port on a non-Cisco Core switch to an NP port on a Cisco NPV switch and use a Trunk (using EVFP).
Displaying NPV Information
We can display information about interfaces.
SWITCH#show npv status npiv is disabled disruptive load balancing is disabled External Interfaces: ==================== Interface: fc1/1, VSAN: 10, FCID: 0x03c200, State: Up Number of External Interfaces: 1 Server Interfaces: ================== Interface: fc1/2, VSAN: 10, State: Up Number of Server Interfaces: 1
List of devices and their NP link (external interface).
SWITCH#show npv flogi-table -------------------------------------------------------------------------------- SERVER EXTERNAL INTERFACE VSAN FCID PORT NAME NODE NAME INTERFACE -------------------------------------------------------------------------------- fc1/2 10 0x03c110 51:40:2e:c0:00:ff:83:ac 51:40:2e:c0:00:ff:83:ad fc1/1
Around NPV, we can display various other information. Error events, messages, or external interface usage may be useful. Alternatively, a detailed technical output.
SWITCH#show npv internal errors SWITCH#show npv internal msgs SWITCH#show npv external-interface-usage SWITCH#show tech-support npv
To display FCNS, we need to connect to the Core switch and display information from all Edge switches there.
SWITCH#show fcns database
There are no comments yet.