Basic Terms
Fibre Channel and Protocols
Fibre Channel was standardized as early as 1994 and gave rise to SAN networks. For the enterprise environment, this was a clear solution for storage, although it was quite costly (and more complicated when connecting geographic locations). After a while, iSCSI emerged as a cheaper alternative, but it has now clearly penetrated the enterprise environment as well. To this day, it is an interface with low latency and high throughput (although Ethernet networks are still faster). FC runs on the same layer as Ethernet (L2), but they are completely incompatible with each other.
FC - Fibre Channel - is a full-duplex, serial, block-oriented, point-to-point communication interface (network technology), designed for high-speed data transmission (speeds of 1, 2, 4, 8, 16, 32, 128 Gbps, currently mostly 16 and 32 Gbps). It is commonly used for connecting disk arrays (storage devices) to servers in SAN networks, so it is a block transfer and we connect directly to the disk space (it appears as a local disk on the server). Typically, optical fibers are used as the physical medium for transmission. FC is designed with high reliability, no frame loss can occur and frames must be delivered in the correct order.
FCP - Fibre Channel Protocol - a transport protocol that transmits SCSI commands over an FC network. FCP works over FC, like TCP/IP over Ethernet. Compared to TCP/IP, FCP is specially designed for one purpose, so it is more optimal and leaner (less overhead).
FCoE - Fibre Channel over Ethernet - a standard that allows encapsulation of FC frames into Ethernet networks, typically using copper cables (Twisted Pair Copper Wire) and a speed of 10Gbps. This reduces the complexity of the data center, because we can use standard switches and cabling, while maintaining the properties of FC. Unlike iSCSI, which runs over TCP/IP, FCoE runs over Ethernet (on the same layer as IP - L3). It therefore has less overhead, but the problem is that it is not routable through TCP/IP WAN. It supports standard NICs, but often a special Converged Network Adapter (CNA) is used. Cisco is trying to promote FCoE and supports it on Nexus switches, some of which support Unified Fabric, so they have Unified ports that can be switched between Ethernet, FC and FCoE. FCoE is able to look into the FC layer and acts as an FC switch.
Network Topology, Components and Terms
For Fibre Channel (FC) we need a special Host Bus Adapter card (something like a network card) in the servers and dedicated FC switches. For addressing, long World Wide Names (WWN) are used, which is the equivalent of MAC addresses, and nothing like IP addresses or DNS names is used (so addresses can't be used from memory). The FC network is flat and usually it is said that FC switching is used and there is no routing.
Fabric - Fibre Channel Fabric or Switched Fabric, it is a network topology used in FC networks (the most commonly used of the three possible). Fabric is a single closed network (SAN). It consists of one or more switches (FC switches), to which end devices (servers and storage) are connected. It is very similar to a classic LAN network, multiple devices can be active at the same time, the medium is not shared. For high availability, the SAN network is usually built with two separate fabrics (two mutually disconnected groups of switches, end devices are connected to both), i.e. two independent paths.
VSAN - Virtual Storage Area Network is the equivalent of VLAN (Virtual Local Area Network) from Ethernet (created by Cisco and currently an ANSI standard). Using VSAN, we can group ports and create a virtual fabric. A single port can belong to multiple VSANs and ports from different switches can be assigned to the same VSAN. Within a VSAN, we have separate traffic, as well as security policy, zones, membership, etc.
NIC - Network interface card (NIC) - a network card, a component that connects a computer to a computer network.
HBA - Host Bus Adapter - the equivalent of a network card, a component that connects a computer to a SAN network. The term is most often used for an FC interface card, but also for a specialized card for iSCSI. The HBA implements various common operations in hardware and thus relieves the CPU (e.g. offloads TCP/IP and iSCSI protocol).
WWN - World Wide Name - a unique identifier used in the Fibre Channel world, the equivalent of an Ethernet MAC address, the length is 8 bytes (64 bits).
WWPN - World Wide Port Name - WWN assigned to a port in the Fibre Channel Fabric, Cisco sometimes uses the term Port WWN, example 50:40:20:c0:00:f6:93:da
WWNN - World Wide Node Name - WWN assigned to a node (end device, disk array, switch) in the Fibre Channel Fabric, the same WWNN can be seen on all ports of the device, but a different WWPN each time, example 50:40:20:c0:00:f6:93:db
Zoning (masking) - VSAN/SAN zoning, we create zones and only devices in a given zone can communicate with each other. It is set on the switch. It is not mandatory to use, but in practice it is always used for security reasons.
Multipathing - if there are multiple paths between the server (initiator) and the array (target), it presents them as one and uses High Availability and Load Balancing.
Configuring FC on Windows Server
For FC SAN, almost no configuration is done on the server side, everything happens on the array side (creating and assigning disk space) and the network side (zoning). We need an FC HBA in the server and the appropriate driver installed. The server automatically detects the LUNs it has available on the network. If we use multiple paths, we only need to set up Multipath, otherwise we will see the presented disk multiple times (see picture). There is no need to configure anything like an iSCSI Initiator.
Note: For testing communication in the FC SAN network (e.g. verifying that we have zoning configured correctly from the server to the array), the standard tool used is fcping, which is available on Linux, in most FC switches, but unfortunately not in Windows. A while ago, such a utility could be downloaded, but I couldn't find a working one now.
Determining WWN on the Server
To configure zoning on FC switches and also to be able to assign LUNs to the server on the storage array, we need to know the FC WWPN (Fibre Channel World Wide Port Name) of its HBA ports. On a Windows Server, we can determine this in several ways.
Starting from Windows Server 2003, Microsoft has a downloadable Fibre Channel Information Tool (MSI package to install), the fcinfo tool. On Windows Server 2012 R2, it gives an error on installation, and I didn't try to get it working further.
C:\>fcinfo
There are 2 adapters:
com.qlogic-QLA2342-0: PortWWN: 21:00:00:e0:8b:8e:8e:8e \\.\Scsi2:
com.qlogic-QLA2342-1: PortWWN: 21:01:00:e0:8b:ae:8e:8e \\.\Scsi3:
C:\>fcinfo /ports
com.qlogic-QLA2342-0, num: 3
-----------------------------------------------------------------------------
| Type State Speed WWN PortWWN |
-----------------------------------------------------------------------------
N_Port, online, 2gbs, 20:00:00:e0:8b:8e:8e:8e, 21:00:00:e0:8b:8e:8e:8e
N_Port, online, unkn, 50:00:1f:e1:50:07:a3:70, 50:00:1f:e1:50:07:a3:79
N_Port, online, unkn, 50:00:1f:e1:50:07:a3:70, 50:00:1f:e1:50:07:a3:7d
com.qlogic-QLA2342-1, num: 3
-----------------------------------------------------------------------------
| Type State Speed WWN PortWWN |
-----------------------------------------------------------------------------
N_Port, online, 2gbs, 20:01:00:e0:8b:ae:8e:8e, 21:01:00:e0:8b:ae:8e:8e
N_Port, online, unkn, 50:00:1f:e1:50:07:a3:70, 50:00:1f:e1:50:07:a3:78
N_Port, online, unkn, 50:00:1f:e1:50:07:a3:70, 50:00:1f:e1:50:07:a3:7c
C:\> fcinfo /ports /details
In Windows Server 2012 R2, the PowerShell cmdlet Get-InitiatorPort was added.
PS C:\> Get-InitiatorPort -ConnectionType FibreChannel | FT -AutoSize InstanceName NodeAddress PortAddress ConnectionType ------------ ----------- ----------- -------------- PCI\VEN_1077&DEV_2432&SUBSYS_1705103C&REV_03\4&370d11c1&0&0038_0 50014380120ed0f5 50014380120ed0f4 Fibre Channel PCI\VEN_1077&DEV_2432&SUBSYS_1705103C&REV_03\4&370d11c1&0&0138_0 50014380120ed0f7 50014380120ed0f6 Fibre Channel PS C:\> Get-InitiatorPort | FL *
Another option is to use the management software from the HBA manufacturer or look in the HBA BIOS during server boot. On Windows Server 2008, the Storage Explorer application is available, where we can see a lot of information including WWPN.
Configuring Multipath
For Fibre Channel Multipath, the Microsoft Multipath I/O (MPIO) feature is part of Windows Server 2012 R2. This is the same feature that we can also use for iSCSI or SAS (FC is a basic feature, the other two must be enabled). It includes a generic Microsoft DSM (Device Specific Module), which supports storage arrays with Asymmetric Logical Unit Access (ALUA) controller model (defined in SPC-3) and Active/Active controller model.
Adding the MPIO Feature
MPIO is a system feature that we need to add:
- open Server Manager - Add roles and features
- navigate to the Features section and check Multipath I/O
- complete the wizard
Configuring MPIO
- launch the configuration from Control Panel - MPIO (or also find MPIO under Administrative Tools or in the Start menu -
mpiocpl.exe) - switch to the Discover Multi-Paths tab
- we should see the devices detected in the SAN network in the list
- click Add, the device will be installed and a dialog will appear stating that a restart is required (when I had Disk Management open, the duplicate disks were already removed at this point)
Note: We can also try the command mpclaim.exe -e, which will list the detected Storage systems.
More information on Multipath and disk operations is described in the article iSCSI SAN network and configuration on Windows Server 2012 and it is very similar for FC.
There are no comments yet.