The SAN has been a mainstay of IT storage infrastructure discussions for well over a decade. Originally, the only storage networking option for creating a SAN was Fibre Channel, but a few years later,
Fibre Channel is the original dedicated storage networking technology; indeed, the term SAN was popularized with the first Fibre Channel products. The Fibre Channel Protocol (FCP) transports SCSI commands over fiber optic and twisted-pair copper cables and can be implemented in a point-to-point, arbitrated loop and switched fabric topology, the latter becoming the most common in SAN applications. FC is also a drive interconnect technology (used to connect drives to servers and to controllers in dedicated storage systems) although it’s been largely replaced by SAS in this application.
Fibre Channel provides signaling speeds as fast as 16 Gbps -- although 4 Gbps and 8 Gbps FC is still very common -- and requires dedicated host bus adapters (HBAs) for all endpoint devices. FC fabrics are extremely flexible and can span distances up to 50 kilometers with the right components.
Fibre Channel over Ethernet (FCoE) is an adaptation of FC that allows it to run over Ethernet networks. A key use case is large data centers that are interested in simplifying their network infrastructures by converging their FC and Ethernet networks. FCoE is not a good SAN option for SMBs, since many don’t already have a Fibre Channel SAN that they’re looking to simplify.
InfiniBand is an interconnect protocol that offers point-to-point, bidirectional serial links designed mainly for high-speed applications. Each connection provides signaling rates of up to 2.5 Gbps in each direction, and links can be aggregated to increase bandwidth. InfiniBand is used as an interconnect for high-performance computing infrastructures, scale-out storage clusters and high-speed direct-attached storage. From an SMB perspective, InfiniBand is probably not an appropriate SAN technology.
iSCSI is a storage networking standard that transmits SCSI storage commands over IP networks. iSCSI uses an initiator installed on each host to connect over a standard Ethernet LAN to a target device, typically a storage system. Initiators can be software-based, embedded in the operating system, or hardware-based, using a dedicated iSCSI HBA. Compared with FC, iSCSI involves more overhead as it must process the TCP/IP stack. For companies interested in network performance, this overhead was somewhat of a concern with early iSCSI implementations. However, with the processing power that’s currently available on most computers, the issue of overhead has faded to a large degree.
An iSCSI SAN is relatively easy to set up, since most servers include iSCSI initiators in the OS and most storage devices are also set up to be iSCSI targets. This makes iSCSI a good choice for companies without a high degree of networking expertise and those that want to consolidate existing storage resources onto a SAN, common characteristics in the SMB space.
ATA over Ethernet (AoE) is a network protocol that encapsulates ATA commands in Ethernet frames and transmits them over an Ethernet network but at the Data Link layer (Layer 2 of the OSI model), below the Network and Transport layers that are used by LANs and the Internet (layers 3 and 4). This means that AoE uses standard Ethernet hardware (cabling, switches and NICs) but doesn’t use IP and can’t be accessed over the Internet or other IP networks.
AoE requires that a host initiator be installed on the server and that the storage system it’s accessing be an AoE target. The AoE driver is included in the Linux kernel, and drivers are available for Windows, Mac OS X and some Unix variants. UK-based Coraid is the leader in AoE technology and currently offers several storage array products.
AoE storage systems use ATA drives, rather than the more expensive SCSI drives used by the SAS protocol or Fibre Channel drives. This, along with its use of standard Ethernet networking, keeps costs down, making AoE one of the most cost-effective SAN technologies, especially when compared with FC.
Serial-attached SCSI (SAS) has replaced FC as the disk drive connectivity of choice for performance applications. With the introduction of 6 Gbps SAS, SAS switching has emerged as another SAN-enabling protocol. SAS switches use “SAS expander” technology to connect independent SAS storage systems to multiple host servers and create what’s been called “shared direct-attached storage.” Each “wide port” on a SAS switch provides 24 Gbps total bandwidth (four “lanes” of SAS times 6 Gbps) and can be zoned for secure access.
SAS SANs are limited by SAS cable lengths (10 meters is standard, but 25 meters can be achieved with “active” cables), but in the SMB space this isn’t an issue since SANs are typically implemented in a single location. LSI is the leader in the SAS switching space with several switch models on the market, including at least one OEM product sold by a major server vendor.
Which one makes sense for your SMB customers?
For smaller companies, iSCSI, SAS and AoE are probably the most appropriate SAN solutions. Fibre Channel is more affordable than in years past, but it still requires its own networking infrastructure. And every small company already has an Ethernet LAN and at least a cursory understanding of Ethernet networking.
iSCSI probably represents the mainstream solution. It’s been available for over a decade and is the most widely supported. iSCSI can be used with any existing block storage that’s accessible on the network, so it’s a good solution if the objective includes consolidating existing storage into the SAN. It will require a dedicated network, but so do the other protocols. That said, SAS and AoE SANs are options worth exploring for smaller companies, especially if a new storage array is planned.
SAS use cases in SMB environments are often situations where shared storage is needed for a few critical applications, such as extending a direct-attached storage system so it can be used by two servers for high availability or enabling multiple hypervisors to share a SAS storage system to support VM migration. For a new SAN, a SAS switch needs to be purchased, but existing SAS disk arrays can be used. Since the SAS protocol supports SATA drives, a SAN built with SAS switching can be used to create a tiered storage infrastructure, with SAS drives for performance and SATA drives for capacity. And its distance limitations won’t be an inhibitor for most SMBs.
Originally a Linux-based solution (the AoE driver has been part of the Linux kernel since 2004), AoE is now making its way into Windows environments, which is essential for small companies. That said, AoE may be more of a small and medium-sized enterprise (SME) play, since its primary target seems to be existing FC implementations or environments that are considering FC. It offers equal or better performance and a potentially simpler implementation than a traditional Fibre Channel SAN. Like iSCSI, it doesn’t have the distance limitations of SAS switching and doesn’t require dedicated switches, although it does need a dedicated Ethernet subnet.
Eric Slack is a senior analyst with Storage Switzerland.
This was first published in December 2011