Problem solve Get help with specific problems with your technologies, process and projects.

Designing storage area network for multiple fabrics

When reviewing a customer's storage area network (SAN) topology, determine if, when and how you'd want to scale beyond a single SAN fabric.

If you're currently reviewing a customer's storage area network (SAN) design and considering scaling beyond a single fabric, there are a variety of technologies and products that can help.

First we'll review reasons why you'd scale out a SAN beyond a single fabric on a local or wide-area basis.

Why scale out a SAN

  • Consolidation of servers, storage, backup, applications and data centers.
  • Data protection and business continuance, including regulatory compliance.
  • Technology replacement combined with business growth and new application needs.
  • Ability to support more devices than a single fabric can support.
  • Isolation and segregation of workloads, applications and customers.
  • Interoperability and coexistence of heterogeneous technologies.
  • Understanding your customer's storage networking objectives, along with what level of scalability and resiliency they require, is integral to developing the storage area network design. Attributes and service requirements associated with scaling a storage network include:

  • Performance (bandwidth and latency) including lack of congestion.
  • Support for more servers and storage devices for growth.
  • Support for backup, recovery and business continuance.
  • Block access, file access (NAS), or combination of block and NAS access.
  • Ease of management, diagnostics and provisioning of resources.
  • Span distances to maintain data accessibility on a local and regional basis.
  • Now that you've determined when you'd scale beyond a single fabric, you must decide how you're going to go about accomplishing this goal. Here are your options:

    How to scale out a SAN

  • Establish additional independent fabrics consisting of one or more switching devices with various numbers of ports per switch.
  • Interconnect fabrics with inter-switch links (ISLs) creating one large physical and logical fabric. While this approach enables any to any connectivity, it does not increase the addressing range and number of devices supported per fabric. This approach also increases the amount of management traffic moving around a fabric. Interconnect additional and existing fabrics with ISLs attached to a storage network router that supports fabric segmentation. The storage network fabrics are physically connected, yet logically isolated similar to how TCP/IP-based Ethernet LANs are often configured. This approach has the benefit in that it can be deployed on a local and wide area basis enabling many devices to be part of a larger storage network, yet logically isolate devices and fabric resources.
  • For spanning distances, technologies including TCP/IP (IP), SONET/SDH, and optical multiplexing (CWDM/DWDM) can be used. FCIP and iFCP can be used to connect fabrics together over distances using TCP/IP based networks.
  • Protocol conversion enables devices and servers outside of a fabric to access storage resources in a fabric. An iSCSI server can access resources such as disk space capacity or tape backup capability located in a Fibre Channel SAN fabric.
  • Switch partitioning is a relative new capability particularly for Fibre Channel- and FICON-based switching products. Partitioning differs from segmentation in that with partitioning, a single large switch is subdivided into multiple smaller logical switches (domains). Each logical domain functions with its own fabric resources (name space) and identity including domain ID. Specific ports and internal bandwidth are allocated to the different local switches. This approach enables a physical single switch to have different logical switches be part of different fabrics. Since each of the fabrics that a logical switch can be part of is autonomous, some form in routing function would be required. The routing can be performed using an external router, be it can be implemented as part of switching devices hardware and firmware feature set.
  • Key technologies for scaling out a SAN

    A key technology for enabling storage networks to span single fabrics and locations is a storage network router. Vendor and marketing hype aside, there are generically three types of functions that storage networking routers perform:

    • Segmentation and inter fabric routing (not to be confused with partitioning).
    • Protocol conversion (not to be confused with mode conditioners).
    • Storage over distance enablement using IP, SONET/SDH and optical networks.

    The emergence of SAN segmentation and inter-SAN fabric routing capability is enabling SAN sub networks to be created. For example, if you connect two SAN fabrics together, they will be merged into a single fabric. The result will be a combined SAN fabric that could have device and address conflicts and a proliferation of state change and management traffic across both networks. Segmentation also can be used to isolate management traffic from different fabrics and keep traffic local. Some products only support a single function such as protocol conversion or distance enablement. Other products support two or more functions and others from vendors including Brocade (Multi-Protocol Router), Cisco (MDS 9216i) and McData (Eclipse) support all three basic routing functions.

    The following table shows technologies that can be used to support different storage functions for scaling on a local and remote basis, for example using iSCSI and FCP for block storage access and NAS for file-based access. Not shown in the table below are vendor-unique technologies that support such functions as segmentation and partitioning.

    Storage Network Scaling Technologies
    Block Storage Access No Yes No Yes No
    File Storage Access No No No No Yes
    Storage Sharing No Yes No Yes Yes
    Storage over Distance Yes Yes Yes Yes Yes
    SAN Fabric Segmentation No No Yes No No
    Tunneling Fibre Channel Yes No No No No

    About the author: Greg Schulz is founder and senior analyst of The StorageIO Group in Stillwater, Minn., and author of the book Resilient Storage Networks (Elsevier).

    Storage over distance support includes FCIP tunneling based products, iFCP, and storage over SONET/SDH generic framing protocol (GFP) products. Some products that support a single function, for example, protocol conversion, are also known as bridges and gateways. Protocol conversion includes Fibre Channel (FCP) to Ethernet (iSCSI), Fibre Channel (FCP) to parallel SCSI, FICON to Fibre Channel (FCP), and ESCON to FICON Bridge. Segmentation products enable independent fabrics to be physically connected, yet remain logically isolated.

    Dig Deeper on Primary and secondary storage