Designing storage area network for multiple fabrics

If you're currently reviewing a customer's storage area network (SAN) design and considering scaling beyond a single fabric, there are a variety of technologies and products that can help.

First we'll review reasons why you'd scale out a SAN beyond a single fabric on a local or wide-area basis.

Why scale out a SAN

  • Consolidation of servers, storage, backup, applications and data centers.
  • Data protection and business continuance, including regulatory compliance.
  • Technology replacement combined with business growth and new application needs.
  • Ability to support more devices than a single fabric can support.
  • Isolation and segregation of workloads, applications and customers.
  • Interoperability and coexistence of heterogeneous technologies.
  • Understanding your customer's storage networking objectives, along with what level of scalability and resiliency they require, is integral to developing the storage area network design. Attributes and service requirements associated with scaling a storage network include:

  • Performance (bandwidth and latency) including lack of congestion.
  • Support for more servers and storage devices for growth.
  • Support for backup, recovery and business continuance.
  • Block access, file access (NAS), or combination of block and NAS access.
  • Ease of management, diagnostics and provisioning of resources.
  • Span distances
    • Requires Free Membership to View

    • to maintain data accessibility on a local and regional basis.
    • Now that you've determined when you'd scale beyond a single fabric, you must decide how you're going to go about accomplishing this goal. Here are your options:

      How to scale out a SAN

    • Establish additional independent fabrics consisting of one or more switching devices with various numbers of ports per switch.
    • Interconnect fabrics with inter-switch links (ISLs) creating one large physical and logical fabric. While this approach enables any to any connectivity, it does not increase the addressing range and number of devices supported per fabric. This approach also increases the amount of management traffic moving around a fabric. Interconnect additional and existing fabrics with ISLs attached to a storage network router that supports fabric segmentation. The storage network fabrics are physically connected, yet logically isolated similar to how TCP/IP-based Ethernet LANs are often configured. This approach has the benefit in that it can be deployed on a local and wide area basis enabling many devices to be part of a larger storage network, yet logically isolate devices and fabric resources.
    • For spanning distances, technologies including TCP/IP (IP), SONET/SDH, and optical multiplexing (CWDM/DWDM) can be used. FCIP and iFCP can be used to connect fabrics together over distances using TCP/IP based networks.
    • Protocol conversion enables devices and servers outside of a fabric to access storage resources in a fabric. An iSCSI server can access resources such as disk space capacity or tape backup capability located in a Fibre Channel SAN fabric.
    • Switch partitioning is a relative new capability particularly for Fibre Channel- and FICON-based switching products. Partitioning differs from segmentation in that with partitioning, a single large switch is subdivided into multiple smaller logical switches (domains). Each logical domain functions with its own fabric resources (name space) and identity including domain ID. Specific ports and internal bandwidth are allocated to the different local switches. This approach enables a physical single switch to have different logical switches be part of different fabrics. Since each of the fabrics that a logical switch can be part of is autonomous, some form in routing function would be required. The routing can be performed using an external router, be it can be implemented as part of switching devices hardware and firmware feature set.
    • Key technologies for scaling out a SAN

      A key technology for enabling storage networks to span single fabrics and locations is a storage network router. Vendor and marketing hype aside, there are generically three types of functions that storage networking routers perform:

      • Segmentation and inter fabric routing (not to be confused with partitioning).
      • Protocol conversion (not to be confused with mode conditioners).
      • Storage over distance enablement using IP, SONET/SDH and optical networks.
      Storage over distance support includes FCIP tunneling based products, iFCP, and storage over SONET/SDH generic framing protocol (GFP) products. Some products that support a single function, for example, protocol conversion, are also known as bridges and gateways. Protocol conversion includes Fibre Channel (FCP) to Ethernet (iSCSI), Fibre Channel (FCP) to parallel SCSI, FICON to Fibre Channel (FCP), and ESCON to FICON Bridge. Segmentation products enable independent fabrics to be physically connected, yet remain logically isolated.

      The emergence of SAN segmentation and inter-SAN fabric routing capability is enabling SAN sub networks to be created. For example, if you connect two SAN fabrics together, they will be merged into a single fabric. The result will be a combined SAN fabric that could have device and address conflicts and a proliferation of state change and management traffic across both networks. Segmentation also can be used to isolate management traffic from different fabrics and keep traffic local. Some products only support a single function such as protocol conversion or distance enablement. Other products support two or more functions and others from vendors including Brocade (Multi-Protocol Router), Cisco (MDS 9216i) and McData (Eclipse) support all three basic routing functions.

      The following table shows technologies that can be used to support different storage functions for scaling on a local and remote basis, for example using iSCSI and FCP for block storage access and NAS for file-based access. Not shown in the table below are vendor-unique technologies that support such functions as segmentation and partitioning.

      Storage Network Scaling Technologies
      Block Storage Access No Yes No Yes No
      File Storage Access No No No No Yes
      Storage Sharing No Yes No Yes Yes
      Storage over Distance Yes Yes Yes Yes Yes
      SAN Fabric Segmentation No No Yes No No
      Tunneling Fibre Channel Yes No No No No

      About the author: Greg Schulz is founder and senior analyst of The StorageIO Group in Stillwater, Minn., and author of the book Resilient Storage Networks (Elsevier).

      This was first published in November 2006

    There are Comments. Add yours.

    TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

    REGISTER or login:

    Forgot Password?
    By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
    Sort by: OldestNewest

    Forgot Password?

    No problem! Submit your e-mail address below. We'll send you an email containing your password.

    Your password has been sent to:

    Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.