Channel Explained: 10 Gigabit Ethernet

Businesses are demanding more from their networks, and technology is responding. With the availability of 10 Gigabit Ethernet (10 GigE) and compatible products, should you be upgrading your clients' infrastructures? In this Channel Explained you'll learn the ins and outs of 10 GigE as well as the types of clients that may be interested in the technology.

Steve Bigelow, Features Writer

Networks are experiencing an unprecedented demand for bandwidth, thanks to data center consolidation and a growing number of data-intensive applications like video production, managed services, storage, VoIP and streaming media servers. Network technology has responded, moving from 10/100 Mbps to 1 Gigabit Ethernet (1 GigE). Now 10 Gigabit Ethernet (10 GigE) transmission technology is on the horizon, buoyed by a suite of IEEE standards and pushed by an increasing array of vendors eager to take the next step in network evolution. This article highlights the basics of 10 GigE and offers insights that can help solution providers bring this technology to their clients.

Introduction to 10 Gigabit Ethernet

10 GigE is not a new technology. It actually started development in 1999 when the IEEE 802.3 Higher Speed Study Group formed to create the successor to 1 GigE released the year before. The primary goal was to increase speed at a reasonable cost while maintaining IEEE 802.3 Ethernet frame formats and compatibility to IEEE 802.3x flow -- basically ensuring backward compatibility with previous Ethernet standards and cabling infrastructure.

Perhaps the biggest difference between 10 Gigabit Ethernet and older Ethernet standards is the abandonment of half-duplex operation in favor of full-duplex. Half-duplex network communication dates back to the original version of Ethernet, relying on packet contention on the wire (also known as Carrier-Sense Multiple Access/Collision Detection or CSMA/CD) to determine whether the cable was free for transmission. This simple solution to a complicated networking problem was a key attribute to Ethernet's early success, but it's simply too inefficient to use at 10 GigE speeds. There is no contention in full-duplex operation, and a network node can essentially send packets at any time. However, this necessitates the use of switches to interconnect network nodes since hubs no longer work to connect network devices.

10 GigE was first ratified in 2002 as IEEE Standard 802.3ae-2002, using -SR, -LR, -ER and -LX4 optical fiber to connect network nodes and switch ports. Since then, the 10 GigE standard has been amended several times to accommodate a variety of cable types. For example, 802.3ak in 2004 specifies the use of -CX4 copper InfiniBand cabling, 802.3an in 2006 specifies familiar-looking copper twisted pair cabling (also known as "10GBaseT," "Category 6" or "Cat 6" cable), 802.3aq in 2006 denotes -LRM fiber with improved signaling, and 802.3ap in 2007 includes the use of -KR and -KX4 copper backplane. Many of the amendments are expected to be consolidated into a single IEEE Standard 802.3-2008, which should be released this year.

10 Gigabit Ethernet compatibility issues

Compatibility is still a gray area for 10 Gigabit Ethernet, and there are numerous alternatives for the physical layer (or PHY) -- the means of connecting network devices at 10 GigE. With only about 1 million 10 GigE ports shipped in 2007, it's still too early to tell which physical layer standard will be accepted by the market. High-end 10 GigE networking devices (such as switches) accommodate different physical connectivity types through the use of plug-in "PHY" modules. For example, if -LR fiber connections are called for, a solution provider can recommend the use of corresponding switch modules and NICs for their clients.

Backward compatibility is another unknown that any solution provider should consider before mixing Ethernet standards. Price is certainly one important factor. While a 10 GigE port should ideally sense and negotiate the slower connection, the current cost of a 10 GigE port makes it economically pointless. "You wouldn't plug a 1 GigE server port into a 10 GigE switch -- it would never happen," said Adam Gray, chief technology officer of Novacoast, an IT professional services and product development company headquartered in Santa Barbara, Calif. However, it's still too early to say with certainty that a 1 GigE NIC from manufacturer A would interoperate properly when connected to a 10 GigE switch port from manufacturer B (or vice versa).

Still, 10 GigE and 1 GigE switch ports should operate together within the same switch. For example, a manufacturer's switch with a mix of 10 GigE ports and 1 GigE ports should work properly. The 1 GigE port(s) would likely interface the switch to slower devices and the remainder of the client's 1 GigE infrastructure.

Ramping up to 10 Gigabit Ethernet

While the 10 GigE standard is hardly new and a limited number of products have been available for years, it's only starting to show signs of adoption. The principle limitations of 10 GigE adoption have been price, need and interoperability. Early 10 Gigabit Ethernet products cost several thousand dollars per port. The bandwidth needs of most organizations did not even begin to justify the corresponding investment.

More Information
10 GigE is a huge, complex standard that encompasses a myriad of physical, signaling and logical issues. To acquire all of the technical details about 10 GigE, visit the IEEE  to download the 802.3 standards, or check the Ethernet Alliance for current news and developments.

But even while most businesses have not yet realized the full potential of their 1 GigE infrastructures, solution providers note that bandwidth needs are constantly growing. More organizations are adopting Voice over IP and other bandwidth-intensive tasks like streaming audio and video. And more application demands are being compounded with data center consolidation initiatives.

As data centers are consolidated, the demand on each data center's core network increases, according to Bob Laliberte, analyst at the Enterprise Strategy Group, an industry analyst firm in Milford, Mass. Laliberte pointed out that Hewlett-Packard, for instance, went from 85 data centers down to six, each with more traffic and greater bandwidth needs than before.

Server virtualization is another bandwidth driver, forcing the traffic from multiple virtual machines (VMs) through the I/O resources of a single physical server. Even the future emergence of Fibre Channel over Ethernet (FCoE) is seen as a potential driver for 10 Gigabit Ethernet technology.

Vendors such as Extreme Networks Inc., Foundry Networks Inc., Alacritech Inc. and Broadcom Corp. are supporting 10 GigE products. You can peruse a listing of manufacturers currently involved in 10 Gigabit Ethernet product interoperability testing. Product competition not only provides a wider selection of interoperable products for solution providers to recommend, it also precipitates the price pressure that makes 10 GigE more affordable and easier for clients to justify -- especially now that standard Category 6 copper cabling is supported in addition to various optical fiber types. "Now we're starting to see companies coming out with 10 GigE products that are priced at $900 per port or less," Laliberte said, indicating that pricing is expected to decline to about $500 per port by the end of 2008.

Making the case for 10 Gigabit Ethernet

Only the largest data-intensive organizations, such as video production houses, Internet service providers and FCoE early adopters, are currently using 10 Gigabit Ethernet technology. "For us, it's shops that produce such a volume of traffic that they need the connectivity between switches or to very large server farms," Gray said, noting that standard business applications have not been a driver for 10 GigE at this point, and it's still more cost-effective to trunk multiple 1 GigE links when additional bandwidth is needed at key points in the network.

Until the economics of 10 GigE allow for deployment with routine technology refreshes, solution providers can find opportunities in client consolidation initiatives. Today's 1 GigE links can be strained as companies use virtualization to tackle more work with fewer servers. Replacing those 1 GigE links with 10 GigE technology can ease congestion, providing bandwidth for future growth through more applications or additional VMs. 10 GigE NICs with TCP/IP offload cards can help streamline I/O on busy virtualized servers.

Expect to see 10 GigE deployment follow the same pattern as 1 GigE, with deployments starting in the core network of the data center and gradually working their way to the edge of the enterprise -- though the need for 10 Gigabit Ethernet at the edge may take years to arrive. "You won't see it pushed out to the desktops yet because I'm doing everything I need to do with 1 GigE," said Christopher Black, business development manager of data center solutions at World Wide Technology, a reseller headquartered in St. Louis, Mo. "It's not [yet] something that I'm going to get a big cost benefit from."

One of the major selling points of 10 GigE is familiarity -- clients already know the technology and have seen it evolve from 10/100 Mbps to 1 GigE, and 10 GigE should not present any type of culture shock. Still, some 10 GigE upgrades will be disruptive because downtime will be needed to install new NICs and deploy new switch ports. Solution providers must be sensitive to network disruptions and downtime, and take extra care to ensure interoperability in a test setting before deploying 10 GigE devices at the client's site. Black underscored the need for careful vendor selection, ensuring adequate support, OS driver availability and interoperability testing to sidestep potential compatibility issues. Another way to mitigate network disruption and deployment downtime is to implement 10 GigE in small phases starting with the least critical links.

Dig Deeper on Campus area networks and services