From the inception of its conceptual framework in late 1999 to the standardization of IEEE 802.3ae in mid 2001, 10G Ethernet has been one of the most anticipated and exciting technologies on the internetworking scene. Besides boasting throughput 10 times that of the already-blazing Gigabit Ethernet, 10G finally gives Ethernet-based networks the ability to contend with the more-established WAN technologies such as packet over SONET and ATM over high-speed OC/synchronous transfer mode circuits.
To the chagrin of many a network engineer and architect, the first round of 10G products, released less than a year after standardization, proved to be subpar. Some gear barely reached half the throughput defined in the specification, while others suffered from high amounts of latency and jitter. Potential customers quickly adopted an air of weary criticism, making it clear they would wait for more-refined products before making major purchasing decisions.
The wait has been short. Round Two of 10G products are here; the lineup looks promising.
Vendors that did not have Round One offerings are delivering their first, more mature 10G gear, while others already in the market are releasing more-refined versions of their earlier units in hopes of improving benchmarks and winning demand for their products. Most vendors say they have worked out the kinks in throughput capacity, leaving most of the differentiation to features including quality of service (QoS), IEEE 802.3ad link aggregation and reprogrammable ASICs.
The improvements are coming none too soon for a growing number of enterprise network users who need to scale past 1G Ethernet. Their justifications for 10G devices include high-volume storage-area networks and back-up segments, IP telephony, Gigabit-backed main distribution facilities, enterprise metropolitan-area networks (MAN), points of presence interconnected via 1G trunks, real-time audio and video multicasting, and data centers with switching and routing cores leveraging many 1G trunks.
"I would use 10Gig for aggregation at the enterprise core, where multiple data centers aggregate via WAN backhauls to one central border network," says Chris Rogers, a lead network engineer at InfoSpace Inc., a provider of wireless and Internet software and application services in Bellevue, Wash. "I'd also use it anywhere throughput, latency and jitter were major issues."
Likewise for 10G's use in a management or dedicated back-up network on which lots of data needs to move quickly, says Claude Johnson, Unix systems engineer for enterprise hosting firm Digex Inc. "In a network of thousands of systems, during a limited back-up window, I want to move as much data to my back-up servers as quickly as possible and keep those very expensive processors busy," he says.
Now with production-quality 10G units at the ready, enterprise network managers must decide which vendor has the most cost-effective, scalable products and how this fast technology best fits into their networks.
Switch vendors will have to strike a balance between price, hardware density and software features to grab the attention of these users. Avaya Inc., Cisco Systems Inc., Extreme Networks Inc., Foundry Networks Inc. and newcomer Force10 Networks Inc. are representative of the renewed 10G efforts. They each rolled out initial 10G Ethernet switches in 2002 that Network World benchmarked earlier this year.
Avaya and the Cajuns
Avaya now has two units on its 10G menu: the Cajun-series P882 MultiService Switch, for carrier cores, and the P580, more suitable for companies with midrange to high-end computing needs. With a 55G-bit/sec backplane and a forwarding rate of 40 million packet/sec, the multilayer switch does not offer much competition in terms of performance. And while users should expect to see the Cajun-series switches priced less expensively than 10G units rolled out from more-established vendors such as Cisco and Foundry, the cost is still relatively high with respect to performance and density. This is especially the case when taking into account that Force10 offers line-rate 10G ports at half the cost.
The P580, which is supposed to have a 20% speed increase over the original P550, still harbors the cross-bar chassis design that inhibited its throughput levels when tested earlier this year. It will take new testing to know whether the upgraded chassis really has resolved congestion issues in the crossbar or the 8G-bit/sec cap on the switch fabric, as Avaya says. However, it seems doubtful considering that the only major difference appears to be a boost in CPU clock speed.
The P580's scaling attributes also are somewhat weak, with a maximum of eight 10G ports, 48 1G ports and 288 10/100M bit/sec ports.
But on more positive notes, the Cajun line offers a solid set of software features, most notably for QoS. And benchmark testing against traffic shaping and queuing features has been positive, with the Avaya code enforcing traffic classes and queuing policies with strict precision. The same goes for jitter and delay.
The Cajun P580 might not be the best choice for the corporation that's looking for high scalability and full 10G performance benefits at the core and distribution layers, but it could serve as a relatively cost-effective solution for firms looking to leverage wide-scale, QoS-enhanced voice-over-IP deployments and real-time videostreaming.
The Cisco Catalysts
Opposite to Avaya on the 10G performance gradient lies Cisco, which offers a promising line of 10G products for the distribution and core layers of an enterprise campus, MAN or WAN. The most noteworthy are the four-port WS-X6704-10GE and two-port WS-X6802-10GE modules, for the Catalyst 6500-series switches and the Cisco 7600-series routers. The 6500-series switch is probably a better fit than the 7600-series router for most companies with midrange to high-end computing needs because of the scalability (48 to 576 ports of 10M, 100M or 1G Ethernet), diversity and redundancy it can provide as one manageable unit.
Combined with the new Supervisor Engine 720, which allows for an integrated 720G-bit/sec backplane, the 6500 series can accommodate up to 32 10G ports. This gives it one of the highest 10G densities available. Most startling, however, is switching performance. When the 10G modules are upgraded with distributed forwarding cards and used in combination with the Supervisor Engine, it is possible to reach a sustained forwarding rate of 48M packet/sec per card and a sustained aggregate forwarding rate of 400M bit/sec.
The new 10G modules also offer myriad QoS and fault-tolerance features. While both modules have the traditional class-of-service-based selection criteria for setting traffic policies, the WS-X6704-10GE also supports virtual LAN queues for defining multiple policies on a per-VLAN basis. According to Cisco, the 6500-series platform, combined with the new 10G modules, should be able to trunk 10G links with 802.11ad link-aggregation. Failover times should be greatly improved via Spanning Tree Protocol enhancements such as Per VLAN Spanning Tree Plus and Resource Reservation Protocol, reducing the failover time gap between 10G and traditionally more resilient WAN links (traditional STP limits Ethernet path recalculation to a matter of seconds, while OC-48 and OC-192 can failover in a matter of nanoseconds).
Cisco's 10G products are competitive for core and data center deployments, but somewhat pricey. The Supervisor Engine 720 starts at US$28,000 and the 10G modules range from $20,000 to $60,000. Enterprise users gain some flexibility, however, because the 6500's adaptive and flexible chassis design should let them add modules as needed.
Those Extreme BlackDiamonds
Extreme is delivering 10G with its BlackDiamond 6800 Series switches. The 10GLRi modules coupled with Extreme's Triumph ASIC technology result in an aggregate solution that could provide a fair amount of competition to Cisco and Foundry gear.
The BlackDiamond series includes the 6804, 6808 and 6816, which vary mainly by density and scalability options. While the 6816 offers only half the number of 10G ports (16) as Cisco's Catalyst 6500, it can deliver up to 1,440 100M/1G ports fully populated, providing more lower-end density than its competitors.
The BlackDiamond lower-end 6804 and 6808 switches max out at 256G and 384G bit/sec, respectively, making them less flexible than comparable competitive switches. But the 6816's 768G-bit/sec backplane is beefier than the Catalyst 6513's (the latest 6800 Series backplanes resolve the 8G-bit/sec cap of earlier models, allowing for full 10G throughput).However, the BlackDiamond's maximum forwarding rate of 192M packet/sec doesn't come close to that of Cisco or Foundry switches. This wouldn't be such a drawback if pricing was more competitive, with the 6816 base chassis costing more than the 6513's, and 10G modules at $29,000 per port.
To keep up with hardware-based access control list caching technology, BlackDiamond's feature set includes Triumph silicon technology that allows for wire-speed access control list (ACL) checking (reduces CPU overhead and filtering-related bottlenecks). It also does standard QoS traffic classifications and 802.11ad link-aggregation, allowing for 1G trunks.
While the BlackDiamond feature set is more competitive than most other vendor lines, the price vs. performance numbers hurt Extreme. Perhaps resolve might come from Mariner, expected to be released by year-end with six-port 10G blades at $8,000 apiece, and new silicon technology, dubbed "T-Flex," that will allow for updating ASICs with new configurations on the fly. With such pricing, Extreme will be a vendor to look to for customers on a tight budget with high-density requirements.
The Foundry for BigIron
Foundry says it has fixed the initial 8G limitation of the NetIron backbone routers and the BigIron Layer 3 switches with scorching throughput levels - 1.2 terabit/sec backplanes and forwarding rates of up to 480M packet/sec (second only to Force10), outrunning Cisco by 80M packet/ sec. While the NetIron routers are attractive for the service provider edge, enterprise clients should look mainly at the BigIron switches for their 10G needs. The BigIron switches come in a staggered chassis line (MG8, 4000, 8000 and 15000) and the same backplane capacity and forwarding rate, allowing entry-level chassis options and modules that will scale as network aggregation and performance requirements increase. Foundry harbors up to 32 10G ports on the BigIron MG8 (comparable to the Cisco Catalyst 6513) at $14,000 to $20,000 per port.
Innovations trickle down throughout the BigIron platform, from the 20G-bit/sec dedicated management backplane (that isolates administration traffic from production traffic) and wire-speed QoS to software features such as Rapid Spanning Tree Protocol (RSTP), sFlow (comparable to NetFlow on Cisco), 802.3ad link aggregation and SNMPv3. Enterprise clients looking to deploy highly scalable, cost-effective 10G solutions should keep their eyes on Foundry.
A Force10 to be reckoned with
Despite its youth as a company, newcomer Force10 offers a 10G switch that outperforms Foundry's and Cisco's. Specializing strictly in 10G, Force10 offers two main products: the middle-end E600, with a 600G-bit/sec backplane and a forwarding rate of 250M packet/sec; and the high-end E1200. This flagship switch boasts throughput attributes that place it at the top of the performance list for high-end products. While its density is tapered slightly at a maximum of 28 10G (or 338 1G) ports, the E1200 will leverage a 1.2 terabit/sec switch fabric and an unprecedented forwarding rate of 500M packet/sec, allowing 48G-bit/sec throughput to each module (the Cisco, Extreme and Foundry switches cap out at 40G bit/sec per module) and wire-rate speeds on all ports, even with ACLs applied. Such performance numbers are enough to make any enterprise network engineer happy, especially when taking into account that Force10 has undercut the 10G market substantially, with pricing at $17,000 per port.
Force10's operating system, FTOS, should compete well, with a feature set including protocols leveraged on leading vendor products (RSTP, 802.3ad link aggregation, Virtual Router Redundancy Protocol ). A variable to consider with Force10 is its freshness as a company: Code base maturity, product support and documentation take time to develop.
A matter of when
As the 10G arena unfolds, enterprise managers and network architects will have to take many things into consideration before making purchasing decisions. The wrong choice could make or break a network.
- Fierstadt is a senior network engineer at eUniverse.com Inc., an online entertainment and media network. He can be reached at lucid@switchninja.net.