FRAMINGHAM (07/31/2000) - The ever-increasing demand for bandwidth is forcing IT to weigh its high-performance backbone options. Gigabit Ethernet is one of the most attractive approaches, offering 10 times the throughput of Fast Ethernet. More than 20 vendors offer Gigabit Ethernet switches, which they either manufacture or purchase from manufacturers. The Dell'Oro Group last year projected that the Gigabit Ethernet market will grow from 1.8 million ports in 1999 to 18 million ports in 2002.
Attracted by the promise of 10 times the speed at less than 10 times the cost, many network managers eagerly look forward to Gigabit Ethernet switching, but choosing from the growing number of competitors is problematic. We tested eight competing switches, Layer 2 and Layer 3, grouping several factors into four categories: performance; management, administration and configuration; features; and installation, ease of use and documentation.
The winners in our tests were the identical twins, Hewlett-Packard Co.'s ProCurve 9304 and Foundry Networks Inc.'s BigIron 4000. HP is an OEM customer of Foundry's, selling the BigIron 4000 with the HP name, support and documentation. The HP switch scored ever-so-slightly higher on installation, ease of use and documentation. But the margin was so thin that we decided to give both switches our World Class Award.
The players
The focus of this review was on fiber gigabit switches, and we limited our evaluation to players in that field. Twenty vendors were invited to provide switches for the review. Of that number, 11 did not respond, and Cisco Systems Inc. and Extreme Networks Inc. declined to be evaluated. In addition to the HP and Foundry switches, we reviewed Allied Telesyn International's AT-9006SX/SC, Intel Corp.'s NetStructure 480T and NetStructure 470F, Nbase-Xyplex's OptiSwitch-800F and SMC Networks Inc.' TigerSwitch 1000. Network Peripherals had agreed to participate but did not send a switch. HP also entered its ProCurve 6308 for review.
Performance
To test the performance of each switch, we measured the critical criteria network administrators should look for in a gigabit switch: packet loss, latency and congestion control. Our test conditions were artificially rigorous to measure the "outer envelope." Switches would rarely encounter these circumstances in real work environments. However, it is expected that, in the future, such outer envelope performance will become essential to many networks.
For now, you should expect somewhat better performance in actual usage than those we measured in our rigorous lab setting.
We tested only six ports of each switch, so our focus was more on port performance than on aggregate and backplane performance. In other words, we did not stress the backplane of the switch. Given the high backplane speeds noted by the vendors, we don't think backplane speed is an issue for users.
The first packet loss (what some call throughput) test was a port-to-port, full-duplex frame loss test. We tested at Level 2 and 3 (if appropriate). The Foundry, HP and Intel Layer 3 switches performed flawlessly at 100% capacity.
The remaining switches did not perform as well. The Nbase-Xyplex, SMC and Allied Telesyn switches suffered packet loss with 64-byte packets. The Allied Telesyn switch even had some difficulty with 512- and 1,518-byte packets. All the packet loss tests were performed using 64-, 512- and 1,518-byte packet sizes.
In the second packet loss test, we produced a full-mesh frame loss test. In general, Layer 3 switches tended to do better than Layer 2 switches: All Layer 3 switches achieved perfect scores. We had to increase the queue depth on Foundry's BigIron 4000 and the two HP ProCurve switches, using a simple Command Line Interpreter (CLI) command, as advised by vendor engineers, after we ran into some frame loss during the full-mesh trials. According to both vendors, all switches now include this CLI change before shipping.
The full-mesh test, a grueling exercise where all ports are transmitting at the same time to all other ports, simulating 100 users per port, turned out to be what separated the men from the boys. The Foundry switch, both HP switches and Intel's NetStructure 480T came out unscathed, delivering 100% wire-speed throughput with all three frame/IP packet sizes. The Layer 2 switches did not handle the load as well, showing various amounts of frame loss.
The last of the packet loss tests was a two-ports-to-one, multiple-pair frame loss test. The BigIron 4000, ProCurve 9304 and both NetStructure switches performed flawlessly at 100% load. The remaining switches suffered some frame loss with 64-byte packets.
The second series of performance tests was aimed at determining switch latency, which is the time it takes for a packet to cross the switch. Here, again, the switches started to differentiate themselves. The Intel NetStructure 480T came in at 5.7, 7.8 and 15.4 msec latency at the various packet sizes. Results for the NetStructure 480T were similar for Layer 2 and 3.
Foundry's BigIron 4000 had a fairly consistent 5.5 msec latency with 64-byte packets. Latency for this switch at 512 and 1,518 bytes was 13.1 and 29.3 msec, respectively. Latency was the same at Layer 2 and 3. The ProCurve 9304 and BigIron 4000 switches had similar times.
The NetStructure 470F, a Layer 2 device, was slightly slower than the NetStructure 480T and had some difficulty with 64-byte packets. The ProCurve 6308 was slightly slower than the ProCurve 9304.
The AT-9006SX/SC, the TigerSwitch 1000 and the OptiSwitch-800F all exhibited unique latency curves. As we approached 90% load on each switch (including those from Intel, HP and Foundry), especially with the larger packet sizes, latency increased dramatically. HP's ProCurve 6308 and Intel's NetStructure 470F showed considerable latency increases as the 64-byte packet load approached 100%.
The third series of performance tests looked at how the switches handled "anomalous" situations. Specifically, we inspected error handling and head-of-line blocking (HOLB). For the HOLB test, we saturated one outgoing port to see if this would affect the throughput of other outgoing ports. In essence, we tested the inbound and outbound port buffer memory handling design. We could not force HOLB on any of the switches.
For the error test, we sent frames with cyclic redundancy check errors at the incoming port to see how the switch handled this condition. All tested switches immediately dropped (and logged) the bad frames at the ingress port and did not hiccup in their handling of other packets.
The management challenge
We looked closely at how easy it was to manage, administer and configure the switches. We looked specifically at the ease of configuration of the boxes.
Foundry's BigIron 4000 and HP's ProCurve 9304 switches were the best in this category, primarily due to their configuration flexibility and the scalability and options available for the switches. The switches came configured with 16 1000BaseSX ports on two gigabit cards. There were four expansion slots that could be populated with a variety of modules. Although both switches come with a single removable power supply, a slot for an optional redundant power module is included on each.
The switches had a fairly straightforward CLI interface, with help accessible anywhere in the interface. Having easily accessible CLI help with command-line completion saves a lot of headaches and shuffling through the documentation.
Although the switches' Web interfaces were not the best we reviewed, they were intuitive and straightforward. Add to this Layer 2 through 4 filtering options (global and per-port), over-temperature autoshutdown, unicast and multicast filtering, and configurable management access policies, and we see switches that will make network administrators happy. Adding help to the Web management portion and clarification of some CLI commands would further improve the interfaces for both switches.
The ProCurve 6308 management was identical to the BigIron 4000 and ProCurve 9304, but it was configured differently. The ProCurve 6308, a fixed-port switch, had eight 1000BaseSX ports. The ProCurve 6308 also came with dual power supplies.
The next contender in the management category was Intel, with two switches. The console interface was menu-driven, uncluttered and well-organized. The Web interface on the NetStructure 470F was the best of all the switches reviewed.
It was very thorough and user-friendly and had good online help. Unicast and multicast filtering options were available, along with port security options to disable media access control (MAC) learning on individual ports. The 470F also had configurable per-port broadcast storm suppression.
The NetStructure 480T, a Layer 3 switch, came configured with 12 autosensing 10/100/1000Base-T ports and four 1000BaseSX gigabit interface converter ports.
A single power supply was shipped, but the switch may also be configured in a dual power supply configuration. The NetStructure 470F (Layer 2) came with eight fixed gigabit ports and a single power supply. This switch also ships in a six-port gigabit copper version with two 1000SX gigabit interface converter ports.
Nbase-Xyplex's OptiSwitch-800F came with eight slots that can be populated with gigabit fiber single- or dual-port modules, eight-port 10/100Base-T modules or two-, four- or eight-port 100BaseFX modules. The switch included two removable load-sharing power supplies.
We enjoyed the CLI interface with this switch because the commands were well arranged in logical groupings. Each grouping of commands had help available, and navigation in the CLI interface was easy, although some of the CLI commands were rather cryptic. The switch also had configurable MAC security for individual ports and configurable MAC filters. The switch did not come with a Web management interface, but the vendor sells a graphical user interface for an extra $500.
SMC's TigerSwitch 1000 came with eight 1000BaseSX fixed ports and a single power supply. An optional external power supply is available. The console interface was menu-driven and usable but could have been better organized. Some options that were not available on the switch appeared on the menu. The Web management interface was fairly straightforward, easy to use and included a graphic of the switch on the front page that showed the port status. You could click directly on the graphic to configure any port.
Allied Telesyn's AT-9006SX/SC switch came with six fixed 1000BaseSX ports, two expansion gigabit interface converter slots and a single power supply. A dual power supply is optionally available.
The AT-9006SX/SC provided poor management compared with the other switches we reviewed. The menu-driven console interface was slow to respond, and we weren't impressed with the presentation of the console interface. Every time an option was changed, the screen took about 8 seconds to refresh. We also found that the supplied Web interface was inconsistent in its presentation and use. Online help was accessible only through the Internet. We also discovered that the per-port broadcast storm controls were inadequate.
Features
We looked at the feature set each switch manufacturer provided, but we did not test all the features. Our testing was designed primarily to evaluate performance, rather than for checking the breadth of available features. These features and unique characteristics differentiate the switches and may justify the price differences between the devices depending on how you want to deploy them in your network.
Although each switch had its own strengths, in this section the clear winners were the ProCurve 9304 and the BigIron 4000. These devices did all we imagined a switch could do. Not all features were tested, although the features appeared as options in the management interface. They routed IP and IPX in hardware, routed AppleTalk in software, supported all major routing protocols, implemented Virtual Router Redundancy Protocol, had the capacity for more than 32,000 MAC addresses and included some interesting security features.
As standard features, all switches possessed SNMP management, Spanning Tree support, BootP support, port mirroring and flow control. All switches also supported port-based and 802.1Q virtual LANs and 802.1p tag-based quality of service (QoS).
The installation blues
As part of our emphasis on user-centered design, we looked at the ease of installation and quality of documentation.
The switches were all relatively easy to install. The Layer 2 switches were pretty much plug and play. All we needed to do was set an IP address for Web management. The Layer 3 switches require a bit more work if you want all the Layer 3 functionality.
However, Intel's NetStructure 470F switch does not support autonegotiation.
Lack of support for autonegotiation was noted in Intel's "Late breaking news," an information sheet that was attached to the switch. Although we don't believe lack of autonegotiation to be of much significance in a Gigabit Ethernet switch - it could cause a hiccup for users who do not read the documentation.
IP configuration on the NetStructure 480T switch, a Layer 3 device, was not as straightforward as it was on the other Layer 3 switches. Instead of assigning an IP address to each port, the NetStructure 480T required assigning an IP address to the default VLAN, which made the switch addressable from every port.
Allied Telesyn's AT-9006SX/SC came with a female DB-9 console port, although the de facto standard is a male DB-9. This was an unnecessary complication and was solved by a gender changer.
With regard to usability, the switches had console ports located on the front (port side) or on the back of the switch. The port location could be a key variable for users. The switches could be controlled through the console port (via a CLI) or through one of the switch ports (via a Web interface).
All switches were rack- and table-mountable, and the differentiation between physical switch installations was minimal.
Documentation
We reviewed the supplied documentation, written and CD-ROM, included with each switch and judged it based on the ease of locating information, suitability for various audiences (novice and expert) and organization of material.
The HP switches earned the top score for documentation, although only four-tenths separated the ProCurve 9304's total score from that of the BigIron 4000. The material for both HP switches took a layered approach, presenting the information for two target audiences: the novice and the expert.
Foundry's BigIron 4000 came in a close second, losing points for the way the material was organized. Additionally, the HP switches included the documentation on CD-ROM, while the BigIron 4000 did not.
We also liked Intel's documentation because it got us started in four easy steps and always explained why each task was being performed. Examples of the necessary commands were accurate and clearly laid out. We especially liked the troubleshooting section, which was laid out in a way a novice can work through.
Documentation included with the other switches was not as good. For example, the Nbase-Xyplex Product Guide for the OptiSwitch-800F read more like a command reference guide than a user's guide and was written in a less accessible style.
And only a CD-ROM version was included. The SMC documentation was fairly well presented, but the Management Guide was somewhat lacking in clarity, especially for the novice.
Allied Telesyn shipped us a preproduction manual, which was, for the most part, a prototype. It contained quick installation notes but lacked the depth the expert user would want and expect.
Conclusion
HP's ProCurve 9304 and Foundry's BigIron 4000 switches, virtually identical products, were the big winners, based on performance and features. HP's ProCurve 9304 had a slight edge over the BigIron 4000 as a result of somewhat better documentation. Intel's NetStructure 480T, with a score of 9.1, had the best overall performance of all switches tested but did not score as well as the Foundry or HP switches in the areas of management, features and documentation.
The ProCurve 6308 came in first among Layer 2 switches, which are less expensive and have smaller feature sets. The NetStructure 470F came in a fairly close second among Layer 2 switches. The remaining switches achieved lower scores, based in some part on each scoring criterion, but more because their performance paled in comparison to Intel's and HP's Layer 2 switches.
The switches we tested covered a broad price range. Switch features and performance roughly paralleled that of pricing, with the more expensive switches scoring better overall than the lower priced switches. In the end, deciding which switch best fits a company's needs involves deciding what is most important - an inexpensive switch with fewer features and lower performance or a more expensive switch that promises more reliability, more features and the ability to do more complex tasks.
Kovac and Gillette are research associates with the Applied Research Institute at Ball State University. The Applied Research Institute is a nonprofit organization focused on assisting the IT field and its vendors. Pelipenko is a graduate assistant with the program. Comments are welcome at rkovac@wp.bsu.edu.
Research assistants were Dan Wlaschinsky, John Brooks, Steve Dobbs, Emerson Almeida and Zach Hinkle.