<< Back to article Print this page Loading page, please wait...

UTM performance takes a hit

IPS, antivirus scanning reduce gigabit firewall speeds to megabit levels

Joel Snyder (Network World)
14 November, 2007 10:24

Because every network requires a different way of measuring performance and most UTM products offer thousands of deployment options, it's hard to draw even general conclusions about how these products will behave in your network. However, we can say that most enterprises will want to proceed cautiously when adding UTM features, such as intrusion-prevention systems and antivirus scanning, to their perimeter firewall boxes, because of their unpredictable impact on total system performance.

In our baseline testing with only the firewalling capabilities turned on, eight of the 14 appliances easily exceeded our 1Gbps measurement goal. When we turned on their UTM features, however, systems that breezed through the 1,000Mbps mark slowed dramatically. Out of 56 test results collected with various UTM features turned on, 36 registered results that were 250Mbps or less. Read the latest WhitePaper - NAC: A Multi-Symptom Remedy

With IPS configuration, your choice of signatures can make the difference between a speedy firewall and a snail. The top IPS performer, IBM Internet Security Systems' Proventia MX5010, shows that you can get a high-speed IPS riding on top of a firewall. Other platforms require careful tuning and an educated selection of what you want to protect before you can achieve predictable and acceptable performance.

Antivirus scanning has a similar cost in most platforms (the Fortinet FortiGate 3600A is an exception) that also makes it a dangerous add-on, taking some platforms to their knees and turning gigabit firewalls into megabit slowpokes.

We ran baseline traffic through the firewalls using Spirent Communications' Avalanche and Reflector load testing products. We set up a load of 1Gbps spread across four ports, with the Reflector serving up Web pages on 20 simulated Web servers on two of the ports, and Avalanche simulating 500 Web clients on the other two.

Deployment question

In each case where we faced a deployment-option question, we optimize for security rather than speed. Yet one man's security might be another man's overkill, especially when the performance penalty for these security features is significant.

Enabling HTTP inspection, a feature that provides some intrusion prevention, caused almost no performance penalty in our Cisco ASA5540, reducing throughput from 660Mbps to about 640Mbps. Enabling HTTP inspection and choosing an advanced feature (such as blocking ActiveX content) caused an 80% drop in total throughput.

Picking a configuration for performance measurements got more complicated when we tested with UTM features enabled. Check Point Software's IPS technology, called Secure Defense, is a good example. With several hundred IPS options for different types of applications and different attacks, there is no way just to turn on IPS. You have to decide which of the signatures you want to use. When you turn on anything above the default settings, the performance impact is huge.

When we tested the Nokia IP290 running Check Point's firewall software with Secure Defense disabled, and then enabled with default settings, we saw a tiny performance hit (from 1003M to 993Mbps). When we followed Check Point's recommended settings for providing IPS for servers (which scans for more attacks), we saw an 85% drop in performance.

To get our IPS performance results, we used two scenarios -- one asking the firewalls to protect servers and one asking them to protect client systems. With server-protective IPS, there are more potential attacks, but the IPS doesn't have to look at as much traffic. For example, in our HTTP testing, it took about 20Mbps of traffic to a server to generate 1000Mbps of traffic coming back from it. Server-protective IPS has to look only at the traffic to the server.

On all firewalls, we set up a modest policy, letting HTTP through between segments with network address translation (NAT) enabled. We weren't trying to find out the top speed for each of the products; most of the boxes we tested had stated capacities faster than our 1Gbps test bed. Our objective was to ascertain how much of a drop we were going to find when we turned on UTM features.

The security features of many of the firewalls we tested comprise a spectrum of options. For example, Secure Computing will let you run the Sidewinder with packet filters or a generic proxy, neither of which have the same security model as the full application-aware proxy it also supports. With packet filters, the Sidewinder maxed out our test bed; with a generic proxy it nearly hits 1Gbps. However, any enterprise paying the US$80,000 price tag would do so for the full proxy capabilities. When we turned those on, raw performance fell to a respectable 826Mbps.

Page Break

Client-protective IPS

With client-protective IPS, there are fewer attacks, but you have to look at the full data stream to find them. For example, if you're looking for an image buffer-overflow attack, that could be at the front, middle or end of the image.

We discovered, with one notable exception, a massive variation in how fast or slow each device ran with IPS turned on. The astonishingly fast exception was the IBM/ISS Proventia MX5010, which handled IPS at just less than 1Gbps in the only configuration it had -- "on."

For all the others, however, the choice of profile makes a huge difference. The IBM System x3650 running Check Point's firewall blew past everything (except Proventia) in the server-protective scenario, turning in a blistering 816Mbps. But when we put everyone into client-protective configurations, Fortinet's FortiGate 3600A and Secure Computing's Sidewinder 2150D led the pack after Proventia, with performances of 624Mbps and 581Mbps respectively.

It is important to note that Check Point asked us to configure the UTM-1 2050 appliances and the Nokia IP290 appliances in active/active mode. All other devices were configured in active/passive mode. This means that the performance numbers reported for these appliances are higher than they would have been in a more traditional active/passive configuration.

Because Check Point and its partners submitted four separate platforms running what was essentially the same software, we allowed this slightly irregular configuration to help show the different options that are available from Check Point's partners.

Security vs. speed

You can easily build an IPS that runs really fast if you don't care how many attacks it blocks or how many false positives it throws. We used Mu Security to help normalize the IPS performance numbers. With the Mu-4000 appliance, we could get a very rough comparison of the ability of each IPS to block attacks.

To generate our scorecard values, we took the speed of the IPS and scaled it by how effectively the IPS blocked attacks. Thus, an IPS that ran at 750Mbps but blocked 10% of attacks was given a lower score than an IPS that ran at 250Mbps but blocked 50% of attacks. Going fast is good, but our scoring favors devices that catch a greater number of attacks.

Again, the Proventia MX5010 turned in such astoundingly good results that we captured a number of packet traces to verify that something wasn't wrong with our configuration. When normalized performance was taken into account across client and server profiles, the MX5010's score was three times higher than that of the next platform (Juniper Networks' ISG-1000). As a superfast, superaccurate IPS, nothing in our testing came close to the Proventia MX5010.

Overall testing showed that while there can be a significant drop in performance when IPS is enabled, careful choice which traffic should be scanned and which signatures are enabled -- along with the right piece of hardware -- lets an IPS and firewall be collocated.

High performers here include Fortinet's FortiGate 3610A, the IBM System x3650 with Check Point's VPN-1 running, Juniper's ISG-1000 with its integrated IPS blade, IBM/ISS' Proventia MX-5010, Secure Computing's Sidewinder and SonicWall's Pro 5060. Also, Check Point's UTM-1 2050 and Juniper's SSG-520M both hit 200Mbps and 400Mbps respectively, as long as we used server-protective IPS signatures.

Page Break

Antivirus results

There was a clear winner both in antivirus and antivirus-plus-IPS speed contests. At 524Mbps of antivirus throughput, the Fortinet FortiGate 3610A blew past the competition. Secure Computing's Sidewinder 2150D was in a distant second place, at 396Mbps, about 20% slower. After that, IBM/ISS's Proventia MX5010 sat in third place with 298Mbps -- just more than half of the FortiGate speed. If you want to do fast antivirus scanning on top of perimeter or core firewalling, Fortinet seems to have the recipe for speed.

After running hundreds of performance tests, we confirmed what several of the vendors told us before we started: Don't do antivirus scanning in an enterprise UTM device. Except for the FortiGate 3610A, which handled firewalling, antivirus, and IPS at 520Mbps, none of the gigabit firewalls we tested could cross the 300Mbps line under the same conditions.

Summary

Read our performance results as a confirmation that neither IPS nor antivirus belong in an enterprise firewall, at least not if you're looking for predictable, gigabit performance. In a few cases, we were able to take otherwise-capable systems to their knees by stressing them in the wrong way. On the other hand, our testing showed that with careful configuration, you can get additional protection in a high-speed firewall. Performance testing is critical, not just when the system is initially installed, but any time new protections are put into place.