Testing So-Called 'Unified Threat Managers'

snydeq writes "The InfoWorld Test Center has released vulnerability testing results for four so-called 'unified threat managers' — single units that combine firewall, VPN, intrusion detection and prevention, anti-malware, anti-spam, and Web content filtering in lieu of a relay rack stuffed top to bottom with appliances. The lab threw nearly 600 exploits of known vulnerabilities in a wide range of popular OSes, applications, and protocols, and despite being designed to thwart such threats, the UTMs as a class allowed hundreds to pass through. Why did the UTMs miss so many exploits? A lack of horsepower to perform the necessary deep packet inspection under load is suspected, as the lab pushed the limits of each unit's throughput with legitimate traffic. 'The upshot is, although the vendors have packed these devices with additional gateway security functions, clearly many UTMs are still strictly firewalls at heart.'"

Comments

general purpose != good

[KingFeanor]

Is it possible that single purpose security applications and appliances do a better job? In combining make various technologies in one device, how watered down was each individual component?

Re:general purpose != good

[houstonbofh]

Part of the solution is the tool, and part is how you use it. The Pix firewall can be very secure, but it is very hard to configure correctly. So many people just opened it up, making it very insecure... If a unified tool can be more easily configured securely than many best of bread applications, it will produce a better result every time. It will also have better cross communication than other applications as it is designed that way, not tacked on to support many other things.

Now could you personally out do that? Probably. Could your typical business person? Not likely...

Re:general purpose != good

[fuzzyfuzzyfungus]

Depends. In practice, I strongly suspect that quality suffers pretty markedly in many cases. If "UTM" is the new hotness buzzword, every last maker of firewalls, IDSes, packet shapers, and god knows what else will be rushing to ship a "UTM". This will probably mean taking their existing product and hacking together, or aquiring, enough other functions to make it qualify. That will typically mean that the result is good at something, with a bunch of other near broken crap hanging around and eating CPU time.

In theory, though, there is nothing about putting multiple functions in the same box that necessarily "waters them down", and there are substantial economies, in terms of space, hardware, and management, to be realized if done correctly. For a critical device, you would certainly want redundancy in the hardware, dual power supplies, that sort of thing. Two or three redundant PSUs running a single box are cheaper than two or three redundant PSUs per box, and multiple boxes. CPU and other system resources are in the same boat. Management is likely even more important. Fewer different styles of interface, fewer distinct login systems to deal with, fewer configuration files to back up, and so forth.

There is nothing fundamentally wrong with putting many functions in the same box; but, at the same time, there is a massive temptation for mediocre and worse "me too" outfits to cram a bunch of crap in and call it an "integrated solution".

Strange

[Reason58]

Focusing on doing one thing well yields better results than trying to do everything. Who'd have thought?

Re:Strange

[Rene S. Hollan]

Disclaimer: I am employed by one of the companies represented in the trial but do not speak for them.

Unfortunately, security is a process and affects all interacting systems. Placing them under one umbrella in a UTM device allows security issues to be dealt with in one place. This is better than having "something else" misconfigured somewhere undo all the efforts one has made in a particular place.

Yes, by layering SPAM filtering, virus scanning, and application protocol validation, one can achieve the same effect, and each appliance can excel in it's area, but this comes at the complexity of having to configure many things independently (not "atomic security changes" spanning multiple issies), adds to complexity (the bane of security), and may give rise to an "end run" if these units run in parallel, instead of sequentially (which yields latency issues).

The bottom line is that the market likes the convenience of unified threat management, and the price to be paid is generally not quality but performance.

Re:Strange

[Rene S. Hollan]

The bottom line is that the market likes the convenience of unified threat management, and the price to be paid is generally not quality but performance.

I dunno.. TFS said it did a pretty crap job keeping things out... I'd call that a cost in quality.

That's a different problem, since signatures can be updated over time. But, now that you mention it, space constraints in a UTM do limit the size of signature databases it can hold.

The answer is, of course, to get a bigger UTM, and address performance with clustered UTMs.

Sadly, one does not have to be perfect, one just has to be better, for some definition of better, than the competition.

No Cisco product?

[zerofoo]

How could you do a credible review of Unified Security Appliances without including one from a tiny little networking company called Cisco?

It would have been nice to see how the ASA5500 series appliances stood up to the test.

-ted

Re:No Cisco product?

[Hyppy]

Could you point us to something with more in-depth information, by all means. All we can find is marketing propaganda from Cisco and Checkpoint. Unbiased, timely reviews with real-world information like this are far and few between.

Re:No Cisco product?

[houstonbofh]

Or IPcop, pfSense, m0n0wall, Shorewall, etc. Why? Because they're not appliances.

monowall is not a UTM, it is a firewall. I am a dev on it, and I should know.
pfSense is also not a UTM, but it has a lot of plugins that can get close. But since it is a lot of plugins, it is not really "Unifiied."
Untangle, and both of the above are available as supported appliances, or installable on standard x86 hardware, or appliance like hardware.
I have not used IPcop or shorewall, so I can't speak on them.

Re:No Cisco product?

[vlm]

Could you point us to something with more in-depth information, by all means.

Your interpretation was backwards. He's looking for less because it's expensive.

When purchasing a $200 graphics card in a corporate environment, the technical staff will read 200 page technical documents, search google for hours, write reports, run simulations, justify the upfront cost vs long term labor savings, basically spend at least a grand or two of labor costs to pick the best $200 card.

However, when purchasing a $30K "buzzword of the month" the decision will be made at a high level by a manager whom is proud of being non-technical based on:
1) What they saw on CSI and/or 24 last night, or maybe Obama's latest speech.
2) Whom has the scariest marketing material (buy this expensive magic widget or you be p0wned)
3) How much he enjoyed the sporting event the sales exec took him to, or how much he enjoyed the sales exec in general.
4) The cheapest, or the first one he saw in a magazine, or perhaps a brand that will offend one of his enemies (you know, like he hates the guy who happens to love Cisco products, so if the enemy of my enemy is my friend, then ...)

Re:Uhm?

[a-zarkon!]

It's the Ron Popeil/Billy Mays/Home Shopping Network sales pitch for IT Security: "It's a firewall, it's an intrusion prevention system, it will filter your web connections, it even provides anti-virus. But wait! It also acts as a router, and it even has a built in gigabit switch module. Now - how much do you think you're going to pay for this? Not $20,000 - not $15,000, not $10,000; no - all this can be yours for the low low price of $9995.95...."

Re:Uhm?

[morgan_greywolf]

Ever heard of computer loaded with *nix and configured as a gateway/router/proxy with snort or something similar loaded? Back before you young whippersnappers came in with your fancy firewall appliances, that's what we had. And we liked it that way!

Flawed by Design.

[canipeal]

The notion of having a single point of failure "security" device contradicts one of the primary foundations of security principle: Defense In Depth. Multiple layers of security is essential in safe guarding your systems, placing them all one one unit is nothing short of moronic.

Re:Flawed by Design.

[ZouPrime]

Defense in depth refers to the principle of having multiple, overlapping security controls. For example, I've seen some companies use dual-firewall configurations where they will use two different brands of firewall. Or they will use a main network firewall as well as host-based software ones. So if one control fail the other is there to protect the asset.

This has nothing to do with UTM, who are about hosting *complementary* controls on the same device. In this case, there is a real benefit in term of management effort. These kinds of devices are especially interesting for small companies who can't bother handling a lot of different appliances and software for something perceived as unproductive as security.

Re:Flawed by Design.

[ZouPrime]

True... but that's not "defense in depth", that's "not having a single point of failure".

I agree that one big box to do everything has its issues. It's certainly not acceptable for corporations. But I think the cost/benefit is worthwile for a lot of small business who most of the time don't have shit (although this is getting less and less true).

It's a bit like those Linksys routers: sure they sucks, but they are so cheap and so commonly available, and so better than being only jacked right in a modem, they are overall a Good Thing.

Neglected to test

[Runaway1956]

They should have used a control for this test. Put each of these unified conglomerations up against one good Sysadmin with a clue.

No one tool will ever be "THE Solution". No matter how many doodads are attached to a Swiss knife, some sack of warm tissue has to fire a few synapsis to put the knife to use. If the sack of warm tissue is lacking in the synapse department, he fails.