Network Intrusion Detection Systems Fail to Impress

TheBongPipe writes "I'm reading a nice test here about 7 commercial IDSs. Who won the prize? Nobody..." They also looked at Snort, but found that all the products generated way too many false alarms.

Comments

$20,000 for crap

[Rupert]

It amazes me that people will pay $20,000 for a product that regularly crashes, doesn't detect all intrusions, and can only be kept up by constant, expensive intervention from the vendor, when for $20,000 less you can have a similar product that doesn't crash, detects just as many intrusions (though not all of them) and can be maintained either by the vendor, or by anyone else with the wit to understand it.

IDS are complex systems. Anyone pretending they have a packaged solution should rot in jail.

False Alarms get annoying for real admins

[Wingchild]

I recall a user we had on our network who thought it'd be cute to install BlackIce on his box, to better secure it. Nevermind the fact that I, and the rest of the admins at my company, had firewalls in place and had never had an intrusion on our network.

Imagine the fun the first time we try to deploy an antivirus package to his desktop just to be blocked for -- are you sitting down? -- an attempted NetBIOS intrusion.

After the second time we tried to deploy (and failed) BlackIce locked down the system so that it couldn't be accessed across the network by any other workstation, despite our having adminsitrative rights. That was cute.

Just throwing up a little real world example of how annoying these false alarms can be.

Who are these people?

[gmhowell]

Just read the article. A bit poorly written. What were the IDS run on? Why no analysis of Snort? I'll say that I find Snort way over my head, but that's because I haven't RTFM enough. Why would one want a GUI on a server? (one of the points they marked it down for). Why did it crash? I've NEVER had a linux box crash. NEVER. I've also very, very rarely had a program freeze up enough to require a kill -9 (other than Netscape Navigator and some other buggy stuff. Not stuff like exim, apache, etc.) As a matter of fact, scroll down, and it seems that the downtime was due to their problem, not Snort (footnote at bottom of uptime table).

There are complaints about false-positives. I've played with Snort and there are ways to decrease the alarms put up. For example, a certain number of bum packets in a certain length of time. Not each and every packet.

Looking at the info at the bottom of the article, the authors should know what they are doing. But given the misrepresentations and inaccuracies releative to Snort, why should I believe their testing of non-Free software was any better?

Maybe it was eWeek or some similar publication about six or nine months ago did a similar check. The article was much longer and more in depth. They were also more appreciative of the programs out there. Now, some will say "just to appease their advertisers". Well... Maybe. But if that is the case, why did Snort get their nod as the best?

IDS only as good as the person reading it

[fruey]

IDS systems generally automate a number of things that hackers do all the time; possible exploits are often false positives but the sysadmin should be able to see a false positive a mile off.

I have used Snort and Qualys (the high priced commercial outsourced IDS) and both give false positives quite frequently. However, proving they are false positives is part of the skill of a good human sysadmin. This is why IDSes will never replace a good sysadmin. He or she should be able to see the report and say without any shadow of doubt in his speech that any particular exploit shown by the IDS is a false postive or not.

This still means that each IDS has its good points; but why anyone would pay a lot for a system that cannot, by definition, be any better than an up to date Snort and human reading of the report, and knowing your network inside out. Those who buy into big commercial IDSes clearly are investing in software when they should be investing in people, training those people, and understanding those people. Too many middle managers think their sysadmin speaks a language they will never learn, and therefore need these things to understand. But a good sysadmin should try hard to find ways to communicate with them, and can if need be annotate a nice little Snort report and be done with it.

Snort GUI...

[Midnight Ryder]

Funny part is, you can take your pick of UI's for snort, on just about any platform (I run snort on WinNT on one network, and snort on Linux on another. And I've got a GUI for both of 'em ;-)

What a wonderful idea!

[Subcarrier]

I recall a user we had on our network who thought it'd be cute to install BlackIce on his box, to better secure it. Nevermind the fact that I, and the rest of the admins at my company, had firewalls in place and had never had an intrusion on our network.

I hate to tell you this but, at this day and age when everything is being outsourced, some users feel they need to protect their machines against the "IT support". ;-)

Re:NIDS will not catch everything

[buffy]

In the past year and a half strides have been made in building anomoly-based detection systems that do not necessarily suffer the weakness of rule lag that signature-based systems do. These systems go about the process a little bit more intelligently by reporting on traffic outside the "norm."

The catch with such a system is that you have to be very careful about measuring what your "norm" is. If you capture a profile on a very noisy network, then a lot of potentially dangerous traffic could go unreported.

As with most things in security and system administration, your solution will only be as good as the person or persons who design, implement and support a system. If you don't have a trained analyst evaluating and tweaking your IDS solution, you're in trouble. There's currently no such thing as a true IDS appliance.

-buffy