Slashdot Mirror
Network Intrusion Detection Systems Fail to Impress
TheBongPipe writes "I'm reading a nice test here about 7 commercial IDSs. Who won the prize? Nobody..." They also looked at Snort, but found that all the products generated way too many false alarms.
What IDS do slashdot users use ?
It amazes me that people will pay $20,000 for a product that regularly crashes, doesn't detect all intrusions, and can only be kept up by constant, expensive intervention from the vendor, when for $20,000 less you can have a similar product that doesn't crash, detects just as many intrusions (though not all of them) and can be maintained either by the vendor, or by anyone else with the wit to understand it.
IDS are complex systems. Anyone pretending they have a packaged solution should rot in jail.
--
E_NOSIG
I recall a user we had on our network who thought it'd be cute to install BlackIce on his box, to better secure it. Nevermind the fact that I, and the rest of the admins at my company, had firewalls in place and had never had an intrusion on our network.
Imagine the fun the first time we try to deploy an antivirus package to his desktop just to be blocked for -- are you sitting down? -- an attempted NetBIOS intrusion.
After the second time we tried to deploy (and failed) BlackIce locked down the system so that it couldn't be accessed across the network by any other workstation, despite our having adminsitrative rights. That was cute.
Just throwing up a little real world example of how annoying these false alarms can be.
Prevention is another way to help secure a network, rather than simply detection.
CycSecure from Cycorp the makers of OpenCyc, the AI reasoning system, helps prevent attacks by using an AI engine to simulate attacks on your network to identify problems.
It's worth looking into.
"The large print giveth, and the small print taketh away" -Tom Waits
Just read the article. A bit poorly written. What were the IDS run on? Why no analysis of Snort? I'll say that I find Snort way over my head, but that's because I haven't RTFM enough. Why would one want a GUI on a server? (one of the points they marked it down for). Why did it crash? I've NEVER had a linux box crash. NEVER. I've also very, very rarely had a program freeze up enough to require a kill -9 (other than Netscape Navigator and some other buggy stuff. Not stuff like exim, apache, etc.) As a matter of fact, scroll down, and it seems that the downtime was due to their problem, not Snort (footnote at bottom of uptime table).
There are complaints about false-positives. I've played with Snort and there are ways to decrease the alarms put up. For example, a certain number of bum packets in a certain length of time. Not each and every packet.
Looking at the info at the bottom of the article, the authors should know what they are doing. But given the misrepresentations and inaccuracies releative to Snort, why should I believe their testing of non-Free software was any better?
Maybe it was eWeek or some similar publication about six or nine months ago did a similar check. The article was much longer and more in depth. They were also more appreciative of the programs out there. Now, some will say "just to appease their advertisers". Well... Maybe. But if that is the case, why did Snort get their nod as the best?
Jesus was all right but his disciples were thick and ordinary. -John Lennon
I have used Snort and Qualys (the high priced commercial outsourced IDS) and both give false positives quite frequently. However, proving they are false positives is part of the skill of a good human sysadmin. This is why IDSes will never replace a good sysadmin. He or she should be able to see the report and say without any shadow of doubt in his speech that any particular exploit shown by the IDS is a false postive or not.
This still means that each IDS has its good points; but why anyone would pay a lot for a system that cannot, by definition, be any better than an up to date Snort and human reading of the report, and knowing your network inside out. Those who buy into big commercial IDSes clearly are investing in software when they should be investing in people, training those people, and understanding those people. Too many middle managers think their sysadmin speaks a language they will never learn, and therefore need these things to understand. But a good sysadmin should try hard to find ways to communicate with them, and can if need be annotate a nice little Snort report and be done with it.
Conversion Rate Optimisation French / English consultant
It's definitely true that this is one of the most notable weaknesses of intrusion detection systems as they exist now. I work in a financial institution where upper management has finally made a sensible decision and devoted a full-time person (me) to network security but that's not the case in many smaller organizations. The vast majority of (external) intrusion attempts are from script kiddies that use pre-fab tools and put forth little effort to conceal their actions. In my opinion, this is justification for most networks to run in a "low paranoia" mode. This would get rid of excessive false-positives and the noise created by Joe Kiddie and his 10,000 buddies who are out there constantly port scanning class A subnets.
I have been to a certain obscure country visiting my relatives where a common joke is this: A car alarm sounding means another dork can't figure out how to enter their car.
The only flaw with this analogy is that car alarms are supposed to alarm people (i.e. draw their attention) who have no personal interest in the safety of your car to pay attention, and alarm the crook that they have been detected so they will hopefully run away without stealing the car.
I am no security professional, but I would expect that such intrusion detection software does not give the cracker any warning that they have been detected by a real person or security system, so they have no reason to leave the system alone. It also does not give the IP of the potential threat to everyone else on the network so that the threat can be DDOS'd.
Funny part is, you can take your pick of UI's for snort, on just about any platform (I run snort on WinNT on one network, and snort on Linux on another. And I've got a GUI for both of 'em ;-)
Davis Ray Sickmon, Jr - looking for something to read? Check out my three free novels at MidnightRyder.org
I recall a user we had on our network who thought it'd be cute to install BlackIce on his box, to better secure it. Nevermind the fact that I, and the rest of the admins at my company, had firewalls in place and had never had an intrusion on our network.
;-)
I hate to tell you this but, at this day and age when everything is being outsourced, some users feel they need to protect their machines against the "IT support".
"I have opinions of my own, strong opinions, but I don't always agree with them." -- George H. W. Bush
You can't simply plug these things in and expect them to work perfectly. If you don't know what you are doing with a high-powered IDS then you don't have any business using or judging them.
You need to take quite a while (based on your network) and OPTIMIZE your rules for a product like SNORT so that you are getting alerted to the types of things you want to know about while minimizing false positives.
It is pretty obvious that the tester didn't do that, and as a result he had nothing but bad things to say.
Let's let an experienced Snort user configure his conf files and then run the test again. I think you may find that the results are different.
dmiessler.com -- grep understanding knowledge
In the past year and a half strides have been made in building anomoly-based detection systems that do not necessarily suffer the weakness of rule lag that signature-based systems do. These systems go about the process a little bit more intelligently by reporting on traffic outside the "norm."
The catch with such a system is that you have to be very careful about measuring what your "norm" is. If you capture a profile on a very noisy network, then a lot of potentially dangerous traffic could go unreported.
As with most things in security and system administration, your solution will only be as good as the person or persons who design, implement and support a system. If you don't have a trained analyst evaluating and tweaking your IDS solution, you're in trouble. There's currently no such thing as a true IDS appliance.
-buffy
2. Offline because of configuration error.
Gee, I wonder if they should learn how to configure Snort before they test it.
People who have witty things here blow.
I suppose this is a good time to plug my university's project, STAT. STAT is an open sourced IDS framework. It allows you to monitor arbitrary events and take arbitrary actions based on them. It's possible to extend the field of STAT's vision by writing extensions to STAT in the STATL language. It's also trivial to write responses to known exploits.
You can find more info about STAT at
http://www.cs.ucsb.edu/~rsg/STAT/
STAT already has 2 extensions, NetSTAT and USTAT that watch for common network and unix-level exploits. Other projects include making java-level IDS's and mobile agent IDS's. It's a great project and it blows everything else out of the water. If you're dissatisfied with IDS's as they are, check out STAT.
Slashdot. It's Not For Common Sense
Well, as the person who got to keep calling the vendors (with some it was more than once per day for multiple days) I can tell you we >did talk to the vendors. We had better support than the average user since we were writing a review. We effectively had an unlimited support contract, as reviews normally do. Nobody involved was "anti-IDS". The fact the fellow from ISS didn't know we were doing the review is a problem between him and Nokia. "Reviewers want the product not to work" is not true, at least not in this case.