Slashdot Mirror
Network Intrusion Detection Systems Fail to Impress
TheBongPipe writes "I'm reading a nice test here about 7 commercial IDSs. Who won the prize? Nobody..." They also looked at Snort, but found that all the products generated way too many false alarms.
What IDS do slashdot users use ?
Those systems are rather difficult to design at times. I think it's odd that 7 systems didn't work to their full-potential.
They also looked at Snort, but found that all the products generated way too many false alarms.
Too many false alarms isn't necessarily a bad thing. In intrusion detection you'd rather take the false positives, than the alternative.
The rate of false fire alarms, and false burgular alarms is VERY high compared to the actual number of real emergencies.
I had a nice experience using snort.
:/ Point and click is not always the best solution...
Come on, reading the article I saw the guy said a Snort disadvantage is not having a GUI. What kind of technical user this guy is?
Fabio - Sumare/Sao Paulo/Brazil/South America/Earth/Solar System/Milky Way/Universe
http://www.morroida.com.br
Like a pregnancy test, I think the false positives are preferable to sitting around thinking you're safe.
Liora
Is not exactly an accurate science; either one has to deal with false alarms, and probably more than a pleasant number of them for the sake of paranoia, or simply be more sure of their systems.
Informal poll who among us uses an IDS, and why? I've always figured them for a way to be lazy, but that perspective is likely not shared. I'd love to see some other opinions!
// -- http://www.BRAD-X.com/ --
Compare with my program that suddenly displays "!!! RED ALERT !!!" at random.
It'd be nice to have some more detail on their results. The chart on the page shows Snort detected all the attacks listed in the chart except the SYN flood. And the footnote on that entry says Snort was down because of "configuration error."
Gee, whose fault is that?
They also go on to mention all ask too much of their users in terms of time and expertise to be described as security must-haves. IDSs are not screen-savers. Those who are setting up an IDS better have a good understanding of how they work and how to configure these applications. Point-and-click doesn't really apply to something this involved.
Like Car Alarms, if it goes off all the time, people will just ignore it -- At some point, the noise drowns out the signal.
You would hope that the increase in false positives decreases the number of false negatives but that isn't necessarily true either.
I am not a number! I am a man! And don't you
Crying wolf: False alarms hide attacks
Eight IDSs fail to impress during the monthlong test on a production network.
By David Newman, Joel Snyder and Rodney Thayer
Network World, 06/24/02
One thing that can be said with certainty about network-based intrusion-detection systems is that they're guaranteed to detect and consume all your available bandwidth. Whether they also detect network intrusions is less of a sure thing.
Those are the major conclusions of our first-ever IDS product comparison conducted "in the wild." Unlike previous tests run in lab settings, we put seven commercial IDS products and one open-source offering on a live ISP segment to see what they'd catch.
What we found wasn't encouraging:
Several IDSs crashed repeatedly under the burden of the false alarms they churned out.
When real attacks came along, some products didn't catch them and others buried the reports so deep in false alarms that they were easy to miss.
Overly complex interfaces made tuning out false alarms a challenge.
Because no product distinguished itself, we are not naming a winner (See "No cigar"). The eight products we tested - from Cisco, Intrusion, Lancope, Network Flight Recorder (NFR), Nokia (running on OEM version of Internet Security Systems RealSecure 6.5), OneSecure, Recourse Technologies and the open-source Snort package - all ask too much of their users in terms of time and expertise to be described as security must-haves.
That's not to say IDSs have no place in corporate networks. They can be valuable tools for learning about network security and can validate that other security devices are doing their jobs. But setting up the current generation of IDSs requires a substantial time investment to ensure they'll flag only suspicious traffic and leave everything else alone.
We used the production network of Opus One, an ISP in Tucson, Ariz., as our testbed. Opus One offers Web hosting and leased-line, DSL and dial-up Internet access services to 50 small to midsize businesses. The backbone infrastructure includes nine T-1 circuits with an average utilization in the range of 9M to 12M bit/sec.
To spice things up a bit, we deployed four "sacrificial lambs," systems running old, unpatched versions of Windows 2000 Server and NT 4.0 Server, Red Hat Linux 6.2 and Sun Solaris 2.6. Putting plain-vanilla versions of these operating systems on the Internet is just asking to be attacked. Past studies have shown that unpatched systems get owned in a matter of minutes, thanks to automated scripts that find and exploit well-known vulnerabilities. We figured the IDS sensors couldn't miss seeing these attacks.
All IDSs consist of at least one sensor that monitors traffic and sends alarms whenever suspicious behavior occurs. There are two major methods of detecting problems: signature detection and anomaly detection. Signature detection, used by all products in this review except Lancope's StealthWatch, will generate an alarm whenever traffic matches a known attack pattern. With anomaly detection, the IDS compares current behavior against a baseline of "normal" traffic on that network and flags anything out of profile as an alarm.
"They also looked at Snort..."
No they didn't. It was posted by "TheBongPipe" and I don't see how you could possibly snort that
Oooooh... they were particularly hard 'testers' " because no product distinguished itself, we are not naming a winner. " The eight products they tested were from Cisco, Intrusion, Lancope, Network Flight Recorder (NFR), Nokia (running on OEM version of Internet Security Systems RealSecure 6.5), OneSecure, Recourse Technologies and the open-source Snort package. Definitely a MUST READ article in these times where everyone is looking for silver bullets and according to this article there are a few 'blanks' being fired around...
The problem with many of the statistics from this test is that the management software was considered equal to the actual IDS machines. The "uptime" was actually garnered from the management software staying connected for the entire period. Given all of the complaints about java consoles being sluggish, I can only wonder at what the console machine was...
From reports of the test the "wu-ftpd" exploit used wasn't an actual exploit, but was only a replay of one of the signatures of a decade old exploit. Since not all of the systems use signature detection, and since the "exploit" didn't actually exploit anything (*gasp*) some of the IDS systems didn't pick it up.
It didn't crash the entire time.
According to the chart, it also didn't detect code red worm, SYN flood, or wu-ftp exploit.
Was this box even operating correctly?
It amazes me that people will pay $20,000 for a product that regularly crashes, doesn't detect all intrusions, and can only be kept up by constant, expensive intervention from the vendor, when for $20,000 less you can have a similar product that doesn't crash, detects just as many intrusions (though not all of them) and can be maintained either by the vendor, or by anyone else with the wit to understand it.
IDS are complex systems. Anyone pretending they have a packaged solution should rot in jail.
--
E_NOSIG
I recall a user we had on our network who thought it'd be cute to install BlackIce on his box, to better secure it. Nevermind the fact that I, and the rest of the admins at my company, had firewalls in place and had never had an intrusion on our network.
Imagine the fun the first time we try to deploy an antivirus package to his desktop just to be blocked for -- are you sitting down? -- an attempted NetBIOS intrusion.
After the second time we tried to deploy (and failed) BlackIce locked down the system so that it couldn't be accessed across the network by any other workstation, despite our having adminsitrative rights. That was cute.
Just throwing up a little real world example of how annoying these false alarms can be.
This article came from the point of view of a normal administrator trying to also manage security. It is mostly based on the assumption that you use the default ruleset (there's no mention of what ruleset is put to use).
Nowadays you really have to be selective about what ruleset you use, logging too much isn't a good thing. This is part of the reason you need a qualified Intrusion analyst who have the expertise to determine which ruleset is useful and which isn't.
The worst thing that can happen (which does happen quite often) is after paying for the expensive distributed sensor IDS system, the logs are never processed or read by anyone.
As stated by the article, an IDS is suppose to log anomalies, that is any abnormal behaviour. But anomalies is only useful if you have a technical guy capable of analysing the traffic. In fact, I would rather have a faulty IDS system that misses packets than to have a good IDS system and all logs go down the drain at the end of the day.
Better to be safe than sorry. Better to punt any wierdness off your network quickly and keep things running right rather than have it taken down. What if it was the other way around, letting all sorts of attacks slip under. If a host on your net is sending strange packets - terminate with extreme prejudice. Hell, plonk that whole network segment if you have to.
But Opus One's servers run OpenVMS, not Windows. Even though it is trivially easy to figure out what operating system a Web server uses, not one of the IDSs did so. Instead, they collectively generated literally millions of alarms about attacks that never happened.
That's an unrealistic expectation to place on an IDS, from the start. You get an IDS to log attack attempts first, not the attacks themselves - if the attacks are known (have signatures) your machines should be protected against them in the first place.
Hands in my pocket
Yeah, I'm going to take what this guy says with a grain of salt, and maybe some doritos too.
Prevention is another way to help secure a network, rather than simply detection.
CycSecure from Cycorp the makers of OpenCyc, the AI reasoning system, helps prevent attacks by using an AI engine to simulate attacks on your network to identify problems.
It's worth looking into.
"The large print giveth, and the small print taketh away" -Tom Waits
So, anything out there besides Snort? I just installed 1.8.7 on my Linux machine and was (un)pleasantly surprised to find that my (un)favorite Snort feature was brought back: random mysterious death of the snort daemon, with no logging or other diagnostic. But only in daemon mode, mind you, making the problem fun to debug.
Luckily it doesn't do it on FreeBSD which is where I really need it running, but it is really frustrating and doesn't instill a lot of confidence. Grumble grumble, bitch, moan, etc.
This review wasnt done very well. There was a lot of discussion on the Security Focus Focus-IDS list. Robert graham, main craeted of the BlackICE engine (and the guy who wrote altivore) summed it up nicely in this posting (text below): http://online.securityfocus.com/archive/96/279595. Also, the entire thread can be found at: http://online.securityfocus.com/archive/96/280125/ 2002-07-08/2002-07-14/1
u rity1.html
Actually, most of his posts tend to have interesting (and qualified) views on IDS> sure he is biased (a vendor) but his commentary is usually thought out and not vendor-ish.
> From: Andrew Plato [mailto:aplato@anitian.com]
> In-Reply-To:
> >http://www.nwfusion.com/techinsider/2002/0624sec
> Next time they should do RealSecure on one of my Win2k
> appliances.
No.
While it is true that the reviewer found a bug with the Nokia platform that
doesn't exist on Windows or Solaris, there wasn't anything especially wrong
with the platform.
The issue is that the reviewer was hostile towards IDSs. A customer wants
his product to work, so when they don't, they will keep calling tech support
until it does. Reviewers want the products not to work, so they will
construct the nature of the test in order to make sure this happens. The
reviewer, in this case, never called ISS; the first we heard about him was
at the end of this review, not at the first crash of the Nokia box.
RealSecure has a unique feature called "audit" events. These are supposed to
trigger on normal traffic, such as every HTTP GET request. These are useful
either to create audit trails, or as "anomaly detection": turn on all
audits, then turn off those that trigger normally on your network.
This reviewer turned on audit events, which flooded the console. The setup
that Nokia provided them (256-megs of RAM and a database limited to
2-gigabytes) is perfectly reasonable for the network they had, but not if
all audits were turned on. (The Nokia bug we fixed was related to the fact
that it didn't have enough memory to handle the event load). The reviewer
complained about an overload of false-positives and the box crashing, but
this was because the reviewer drove the product to the point where this
happened.
In truth, it isn't always obvious which of our events are "Audits" and which
ones are "Attacks"; this is an issue fixed in 7.0 of our product. I doubt
this would have made a difference in the review: 7.0 has a lot more audits,
allowing reviewers to overload the product even more if they desire.
Imagine a review of automobiles, where a reviewer grabs a Ford Explorer and
starts complaining that it still crashes, even with the Firestone tires
fixed. One might ask if the there is a problem with the Ford, but one might
also ask if the reviewer intentionally drove the car until it crashed. Next
time you are driving down the freeway, violently jerk the steering wheel all
the way to the right. If you survive, you'll understand what I mean.
I'm not saying the review is wrong. As the reviewer said, he learned a lot
about IDS during the process of reviewing these products. If you, too, don't
know much about IDS but are planning to install one, you will likely get the
same experience: being overwhelmed with alerts that are "false-positives",
and a general sense that the product isn't working. The first few months of
running the IDS are likely to be particularly frustrating. I suggest (a)
working with a consultant to tune the system, (b) working with the vendor's
support in order to get suggestions from them, (c) learning more about the
system. You are going to do (c) anyway: after a few months, you are going to
have learned a heck of a lot more about hacking and defense then you ever
dreamed possible. Read the review: take it with a grain of salt knowing the
reviewer wanted all the products to fail, but realize that this likely to be
your experience the first few months after installing the product, you are
likely to be overwhelmed with events and unlikely to be impressed during the
first few months of ownership.
Robert Graham
Chief Architect
Internet Security Systems
Just read the article. A bit poorly written. What were the IDS run on? Why no analysis of Snort? I'll say that I find Snort way over my head, but that's because I haven't RTFM enough. Why would one want a GUI on a server? (one of the points they marked it down for). Why did it crash? I've NEVER had a linux box crash. NEVER. I've also very, very rarely had a program freeze up enough to require a kill -9 (other than Netscape Navigator and some other buggy stuff. Not stuff like exim, apache, etc.) As a matter of fact, scroll down, and it seems that the downtime was due to their problem, not Snort (footnote at bottom of uptime table).
There are complaints about false-positives. I've played with Snort and there are ways to decrease the alarms put up. For example, a certain number of bum packets in a certain length of time. Not each and every packet.
Looking at the info at the bottom of the article, the authors should know what they are doing. But given the misrepresentations and inaccuracies releative to Snort, why should I believe their testing of non-Free software was any better?
Maybe it was eWeek or some similar publication about six or nine months ago did a similar check. The article was much longer and more in depth. They were also more appreciative of the programs out there. Now, some will say "just to appease their advertisers". Well... Maybe. But if that is the case, why did Snort get their nod as the best?
Jesus was all right but his disciples were thick and ordinary. -John Lennon
I would also like to remind everyone having pride in their own IDS that NIDS will never catch every single attack. (At least for the next little while)
Signature based detection is only good if the attack utilize abnormal or unique traffic to exploit the vulnerability. It will not pick out attacks that uses normal common traffic (for obvious reasons).
IDS evasion techniques are also heavily worked on, plus all application level evasive techniques (eg. sidestep). We can just never be totally dependent on the NIDS for telling us intrusion has occured. It works for most attacks but will fail for some.
Not having a GUI?!?
/etc/snort/snort.conf
I've been running Snort for some time now, and love it! I'm using MySQL logging with ACID and ADODB under Apache for a front end. You just can't get any easier than fill-in-the-blanks SQL querys and intuitive packet layouts. Obviously, they want a strictly out-of-the-box product, and aren't willing to invest any time to make a solid IDS.
As to the false positives, I can concur that in the beginning it was daunting seeing the flood of alerts, but in time, you figure out what is normal and what is not. A little restructure, or a few rule overrides, or rewritten rules, and it's seamless. All it takes is time. This is akin to bitching that your fresh *nix install doesn't have everything just the way you want it, with all your custom apps and modules. You can easily reduce the number of snort alerts by passing the command option as:
snort -D -o -i eth2 -c
This (the -o) changes the rules order to Pass:Alert:Log killing home network normal activity before alert processing. It helps immensely!
In the same vein this article http://www.gigaweb.com/mktg/man_sec_mon/cpane2.asp compares various managed security services, which also offer and utilize some of the various IDS systems you've mentioned.
I have used Snort and Qualys (the high priced commercial outsourced IDS) and both give false positives quite frequently. However, proving they are false positives is part of the skill of a good human sysadmin. This is why IDSes will never replace a good sysadmin. He or she should be able to see the report and say without any shadow of doubt in his speech that any particular exploit shown by the IDS is a false postive or not.
This still means that each IDS has its good points; but why anyone would pay a lot for a system that cannot, by definition, be any better than an up to date Snort and human reading of the report, and knowing your network inside out. Those who buy into big commercial IDSes clearly are investing in software when they should be investing in people, training those people, and understanding those people. Too many middle managers think their sysadmin speaks a language they will never learn, and therefore need these things to understand. But a good sysadmin should try hard to find ways to communicate with them, and can if need be annotate a nice little Snort report and be done with it.
Conversion Rate Optimisation French / English consultant
If you look at the table, snort looks like it was doing great, except that it somehow missed the SYN attack. So, based off that chart, none of the IDS corrected detected all of the attacks... however, you read on a bit further, and..
Snort was off the air at the time of the attack because of misconfiguration on our part.
I don't have a lot of confidence in their results.
Too many false alarms isn't necessarily a bad thing. In intrusion detection you'd rather take the false positives, than the alternative.
Spoken like someone who does not carry the IDS support pager at nights and on week-ends!
The problem with too many false IDS alarms is that the staff tend to treat it like the boy who cried wolf. After awhile, you disregard the pages or treat them with less consideration because the last n pages have all been false alarms.
I think that IDS is important, but if there are too many false IDS alerts, it becomes difficult to put up with. Because they are strictly reactive systems, it is improbable that there will ever be a perfect IDS that never raises false alarms, but clearly there is a lot of work to do. I am surprised that Snort did so poorly, since it really is a nice system, but it takes a long time to build up a good set of heuristics...
The rate of false fire alarms, and false burgular alarms is VERY high compared to the actual number of real emergencies.
That's right. And in my area, if the police department are called out to the same location for three false burglar alarms in one year, they will not respond to any subsequent alarms automatically. And the fire department charges a fine of $300.00 per incident if they receive more than three false fire alarm calls to the same location in one year. Why? Because, as you said, the number of false alarms is much higher than the number of actual emergencies. The false alarms cost time and money and if all the resources are busy dealing with false alarms, there is nobody left to help when a genuine emergency occurs.
*** Where are we going? And what's with this handbasket?
because they make their IDS too complex # cat shoot_em_up_bad_guy_ids.conf #
vodka, straight up, thank you!
It's definitely true that this is one of the most notable weaknesses of intrusion detection systems as they exist now. I work in a financial institution where upper management has finally made a sensible decision and devoted a full-time person (me) to network security but that's not the case in many smaller organizations. The vast majority of (external) intrusion attempts are from script kiddies that use pre-fab tools and put forth little effort to conceal their actions. In my opinion, this is justification for most networks to run in a "low paranoia" mode. This would get rid of excessive false-positives and the noise created by Joe Kiddie and his 10,000 buddies who are out there constantly port scanning class A subnets.
Homer: Now, here's my "Everything's O.K."alarm!
[Homer flips a switch on the device, and it begins to emit a high pitched, incredibly loud beep. The rest of the Simpsons cover their ears as Homer speaks up]
Homer: This will sound every three seconds, unless something isn't okay!
Marge: Turn it off, Homer!
Homer: It can't be turned off! [alarm fizzles out] But it, uh, does break easily.
-- "The Wizard of Evergreen Terrace"
This sounds about as useful.
Carthago delenda est!
Funny part is, you can take your pick of UI's for snort, on just about any platform (I run snort on WinNT on one network, and snort on Linux on another. And I've got a GUI for both of 'em ;-)
Davis Ray Sickmon, Jr - looking for something to read? Check out my three free novels at MidnightRyder.org
Trollem mirabilem hanc subnotationis exigiutas non caperet
I recall a user we had on our network who thought it'd be cute to install BlackIce on his box, to better secure it. Nevermind the fact that I, and the rest of the admins at my company, had firewalls in place and had never had an intrusion on our network.
;-)
I hate to tell you this but, at this day and age when everything is being outsourced, some users feel they need to protect their machines against the "IT support".
"I have opinions of my own, strong opinions, but I don't always agree with them." -- George H. W. Bush
IDS's are just plain silly. Do the following:
1. Turn off unused services, and disable suid software. If this is not possible, then don't use the application.
2. Take steps to minimize compromise of at-risk services. Use chroot, aplication specific uid/gid.
3. Leran how to apply packet filters to ingress/egress points.
4. Apply patches for applications and O/S
5. Stop going to expensive, lame 'security' seminars.
fini!
IDS will someday approach NMS systems in uselessness
IDS Systems are only as good as the people (i.e. admins) using them... ged68
... you may as well go with Snort, which is free. All but the $2,500 Nokia are $12,500-$25,000. Excellent article.
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
IDS systems need to be tuned! Don't have any NT machines on that subnet? Turn off all of the NT related signatures! Get tons of false alarms on a particular alert which isn't applicable? Turn it off! It's a matter of risk assessment. Are you more likely to miss something important because of this alert which goes off all the time and has a low probability of being legitimately triggered? Turn it off! You won't catch everything this way but the goal is to at least catch SOMETHING that you would not have if you didn't have the IDS!
IDSes are NOT meant to work out of the box. Snort's FAQ specifically states that you should disable rules for things you don't need! By default, it includes a lot of stuff. Luckily, the rules are neatly organized into files, so you can comment them out, and stop getting warnings you don't want! Likewise, using Snort without Acid is well... not very common. Yet, there is no mention of Acid in this article. I can only imagine that the rest of this article is flawed due to the reviewer's lack of knowledge.
I created and help to operate a large-scale Intrusion Detection System running at multiple disparate sites throughout the Internet. All of these sites have their own unique network environments, and false positives occur as a normal part of network traffic.
Through the use of aggressive filtering, it is very easy to eliminate most false positives and false negatives occuring in your intrusion detection environment. It is often easier to focus on incoming traffic rather than traffic emanating from within your network, concentrating on external entities attacking you. If you know for instance that there are no Qmail installations on your mail server and a signature is being tripped accidentally, filter that signature from the Internet to your mail server. It should take several months to properly train an IDS and even longer to train a network of them, but the end result is a very effective device.
The reason I mention this is that the NWFusion fails to talk about filtering, when this is decidedly a part of the game when administrating an IDS network.
-biv
haha... "TheBongPipe".. that's an awesome nick. smoke some of these Northern Lights, bro. Got some maui wowwie, too (seriously, i do, though I ran out of northern lights).
To wit, we've used all of the products they complain endlessly about, and all I can say is RTFM. All of the problems they encounter are either configuration problems or worse, PEBKAC.
If you want to really learn about IDS, and you don't have the budget to buy a commercial IDS, download a copy of snort and learn for yourself. This report strikes as the type of complaing you get from an IT customer that wants to buy a product, turn it on, never configure it and expect it to magically work.
Wow! What a revelation! You mean you have to know what you're doing and it actually takes time to configure these powerful tools?! In a word, DUH. IDS'es must be tuned. IT products must be configured properly. These things take time, sometimes a lot of time. The core of their complaints revolve around their inability to do either of these things well. Given that lots of people manage to do this effectively everyday and have been for years and years, we're left to conclude that these reporters were not up to the task. And here it is:
These folks actually expected NIDS to be plug-and-play, and thats what they seem upset about. NIDS are powerful sniffers, they need to be tuned, they need to be configured and yes, this IS an ongoing process - but they are not plug-and-play devices.Futhermore, all of IT is an ongoing process. A big, circular, ongoing process that requires competent personnel to manage, maintain, tune, test, patch, configure , deploy and yes, spend TIME on. Anyone that expects to be able to deploy close to a dozen different IDS products as plug and play devices into a production network in 90 days with questionable expertise is fooling themselves.
And then they say as much. Again, this report is total waste of time. Its overly sensationalized and stems from a lack of expertise on the products in question. Skip it, download snort or buy one of the commerical products, take a class, read a book and learn for yourself. You won't learn much from this report that common sense wouldn't have told you already.Python
Was this test run outside of a firewall/filter? Because it would seem like the IDS is best used behind a full strength firewall in the first place; that is, most of the attacks should never reach the IDS anyway. We use Snort to monitor our networks, and under proper configuration the number of alerts is manageable / predictable under normal circumstances.
This article sucked.
It would have been a lot more interesting and useful if they had concentrated on getting the IDS' set up right in the first place, and then wrote about the results. The whole article basically describes their learning curve with an IDS, 7 times over. For instance, of course you need to tune the alerts. You can't possibly log everything, and likewise, the vendor has no idea what you are interested in watching in your environment.
It was the reviewers own expectation for 'plug and play' which ruined the test. It sounds like they installed the software, turned it on, and didn't bother to read or try to understand what an IDS even does before starting to 'test'. Pretty worthless read, IMO.
Too bad, it could have been interesting. Sounds like an interesting environment to do this test.
I would not get or avoid any of these IDS' based on this review. Its just not helpful.
If you want real security, you should have your IDS monitored by Riptech. Ok, I work for Riptech, but we really do offer a good service. We know more about IDS's than you, so we can help you set them up properly, and we essentially mine the data coming off your IDS in real-time, and our software is VERY good at eliminating false-postives.
Why is Snort the clear winner? Because it's the only one that doesn't cost anything. If none of them work as well as they should, at least with Snort you aren't blowing money on the software :)
This sig has been temporarily disconnected or is no longer in service
I am the Director of Managed Security for a company on Hawaii, and we rovide managed Security Services to various companies around the state based on Snort. Snort truly is a very good IDS, and if configured properly it will generate few if any false positive alerts. Most of the reason that people say bad things about a product is due to their own lack of experience in setting it up.
Wherever you go, there I am...
The author of the article complained that the majority of the systems reported results by IP and not domain name.
This statement alone gives me reason to doubt their abilities. Combine this large amounts of traffic they have with a reverse-DNS lookup for each and you would have crippled your DNS servers.
This is configurable in Snort, and mentioned in detail in the Snort docs.
A good solution is to either dedicate a DNS server to the IDS box, or use a script/utility to do reverse-lookups on the items you are interested in. Not live, but when a human is looking at things.
They are right about one regard -- IDS configuration, monitoring and maintenance isn't for non-professionals. You *need* to know what you are doing.
Learning HOW to think is more important than learning WHAT to think.
IMHO, the recent review performed by NSS reveals a more advanced understanding of IDS technology. They haven't evaluated quite as many NIDS in their review and instead have opted to include a few HIDS for good measure. -Gordo
You can't simply plug these things in and expect them to work perfectly. If you don't know what you are doing with a high-powered IDS then you don't have any business using or judging them.
You need to take quite a while (based on your network) and OPTIMIZE your rules for a product like SNORT so that you are getting alerted to the types of things you want to know about while minimizing false positives.
It is pretty obvious that the tester didn't do that, and as a result he had nothing but bad things to say.
Let's let an experienced Snort user configure his conf files and then run the test again. I think you may find that the results are different.
dmiessler.com -- grep understanding knowledge
Those guys are completely clueless when it comes to IDSes. They tried to run them with all alarms turned on, which is not what anyone would do on a production network.
Really, that article is ridiculous. It is obvious they never, EVER, used an IDS in production.
I wish that kind of articles to be written by more knowledgeable people.
This article would have benefited greatly from a diagram or two. IDS behind the firewall? IDS in front of the firewall? Possibly they mentioned it, but I failed to find reference in the article. Myself, I prefer to keep the IDS behind the firewall as I only care about packets that get through. How do /.'ers deploy?
From the article:
"On any of the sacrificial lambs, outbound traffic of any kind is a sure sign that the machine has been compromised."
So, based on this metric aren't 90% of the "modern" Microsoft applications compromised, right out of the box?
2. Offline because of configuration error.
Gee, I wonder if they should learn how to configure Snort before they test it.
People who have witty things here blow.
It isn't a matter of configuring your IDS that
makes your life peaceful, it is understand what is
normal on your network. The fact that you see 3000
of something that your IDS says is attack xyzzy
doesn't mean that one of those couldn't be real.
You cannot simply ignore that attack if you have
systems on your network that could be vulnerable
to that attack.
You can tell the IDS to ignore everything that for
which you do not support the vulnerability. What
you are left with is hopefully a more manageable
set of possible attacks. On those remaining alarms
you have to learn what is normal for your network
in order to detect a real attack. You need a way
to pull the signal, i.e., the real attack, from
the noise of the backdrop of false positives.
I suppose this is a good time to plug my university's project, STAT. STAT is an open sourced IDS framework. It allows you to monitor arbitrary events and take arbitrary actions based on them. It's possible to extend the field of STAT's vision by writing extensions to STAT in the STATL language. It's also trivial to write responses to known exploits.
You can find more info about STAT at
http://www.cs.ucsb.edu/~rsg/STAT/
STAT already has 2 extensions, NetSTAT and USTAT that watch for common network and unix-level exploits. Other projects include making java-level IDS's and mobile agent IDS's. It's a great project and it blows everything else out of the water. If you're dissatisfied with IDS's as they are, check out STAT.
Slashdot. It's Not For Common Sense
They seem to want a NIDS that somehow magically knows about the systems and services on their network and ignores all attempted attacks that aren't against vulnerable systems. This is like ignoring all attempts at burglery that fail to get inside the door, giving a pass to all the burglers who case your company, or don't have the right tool to defeat the specific lock you use. Yet.
The better attitude is to report all these ineffectual attacks to the likes of Dshield, and help clean up the neighborhood.
They also looked at Snort, but found that all the products generated way too many false alarms.
Curses, foiled again! If it weren't for that pesky "not too many false alarms" requirement, I'd be able to create terrific security software. I'm picturing a system that generates a "WARNING: NETWORK SECURITY BREACH" message every five minutes, rain or shine. Keeps the sysadmins on their toes, and foils all network intruders who aren't fast enough to be in and out in five minutes.
It's the only way to be sure.
It is really too bad the Enterasys product wasn't available. I've implemented that on a _very_ large government department network with dual T-3 pipes and collecting >1Gig of data per day. Yes, still many false alarms to sift through but the uptime was measured in months not days or hours. This same gov't department has had other IDS vendors try to bring in their products to no avail because none of them can stay up >24 hours.
Do really dense people warp space more than others?
Where I work they try to mandate anti-virus software running on all the PCs. Only problem? The damn anti-virus software locks up my PC almost daily.
The really shitty part is that just when I get the anti-virus off my system, IT (in their infinite wisdom) pushes an "update" onto my system.. sending me back into blue-screen land.
I finally installed BlackIce on my system and set it up to deliberately block the fuckers. Yeah, IT gets pissed and thinks I'm stupid or something, but at least my computer doesn't consistently bluescreen anymore.
Let's see, I'm going to spend 5 digits up to protect way upwards of 7 digits in information. Therefore I should insert the CD, click the mouse, and that's all the expertise I need.
So how come alarm companies exist? According to this logic, everyone should send a co-op to Home Depot and have them install the company alarm system. Maybe it'll take a day.
Where have I seen this attitude before? Oh yeah, the guys who turned on their new computer, plugged it in, and called it their web server. The ones that scan my boxes' port 80. The ones that are owned by Code Red, Nimda, and Klez. The "network administrators" with an MSCX certificate on the wall. The Windoze users.
The company that makes the opensource security tools portsentry, logcheck, and hostsentry released something a few weeks back that they claim helps with this problem. The tool is called ClearResponse and it (supposedly) will investigate each attack seen by your IDS to see if the alarm is real or false and will then downgrade or upgrade the alarm and report to the admin. It looks pretty cool and we're getting an eval to check out. Here is their website:
ClearResponse
My brother used to work at a naval shipyard. The fire alarm for one part of the site also rang at the central telephone switchboard, where the operator had to call the fire brigade manually. One day, the alarm sounded, she called the fire brigade, they responded, and it turned out to be a false alarm. Later, it sounded again, again she called it it, and again it was a false alarm. The third time it sounded in a matter of hours, she decided it must be another false alarm, and made the decision not to call the emergency services. It turned out that she was correct and it was another false alarm. That telephonist was fired the same day, and an automatic system was subsequently installed.
bleh, thats like hiding your head in the sand. 'They can see me so i think im safe'. No, your a victim who does know it.
If your talking about one or two computers at home, AND you know your systems well, AND you devote the tike to finding out about vulnerabilities, AND then patch your systems, great this works.
If your a business with several systems, diverse needs, customers, and have anyone touching the systems other than you, this probably will not hold up. PLUS - the main threat - insiders. Forget the external attackers. ignore them for the sake of this. Where is your threat? Most threat/risk (70-90% per most survey) is INSIDERS. IDS can tell you when that consultant you just hired decides to do a network discovery to see whats there. It can detect that 'funlove' or Klez/ElKern virus as it spreads across the network.
I have seen many posts about IDS creating to many alerts. Idiots. What is a false positive? An alert that indicates something unsual, but is not a hacker? No. This is a network anomoly. Collect these and you can learn very interesting things about your network. Find attackers, yes. Inside and out. Also find misconfigured systems. And dont give me that crap about 'smart hackers will just bypass it'.
Yes, that is true about an advanced hacker/cracker. But IDS WILL detect the imbicile IT/Finance/Engineer/Etc employee that decides to do something malicious. Those exist in a large organization in abundance. It WILL detect the idiot developer that decides to test something on your product network and unkowningly mucks up your deadwork and takes critical servers offline.
It is not a panecea that you 'buy' and protects your network.
Things I like about it:
installation script is truly a marvel (installs snort, mysql, apache, perl modules)
Login screen/authentication
Big Brother like monitoring
File integrity checking
IDS using Snort sensors
free to use for non-commercial use
no I don't work for them I just like the software.
Fact is any IDS, now or in the forseable future, assuming we do not crack that artifical intelligence thing, is going to require a qualified and skilled security professional to monitor and tweek any IDS.
You are not gonna find security out of a box. There is no software out there that alone will outsmart even the most lame brained...you gotta have people there to protect your system or sooner or later you will be owned.
Snort was only 1 of 2 that don't cost over $10,000 (U.S. I imagine). The other that was under $10,000 only started at $2,500 and goes up depending on your options. Gee, wish I had $10,000+ to blow on some hardware, then use snort on $100 worth of hardware in the form of a kinda cheapy PC. To apply the motto at my place of employment, "TWMS" which stands for THAT WOULD MAKE SENSE. Hardly anything is done that makes sense at my work. Oh well...
ZERO ZERO ONE ZERO ONE ZERO ONE ONE! Just brushing up for my next big invention: Ethernet over Voice (EoV)
You mispelled imbecile. My point was that an IDS provides little in added security if the steps described above are taken. Use tcp_wrappers to listen on a bogus port and report port scans, if that makes you feel better. Even use Snort---but please don't waste $30k of your company's money on a commercial system.
I don't trust any "real world" shootout that doesn't show how the IDS were plugged into the network, how they determined an attack, and other such key points. You can't just say "we plugged it in and nothing worked." IDS are much more complicated than that. How and where were they plugged into the network fabric? Were they using switch port mirroring or passive ethernet taps at the uplinks? How do they know these attacks happened without initiating them themselves? That last one is the biggest single problem with "real world" testing. Unless you're launching the attacks yourself you do not know, and unless I missed it, they were relying on attacks to just happen out of the blue.
Now, they do raise some important issues with the backend storage of events and the need for clarity with the false positives and false negatives, but many of these can be dealt with by implementation of a real-time security console that does some form of event correlation from multiple security devices that says "The IDS sees this as a problem, the firewall sees it as a problem, and the target sees it as a problem. It's probably a problem. RED ALERT!" It's a much more intelligent way of dealing with events than just forwarding each one to a pager.
We've always said security is a process which must be maintained and firewalls/IDS systems are not a panacea to network security. As someone who's been responsible for a large scale IDS roll-out at Enron Broadband Services, where we were ISS' single largest customer for RealSecure before everything went to hell, I feel confident saying that Network IDS is a very useful tool, provided you keep it out of the hands of people who have absolutely no clue what they're doing with it, like the three gentlemen who are responsible for this article.
JosephLet me suggest an alternative. Since you can build for $5000 or buy for under £3200 a terabyte disk store, why not record all network traffic in a rolling log for a few days or weeks. A background process can look for attack signatures, and see what the response was. So an IDS attack signature of "cat /etc/passwd" won't be reported unless it appears that a password file actually was returned.
The NSWC Shadow project is similar, but does not necessarily record all traffic.
One great advantage of having a historical log is you can see if you had been attacked, during the interval between an exploit being discovered and being notified with a new IDS signature. If you know you were not exploited, you save a lot of time not needing to check systems or reinstall to be on the safe side.
Anyone like to build me one?
Andrew Yeomans
There is a misconception among IDS noobs that alerts are like alarms: alerts generate alarms. An IDS generating 1000 alerts should not mean that an admin would receive 1000 alarms/pages.
For example as an IDS admin I want to see alerts for failed telnet attempts internally. If it's 5 within one minute directed at one host, it's not a problem, probably just human error. If it's 1000 per minute, then I want to see an alarm, and get paged. The "reviewers" of the IDS products would have understood this, and taken it into consideration had they ever deployed an IDS in a production environment.
Alerts are not a bad thing even if they are false positives. False positive alarms are a bad thing, especially when they wake you up at 3AM. Again it comes back to tuning.
I can however verify what they experienced with ISS. At my last company we had ISS come out and install, configure and tune Real Secure. I can tell you from first hand experience, that the ISS products suck. I ended up installing Snort in order to keep the ISS products honest. My experience was that ISS had a nasty habit of dropping packets. Snort had no problem keeping up with our frational DS3 (30 Mb).
More recently I also had the priveledge of seeing Real Secure crash repeatedly on a Nokia 530 (installed and configured by ISS engineers) during an IDS pilot. We kicked ISS to the curb and are now in the process of installing Snort with support for ACID, and MySQL.
IMO the reviewers were idiots. They obviously didn't spend much time with any of the products. Which is unfortunate, because some of the good products got lumped in with some of the bad ones, which were all failed together because some reviewers obviously didn't RTFMs.
FYI there is also a really pretty GUI for Snort:
www.demarc.org
If people are expecting security-in-a-box from an IDS, of course it's not going to live up to their expectations.
An IDS is nothing more than something to alert you to any abnormal conditions. It's a tool to help filter out the noise and show you what you want to know.
It really gets my blood boiling when journalists review security technology poorly. IDS is a powerful TOOL to help you secure your networks. It detects and helps you respond to security incidents (with TCP resets and "shunning" of attackers when integrated witha firewall). IT HAS TO BE TUNED! You can't go down to your corner computer store and ask, how much will security cost me? There's no magic box you can buy! This is what Journos expect, a magic box. Journalists when they do a wide comparision of security products usually do a few things: Install software on standard hardware (click next a lot). Look at the pretty GUI. Click on all the buttons they don't understand. Measure throughput/uptime as it is easy to do in between sneaking off to the bar. Do some half-baked "real world" test. It is really highlighted in this article, how they did not understand what role IDS plays in an organization. IDS is your eyes and ears in a corporate security setting. It lets you know what's going on...who's runing peer to peer file sharing, who's scanning networks for vulnerabilities. They installed it in an ISP for A security professional is tuning their IDS EVERY day, modifying the policies on the IDS to reflect the constantly changing environment that it is installed. Watching the console at every spare minute! When we review security technology for use with our clients some of our criteria are: 1. Ease of deployment (product availability, channel relationships, ease of install) 2. Ease of management (most security incidents are due to misconfiguration) 3. Availability 4. Performance 5. Market Share of company 6. Level of technical support provided by vendor. Again and again it annoys me how journos write off good products!
None of the IDS systems they named "detect" intrusions. NO IDS system on the market really detects intrusions. They either look for known signatures of various exploits, probes, what have you, and report on them, or they do some form of anomoly detection based on a "baseline" for the network they're observing.
Neither of them is flawless OR a complete solution.
NONE of them are going to be perfect out of the box. It takes skill and experience to know what's important and what's not. None of these IDS systems are going to catch the guy doing a slow map of your IP space. ALL of them will false positive on some things, and miss others completely.
It takes a human at the other end to look and see and decide what's a threat and what's not.
We won't go into the lack of information on how they configured each one of these IDS systems. I use snort at home on my LAN, but have worked with NFR and Cisco's Netranger - and each has it's advantages and disadvantages. If you're SERIOUS, you're combining something from a commercial heavy duty IDS, with Snort, with dumps from all your syslogs, some kind of host based IDS, and putting together the individual pieces to see what's happening. Then you might be able to detect a skilled intruder.
It's NOT for the faint of heart, the clueless, or, it seems, the media pundits.
Never attribute to malice what can as easily be the result of incompetence...
at least my computer doesn't consistently bluescreen anymore
You brought your computer to work? That PC probably doesn't belong to you, but to your company. If the antivirus is causing a problem, open a problem ticket with your company's IT department, or complain to your manager if the IT department is unresponsive.
Antivirus sucks - get used to it. I recently rebuilt a machine for a client who was hit with the 'E' variant of klez and it wrecked every data file and most system files on his local drive. If properly deployed on decent hardware, AV software (I recommend NAV) is tolerable - and NECESSARY!
Corporations have no choice but to deploy AV software, and that is their decision, not yours. If your uptime is affected, get used to opening problem tickets and putting your feet up until it gets fixed - or find a job working for a company whose IT department has a damn clue.
/set bofh_mode on
<flamebait>
Personally I use dragon and nothing else. I've baked-off all the major packages, and nothing even comes close. I know this is going to rub the open source community raw, sorry, I would't trust my network to snort/acid for a minute. Yes Snort is a good NIDS, *WHEN CONFIGURED PROPERLY*, and does a better job than Network Fud Recorder in the hands of a *TRAINED OPERATOR*. Dragon has never failed me. Yes it returns false positives on a daily basis. Do I waste my time on all matches? Not hardly. All IDS do and will continue to do so. Experience, lots of SANS courses, GIAC, and talking to the *MAN* (you know who if you do network ID) of ID, taught me what should raise my eyebrow and what shouldn't. It will do the same for anybody. I asked the *MAN* of NIDS once, "what IDS would you use if he went into battle tomorrow?" He told me Dragon. Using it ever since....
I use Dragon because of the extensive library of traces that Interasys has managed to compile. Snort, ISS, NFR, Cisco, all lag behind with their trace libraries. Dragon is also the *ONLY* NIDS out there that can operate even semi-effectivly in a GigE network. IF you have alot of bandwith in your operation it's the only thing that I'd recommend. It's easy to configure, easy to use, returns a lower rate of false positives (in the hands of a trained operator), and the support is great.
When I first started to IDS, I learned the *trade* using Shadow. Shadow taught me the value of using and writting good filters. Shadow also taught me what the *LIMITATIONS* of an IDS are. Shadow is a good tool, but the bottom line is that I don't have the time nor desire to sit down and write filters for *everything* on an weekly and sometimes daily basis. I don't want to be married to my IDS.
The DuCk
</flamebait>
Seriously, I've yet to see one that doesn't generate false alarms.
:p
:P
False alarms waste time, and have other effects, such as admins paying no attention to the IDS anymore because, "Damnit, I don't *give* a shit if Bob in HR has problems remembering his password. Do NOT BOTHER ME! *pager off*"
Not to mention, it's software. It can be fooled.
You know how to secure a network? Grab an admin, lock him in a room, and tell him to secure the network and keep it secure, or you'll make him administer the Win 2k boxes.
Wait a few decades, maybe we'll have hostile ice and fried skript kiddiez. But even then, a human brain checking things out will still be superior
We use Manhunt IDS from Recourse Technologies. It works very well, 1 box (a Solaris x86 system) is monitoring 4 network segments with a total average throughput of 25 Megabits per sec. It peaks at about 40 on occasion. It has not crashed once, and after about 3 days of killing false positives, it works extremely well. I use the Java interface on Linux, and while is sluggish to start, once its running, it is extremely usable.
IDS' have serious issues... but Recourse is definately better than the other options we tested.
Spoken like somebody who never has to get any
work done. Let's see, risk getting fired because I'm better at protecting my machine than the IT department & they are whining about me giving them the metaphorical finger, or get fired because I'm not getting any work done & all my boss hears is how the "IT department is full of incompetents".
I believe I'll take the risk over the sure thing. I even get to keep some of my self-respect.
The problem isn't that you are fixing your machine. The problem is that IT should be preventing you from doing so, and should be held accountable if they are unable to do it themselves.
Your company *ought* to have a computer use policy that prohibits you from installing software, making changes that exceed your privileges, or trying to escalate privileges yourself beyond those provided. You should always be able to request privilege escalation when your job functions require it, but your system should be locked down to prevent malicious user activity.
Most compromises of computer and network security come from within a company and your company is apparently not addressing that fact. Not to mention they can't fix a problem in the first place. Sorry to hear that.
My post was deliberately satirical of Mr.-IT-is-always-right, and was written as if it might have come from one of his users. I was being entirely facetious.
Nathan
No, the problem _IS_ fixing the machine. I need the machine working to get my job done. The IT department is NOT fixing the machine, therefore to get my job done, I have to fix it myself. (The IT department wasn't exactly incompetent, just understaffed and overloaded. They were perfectly happy that somebody was able to fix their own machine.)
Idiot blanket policies not allowing anybody to alter anything on any machine just prevent _me_ (supposedly an expensive company resource) from getting my job done.
BTW, if you think that kind of policy is useful at stopping _malicious_ user activity, you're completely in dream land. Users have _PHYSICAL ACCESS_ to their machines. There's nothing that the IT dept can do to stop them from installing or using anything they want on their machines. A competent malicious user will do anything they want on that machine.
All the IT dept can do is try and limit the fallout from _accidental_ user mistakes, set up a good secure network architecture & provide some competent monitoring to try and discover if anything out of the ordinary is occurring.
One thing is for sure, if the IT department thinks that establishing complete control over everyone's machine is more important than actually doing the work that keeps the company alive, then everyone in the IT department needs to be fired. They've got to find the balanced solutions that provide decent security while minimizing the inconvenience to the people actually doing the work.
I'm not with that company anymore (left amicably), but when I was, it was doing just fine, thank you for the concern. Most of the reason is because most of the managers were more interested in helping us get our job done rather than thinking they had to maintain absolute control over all our actions.
The _best_ way to reduce the probability of malicious insider activity (or to increase the probability of discovering such malicious activity) is to make sure that everyone knows what everyone else is doing (transparency). (Not everyone in the company, obviously, but a wide enough circle of peers to provide decent self-monitoring.) This also has the added benefit of improved communication between team members & less unproductive screwing around (to avoid looking bad in front of your peers).
if you think that kind of policy is useful at stopping _malicious_ user activity, you're completely in dream land. Users have _PHYSICAL ACCESS_ to their machines. There's nothing that the IT dept can do to stop them from installing or using anything they want on their machines. A competent malicious user will do anything they want on that machine.
The purpose of a policy is not to prevent, but to prohibit. If a user violates the policy, there is something in writing that says they can be terminated.
Like it or not, most compromises come from inside, most likely for the very reason you mention - users have physical access to their machines. However, you are incorrect that nothing can be done about that. Bios passwords can be set, cases can be locked, floppy drives can be disabled, and machines can be physically located in full view of supervisors and other employees to ensure that no one tampers with them. These measures are not meant to assume that everyone is going to try to go around them, but to offer deterence when someone thinks about trying.
Access control can and should be used to prevent users from having privileges to install applications on their own.
All the IT dept can do is try and limit the fallout from _accidental_ user mistakes, set up a good secure network architecture & provide some competent monitoring to try and discover if anything out of the ordinary is occurring.
This comment is enough reason to prevent you from working in computer security at any company. Management can't rely upon good intentions of the workers. Monitoring is important, but that does not replace the importance of controlling access.
How do you expect HIDS to prevent virii from infecting local machines when any user can bring any program into the building and install it from floppy or CD?