Snort up For Revamp, says Creator

A reader writes:"The creator of Snort, the open-source network-based Intrusion Detection System (IDS), says the software is up for an overhaul. Martin Roesch has told the AusCERT conference IDS has failed to impress the market, citing the inability of many to minimise the number of false alarms triggered by the monitoring devices. The next iteration will include "passive discovery" features."

Comments

Worked ok for me

[HogynCymraeg]

I used snort on an IPCOP box. Worked ok for me.

Re:Worked ok for me

[justsomebody]

It's not the problem with working or not.

Snort just isn't ready for serious use in enterprise. Sounds silly, but... Too many false positives may very well lock network.

Here is a simpler case. On Port detectors.
Why don't port detectors use iptables lock out of port scanners (or even better why it is not suggested).

Most of port scan detectors count actual SYN,FIN and other blocked TCP blocks which are marked as invalid on your firewall. Not even one of them doesn't take to account that some features are completely valid even though they are not. (here is the case: You're running ftp server, that means port 21 is open, does it really matters if SYN,FIN occurs on that port???) Secondary they don't take to account some stupidity. (You don't really care if one machine has scanned your port XXX (yes, your pr0n port) for 50 times), result was always the same. This should count as 1 and not as 50. With this relatively simple logic port scan detectors would exclude false positives in a very simple way. And you could be sure thatsomeone that scanned 50 ports (different and not public open) is really port scanner.

Now go to second level. Porscan detector is just one of many functions that snort provides. And even here there's many false positives.

Actualy I use Snort very differently. Redirected logs to my external processor and this processor excludes invalid information, after that reaction on firewall or server follows. That way was the only possible way to get fairly restrictive measures with not too many false positives.

Not funny

[SirChris]

not meant as a joke but i think it could also have to do with the psychological fact that Snort just sounds like something you want to distance yourself from. I wouldn't associate myself with a program called mucus-mouth no matter how good it was.

Re:Not funny

[dallask]

maybe they can rename it "Blow" :)

Re:Not funny

[flynt]

Martin Roesch has told the AusCERT conference IDS has failed to impress the market,

He's not saying Snort has failed to impress the market, but rather that in general, all IDS systems have, which is true.

Re:Not funny

[Kaa]

A sense of humor is a wonderful thing, in case you were wondering...

And I associate the word "snort" with two things: either a sarcastic disbelieving chuckle, or an angry rhino. Umm... maybe you have something there about wanting to distance myself from... :-)

Cool, but effective?

[Allen Zadr]

From the article:

"The idea is to take a policy like 'thou shalt not run OS X on the network,' and then if someone with a Mac plugs into our network... it can tell the firewall to [block them],"...

While this would be cool, the nature of TCP/IP says that it will be quickly defeated. There are already programs out there that will make your Linux box masquerade as another type of computer.

If a policy says, thou shalt not run P2P - then the P2P will be reached through proxy. If you use snort regular expression detection (one of the coolest features) then new protocols will be written to look like an innocuous service (P2P though ICMP/Ping).

The worst part, and my buddy Zero Hex could talk about this forever, is when ISPs start using this to enforce their will on users. Thou shalt not connect without Windows.

Basically, it's not likely to enforce policies among those who actively want to get around them. Instead, it will enforce policies that push an agenda.

Re:Cool, but effective?

[Kenja]

"If a policy says, thou shalt not run P2P - then the P2P will be reached through proxy. If you use snort regular expression detection (one of the coolest features) then new protocols will be written to look like an innocuous service (P2P though ICMP/Ping)."

A GOOD firewall will be doing more then just blocking ports. It will analyze packets to determine the type of comunication being used. Which is not to say such things can't be circumvented, but it is much harder then just using a proxy.

The problem, and what this article is in many ways about, is dealing with false positives when checking for spacific types of network traffic.

Re:Cool, but effective?

[wfberg]

A GOOD firewall will be doing more then just blocking ports. It will analyze packets to determine the type of comunication being used. Which is not to say such things can't be circumvented, but it is much harder then just using a proxy.

Not quite. Case in point; try blocking instant messengers on your network. Turns out that if you block specific ports, you'll find that they start using port 80.

Ok, block any IM content on port 80, and they move to port 443, that's HTTPS, encrypted.

Ok, so you block some IM server hostnames (there are many) on your DNS server and block access to outside DNS and proxies. Then you find out that there are apps such as htthost/httport that will happily run on a box outside your network accepting encrypted traffic on the HTTPS port and with HTTPS headers, but that are actually proxies (similar things can be achieved on a linux box with a simple enough shellscript). This works easily enough to be downloaded by your smarter-than-average bear.

P2P programs could easily go the HTTPS route if blocking becomes enough of a nuisance. They went route 80 (HTTP port) a long while ago.

So what are your alternatives? Perhaps degrade network performance by interrupting (apparent) HTTPS sessions once in a while so that people won't be able to use certain applications? Or disallow any kind of encrypted communications?

Creative people will always find a way around it. You're better off dealing with those sorts of threats from the inside by dealing with the people rather than the technology. That's probably also true for outside hackers, script-kiddies and virusauthors, but those you typically don't know.

Re:Cool, but effective?

[Allen Zadr]

Sure, but gnutella (for instance) is already implimented using a structure that very closely follows the http protocol. So gnutella, across port 80, is very difficult to detect.

So, even if I get 'smart' detection, how will this better protect me from getting false positives for P2P by users whom are hitting IP dotted addresses to find legitimate web sites.

Computers can only get so smart, before they become smarter than you are...

An example I would call on is Word. When I want to misspell a word on purpose like, recieved. Word knows better than I do, and will change it back, automatically. This is not so bad, until you start dealing with multi-page columns in a document. I know what text I want to show up in each column. I type it where it goes, Wysiwyg style, but Word knows better. It will change things around, and put my text where it wants.

This is similar to false positives. Eventually, the program is written to think it knows better than the person running it.

My point is - when will a SNORT type product decide that my Windows machine cannot work on an ISP of Windows machines because I pipe it's traffic through a virtual interface that coincidentally looks like I'm running OSX or Amiga?

Re:Cool, but effective?

[homer_ca]

You make a good point about people vs. technology. In security, policy is as important as firewalls. If IM's are prohibited by company policy and blocked so that advanced measures like httport are required to circumvent your block, you have good cause to reprimand someone found using IM.

Re:Cool, but effective?

[Allen Zadr]

If you know of something that can block MSN Messenger effectively, let me know. It installs as part of windows, and without user intervention, tries very hard to bypass detection and get through to it's home servers.

I can have a policy - don't install this - don't use this, but most people do anyway just to make that damned message go away. "Wouldn't you like all the benefits of adding a .NET password to XP?". Sure, I can remove it, but the service packs put it back again. I turned it off through the registry, and a security update restored it. MSN Messenger is pervasive, and annoying. No user intervention necessary.

Back to "smart detection" -- After the first blocked attempt, it talks using standard http then as https (also over the correct ports). I don't want to block any web page that 'could' actually be a web page though.

Re:Cool, but effective?

[rograndom]

Ok, block any IM content on port 80, and they move to port 443, that's HTTPS, encrypted.

Hold on now, just because something is using port 443, which also happens to be the standard HTTPS doesn't mean that it's automatically encrypted. Both sides of the connection have to be using an agreed upon encryption method. If they IM program was going to jump to port 80 just to run encryption, it could've done it just as easily on port 80. It's probably using 443 because that's the next "most-common" available port since port 80 was blocked.

Re:Cool, but effective?

[Aliencow]

Geez just set a GPO to disallow running Msn Messenger and it'll stop bugging them!

been done

[Triumph The Insult C]

http://cvs.openbsd.org/faq/pf/filter.html#osfp

Yep, Snort is great, but...

[justsomebody]

Too many features might really mean to many false alerts (logs or mysql tables can get pretty crowded). But in any case it's usable to detect default signatures of attacks pretty well.

Should be used? Yes, except some functions should be disabled
Should be remodeled? Yes

It has the same flaw as port scan attack detectors.

the problem with IDSes

[Keruo]

the problem with IDS systems is encrypted traffic
if someone wants to attack your network, they can easily implement proxy which will encrypt all the traffic they transfer and thus disabling the IDS's ability to analyze the traffic

Re:the problem with IDSes

[tkrabec]

umm no, not completely, in order to attack your network the data has to become unencrypted at some point in time. When it does it becomes traceable and you'd then be able to analyze it. It may be a bit harder and require an IDS box on the inside of the network, too.

-- Tim

Open Source IDS Correlation

[dcgrigsby]

Some of what Martin says regarding minimizing false positives by correlating an attack with the correct platform, etc. is already being done by the open source IDS correlation project QuidScore:

http://quidscor.sourceforge.net/

Re:Open Source IDS Correlation

[_dl_]

Well, you can still download the source and adapt the concept to use another vulnerability assesment tool, not just Qualys'.

Or in general, this shows that there are ways to enhance both tools efficiency by combining them.

ps: I was the lead on Quidscor, so yes, I'm biased :-)

a way to improve....

[millahtime]

a way to improve this might be to take principles from the advanced diagnostics industry. There were a lot of false alarm rates with diagnostics for many years but now it is pretty well nailed down. They use some advanced methods to do diagnostics now. From the way monitors are used to the types of algorthyms.

I know I can't spell. That's why I am an engineer.

Will they disable some attack triggers?

[k.panik]

"...If the new software detects an Apache server running on Linux, it will only look for attacks relevant to that configuration, instead of monitoring the device for an attack that would affect a Cisco router or Windows server..."

This have 2 serious drawbacks:

1. If someone is trying to brute-force attack your servers sending probes for every known exploit (aka. nessus), disabling alarms for software/services you don't run will not show the real size of the attack.

2. In case of an infection similar to code red you won't be able to know wich infected servers are "attacking" you, so there is no way to block them in the router, firewall or reporting the virus-generated traffic to their ISP.

Akamai Mirror

[karmatic]

It's fast, it's friendly, and it's fun!

Mirror Here.

Snort Internals

[cerberusss]

I analysed the Snort source during my study. It has been some time ago, so I don't know how much of the core code has changed. If anyone's interested, look here and go to chapter 8. Snort ROCKS!

Intrusion Prevention System is the key

[Dark Coder]

Seems like most everyone needs to get off the IDS fence and go over and sit on the IPS fence.

For the uninitiated, IPS stands for Intrusion Prevention System. What's the main difference?

#1) IDS doesn't block bad traffic. IPS does. #2) IPS handles anomaly variants, IDS doesn't.

IPS is a new technological way of filtering traffic over the simple brain-dead IDS method.

You need to visit many of Tippingpoint's white papers to get the grift. (registration req. Just fake your email... I know, this is not an official endorsement, but I used to write IPS filters for them and my working real world experience shows that this IPS filter is more effective than any of Snort's filter.)

I would love to write more IPS variant-resistant filters for SNORT but I'm afraid to tread on TPTI's handiwork (much less if I step on the same filter). Nonetheless, the defense industry picked me up. Go figure.

IDS is truly dead. Stop beating a dead horse. Get over it, bud. IPS is your savior.

Re:Intrusion Prevention System is the key

[bamm]

A cut -n- paste job from a previous post. Sorry, but I am too lazy to rewrite the thing.

My personal opinion is IPS's have been mislabeled since the beginning (aren't marketers wonderful). Take this definition I found in some Usenet archives (circa 1992):

"a combination of a security policy with some of the components
above. Specifically, an implementation of the given policy that
is enforced by a combination of screening and/or routing." [1]

Geeze, seems like IPS would fit right in there. Now the final jeopardy question, what was that a definition of? If you guessed "firewall" then you get the big prize. So that's it, you heard it here folks, an IPS isn't the evolution of a IDS, but instead part of the evolution of a firewall. If you look at the history of firewalls, you'll see that early on there were huge flame wars over Packet Filtering and Application Firewalls. In the end, the packet filtering FW won out. Seems to me packet filtering FWs used less resources and could handle faster networks and as those speeds approached full duplex 100mb links, application FWs got left in the dust.

Fast forward to 2003 and the designers of IDS software have made huge progress in detecting potential attacks, system's CPU/RAM/etc have increased phenomally, and the 'normal' speed of network have sorta leveled out. So, application FWs are back in the picture. Vendors with short term memory loss label this 'new' product an Intrusion Prevention System and advertise it as the replacement for your IDS. Those vendors give it a new label for good reason. There is no way they want to bang heads with the big FW companies and more importantly, their implementations of IDS have been huge failures within their customers networks and they need something to market as 'new and improved' (again).

I say (most) vendors of IDS and 'IPS' products failed because they sold the product as an INTRUSION Detection System when they really had an ATTACK detection system. An INTRUSION Detection System implies the IDS can detect an event and determine its nature (malicious vs non-malicious). If the attack was malicious, an IDS will help you determine if it successful. If the attack was successful, the analyst should be able to use the data collected by the IDS to determine the impact on the system in question and finally what steps are needed for remediation. The 'IDS' vendors instead force fed us near worthless systems that can display an 'event'. Many won't give us any details on how they determined it was an 'event' and most can't give us any supporting data about 'attack' beyond a src/dest IP addr and port. If we are lucky, we get a whole packet too. No analysis can be done with the console, instead one must go to the targeted machine and pull out his/her host forensics kit or pay a 'Security Consulting' firm $600/hour to recommend you wipe and rebuild the system.

Soon customers begin to ask "what do I do with this event" and later "I spent XXX hours tracking this down only to find the attack didn't happen or wasn't successful". The vendor noticing the agnst in his customer's voice replies with "we are working on ways to reduce 'false-positives' and in the future we will use IPS technology to prevent attacks too." and thus the birth of "IDS is Dead". I expect FW vendors to incorporate more and more attack detection features from IDSes (duh) and have true hybrid Packet Filtering/Application FWs, but the fact is we will still need IDS. IDS done the right way of course (we call it Network Security Monitoring), but that is a whole other rant.

Bammkkkk

I haven't found this to be true

[VAXGeek]

I use Snort all the time. The only thing I really get false positives for in great numbers are portscans, but even that is easily tweakable in the config file. I'd rather see one too many security alerts than one too few.

Re:a new DOS tool in the making

[karmatic]

Of course, that only works if the attacker has the firewall password/snmp info anyway. And if the attacker has the password in the first place, he won't need snort anyway.

"But what if he _installs_ a firewal?" If he has access to the cabling, all he has to do is cut it. Perfect internal DoS.

Besides, if you really want to do that, go grab a copy of FIRE (no, I won't find the URL for you). Give ettercap a go. A couple of ARP packets, and you can take any windows machine off the local network (provided they aren't more than one router away).

It is already in the works - I've seen it

[Afroplex]

Roesch works within Sourcefire (which puts a lot of development into Snort) as their lead engineer. I've talked with him over a teleconference call and I got the feeling that he loves working with the technology and tries to avoid the sales side of business. Also discussed during the conference call was exactly what this article pertains to.

If Sourcefire's engineering puts out something like this and not their sales reps, then this is really close to being reality. Take a look at Sourcefire's website, you'll see something called RNA. RNA can do passive monitoring of a network and find what machines do what, and what they are running. I've worked with RNA on a production network - it does as advertised very well and even determines patch levels of some machines just by sniffing network traffic. It doesn't take a rocket scientist to put 2-and-2 together that Snort and RNA are on a collision course to work together considering they are from the same company. I would expect something before the end of the year.

RNA though isn't open source, so I'm curious to this announcement if the underlying engine to that product will eventually be opened up.

Hank: the response to snort

[phoxix]

The OSS app known as Hank was pretty much written as a reponse to the short-comings of Snort.

It supports XML based network rules, and has really advanced things like an ACBM implementation

Sunny Dubey

False positives are not the primary IDS problem

[Helevius]

The problem with IDS is not false positives. The problem is knowing what to do with an alert once it appears. If you don't have enough information to make sense of the alert, why bother triggering it in the first place?

Most IDS vendors focus on ever more accurate alerts, but once they trigger they wash their hands of the problem. The end user must decide if the alert is truly significant to their situation and priorities. It's like having an anti-virus product cry wolf but never give any reasons for its identification of malware or background on its findings.

An alternative to the "alert-centric" point of view is "Network Security Monitoring," which concentrates on giving analysts information to conduct at least rudimentary network-based investigation. Where most IDS care only about alerts, NSM-centric operations combine alert, session, full-content, and statistical data to give analysts a chance to identify and escalate incidents.

A tool which uses Snort to generate alert data, combined with session and full content data from other sources, is Sguil.

The April 2004 Sys Admin magazine features Sguil and a few other NSM tools.

A book due in July, The Tao of Network Security Monitoring (also at Amazon.com) is all about NSM.

Anything vendors can do, like Sourcefire's work with Snort, helps with more accurate identification. Just remember creating alerts is only the first step.

All of the IPS fans out there should remember that their "prevention" depends on correctly identifying intrusions. All IDS and IPS products can be bypassed, which drives the need for audit-centric tools (especially using session data) which are content neutral and don't care about triggers, encryption, and so on.

Helevius

Re:Man bites dog.

[dilettante]

I work on an Air Force-funded project for disseminating tactical intelligence info. The IDS that this system uses: Snort.

Then again, maybe the government doesn't have enough money for the better-quality commercial IDS.

Distributors

[Gothmolly]

Could this revamp be due to pressure from companies which have built commercial offerings on Snort? Guardent's SDA tool is basically a Snort box, x86 linux on commodity hardware. How many other money-making ventures out there depend on Snort, and what influence do they have over the Marty?

Log Monitoring/Notification tools have same issue

[xmas2003]

The false positive problem also affects log analysis programs ... although argueably, it's more a matter of how you use and tune it. I.e. most people expect to install Swatch (or other log scraper) and have it just "work outa the box" ... but in reality, you have to understand what is "normal" in your network and then tune out the false positives accordingly. Same things with network IDS tools such as Snort (MartyR is one smart dude BTW) ... although I "cheated" in that comparision because log scrapers are argueably host-based IDS's! ;-)

If you are interested, read more about how Swatch and syslog are used in a large production environment.

Lack of functional cohesion to industry-wide IDS

[Dark Coder]

With the IDS technology lagging today, several IPS vendors stepped in with several technological enhancement toward IDS.

But the key issue confronting the IDS industry today is the lack of functional cohesion (or double-speak for functional capabilities working together).

Some of the basic building blocks of network-based inline IPS feature set that is needed to work together perfectly are:

1. Host-OS-based anomaly decision. Both passive and active scan are recommended to be default on.

2. Deep high-speed REGEX support. Some REGEX chip market didn't materialized as robustly as they should (SafeNet/Raqia)

3. Large-scale TCP connection tracking. This has to work at high-speed as well. Goes to protect against DoS, unwarranted connections and terminations of a pattern-hits' connection.

4. Anchored, unanchored and floating pattern match hardware-assist are needed to work together to cover the variety of algorithms set forth today. This would be a current "1000-watt" hardware issue.

5. Basic issue of quick sub-millisecond table update of content-search memory remains undauntedly elusive. Most H/W content-search engine requires intensive compilation of fancy tr[e|i]e algorithms floating around.

How about weaning yourself of SNORT and start coalescing these incoherent IDS functional cohesions into an IPS?

general IDS probs

[graf0z]

Network IDS have to fight several problems:

• false positives: if this problem does not get solved, IDS won't work on larger sites. One solution could - maybe - be the interaction of an network IDS with a vulnerability scanner (eg packetalarm which combines snort with nessus or the above mentioned quidscor) Roesch indicated. Smart attacks try to hide the actual exploit inside the intentional white noise of false positives.
• false positives.
• false positives.
• encrypted traffic: if an ipsec roadwarrior attacks a service or an attacker targets the https-port of a webserver (not using detectable openssl-overflows :-), the IDS sensor has to sit between encryption endpoint and the target. That means that You may have to terminate https with stunnel at your IDS-sensor one hop in front of your apache instead of using mod_ssl ... And how do You detect malicious content in gpg-encrypted mails?
• "protocol tunneling" or "firewall piercing": how do You detect an trojan inside a corrupted internal workstation calling home through harmlessly looking traffic like http (or even https)? It could craft the http-requests to be indistinguishable from real user traffic (using same proxy and proxy-credentials as the user if nesseccary).
• everyday a new way of obfuscating exploits. Modern IDS know tcp-segment-reassembly, ip-fragment-reassembly, hundred ways of quoting and encoding (like unicode), but it's hard to catch up hackers creativity.
• polymorphic attacks: we've already seen polymorphic shellcode. There will be polymorphic attacks as well. Your IDS engine will have to be much smarter than matching regex to detect those!
• hand crafted variations of known exploits: there are open source exploits which can be modified such that the according snort rules does not strike any more.
• there are myriads of exploitable cgi/php/.. scripts: once coded by a poor student or - worse - a webdesigner, never updated, not publicly known. The chance is not too bad for an experienced hacker to construct a never-seen-before exploit against some ancient webshop. If he avoids the well known evidences (like typing "id" into an unencrypted bindshell) the IDS won't scream.

I fear that when attackers learn to make heavy use of triggering massive false positives, crypto & steanography, protocol-tunneling and start to build exploit-engines producing polymorphic code the days of pattern matching IDS are count. Maybe anomaly-detection (using statistics or neural networks) will help.

Just my 2ct. /graf0z

Snort on ACID

[AgentPhunk]

During a recent interview I was asked about my experiences with IDS systems. I joked with the guy interviewing me (also the hiring manager) that I was using Snort on ACID*, and how that wasn't a phrase you should use in the elevator or when around non-geek types. He agreed and laughed.

Sad part is, I just got the call 5 minutes ago saying that I didn't get the job. :-(

* (Analysis Console for Intrusion Databases)

Just need the right config

[Omega1045]

I haven't used Snort for a while. We used it at a little ISP I worked at to protect various parts of our network. If I recall we were running Snort on some small linux build like trinix. We had old desktop machines running snort with rulesets that were mean for each segment i.e. the Snort box sitting on front of our NT machine was watching for NT attacks, etc. I never recall having performance issues; Snort was very fast even on old machines.

We ended up putting together a little access db that we could generate rules for snort based on critieria like port, os, etc. Eventually we turned this into the first Snort rules site snort.rapidnet.com which is now down. I would imagine that any problems someone might have with Snort (or other IDS) is the correct config for a given scenario or situation.

You have to give props to guys like Marty who make a really great, free product that the little guys can use to conduct homegrown (not homeland) security.

The real problem is not false alerts...

[greyfeld]

The real problem with Snort, and this is coming from someone that has administrated Snort systems in two major companies, is management's lack of understanding that it takes labor to maintain these systems. They want something that they can just pay for up front and will work with no additional tuning or labor costs.

This is the true failing of Snort and other IDS systems as well. They require labor to tune the ruleset and configuration to a network. They require constant updates and someone that can create signatures on the fly. They require someone that has a knowledge of TCP/IP protocols, routing, networking and the ability to analyze data and follow leads.

Working with Snort is kind of like being a detective. The alerts are clues and you have to dig through a lot of other logs, traceroutes, whois, calling people on the phone and find out what they are doing, etc. It's all labor intensive and no one in management wants to dedicate the resources necessary to make it really work.

I could spend all day working on Snort, but I have to monitor firewalls, email, viruses, go to meetings, train people and type on slashdot once in a while. And IPS is no different, it is not something you can just put in and leave forever and feel safe.

Management needs to realize they need people on site to deal with the New World Order of constant hacking attempts. IDS admins are jobs needing to be filled, that's why Snort is not living up to the "promise". Management somehow twisted the promise of Intrusion Detection into some automaticlly, always upgraded intrusion prevention system that requires no labor, no upkeep and you never have to spend any more on it. They continue to live in a fantasy world and one day will end up hacked even though they got a raise for cutting their security budgets by 25% for the year.

weird?

[Anonymous Coward]

whinge, whinge,

* too many false positives, then tune your sensors - but then again _YOU_ will have to know and understand your network and its traffic.

* requires too much time/labor/knowledge to use/setup/maintain - since when did the security industry stay static, or more likely since when did the otherside put their feet up and say, "enough is enough", we have created enough virii/worms/ddos apps/exploits...

LOL, 'greyfeld' had it right in the last paragraph. Spend some money on Security you tight-fisted sods and stop bleating about how it hurts your budget so much - or your job will be on the line not mine; when the site is hacked and the customer databases are ripped, sales figures erased, and the backups failed (as they always do).

I use (daily) a few of the top commercial IDS/IPS apps depending on the customer, and snort is still a favorite for what it is.

So to get a feel on how ppl are comparing this to other IDS/IPS apps, a few questions;
Q1. So how much money was invested in the latest McAfee IPS, and how much was that compared to the latest version of Snort?
Q2. How many IDS/IPS style companies did snort.org take over in the last few months compared to Cisco.com?

Moh.

Setting the record straight

[martyroesch]

The article missed a few key points so I'll try to set the record straight here.

First off, my presentation was about making the case for Passive Network Discovery Systems (PNDS), a "new" technology that I created over at Sourcefire. The basic idea of a PNDS is to discover the composition and topology of your network via a mix of passive OS fingerprinting and passive application layer protocol discovery and the other information that you can infer from that data, such as network topology and asset vulnerabilities. I sought to show how that technology could improve a variety of network security technologies by using the example of how Snort (and other IDS) works today and how it could be improved by integrating the information that comes from a PNDS.

Sourcefire has developed a product called RNA that performs the PNDS functions that I outlined during my talk. Note that it is a proprietary technology that we developed commercially and it is a completely separate product from Snort or the Sourcefire IDS sensors. We are not going to be integrating the functionality of RNA into Snort, we're going to be modifying Snort to take advantage of the information that a system like RNA can generate. In the best case scenario, RNA has a very different deployment profile than an IDS.

I said that IDS has had trouble in the market because of its complexity and the requirement that users perform extensive tuning of IDSes in general in order to get maximum benefit from them. There are a lot of things that factor into this problem, but the root cause of almost all IDS problems today is that we don't have automated methods for provisioning them nor do we have effective methods of data reduction available that are automated, persistent and real-time. PNDS addresses that problem head on in a way that is appropriate for real-time processes like IDS in ways that traditional scanning technologies have a very tough time providing.

I then went on to say that we're planning on making changes to Snort to enable it to leverage the information that a system like RNA provides and make it into a true target-based IDS, redefining how IDS operates and hopefully revitalizing it as a technology. Snort will still be available for free and will still operate in "classic" mode where it doesn't leverage this info for people who don't have passive discovery technologies (or even active ones) so that they can still continue to use it.

Snort is not going to be doing the configuration policy enforcement (i.e. the "block OS X on my network" function), RNA is. RNA is capable of seeing devices on the network and discovering their attributes in real-time and communicating that data to our management console where it can be analyzed for policy compliance and where appropriate remediation responses can be executed. Not to get too deep into the marketing, but there are good engineering reasons for wanting to do this that include worm/virus containment, real-time IDS policy updates and some other really useful mechanisms for performing policy enforcement.

We're making mods to Snort because we believe that we can make a truly next-generation IDS capability that is easier to deploy, manage and get valuable information out of due to the effect of RNA. This approach directly addresses all the arguments of the "IDS is dead" crowd while at the same time making IDS a much more impactful technology while greatly reducing the overhead requirements on users.

I hope this clears things up for people!

One more thing

[martyroesch]

IDS != IPS, IPS !>= IDS.

Once again, with feeling:
IDS is a network monitoring technology

IPS is anaccess control technology

We use IDS to let us know what's happening on our networks, how our policy is being enforced by our access control mechanisms and when there are security failures.

We use IPS to "shoot down" attacks that are in flight before they can complete and affect the target.

Confusing the two is the name of the game for IPS vendors because the FW vendors have deep pockets and the IPS guys didn't want to rock the boat at first. In-line network IPS is only useful as long as you have time to provision new detection signatures before attacks/worms come out, they are deterministic and therefore have a very tough time dealing with the unknown (and yes, I know they have the ability to do rate-based blocking in some cases, that's deterministic too). The natural progression for IPS technology is as a feature on a firewall, not as a stand alone independent product, it's just an enhancement to access control technology after all. The natural progression of IDS will remain as a stand alone product or perhaps it will disappear into the infrastructure of the network itself (e.g. switches), but it is going to be a necessity as long as people need to have visibility into what's happening outside the purview of their access control technologies. In-line network IPS only watches/defends your peering points, NIDS monitors everything if deployed properly.

To claim that IDS is "dead" is to basically say that people should put on blinders and only watch the peering points, not a very realistic proposition in my opinion. IPS is not a replacement for IDS, those who say so either don't understand the role of IDS or they're selling something.