Snort up For Revamp, says Creator

A reader writes:"The creator of Snort, the open-source network-based Intrusion Detection System (IDS), says the software is up for an overhaul. Martin Roesch has told the AusCERT conference IDS has failed to impress the market, citing the inability of many to minimise the number of false alarms triggered by the monitoring devices. The next iteration will include "passive discovery" features."

Comments

Cool, but effective?

[Allen Zadr]

From the article:

"The idea is to take a policy like 'thou shalt not run OS X on the network,' and then if someone with a Mac plugs into our network... it can tell the firewall to [block them],"...

While this would be cool, the nature of TCP/IP says that it will be quickly defeated. There are already programs out there that will make your Linux box masquerade as another type of computer.

If a policy says, thou shalt not run P2P - then the P2P will be reached through proxy. If you use snort regular expression detection (one of the coolest features) then new protocols will be written to look like an innocuous service (P2P though ICMP/Ping).

The worst part, and my buddy Zero Hex could talk about this forever, is when ISPs start using this to enforce their will on users. Thou shalt not connect without Windows.

Basically, it's not likely to enforce policies among those who actively want to get around them. Instead, it will enforce policies that push an agenda.

Re:Cool, but effective?

[wfberg]

A GOOD firewall will be doing more then just blocking ports. It will analyze packets to determine the type of comunication being used. Which is not to say such things can't be circumvented, but it is much harder then just using a proxy.

Not quite. Case in point; try blocking instant messengers on your network. Turns out that if you block specific ports, you'll find that they start using port 80.

Ok, block any IM content on port 80, and they move to port 443, that's HTTPS, encrypted.

Ok, so you block some IM server hostnames (there are many) on your DNS server and block access to outside DNS and proxies. Then you find out that there are apps such as htthost/httport that will happily run on a box outside your network accepting encrypted traffic on the HTTPS port and with HTTPS headers, but that are actually proxies (similar things can be achieved on a linux box with a simple enough shellscript). This works easily enough to be downloaded by your smarter-than-average bear.

P2P programs could easily go the HTTPS route if blocking becomes enough of a nuisance. They went route 80 (HTTP port) a long while ago.

So what are your alternatives? Perhaps degrade network performance by interrupting (apparent) HTTPS sessions once in a while so that people won't be able to use certain applications? Or disallow any kind of encrypted communications?

Creative people will always find a way around it. You're better off dealing with those sorts of threats from the inside by dealing with the people rather than the technology. That's probably also true for outside hackers, script-kiddies and virusauthors, but those you typically don't know.

Re:Cool, but effective?

[Allen Zadr]

Sure, but gnutella (for instance) is already implimented using a structure that very closely follows the http protocol. So gnutella, across port 80, is very difficult to detect.

So, even if I get 'smart' detection, how will this better protect me from getting false positives for P2P by users whom are hitting IP dotted addresses to find legitimate web sites.

Computers can only get so smart, before they become smarter than you are...

An example I would call on is Word. When I want to misspell a word on purpose like, recieved. Word knows better than I do, and will change it back, automatically. This is not so bad, until you start dealing with multi-page columns in a document. I know what text I want to show up in each column. I type it where it goes, Wysiwyg style, but Word knows better. It will change things around, and put my text where it wants.

This is similar to false positives. Eventually, the program is written to think it knows better than the person running it.

My point is - when will a SNORT type product decide that my Windows machine cannot work on an ISP of Windows machines because I pipe it's traffic through a virtual interface that coincidentally looks like I'm running OSX or Amiga?

Re:Cool, but effective?

[Allen Zadr]

If you know of something that can block MSN Messenger effectively, let me know. It installs as part of windows, and without user intervention, tries very hard to bypass detection and get through to it's home servers.

I can have a policy - don't install this - don't use this, but most people do anyway just to make that damned message go away. "Wouldn't you like all the benefits of adding a .NET password to XP?". Sure, I can remove it, but the service packs put it back again. I turned it off through the registry, and a security update restored it. MSN Messenger is pervasive, and annoying. No user intervention necessary.

Back to "smart detection" -- After the first blocked attempt, it talks using standard http then as https (also over the correct ports). I don't want to block any web page that 'could' actually be a web page though.

Yep, Snort is great, but...

[justsomebody]

Too many features might really mean to many false alerts (logs or mysql tables can get pretty crowded). But in any case it's usable to detect default signatures of attacks pretty well.

Should be used? Yes, except some functions should be disabled
Should be remodeled? Yes

It has the same flaw as port scan attack detectors.

the problem with IDSes

[Keruo]

the problem with IDS systems is encrypted traffic
if someone wants to attack your network, they can easily implement proxy which will encrypt all the traffic they transfer and thus disabling the IDS's ability to analyze the traffic

a way to improve....

[millahtime]

a way to improve this might be to take principles from the advanced diagnostics industry. There were a lot of false alarm rates with diagnostics for many years but now it is pretty well nailed down. They use some advanced methods to do diagnostics now. From the way monitors are used to the types of algorthyms.

I know I can't spell. That's why I am an engineer.

Snort Internals

[cerberusss]

I analysed the Snort source during my study. It has been some time ago, so I don't know how much of the core code has changed. If anyone's interested, look here and go to chapter 8. Snort ROCKS!

I haven't found this to be true

[VAXGeek]

I use Snort all the time. The only thing I really get false positives for in great numbers are portscans, but even that is easily tweakable in the config file. I'd rather see one too many security alerts than one too few.

Re:Worked ok for me

[justsomebody]

It's not the problem with working or not.

Snort just isn't ready for serious use in enterprise. Sounds silly, but... Too many false positives may very well lock network.

Here is a simpler case. On Port detectors.
Why don't port detectors use iptables lock out of port scanners (or even better why it is not suggested).

Most of port scan detectors count actual SYN,FIN and other blocked TCP blocks which are marked as invalid on your firewall. Not even one of them doesn't take to account that some features are completely valid even though they are not. (here is the case: You're running ftp server, that means port 21 is open, does it really matters if SYN,FIN occurs on that port???) Secondary they don't take to account some stupidity. (You don't really care if one machine has scanned your port XXX (yes, your pr0n port) for 50 times), result was always the same. This should count as 1 and not as 50. With this relatively simple logic port scan detectors would exclude false positives in a very simple way. And you could be sure thatsomeone that scanned 50 ports (different and not public open) is really port scanner.

Now go to second level. Porscan detector is just one of many functions that snort provides. And even here there's many false positives.

Actualy I use Snort very differently. Redirected logs to my external processor and this processor excludes invalid information, after that reaction on firewall or server follows. That way was the only possible way to get fairly restrictive measures with not too many false positives.

It is already in the works - I've seen it

[Afroplex]

Roesch works within Sourcefire (which puts a lot of development into Snort) as their lead engineer. I've talked with him over a teleconference call and I got the feeling that he loves working with the technology and tries to avoid the sales side of business. Also discussed during the conference call was exactly what this article pertains to.

If Sourcefire's engineering puts out something like this and not their sales reps, then this is really close to being reality. Take a look at Sourcefire's website, you'll see something called RNA. RNA can do passive monitoring of a network and find what machines do what, and what they are running. I've worked with RNA on a production network - it does as advertised very well and even determines patch levels of some machines just by sniffing network traffic. It doesn't take a rocket scientist to put 2-and-2 together that Snort and RNA are on a collision course to work together considering they are from the same company. I would expect something before the end of the year.

RNA though isn't open source, so I'm curious to this announcement if the underlying engine to that product will eventually be opened up.

Network Nazis!!!

[Anonymous Coward]

"The idea is to take a policy like 'thou shalt not run OS X on the network,' and then if someone with a Mac plugs into our network... it can tell the firewall to [block them],"

Lack of functional cohesion to industry-wide IDS

[Dark Coder]

With the IDS technology lagging today, several IPS vendors stepped in with several technological enhancement toward IDS.

But the key issue confronting the IDS industry today is the lack of functional cohesion (or double-speak for functional capabilities working together).

Some of the basic building blocks of network-based inline IPS feature set that is needed to work together perfectly are:

1. Host-OS-based anomaly decision. Both passive and active scan are recommended to be default on.

2. Deep high-speed REGEX support. Some REGEX chip market didn't materialized as robustly as they should (SafeNet/Raqia)

3. Large-scale TCP connection tracking. This has to work at high-speed as well. Goes to protect against DoS, unwarranted connections and terminations of a pattern-hits' connection.

4. Anchored, unanchored and floating pattern match hardware-assist are needed to work together to cover the variety of algorithms set forth today. This would be a current "1000-watt" hardware issue.

5. Basic issue of quick sub-millisecond table update of content-search memory remains undauntedly elusive. Most H/W content-search engine requires intensive compilation of fancy tr[e|i]e algorithms floating around.

How about weaning yourself of SNORT and start coalescing these incoherent IDS functional cohesions into an IPS?

general IDS probs

[graf0z]

Network IDS have to fight several problems:

• false positives: if this problem does not get solved, IDS won't work on larger sites. One solution could - maybe - be the interaction of an network IDS with a vulnerability scanner (eg packetalarm which combines snort with nessus or the above mentioned quidscor) Roesch indicated. Smart attacks try to hide the actual exploit inside the intentional white noise of false positives.
• false positives.
• false positives.
• encrypted traffic: if an ipsec roadwarrior attacks a service or an attacker targets the https-port of a webserver (not using detectable openssl-overflows :-), the IDS sensor has to sit between encryption endpoint and the target. That means that You may have to terminate https with stunnel at your IDS-sensor one hop in front of your apache instead of using mod_ssl ... And how do You detect malicious content in gpg-encrypted mails?
• "protocol tunneling" or "firewall piercing": how do You detect an trojan inside a corrupted internal workstation calling home through harmlessly looking traffic like http (or even https)? It could craft the http-requests to be indistinguishable from real user traffic (using same proxy and proxy-credentials as the user if nesseccary).
• everyday a new way of obfuscating exploits. Modern IDS know tcp-segment-reassembly, ip-fragment-reassembly, hundred ways of quoting and encoding (like unicode), but it's hard to catch up hackers creativity.
• polymorphic attacks: we've already seen polymorphic shellcode. There will be polymorphic attacks as well. Your IDS engine will have to be much smarter than matching regex to detect those!
• hand crafted variations of known exploits: there are open source exploits which can be modified such that the according snort rules does not strike any more.
• there are myriads of exploitable cgi/php/.. scripts: once coded by a poor student or - worse - a webdesigner, never updated, not publicly known. The chance is not too bad for an experienced hacker to construct a never-seen-before exploit against some ancient webshop. If he avoids the well known evidences (like typing "id" into an unencrypted bindshell) the IDS won't scream.

I fear that when attackers learn to make heavy use of triggering massive false positives, crypto & steanography, protocol-tunneling and start to build exploit-engines producing polymorphic code the days of pattern matching IDS are count. Maybe anomaly-detection (using statistics or neural networks) will help.

Just my 2ct. /graf0z

Re:One more thing

[Anonymous Coward]

I'm sure you've thought of it this way, oh wise oracle of Snort, but maybe turn your thinking of FW vs IPS around: An IPS is a firewall-plus. In-line IPS's can eliminate a need for firewalls, since it's trivial to set firewall-ish rules there. Is there some reason against this, other than customer inertia? I understand the performance gap between software and ASIC-based firewalls, but performance seems sufficient for most uses at this point. Put another way, I'd say the natural progression is to move from firewalls to IPS, with simple firewalls becoming obsolete.

Further, while I respect you conservatively saying IPS won't help against zero-day stuff, IPS is useful immediately for all former signatures. There are so many compromised machines yammering out old attack patterns, a handful of base rules can silence them forever in ways a simple firewall can't do. People love that, where we've set it up.

As for 'provisioning new signatures', I've been waiting for you to boil rules down to another service-oriented mechanism, providing updates much like AntiVirus, for an annual fee. To be honest, there are several damn-useful technologies that I can't recommend to small businesses mostly because there isn't an inexpensive source for expert maintenance/support. Automation, trust, and expertise in ensuring/responding to zero-false-positive issues would be why customers would pay you for autodeploying these 'free' rules. Oh, and for the call center for supporting false-neg/false-pos questions.

I do agree that IPS vendors aren't yet doing a remarkable job of tackling the firewall-vendor hegemony (pauses, considers setting the 'post as anonymous' flag). Perhaps modularizing some rules then sticking them into a GUI that resembles firewalls is part of the answer. Familiarity is a big selling point when the customer doesn't have time to spare. Or maybe the answer is redeclaring the battlefield itself: stop calling it IPS and embrace the firewall name with "100% firewall functionality plus (insert buzzword like 'in-packet inspection', akin to 'stateful inspection' improvements to firewalls that have become standard). Everyone knows they need a firewall. Make them want a 'firewall with subpacket filtering' or whatever you name it. Otherwise, it'll be like a project I'm doing now, with the packets flowing (redundantly) thru an IPS to a firewall to the protected equipment.

Keep up the good ^h^h^h^h great work.