New Open Source Intrusion Detector Suricata Released

richrumble writes "The OISF has released the beta version of the Suricata IDS/IPS engine: The Suricata Engine is an Open Source Next Generation Intrusion Detection and Prevention Engine. This engine is not intended to just replace or emulate the existing tools in the industry, but will bring new ideas and technologies to the field. This new Engine supports Multi-Threading, Automatic Protocol Detection (IP, TCP, UDP, ICMP, HTTP, TLS, FTP and SMB! ), Gzip Decompression, Fast IP Matching and coming soon hardware acceleration on CUDA and OpenCL GPU cards."

Comments

Innovation

[Reason58]

This engine is not intended to just replace or emulate the existing tools in the industry, but will bring new ideas and technologies to the field.

Sweet! What are some examples of things this does that no other solution provides?

Re:Innovation

[Monkeedude1212]

This engine is unique in the fact that it will determine your external IP assigned by your ISP, and then use that to determine what city you live in. It will then automagically search the local job banks for the best private security professionals, and position one every 10 meters across your network.

It also has some very special features that allow you to choose exactly how you want your security team to look like, if you prefer mustaches or aviator sunglasses, that kind of stuff.

Also, forget Automatic Protocol Detection, they're planning on implementing Brute Force Protocol Detection. Instead of reading the headers or any other piece of data, it merely tries iterating ALL of the data through EVERY kind of protocol. This way, you can secretly encode messages using one protocol (Say SMTP) but send it using a different one (Say HTTP). That way your ACTUAL information can stay completely secure while the hackers are stuck with some Bogus Data they think they broke the encryption for.

Re:Innovation

[Anonymous Coward]

Do you work with me?

I am a network security analyst with both a mustache and aviator glasses...

Re:Innovation

[Anonymous Coward]

It's new and funded by the government. The first beta release probably can't compete with Snort on the first try. If they're claiming to have automatic keyword detection for TLS ("The engine not only has keywords for IP, TCP, UDP and ICMP, but also has HTTP, TLS, FTP and SMB"), then I'm going to call BS. It's encrypted. You might be able to detect a malformed TLS/SSL handshake, but you're not going to magically decrypt IP traffic and detect attacks within it.

Re:Innovation

[newdsfornerds]

hahaha

Re:Innovation

[Anonymous Coward]

...but will bring...

Promising

[zero0ne]

Hardware acceleration with CUDA makes this product worthwhile to watch.

Re:Promising

[FooAtWFU]

That is interesting to me. CUDA can easily provide parallelization of bulk mathematical operations, but it's notoriously weak with conditional logic. Are they doing a whole lot of math on the side -- perhaps with some fancy anomaly-detection algorithms that work by clustering packet attributes in multidimensional spaces, or approximate nonnegative matrix factorization, or such?

Re:Promising

[PatDev]

Maybe I'm missing something, but as someone who has working with the techniques referenced in the parent post - I'm not sure where the funny mod came from. Both clustering packet attributes and nonnegative matrix factorizations could be used for anomaly detection. And as someone who has also worked on CUDA a good bit, I think both of those problems have solutions that fit CUDA's concurrency model.

I get the impression that the mods saw big words and assumed this was a joke about buzzwords, but in fact that's a reasonable approach to this problem.

Re:Promising

[FooAtWFU]

I suppose that ranks as "funny" because the extent of the CUDA parallelism is... probably just to accelerate the aforementioned gzip decompression, or something like that. :(

Re:Promising

[ColdWetDog]

I get the impression that the mods saw big words and assumed this was a joke about buzzwords, but in fact that's a reasonable approach to this problem.

Somebody must be hitting the punch early today. I immediately thought of this when I read the summary:

...ICE patterns formed and reformed on the screen as he probed for gaps, skirted the most obvious traps, and mapped the route he'd take through Sense/Net's ICE. It was good ICE. Wonderful ICE...

...His program had reached the fifth gate. He watched as his icebreaker strobed and shifted in front of him, only faintly aware of his hands playing across the deck, making minor adjustments. Translucent planes of color shuffled like a trick deck. Take a card, he thought, any card.

The gate blurred past. He laughed. The Sense/Net ice had accepted his entry as a routine transfer from the consortium's Los Angeles complex. He was inside. Behind him, viral subprograms peeled off, meshing with the gate's code fabric, ready to deflect the real Los Angeles data when it arrived.

Re:Promising

[Anonymous Coward]

The Sense/Net ice had accepted his entry as a routine transfer from the consortium's Los Angeles complex. He was inside. Behind him, viral subprograms peeled off, meshing with the gate's code fabric, ready to deflect the real Los Angeles data when it arrived.

Ack! This is a terribly designed system! Your firewall should not execute arbitrary untrusted code! Somebody please save these poor computers from themselves.

Re:Promising

[ColdWetDog]

To be fair, I think this is from 1994. You gotta give Gibson some slack on that one.

Re:Promising

[Sulphur]

All your GPU are belong to us.

Re:Promising

[thePowerOfGrayskull]

To be fair, I think this is from 1994. You gotta give Gibson some slack on that one.

If his writing had any redeeming qualities, he'd have earned some slack... but I've just found him to kinda suck. His writing style is torturous reading at best. This holds true in both his newest and oldest work.

I know, I know. I'll be turning in my geek card on the morrow.

huh

[dropadrop]

While there is some information available on the site, it's still pretty sparse. Is this a whole framework? They refer to engine, but do they mean a detection engine or also a correlation engine? This area really needs more open source innovation, commercial solutions are ridiculously expensive for small / mid sized companies, and the only "complete" IDS option I know of for the moment is Ossim (which has extremly lacking documentation).

Re:huh

[richrumble]

From the INSTALL doc:Suricata is compatible with standard Snort rulesets. A sample standard configuration file can be found in the Suricata base directory. This file is called 'suricata.yaml'. I've just now got it installed, going to investigate further...

Re:huh

[Anonymous Coward]

small / mid sized companies and Linux newbies are specifically the target audience for EasyIDS. It's nowhere near as complex as Ossim and is free.

Dangerous

[Anonymous Coward]

The feautres look indeed promising. On the other hand, the more complicated an IDS/IPS gets, the more likely it will become a new attack vector itself.

Hopefully it is implemented well...

On first glance

[joeflies]

I thought that the "Open Source Intrusion Detector" spotted intrusions of open source software in the company. I'm sure that Microsoft would have loved to have one of those for the Windows 7 USB/DVD download tool.

Re:On first glance

[Icegryphon]

Give them time, Maybe it will be part of SP1b.

That's actually an interesting idea...

[Anonymous Coward]

If someone kept a database of all GPLed software and associated signatures, just as people keep databases of virus signatures now, you could probably use an anti-virus program to search for viral licenses as well as viral code. It's just a matter of matching patterns against a database of signatures in either case.

More info

[methamorph]

Since the original site is slashdoted some more info can be read here

Are these useful yet?

[Anonymous Coward]

I'm not trying to be a troll here or anything, but are IDS/IPS systems actually worth while?

We started using Snort back around 2002 when I worked at a hosting provider and it was one of the biggest waste of resources in the NOC department.

The first issue is that there was no way we were going to inject such a box that could ever modify the packets going through the border routers/switches (no server was fast enough for starters), so that eliminated any "prevention" from happening.

The next issue is that it was constantly an issue of which rules to enable vs. the amount of traffic that needed to be sorted through. The IDS servers had more hardware than most of our database servers and they still couldn't keep up with just a fraction of the rules that we would have liked to have enabled. Traffic was increasing at much faster pace than CPU speed was too at that time.

The final nail in the coffin though was that it was a huge time sink, and resulted in almost zero benefits. It took hours to actually go through the alerts being triggered and investigate them in more detail to determine if they were legit or just a false alarm, but then what... Either a server was compromised or it wasn't, and in many cases its not exactly easy to determine if a server was compromised or not, especially if it was a SQL injection that simply modified a users password or something.

Now you could say without the "prevention" part of the formula the usefulness is severely limited, but I just can't see making something like this take a critical roll in a network, as most of them are dead easy to DDOS unless the vast majority of rules are disabled. It would be easy enough for an attacker to send their attack payload in the midst of a minor DDOS from a cable modem or two and the IDS system would have no way to keep up. Heck, you can DDOS most recursive DNS servers with just a few hundred carefully crafted packets per second.

I know some companies have "wire speed" IDS systems, which the definition of "wire speed" and the number/complexity of the rules involved are surely hidden in the fine print somewhere, but those systems would also break most budgets.

Am I missing something?

Re:Are these useful yet?

[Martin Blank]

You are. Your IDS was incredibly poorly-tuned, a very common problem in IT. First guideline: turn off signatures for anything that you're not running. It makes no sense to watch your inbound traffic for Windows signatures if you run Apache on RHEL. If all you have are web servers and they do only HTTP, there's no reason to watch for SMTP.

Making the move to IPS is always tricky. You have to figure out what level of false positives you're willing to accept. If it's zero, well, you don't need an IPS. But odds are that you will come across some strange but innocuous traffic that the IPS doesn't like, and it trips a rule and blocks the traffic.

In addition, you need to get the hardware for the solution. A server-based Snort solution works well for low-bandwidth scenarios, but at most hosting providers, you need a dedicated appliance solution built on ASICs. If you like Snort, you go to Sourcefire. Otherwise, you find solutions from McAfee (Intrushield), Tipping Point, IBM, etc. They have boxes that scale into the gigabit-per-second range, with latencies under 1ms in most cases, and there are a few true-10Gbps solutions out there now. Yes, they can be quite expensive, but the low-end systems (essentially highly-tuned servers) can start at as low as a few thousand dollars.

But in any case, rule tuning is an ongoing item, and anyone that tells you that an IDS/IPS will reduce your time requirements is probably trying to get you to sign a contract. It can reduce overall time requirements by alerting you early in the attempt to compromise a system and save you all the time of recovery, but that is not a certain thing.

Re:Are these useful yet?

[JimmytheGeek]

Try the sguil console, and you'll be happier with handling alerts. It presents the data from full content pcaps, Snort alerts, and session data, together with a handy window to to reverse DNS and whois. It will give you the signature that fired the alert, or, if no alert fired (say someone emailed abuse@yourdomain.tld with an IP and time range) you can look back in time and see what connections your host had open when. It will even help you decide which alerts are useful and which are useless, but you still have to tune the rules yourself. For handling that, I use oinkmaster. Sguil scales to billions of rows.

Some folks have worked on integrating bro (or was it prelude?), which is another interesting alerting engine. It might be possible to integrate with this project.

http://sguil.sourceforge.net/

Re:Are these useful yet?

[TheLink]

Say it's HTTP only. What can they do for you?

So far have there actually been attacks your IPS systems have _stopped at your sites that would have actually caused problems - wouldn't have been shrugged off by your servers being not vulnerable in the first place (patched, not vulnerable to SQL injection etc).

I'm just wondering how much security and availability they'd really add over just having a firewall (some even have IPS features themselves, but let's ignore that for now).

Can they usually spot AND prevent attacks before they are patchable?

It's like network equivalent of antivirus software - need to update "signatures" very regularly. But with a harder performance constraint.

So I'm wondering if they can actually succeed in keeping up in practice.

Re:Are these useful yet?

[Martin Blank]

They can protect in situations involving unpatched vulnerabilities, actually. In many cases, once a vulnerability is publicized, a signature can go out within a few hours, sometimes even within minutes. Whether you add that in, being new, is something for you to decide based on policy. But IDS/IPS is, as you mention, reactive in part, but anomaly engines are getting better.

I come from an environment where we moved from simple port-based firewalls to Sidewinders, which are application proxy firewalls. They check to ensure that the traffic coming through is valid HTTP. But that doesn't mean that they're all that good at spotting SQL injections, which shouldn't be there, but sometimes these things slip through.

And while you may have enough bandwidth to handle a DoS or DDoS, your server may not be able to handle that much on its own, and an IPS capable of noticing and dropping the traffic before it hits your server may keep you online.

It's part of defense in depth. Write the libraries properly to not have vulnerabilities. Write the web apps to not accept invalid input that could find missed vulnerabilities. Filter your traffic to not let odd input get to your web apps. It's kind of a pain, sometimes, but it does keep things interesting.

Apparently it can't stop a DoS

[istartedi]

If it were really that good, it would sniff the referrer on all the HTTP requests and throttle Slashdot.

Re:Apparently it can't stop a DoS

[Reason58]

If it were really that good, it would sniff the referrer on all the HTTP requests and throttle Slashdot.

I'm sure this was meant in jest, but it doesn't work that way. They could instantly drop every packet in a DDoS attack and it wouldn't matter. By the time it hits their network the bandwidth is already gone.

Re:Apparently it can't stop a DoS

[istartedi]

I did lash out a bit there.

Now that you mention it, recognizing the referrer would most likely be of marginal benefit at best. You have to SYN all those connections to do it in the first place, and in a true DoS attack you probably do have your entire network saturated with SYNs alone, nevermind data.

In other words, you're right. The IPS is a doorman; but it can't prevent a crowd from forming outside the door and preventing the band from getting on stage.

Re:Apparently it can't stop a DoS

[Anonymous Coward]

I believe the most probable cause of sites being /. is server/application issues, not network in the sense of a real DoS attack, so I think SYNing the connection should not be a problem in most cases. Most of these could be avoided by using a simple static page with very few images/stylesheets/javascript include files, etc..

Have you noticed the download link doesn't seem to be a static page ?

http://openinfosecfoundation.org/index.php/download-suricata

I won't go into another point that the link reveals...

So I would say you were originally right, or at least they haven't configured their product for a /. like effect ;-))

Re:Apparently it can't stop a DoS

[Bert64]

No, it would help a lot... Most of the bandwidth from being hit by web requests is actually outbound traffic when the server actually responds and tries sending the site content... Also since you would be blocking the first get request, the client would never receive the html content and therefore not try to retrieve any images, css files, javascript etc.

Also, sometimes a site goes down not for lack of bandwidth but because the page is dynamically generated and too complex for its processor to handle so many simultaneous requests.

You are missing something.

[Anonymous Coward]

These products are intended for banks, government institutions, nuclear power plants, and the power grid.

Q: What do all these customers share in common?

A: Those are customers who don't care about actual security.

They don't implement air-gaps, they don't apply basic Windows security patches, and they use allow-by-default firewall policies. What IDS provides is a CYA system for when security breaches occur. The top management points the finger at the head IT guy, who points the finger at the security guy, who points the finger at the outsourced IDS solution provider, who points the finger at the IT guy, who points the finger at another IT guy, who points a finger at the IT guy who left who requested the impossible-to-deal with number of alert triggers be disabled so traffic didn't slow to a crawl.

Re:You are missing something.

[Bert64]

A lot of companies don't know enough to understand security, some well dressed salesman comes through the door and tells them whatever he's selling will make them more secure so they buy it...

I went to a company where every box was running a commercial implementation of SSH for just this reason, none of their staff actually used it, every box also ran telnet and rlogin and that's what the staff used.
They believed that simply by having it there they had improved security, when in fact they had weakened it by introducing an additional listening service that was not being used.

I've also been to several places where they had an IDS, monitoring or logging setup that was running but either not configured or just completely ignored.

Editors?

[shutdown -p now]

What's with all the caps, exclamation marks, spelling etc in the summary? As it is, it wouldn't even pass any sane spam filter:

is new Engine supports Mult-Threading, Automatic Protocol Detection (IP, TCP, UDP, ICMP, HTTP, TLS, FTP and SMB! ), Gzip Decompression, Fast IP Matching and coming soon hardware acceleration on CUDA and OpenCL GPU cards."

Does it page their sysadmin...

[fak3r]

...because their site is failing to load, looks like the DB server or connection is fek'd: Database Error: Unable to connect to the database:Could not connect to MySQL

Multi-Threading is evil

[Dr.Ruud]

Multi-threading is insecure in itself. Stop sharing, start merging.