Slashdot Mirror
Vulnerability Assessment Scanners Comparison
Roberto writes "Network Computing is running
a comparison
between various commercial and vulnerability assessment scanners - and open-source wins, thanks to Nessus,
even though none of the tools could do spot all the vulnerabilities that were present in the test lab."
Ph34r M3 4nd 411 0f my 31337 fr13nd5! W3 kn0w 4b0u7 y0ur ftpd fl4w!
One other valid point missed in the review is the frequency of product updates. What mechanisms exist to check for newly discovered vunerabilities? Nessus can be made to automatically install updated scripts, but I have absolutely no idea of the other products reviewed. An out of date security tool can be worse than no security tool at all, as it installs a false sense of confidence. Would you put all your trust in a 1-year-old security scanner - I wouldn't.
My company now uses open source scanners exclusively. We do, however have our own very competent programming staff capable of reviewing and possibly modifying the code. This gives us some assurance as to what the tool is doing, and the capability of fixing problems quickly if required.
Having a security scanner malfunction due to software error is much more serious than having your word processor freeze. Malfunctioning scanners can crash servers and in general wreck havoc in networks.
If your children ever found out how lame you are, they'd murder you in your sleep
I'm not really sure who you are or why you are posting anonymously, but I've got sitting in front of me here a signed copy of the letter written by Jennifer Carroll, Marketing and Business Development Manager of eEye. It's dated Aug 25th. If you do indeed work for eEye, I'd encourage you to formally address us. Not only was eEye fully aware of our review, eEye gave us a full license key. And yes, we did update the product. I'm continually amazed by the amount of BS posted on these threads. I'll post the serial # if there are any further doubts.
The most interesting part that I find about this entire article is the fact that this magazine (which I subcribe to) is a free subscription. The magazine doesn't make any money off of subscriptions. The magazine effectively makes all of its money from advertisements. The fact that they would review a opensource competitor is surprising in itself. The fact that they gave it the nod, is going to do nothing but hurt their advertising deals with the commercial products that they reviewed.
Of course, that's only one way to look at it. The other way to look at is that they just effectively said that if you want to get all your vulnerabilities detected, you need to buy at least one thing. Combine that one thing with the open source product, and you've got a complete solution.
Is the glass half empty, or half full? Hmmm...
Key to financial independence: Spend less than you earn. Save and invest the difference. Do it for a long time.
It would take a well trained, intelligent human being to discover security flaws. If your need for security is more than the average home-based internet surfer running ZoneAlarm, then you should hire a 3rd-party company specializing in security to evaluate your system.
I would use scanners only to perform automated checks to make sure that known holes have not been opened after the initial check. Periodically, the 3rd-party company should be hired to come back and recheck the system for old holes as well as new ones that have been discovered since the previous system test.
shouldn't that be "between various commercial and open source vulnerability assessment scanners"?
`ø,,ø!
Free Software: Like love, it grows best when given away.
It may be a bit unfair to take the paragraph I cited out of context because the article goes on to do a good job of weighing the individual pros and cons of the highly rated scanners. Nevertheless, I think the article's key finding is that even the best of the tools they evaluated failed to catch all of the vulnerabilities that they had intentionally installed. Every opportunity should be taken to emphasize this point to the readers.
--
Dave Aiello
-- Dave Aiello
nessus-update-plugins would've helped. My guess is that these people used the stock nessus installation without retrieving the latest scans.
The two vulnerability missed by nessus are at
http://cgi.nessus.org/plugins/dump.php3?id=10318 and
http://cgi.nessus.org/plugins/dump.php3?id=10260
Again, I'm no security expert, but these people should've at least updated the list.
The whole point of the test was to see which scanners could be used for proper security tests.
Closed source software showed itself to be just as good, or in this case poor, as open source.
The "only open source can be trusted" argument only holds up if everyone looks at all the source, which rarely happens except in the luckiest projects.
I would trust a competently written and tested closed source product more than a crappy open source one any day. It is a matter of quality. If an open source product is better, or as good as a commercial one then I would use that instead.
Ideology should never be allowed to get in the way of practical concerns, doing so is hjust another way to shoot yourself in the foot.
Seeing as many of the scanners are free and open source.. I would hope that admins would maybe use a combination of 2 or 3 different ones.. which would hopefully capture everything?
Or are these scanners the lazy way out anyway (compared to servere audits..) and running Nessus and Sara would be just too much work?
The problem with slashdot is that most of its users were bullied and stuffed into lockers as kids!
_Network Computing_ is one of the good guys out there. Since Novell lost its dominance in corporate networking, _NC_ hasn't had a majority of its ad pages from any one vendor (that may or may not correlate to ad revenue). They seem to be pretty vendor neutral and willing to call shots as they see 'em. They have certainly p**sed off Cisco several times in the last year, which takes some courage. This is one of the few trade rags I trust.
sPh
Commercial scanners are not produced by "a person off the street". They're produced by professionals that work for companies that have a significant motive for ensuring the accuracy of their products: money
You poor, misguided, niave thing. Have you ever worked for a "company" that has "professional programmers"? Let me give you a clue. Programmers are programmers. Regardless of whether they work for a "company" or are donating time to an Open Source project, they are just as prone to stupidity as the next guy.
Commercial products, contrary to your utopian notion, are produced by professional marketing departments that have a significant motive for ensuring that the average consumer THINKS their product is worth its cost. Accuracy, efficiency, function, and stability have nothing to do with it - the Consumer, more often than not, is never in a position to objectively judge the quality of a product.
I thought that was just win.com... :)
Did the dickhead that posted originally, supposedly from eEye, get back to you or was it a 15 y.o with more time than brains?
If this is the case then why did none of the software detect all 17 of the issues?
I'm one of the authors of the article in question, and would like to address a few points being made: 1. Thanks for the feedback - it will be well received. 2. NWC has a lead time of 2-3 months. There is no way around this - the mag goes to film almost a month before the world sees it. So those tests were done back in October - it's not that we are THAT behind, the content is just a little older. 3. Please don't assume that we are idiots. Yes, we applied all of the Nessus plugins. No, we did not install it from some old Linux distro. Ask Renaud who I am, he'll tell you. Give us some credit here guys.... 4. I've been writing for NWC for 4 years, and I have not ONCE been asked to modify my writings due to advertising. NWC takes editorial integrity very seriously. Take that for whatever it's worth... 5. The point on the update times/procedures is a valid one, and unfortunately not one we had room to address properly (we're limited in word count). Next time we do this we'll attempt to address this. Again, thanks for the feedback. -Greg
You have a very unrealistic, unhelpful and unnecessarily cynical attitude brought on by reading too much Dilbert.
.|` Clouds cross the black moonlight,
Don't bother waking me when you've grown up.
~Tim
--
~Tim
--
Rushing on down to the circle of the turn
Cisco was invited, but declined to participate.
Using a proprietary (closed source) vulnerability scanner is sort of equivalent to asking a person off the street to give your home a security check. Do you know what internal code audits are done on the software? What sort of 'reporting' it may do during 'updates'? I don't mean to sound too paranoid, but all it takes is one programmer...
Another, more down to earth point is the ability to write your own checks for the scanner - are you stuck with paying maintenance fees to a company for updates of dubios quality, or can you go out and write them yourself?
Think outside the... Hey, where'd the friggin' box go?
what about Nmap?
You can only portscan with Nmap, altough it is very sophisticated. Even Nessus uses Nmap to do part of its scans.
Nmap is defnitely *not* a vulnerability scanner.
- In Memoriam: Jeroen de Bruin (1972-2004), bye bro
Renaud Deraison and Fyodor will be present, and this could be a subject of discussion. http://www.osdem.org
Jacco /var/log
---
# cd
-------
Warning: Slashdot may contain traces of nuts.
"Worse, if you are a consulting firm basing your assessment services on these products, you better have some system in place to cover for their shortcomings, as these products don't cut it."
.|` Clouds cross the black moonlight,
Er, yeah? A security consulting firm that uses only a few of these as anything more than a starting-point for further hole-research and criticism is doing nothing that I couldn't do myself, and will not seen on my Pigsty. When I consult, I expect to give proper service, and if I get consultants in, I expect perfection.
"Because all the products failed to identify key vulnerabilities, none of them received our Editor's Choice award."
If a company relies on an Editor's Choice Award to distinguish good from mediocre from bad, it has altogether too many other problems...!
~Tim
--
~Tim
--
Rushing on down to the circle of the turn
PLEASE! Automatic scanning *and* hand auditing *every* service is the only way to do vuln. assement or a pen test!
whisker
-r
Great article. I enjoyed very much putting this on the desk of our "security admin" who keeps insisting that we buy a product because "big corporations make better products"
It should mention how Nmap will help cover the tracks that Nessus misses.
Commercial scanners are not produced by "a person off the street". They're produced by professionals that work for companies that have a significant motive for ensuring the accuracy of their products: money. If Joe Blow Security company said your home was secure but it wasn't, they're not going to get a very good reputation around the industry, will lose customers, and eventually get such a bad name they'll be run out of business. Definitely not something the investors like to see. Capitalism works for good and evil fortunately. Open source scanners are the ones that are more like asking a guy off the street. 95% of the time people are not going to be writing your own exploit modules for them so you have to take the word of that guy off the street you got the scanner and modules from to say that your network is secure. If you ARE skilled in writing modules and evaluating your network, then the best thing would be to use a combination of all of these methods. Get a couple of commercial scanners, get some open source stuff, and write some of your own. With the security of your network at stake there is no such thing as overkill.
Here is a list of good tools