Web App Scanners Miss Half of Vulnerabilities

seek3r sends news of a recent test of six web application security scanning products, in which the scanners missed an average of 49% of the vulnerabilities known to be on the test sites. Here is a PDF of the report. The irony is that the test pitted each scanner against the public test files of all the scanners. This reader adds, "Is it any wonder that being PCI compliant is meaningless from a security point of view? You can perform a Web app scan, check the box on your PCI audit, and still have the security posture of Swiss cheese on your Web app!" "NTOSpider found over twice as many vulnerabilities as the average competitor having a 94% accuracy rating, with Hailstorm having the second best rating of 62%, but only after extensive training by an expert. Appscan had the second best 'Point and Shoot' rating of 55% and the rest averaged 39%."

Comments

Re:The cat and mouse game.

[JWSmythe]

    From what I recall doing this for sites that handled credit card processing (me being in the tested side), those tests are pretty much worthless.

    If you had 1 vulnerability, you'd get pages of false positives or irrelevant information. I recall a particular 10 page report we got back that we were advised to fix or we'd fail on. The only item to fix was the version of the web server was just one behind current. The changelog indicated that it was to fix a vulnerability on a different platform, so it was completely unrelated to us. We'd frequently have points marked off because we couldn't be pinged or portscanned. I'd have to open the firewall up to them, just to be scanned. Our security would identify an attempted port scan as a hostile action, and react by dropping all traffic from them. Sorry my security stopped your scanning, but that's the intention of it. {sigh}

    After opening the firewall to them, and changing the version number on the web server (there were reasons we couldn't do the trivial upgrade), we passed with flying colors.

    For them, they were interested in the version numbers handed off by the server, not what they actually were. For example, if it was Apache, we could have it report Apache version 9.9.9, and that would have made us pass on that part without fail for years.