Slashdot Mirror
Best Security / Vulnerability Testing Firms for Web Apps?
An anonymous reader writes "I'm in charge of a web application that must be extremely secure. Users will be submitting highly sensitive information to each other using the site. Security must be world-class. We believe we've built site in such a way that minimizes security risks and we've implemented numerous policies and procedures company-wide to increase security. We'd like a third-party to perform exhaustive and ongoing security tests: automated tests, application testing, and more, to check for things like cross-site scripting issues, server misconfigurations, form/hidden field manipulation, command injection, cookie poisoning, known platform vulnerabilities, etc. What companies would Slashdot readers recommend for these types of services?"
http://sandsecurity.com/
This is one of the things that SandSecurity does for its clients. Try them out!
Full disclosure: friend of the owner
Whoever stated that signature sizes should be limited to one hundred and twenty characters can just go ahead and kiss my
We'll point out any flaws for ya ;)
[FUCK BETA]
I've had the privilege of meeting Jeremiah Grossman at a security conference. I'd recommend reading several of his white papers and then decide if you want to call his company up. I doubt they are cheap, but the best rarely is.
http://www.whitehatsec.com/home/index.html
Securicon worked on my bank's, sorry - financial institution's online finance application. Good enough for my money- good enough for me.
> ... web application ... extremely secure ...
You contradict yourself.
Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
Financial institutions don't necessarily have the best possible security, there are plenty of precedents to prove otherwise. They may or may not but I wouldn't use that as a standard. (I've worked in the "moving other peoples' money sector for several years so this is an insider's perspective.)
I have nothing against Securicon, they may be great, but I'd try to find out who handles testing for the credit bureaus. I've used a web interface with one of them and it was at least secure enough to take effort to use. We had to import a certificate to verify the client and register the IP with them, which seems like a good start.
B) Eliminate all the stupid users. This is frowned upon by society.
> They are pretty much the standard in most large sized organizations.
Standard doesn't mean good. Windows is also pretty much the standard in most large sized organizations.
Most of the information security consulting companies are relatively small shops (5-50 people is common) with a handful of customers each. There is also a number of security testing divisions attached to some of the largest all-around international consulting firms, but they are relied upon primarily for regulatory compliance needs (meaning: "let's get this over with as soon as possible"), and they usually combine lack of any identifiable infosec talent with outrageous pricing.
So, with small companies serving non-overlapping groups of customers, it is almost guaranteed that no Slashdotter (of whom only a small fraction deals with information security!) can offer a meaningful, first-hand comparison of the services of key players in the field - and even if this is incorrect, there is absolutely no guarantee that the person telling you about their experiences would in fact have a sufficiently advanced understanding of computer security to make the comparison meaningful.
Unless you have enough in-house expertise and set up some controlled experiments, it's very difficult to tell if a positive outcome of a security audit means you are in the clear, or simply that the auditors are incompetent. To make things worse, even observing that auditor A identified n bugs in the setting in which auditor B identified n+m does not really tell you much, unless you truly understand their impact in the context of your services, or the reporting granularity and thresholds used.
What else? Many of the small companies may rely on PR alone, and some might be outright dishonest, for example releasing inflated security research, or simply astroturfing on Slashdot or elsewhere. And some might be run by people with actual credibility in the industry, but running subpar businesses because of poor project or team management skills. Just because they present at Black Hat, post to BUGTRAQ, or have a book published, does not mean a lot (but is a positive factor, of course).
So there's no easy solution. What you need to do is not to rely on Slashdot to give you answers, and instead, collect all the names you can easily find on the web (and in responses to this thread), then spend several days going through all the freely available primers on web application security... and come up with a decent RFQ that inquiries all the companies about their credentials, methodologies, the tools they use, sample reports they provide, and so forth. Ask technical questions, and expect them to be answered by technical people. You then need to set your bullsh*t detector to overdrive, and be wary of vague, dismissive, or nonsensical responses that look as if written by a marketing drone.
Based on this information, you then need to make the call which one would suit your business best. Good luck. It's not easy.
"Do some real good monitoring, real time, and post on a grey hat security board that it can't be hacked.
Except that a company providing an ultra secure website might, I don't know, have a vital relationship with their ISP that would be damaged by this type of action.
Be sure that, whoever does your testing, your company's "policies and procedures" are both satisfactory and being reliably followed by all employees. Social engineering is quicker, cheaper, easier, and more difficult to detect and track, generally speaking, than hacking in through some obscure loophole in the application.
Your people need to know what not to do, what not to say, and whom not to talk to, or your iron-clad web app may as well be tin foil. A top-notch security analysis company should be able to help make sure those bases are covered, too.
Coffee is my drug of choice.
I worked for KPMG for ten years performing penetration tests. For the last several of those years I ran the teams and worked with clients to scope the work.
The following is true for most big companies that have country or regional teams and for any team for that matter: there are good teams and bad teams. You're going to have to talk to the techies to get comfortable with them.
The bad companies will use a lot of automated methods. For example they'll tell you that they have a software product that does the pen test and then they manually review the output. There are a few of those 'pen test in a box' companies out there you should avoid. Or they'll say they know what they're doing and actually run nmap, nessus and then do some poor manual testing.
What you need is someone who will make use of some automated tools but spend a lot of time manually testing the web application. This means they are manually testings various inputs to see what they can do and they have to know what they're talking about. I don't mind companies that rely on products like WebInspect or AppScan, but that should only be a tool and not the main show. Make sure you ask to talk to the techies and not just the salesguy so you can ask them how a web app should be secured and what kind of things you should look for to get your app in shape before a pen test begins. What often distinguished us was that we could give free advice to help improve security even before our testing began.
Besides some of the teams at KPMG and the other big firms (again, you have to vet each team) I would also suggest Corsaire which is a smaller company.
In terms of scoping work you should ask for an infrastructure test and an application test. If you are really unsure of things you should ask for them also to review your architecture and things like your firewall rules. Expect to pay a minimum of 5k USD but depending on how big your app is you may get as high as 30k. After htat you can look at regular scanning but there are a lot of companies that offer that more cheaply (like Qualys)
Ask whoever you choose to first run an automated scan against the site so you can fix those things before they do their work. Give yourself a few weeks for that. You really really don't want them to test your site before it is ready. Otherwise it might be a waste of money. I now work for another global company but on the other side of the table: I use services from companies like KPMG. I'm still impressed with the service they and some of the biggies give us. They find things that I haven't even had a chance to hear about yet. And occasionally we'll have a really crappy B team that misses things I've already found in our apps but didn't tell them. That tends to happen more from some of our smaller vendors who magically got on our approved tester list.
Cenzic will test your web security for you.
www.cenzic.com
Check out their "Click to Secure" service.
They are first class and will tell you if your if things are secure.
Regards,
Doug
Just a tip: ask multiple companies to do the first audit. You'll likely get very different results, go from there.
<shameless plug>
I do pen tests for clients (both government and banking) via my company. I wouldn't call myself the best, but there's always something that can be found.
</shameless plug>
This sig is intentionally left blank
As someone that used to lead teams that did this kind of work, too little attention was paid to the thoroughness of the testing by both client and testers. "What is supposed to be tested", "what was tested" and "what were the results" were simple questions that I would ask but testers and client were only interested in "what vulnerabilities were found". I could have 100 interesting findings but have only tested 1 out of 10 components that were supposed to be tested - and that was fine! You should make a list of what you want tested and demand that you be told what was tested and what were the results of those tests.
I can't answer the question about recommending a testing company. However, I can tell you that you will need to have your app re-tested at regular intervals, as well as after any change (no matter how small) to the code or infrastructure. You need to build that into your plan and budget, and you need to have the tests run against your staging/QA setup so that you can catch problems before they hit the production site, as well as against your production environment.
--Paul
I'm in charge of a web application that must be extremely secure. Users will be submitting highly sensitive information to each other using the site. Security must be world-class.
And your way of approaching this problem is to "ask slashdot"? Ye gods.
... and then they built the supercollider.
I do not think anyone can recommend the "best" company as the criteria for "best" depend on your business needs.
That being said, I would recommend sending a request for proposal (or call for tender, I never know the correct name for this) to 5 companies with local offices so you can meet the ethical hackers if needed. This is good to avoid relying on a bunch of "not so white hackers" with little knowledge of collateral damages and potential impact of the pentest on the information system.
Make sure the intruders do not rely on automated tools. I have seen Eeye/ISS reports labelled as actual pentests reports, sold at pentest prices. A good pentest on a 3/3 application requires at least 8-10 days from my experience. These figures should be adapted to the complexity of the infrastructure of course.
I would also ask for information regarding
- system tests vs application tests. The latter cannot be automated to be effective, but both are necessary for a pentest to be meaningful
- the pentest methodology (do they have anything set or do they do it "as they feel" for each project),
- audit trails gathering (all traffic between the pentest lab and your information system should be archived)
- alert processes (what should they do if a critical vulnerability is discovered) and so on
Many companies with little knowledge of professional penetration testing sell intrusion services, from my point of view it is your job to select the best one, nobody on Slashdot can do that for you.
It is rare that I would get into a discussion like this, since it often will devolve into the equivalent of a perl vs python war, or at a minimum, vendors will try to sell their warez.
When hiring a company for an application penetration test, I like to look towards those who are actively involved in research within the security community, and hire people that contribute to the community heavily as well. For example, does the firm have people on staff that discovered and disclosed new vulnerabilities? Does the company have people that bring new ways of attacking to market, and what tools do they make available to the community.
Quite often this rules out a number of the large companies, like the big auditing firms. (Whilst in some cases they have intelligent people, I have met an awful lot of tool monkies that worked for these companies.
Some companies that I would usually consider include NGS software (David & Mark Litfield ... known for a number of Oracle vulnerabily disclosures), Immunity Security (Dave Aitel, Kostya Kortchinsky, and Nico. These guys are very well known in the community, and are the brains behind Canvas, Spike Proxy, and others...), Security-assessment.com (Paul Craig, released iKat for kiosk hacking.), and finally, insomnia security (Brett Moore, this guy knows heaps about heaps.).
Which of these are the best will depend on the particular assessment you are having performed, and what the goal of the test is. These guys are damn smart, and very professional. Go to their sites and see what they do, and then talk to references. In the end you have to be comfortable with the company.
I hope this helps..
Cy
Your first stop should be OWASP, the Open Web Application Security Project. You'll find there many companies that are experts in web application security, including tools and guides to get a handle on web app sec. I'd also recommend becoming familiar with the OWASP Top 10 http://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project
"Consulting Search Fee Saved, $5000. Giving people the chance to earn Informative Karma, Priceless!"
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
I know it's sometimes a pain and can take time, but you might want to consider putting out an RFP for an application test. Depending on the size of your company and procurement policies, you might be required to put the job out for bid anyway. It also gives you a good idea about what's out there. Let me warn you however, that if you're only looking to satisfy an audit requirement, you're probably wasting your time, as you'll probably be force to choose the lowest bid, which will most likely provide the least value in the long run, not to mention a false sense of security. There are many things to include in the RFP, but the major points that come to mind at the moment are as follows: - Company information (size, qualifications, location (important if testing is on-site), personnel bios, insurance, etc.) - Technical Methodology (as detailed as possible) - Tools used - Reporting (make them include a sample) - References (3 professional references seem to be the norm, which should be past clients) There are many places one can place the RFP, such as magazines (SC, Infosec.), listserves (e.g., securityfocus.com) and of course you can always pick the top-10 replies to your query on slashdot and send the RFP to them. You should get at least 5-6 responses.
You don't do this on the live server. Do it on one you're hosting yourself, in a DMZ, rather than the one your hosting company is running.
Drop some fake information into it so it looks legit, and see what happens.
"City hall" in German is "Rathaus" Kinda explains a few things......
Security Team at Cornell University are amazing IMO. Talk to them.
Security must be world-class.
"WORLD CLASS: A phrase used by provincial cities and second-rate entertainment and sports events, as well as a wide variety of insecure individuals, to assert that they are not provincial or second-rate, thereby confirming that they are."
--unknown
Unexpect the expected!
"We believe we've built site in such a way that minimizes security risks and we've implemented numerous policies and procedures company-wide to increase security."
This is by far your biggest security threat that you should worry about before any penetration testing. The idea that implementing policies and procedures will somehow increase security. Relying on humans to adhere correctly to policies and procedures as a security measure is probably a sure fire way to end up with a security leak.
You should be working to secure your app. so you don't need these policies and procedures. This will likely end up with a system that is a little more frustrating for your users, but if security is your number 1 criteria for the application then that's the compromise.
You may think to yourself, well, anyone not following procedures risks the sack so they'll have to follow them, but this misses the fact that your CEO is as likely to ignore policies and procedures as anyone and he certainly aint going to sack himself, no, it'll be all your fault. It's also worth pointing out that even the threat of the sack isn't enough to convince some people to pay attention and realise the rules are there for a reason not to mention the fact that to be fair on users, sometimes it's just human nature to forget to do something.
IBM AppScan (AppScan or Enterprise) was developed by Watchfire and than acquired by IBM for the sole purpose of Web application security testing. When you bring in a group of security testers they usually use a tool to help them with the automated testing, but if you get this than you can have your developers do their own testing as well as have security consultants use the data to preform their own pen testing. (Posted it under anonymous without thinking)
I would agree with the post in for Jeremiah Grossman at WhiteHat Security. Jeremiah and his team do great work in this space, and their research is top notch.
I also wanted to offer our company's services as well. InGuardians is also well known in the industry. Our team frequently presents at major security conferences, both commercial (BlackHat, SANS, ...) and community (Defcon, Toorcon, Shmoocon, ...). In fact, I'm sure if you spoke with Jeremiah, he would give us a shining recommendation as well. And honestly, I'd say that you'd be hard pressed at finding anyone else in the industry that does better work than InGuardians and WhiteHat Security. You really can't go wrong with either choice.
Full Disclosure. I am a Senior Security Analyst for InGuardians that specializes in network and web app pentests. Another one of our Senior Analysts is Kevin Johnson, who is the author and lead instructor for the SANS 542 "Web App Penetration Testing and Ethical Hacking" course.
http://www.sans.org/security08/description.php?tid=1722
Here is something else to help you out, regardless of who you go with. Kevin and I have a few OSS community projects, one that you'd probably be interested in is our live pentest CD called "Samurai-WTF". It is a live Linux environment that has been pre-configured with the best open source and free tools for testing and attacking websites. Feel free to go download a copy from our website. It works great running from any of the virtual machine products out there, and also works great if you burn it to a DVD. Once you get it running, the login is "samurai" with the password "samurai".
http://samurai.inguardians.com/
I'd love to draft up a proposal for Kevin and I to pentest your website and the network it is sitting on. Please feel free to email me at justin (at) inguardians.com to set up a time to talk about your needs in more detail.
Check out our website if you would like to learn more about our company, the other services we offer, and the other members of our team.
http://inguardians.com/
I'll add another plug in the parade of shameless plugs.
My employer is Fortify Software; we make a static analyzer that performs good quality cross tier analysis of popular languages like Java, JavaScript and PHP.
In addition to the static analysis, we also have a QA assistance tool that uses Java bytecode instrumentation to follow taints dynamically through the application and correlate with the static findings.
Doug
Take off every 'sig' !!