Slashdot Mirror
Creating a Security Test Environment?
Enderandrew writes "Our IT department has been tasked with creating a list of authorized software, and only allowing software to be added to such a list after it has been thoroughly tested. In theory that sounds like a great idea — but how should we test apps to make sure they are secure? We have tools to scan internal websites, and we use MBSA for our Windows servers. However, I'm turning to Slashdot to ask what are the best methods for creating a test environment where I can analyze apps for security vulnerabilities. We're a multi-platform shop, but my main concern is with Windows apps."
You can't be certain unless you have the source code, so tell the folks who are requiring this list to be created that they can say goodbye to Word and Excel.
If yes, then the product is insecure. If no, then the product is secure.
I know that for most federal government institutions, the NSA is involved testing a product for its security. I've never heard of a place doing it in-house.
Is your boss ex-military or something to that effect? If so, he may not understand the complexity of the task he is assigning.
Just how thorough of testing is being asked? Some security flaws escape eyes for years (see DNS flaw). Some flaws are obvious. But, in general, you can never be 100% certain of a program be 'secure' unless you know its has passed some milspec cert.
While optimistic, I think the new policy is a bit misguided in its wording.
Bearded Dragon
The plural of "anecdote" is not "data." Also, usability and security aren't really related. If someone complains about a product, for the most part, it will not be because there is a buffer overflow vulnerability for the 4th input field.
'what are the best methods for creating a test environment where I can analyze apps for security vulnerabilities. We're a multi-platform shop, but my main concern is with Windows apps..'
...
You can't test for security vulns, especially in a Windows environment, as there is so many interlocking components that behave differently depending on the configuration. Introducing a service pack into a previously 'secure' environment and all bets are off. All you can do is patch, patch patch
You could also intalss a second intrusion detection system that monitors the first for unauthorized access and keeps a full audit of access alterations etc. You could also compartmentalize your system so as a breech in a web service doesn't automatically lead to a total system compromise.
davecb5620@gmail.com
You are apparently talking about black-box testing. For starters, you need a security team to perform penetration testing on the apps in a production-like environment. But if you have home-grown software, you need to address the problem of insecure systems being built by your programmers. The programmers need to understand application security. For a somewhat theoretical but still practical treatment, I recommend my own book, High-Assurance Design (Addison-Wesley, 2005). You should also check out Michael Howard's book and his blog. And then there are Gary McGraw's books which address process. - Cliff
This is a no-win situation for the persons assigned to certifying an application. I personally have a very hard time communicating with managers who believe, with an unshakable faith, this is a reasonable solution to a perceived problem. When it blows up, MY head rolls, not theirs. The ages-old "Get IT in my office *now*" blame-shifting game.
The right way to handle this would be to push back hard and explain why this is an epic failure in the making and resources quagmire. That isn't typically a good political solution though.
I would love some advice as to how to change such foolish thinking.
Maybe you've got a good thing going there, but if there have been IT organizational changes recently, this may be a harbinger of things to come.
http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
1) There is no secure software. Just software that doesn't have any known vulnerabilities.
2) Define 'thoroughly tested'.
3) White-listing applications is going to cause your company pain as people wait for required tools to become available.
4) You can't afford to do this yourself.
Look at it this way. When PHB is shouting for his new shiny piece of software to work, you can bet that 'thoroughly tested' gets severely watered down. You'll be left with some software on the list that has vulnerabilities whilst other people can't get their jobs done because the software hasn't been listed yet. Even if you avoid those pitfalls and manage to 'thoroughly test' each application, how many applications does your company use? Say you can test an application in 2 weeks - For one fulltime employee you'll white-list 24 applications in a year. The fact that you are asking /. for help suggests that you don't have much knowledge or experience in this field, so you're going to have to pay for training or for an employee with that training.
This model is doomed to failure, and whoever suggested it should be shown the door and replaced with a CISSP qualified guy (or gal). Security is a process, and what you need are tools to help carry out that process. Someone with a CISSP qualification will know where to find those tools.
Some of those tools might include auditing (to find what applications are in use), firewalls, IDS systems (to detect suspicious traffic), patch management (to mitigate vulnerable software), an intelligence provider (to tell you about new vulnerabilities and patches in a timely manner), a Security Event Management system (to manage security data and drive processes) etc...
For the record, I am a QA lead engineer with experience in enterprise and industrial security. I do this kind of stuff for a (good) living.
This was to have been implemented today in my organization but in three stages to minimize disruption. We must conform to FDCC dictated by DHS. I received a USB drive with some instructions and files that I can use to download and install VMWare to create a sandbox for testing. The instructions are lengthy so I've just skimmed them but it appears that a software package is installed that when run establishes the baseline security of the virtual machine. Then the software to be tested is run in the virtual machine and if the base changes it fails.
I think that DHS or some Federal Agencies have lists of software that is FDCC compliant and this should ease the burden of testing if the lists can be easily accessed. I'll probably test one of my applications this weekend.
At any rate search on FDCC for more informaton.
Nate
The likely goal (from a management perspective) is to establish an authoritative list to allow for efficient management of vulnerabilities. If you know what's out there and "authorized" you can respond to threats far more quickly than otherwise.
Your management would be silly to expect you (unless you are an application analysis company) to thoroughly vet every application that comes through the door. It would be a terrible waste of time.
Do some basic analysis (what ports does it open, does it connect out to other systems, that sort of thing). Beyond that, I believe the value is simply in knowing about the application and being able to track any potential issues.
I think most answers here (at least the reasonable ones) are going to have more questions than answers. Questions like:
Part of the problem is that there isn't any such thing as completely thorough testing that makes sure everything is completely secure from any kind of attack whatsoever. It doesn't exist. So when you're talking about testing security, you have to know what you're trying to achieve.
Are you just trying to meet some kind of regulation placed on your company from an outside body? Because that outside body should be providing you with information, then, that will tell you how to meet their standards. Are you just trying to satisfy someone's paranoia? Then you just need to come up with a bullshit standard that satisfies that person's paranoia.
For most situations, it's enough to protect against generic hacking over the network, worms, etc. In those cases, as far as the software installed, it's probably enough to buy from major vendors or use respectable FOSS distributors, and keep things patched. If you're using Debian/Redhat/SuSE, then you're probably good. Even Microsoft's stuff (despite its bad reputation on Slashdot), if kept up-to-date with patches, is fine. A decent firewall goes a long way, and you can look into IDS stuff.
Another valid concern is employees, so make sure they're locked down as much as possible, not given access to resources they don't need. Ideally you'll audit what you can, keep backups going back so if some data goes missing (or is mysteriously changed), you can recover it.
Or you might just be asking about making sure a given piece of software isn't malware? So you do some research online and find if anyone is complaining about it. Maybe you install it, scan for opened ports, look at the network traffic coming off for anything anomalous.
But if you're talking about auditing the security of Firefox or MS Office for bugs that allow for privilege escalation or something, and you're unwilling to rely on the analysis of other experts.... well, good luck with that.
It certainly cannot be secure because all apps have different functions, outputs and installation requirements. You would have to come up with a "lowest common denominator" for the security requirements that apply for all. And if so, it would be of such low common denomination that the term "secure" becomes useless.
What are you going to categorize all this apps with? With the requirement that they don't crash? That they don't access priviledge info (which is mostly done via OS/infrastructure restrictions). That they don't leak info (whatever that could possibly mean). That'd pretty much all you can use as a requirement for building a semi-usable, non-retarded list.
Furthermore, the security of an app, the criticality of it depends on who uses it, where and how. You don't care if your copy of MS Word crashes on your desktop, but you care if a copy of TOAD is running with autocommit on while accessing a production database.
Any IT person worth its salt would see the major folly of trying to create an authorized list of 'secure' applications for general use without specifying a context (type of user, type of use, type of resource been access, blah blah blah.) The whole idea is ludricuous corporate red-taping process-making, const-increasing hooplah.
Now, a wide and open list of authorized software by license (to prevent copyright violations), that makes absolute sense.
Security is an abused word. Most people have no clue what it means, incredibly and unjustifiably so by individuals who earn their living in IT. They confused 'authorized' with 'secure' and assume the attribute of being secure to be free of context, readily applicable to anything, like a color or something.
I want my database' color to be mauve as it has the most ram... plus it's more secure than blue.
Perhaps he should try getting his software from someone who creates it with their brains instead of their muscle.
Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
Certainly agreed.
You should certainly do all these things. However, some amount of focused, possibly manual, application-specific investigation can also be worthwhile. I *think* this is what the original poster was referring to.
Investigate the tool conceptually. Identify how it works, what it trusts, how it safeguards against problems. Essentially do your own black box security review from what you know about the program.
Consider asking the vendor to comment on steps they take to ensure security. You may get useless answers from the wrong employees, but they may give good data.
A static analysis tool which operates on binaries (usually) won't uncover conceptual design mistakes, but it will tell you if they bothered to do some basic stuff right. This type of tool can identify that the code has definite buffer overflow exploit points by following the flow of input to potentially problematic system and library calls (eg strcpy). Veracode sells tools of this nature, I'm not sure who else.
Consider whether it's reasonable to fuzz-test (send random input to) the tools. Tools that crash on garbage are probably not safe vs a malicous agent.
The most important thing of course is what everyone else is saying: this is just dilligence. I don't particularly like the term "tested" in the security context, since that implies a comprehensive style of exercise similar to what the team developing the tool might use. Comprehensive security testing is probably not achievable, and certainly not without a very large amount of time and is obstructed by not having source code, insight into the project, and so on. I'd prefer some phrase like "sanity checked" or "passed due dillegence efforts".
-josh
so how long will you test? what are the criteria to make it out of the test environment?
It seems a more reasonable approach would be to:
1. pick software that is in wide enough distribution that exploits are published and patched quickly.
2. Set up the production box on a private LAN. Scan the hell out of it. Lock it down. Scan it again.
If you think your team is going to find zero-day exploits... then you are either lost or secretly have the crack team (in which case you wouldn't be posing this question here;)
This leaves you with proven solutions, scan, lockdown/patch, scan. Then of course you'll need to keep the server/apps up to date and backed up. This is where most well intentioned admins fall short...
Seriously. You really, really cannot possibly test for every possible vulnerability in every possible app, especially without the source available! The best you can do, IMHO, is to structure your network and systems to primarily isolate any possible bugs from spreading or from compromising integral data. In this way, if bugs or other breaches of security really *are* arising due to users installing insecure apps, you'll know which users did it because only their own systems will be f*ckd. Further, the users, being employees, need their computers to do their jobs and are not trying to maliciously break them (hard as that may be to believe). Indeed, the guy who's computer is broken all the time obviously can't get his job done, so why would someone sabotage their own job by breaking their own machine? Since I'm sure they really want to *keep* their jobs, many users will be happy to attend more training about good security practices, possibly even taught by you. This has the added bonus that you get to help close the human security loopholes that exist despite your courageous efforts to shelter the users from their own demise: you can remind them not to share their passwords, to check ID of visitors, to keep documents off their desk and screensavers locked when they're away, and all those other *ordinary security* measures without which could give a very malicious or opportunistic person direct access to the hardware on your precious network, a way bigger risk than any software bugs could present...
;)
Give your employer a cost-benefit analysis comparing the migration to either a Linux or Mac server backbone that won't let users f*ck it up so easily, to the zillions of hours of overtime they'd have to pay a whole crew of code-monkeys and network goons to moonlight scrutinize all possible apps for all possible vulnerabilities, and let them decide
-=[You cannot consistently judge this statement to be true.]=-