Slashdot Mirror


Over Half of Software Fails First Security Tests

An anonymous reader writes "Even with all of the emphasis on writing software with security in mind, most software applications remain riddled with security holes, according to a new report released today about the actual security quality of all types of software. Close to 60 percent of the applications tested by application security company Veracode in the past year-and-a-half failed to achieve a successful rating in their first round of testing. And this data is based on software developers who took the time and effort to have their code tested — who knows about the others." Reader sgtrock pointed out another interesting snippet from the article: "'The conventional wisdom is that open source is risky. But open source was no worse than commercial software upon first submission. That's encouraging,' Oberg says. And it was the quickest to remediate any flaws: 'It took about 30 days to remediate open-source software, and much longer for commercial and internal projects,' he says."

36 of 145 comments (clear)

  1. That's great. by cbiltcliffe · · Score: 3, Insightful

    Now they need to test the users.....

    --
    "City hall" in German is "Rathaus" Kinda explains a few things......
    1. Re:That's great. by TrisexualPuppy · · Score: 2, Interesting

      Why is this such a shock to you?

      For secure software, isn't it just a bit subjective? These tests were submitted by people who NEEDED to have their software tested. Much of the software out there doesn't deal with sensitive data, and much of it is too simple to serve as a system security risk, and it isn't submitted. So you this 60% figure doesn't really mean much. Most software isn't submitted for security checks and never needs to be.

      This article is FUD, and the necessary details are not explained. Methinks that Veracode was just trying to get a little publicity. Thanks again, Soulskill!

    2. Re:That's great. by Volante3192 · · Score: 3, Funny

      Of course, 60% of the apps they tested were web applications, leaving 40%...

      (Yeah, yeah, it's unlikely that the only apps that failed were web apps, I just thought it a spiffy coincidence that the % of apps that failed testing also equaled the % of web apps tested.)

    3. Re:That's great. by AlecC · · Score: 2

      Why? Surely, if I was going to send my software for external security testing, I would first test it in house, both more cheaply and less humiliatingly. This is not 60% of all software failing, this is 60% of all software sent for test failing. This suggests that in-house testing is remarkably badly thought through.The sorts of the sorts of tests that Veracode are going to do should be predictable - shouldn't you run them yourself before submitting the software?

      --
      Consciousness is an illusion caused by an excess of self consciousness.
    4. Re:That's great. by Anonymous Coward · · Score: 3, Insightful

      Your viewpoint is a little close-minded. Most software written is never even sold. It is mainly in-house custom apps in companies where it would be pointless to try to exploit it because there are easier ways to get the data. And how about the software that runs completely closed on microcontrollers that are in every single product sold today?? Think before you post. :)

    5. Re:That's great. by ka9dgx · · Score: 2, Interesting

      Yes, the registry sucks, for many reasons.

      Yes, better defaults could have been chosen 2 decades ago.

      Now things have changed, and any system that doesn't let limits get set per task is insufficient. The current choices now are insuring 2 more decades of pain. I'm trying to educate people on the better options available, so that a better choice gets made.

      It's now necessary to think of security with a much finer grain. The user is no longer the natural dividing line. It needs to be per task instance.

    6. Re:That's great. by jsebrech · · Score: 2, Interesting

      These tests were submitted by people who NEEDED to have their software tested.

      I think the software submitted for testing is actually more secure than the average software, because it's made by people who actually know about the problem.

      Much of the software out there doesn't deal with sensitive data, and much of it is too simple to serve as a system security risk

      All web sites need to have good security. Without good security, you can get all sorts of hijacking attacks, where systems that seem harmless are abused to mount attacks on more sensitive systems.

      The biggest problem with security is the degree it is underestimated. Everyone thinks it's somebody else's problem. Collectively though, the web is a one huge gaping security hole, and it's because of this attitude.

      Most of the books on web development I've opened up contain security holes in the code samples. Even something as basic as SQL injection is still very prevalent in the code samples you find online and in print. Things get much worse when you start talking about subtler flaws like XSS or CSRF. And don't even get me started on the programming forums...

      This article is most definitely not FUD.

    7. Re:That's great. by ClosedSource · · Score: 2, Informative

      I agree to the extent that compatibility is an important selling point and it also limits their ability to change their OS.

      I'm not so willing to concede that the registry is an example of a design flaw. You have to consider the design within its context. For an explanation of why the registry was created and a discussion for and against it see http://blogs.msdn.com/oldnewthing/archive/2007/11/26/6523907.aspx

    8. Re:That's great. by TheLink · · Score: 2, Interesting

      > If you can't easily restrict a program to a small subset of your machine, you're forced to trust code you didn't write to get anything done.
      > Nobody should blame the users, if the OS sucks.

      Agreed. And most OSes out there suck in this respect (OSX, Linux, Windows).

      FWIW Windows Vista and Windows 7 kinda suck less - since they actually have some sandboxing with IE8.

      Ubuntu has apparmor sandboxing of firefox as an option that's turned off by default, and even if you turn it on it's not sandboxed enough IMO (firefox can read and write almost anything in the user's home directory with the exclusion of just a few directories).

      As it is, most users are either forced to:

      1) Solve a version of the Halting Problem where they don't and can't know all the inputs and are unable to read the source code (or even know if that's really the source code of the executable they are about to run ;) ).

      2) Use only software from a Trusted Vendor's repository. Not a good strategy for Microsoft given their Monopoly Status, and this approach/philosophy doesn't actually help the OSS cause that much either.

      You can say "download the source and compile it yourself", when even experts have difficulty finding flaws in the software, how would users find them (see also 1) ).

      Users will just skip the pointless steps and go to "make install" (which often requires root permissions).

      As it is I have proposed that applications request for the sandbox they want to be run in. Then the O/S enforces the sandbox.

      It's easier to figure out the danger the application poses, if you require applications to state up front the limits of what they want. If they say "No Limits" you can assume you don't want to run it.

      The sandboxes can be from a shortlist of template sandboxes, or custom sandboxes which are signed by trusted parties.

      Organizations could have Trusted 3rd Parties audit the application's proposed sandbox and sign it if they believe it's OK.

      It is much easier to audit a sandbox than audit thousands of lines of code.

      Furthermore the code audit results will be invalidated if the program can update itself online, or can possibly fetch new instructions from the Internet. Whereas the sandbox audit would still be valid.

      For example, without sandboxing, a code audited program might fetch new instructions and decide to turn on your webcam without your permission. In contrast if the sandbox doesn't allow the program to access the webcam, the program isn't going to be able to access the webcam even if it fetched new instructions.

      Unless of course there's a bug in the sandboxing. But at least this means you can concentrate more resources on getting the sandbox and O/S bugs fixed, rather than try to get the dozens or hundreds of programs security audited and reaudited everytime there's a new update.

      --
    9. Re:That's great. by david_thornley · · Score: 2, Interesting

      The referenced defense of the registry is an article that mostly discusses the weaknesses of Microsoft implementations of ".ini" files, and many of those weaknesses are due to Microsoft design features I consider distinctly suboptimal. I'm perfectly willing to agree that the registry might be better than a botched implementation of rc files, but that is hardly a convincing defense.

      Moreover, even if the registry was the right idea in the 16-bit days, that doesn't mean it's not a problem currently. The biggest strength and biggest weakness of MS Windows is the tremendous number of MS Windows-compatible apps out there, making the operating system very useful and very hard to change.

      --
      "When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
    10. Re:That's great. by tjarrett · · Score: 3, Informative

      We scan selected open source projects on a pro bono basis and reach out to the project teams to share the findings with them.

      Disclaimer: I work for Veracode and was a coauthor of the report.

  2. What emphasis on security? by Jurily · · Score: 4, Insightful

    I thought the only measure of a project was whether it makes the deadline.

  3. Bolting On by Chris+Lawrence · · Score: 3, Insightful

    As Bruce Schneier has said, trying to bolt on security to an existing product or application can be very difficult and time consuming. Sometimes you even have to redesign things. Designing for security and using secure coding practices from the beginning, however, makes it much, much easier.

    1. Re:Bolting On by Anonymous Coward · · Score: 3, Insightful

      Designing for security and using secure coding practices from the beginning, however, makes it much, much easier.

      Sure it does... but that sort of design takes money and expertise. More often software is dreamed up and planned in ad hoc meetings. For example, a person in marketing decides it would be a great idea if their customers can get updates on their phones and Nitwitter accounts. In a 4PM meeting the marketer proposes it to his boss as a necessary value-add function without which the competition would eat us alive (1).

      The next day, a "planning" meeting is called. The marketing manager tells (note, I say "tells" not "asks for input") the programming manager that the company needs mobile updates. The company needs (note, it's changed from the "Marketer wants" to "company needs") it before the next peak retail opportunity. This opportunity is either Valentine's Day or Easter or Summer Break or Thanksgiving or some other arbitrary retail holiday.

      The programming manager tells his programmer, "We need it by end of week."

      The programmer begins to think about the problem. He raises objections to the timeline and lack of design. The marketing manager cries to the CEO. The CEO screams at the CTO. The CTO screams at the programming manager. The manager tells the programmer that he's wasted a day and we still need it by end of week.

      The programmer thinks about coding and how to grab the data he needs. He browses a database and finds a table that he needs. To make it accessible to the web frontend, he opens up some permissions. Maybe he creates a new view that combines multiple tables to make his code easier or faster. This new view now violates PCI and SOX regulations, but he doesn't care.. this is just for testing until he can figure out how to do it properly. He stays up all night and gets a proof of concept working. The next day he shows it to his manager.

      His manager says, "OK, tell them it's done."

      The test software becomes production.

    2. Re:Bolting On by hardburn · · Score: 2, Interesting

      When it comes to security, not necessarily. A good design of classes for the purposes of readability and maintainability does not necessarily mean it's easy to fix a security bug. These are often completing design choices.

      The two biggest errors cited by TFA were cross-site scripting and SQL injection. Generally, XSS can be fixed if you had a decent design--just filter your inputs better. SQL injection even more so. In my experience, good languages have database libraries that make it easy to use placeholders in SQL statements (if you're using some idiot RDBMS that can't handle placeholders, the library can transparently handle placeholders for you in a secure way). If your design started off with a proper database abstraction layer, and you let an SQL injection attack slip through, it should be easy enough to fix.

      However, the third one mentioned is cryptographic implementations. This is much, much harder to solve, and fixes will often result in breaking backwards compatibility. For instance, the RC4 algorithm is considered reasonably secure on its own, but it's also very fragile. If you later decide to use something else, moving your data away from it can have huge backwards compatibility issues; this was exactly the situation faced by WEP. It can still happen for other algorithms, even one's that are sturdier than RC4.

      Making practically unbreakable algorithms was hard, but it's largely a solved problem. Using those algorithms in practice is much, much harder, and it's a problem that has to be re-solved with each new system.

      --
      Not a typewriter
    3. Re:Bolting On by Bert64 · · Score: 4, Interesting

      For another encryption example, look at how windows and linux implement user password hashing...

      Linux takes the plaintext password via an input channel (ssh, telnet, gdm, local console etc), passes it to PAM which loads the corresponding password from the shadow file, encrypts the user input with the same algorithm and salt, and compares the output. The backend (pam, encryption cipher) can be changed without affecting how the frontend, making it easy to use a different encryption algorithm as increases in computing power, or discovery of cryptographic flaws, renders the old ones insecure.

      Windows, in a somewhat misguided attempt to prevent plain texts being sent over the network, effectively uses the encrypted hash (yes its more complicated than that, but the general idea is that only the hash ever gets used and the password isnt sent in the clear - unix solves this at a different layer by using encryption of the plaintext password such as ssh)... Because of this, the hashing algorithm is difficult to change. Older windows used lanman which is laughably weak, while modern windows uses ntlm by default which is somewhat stronger but not great... However, modern windows still has lanman support for compatibility reasons, and until vista/2008 it was still enabled by default. If they change the hashing algorithm, then they will still have to retain the old ones for quite some time in order to have compatibility, and also change the protocols to handle a third possible algorithm.
      The fact that you can use the hash without cracking it first is also a design flaw, this isn't possible on unix or anything else i'm aware of.

      --
      http://spamdecoy.net - free throwaway anonymous email - avoid spam!
  4. Open source doesn't necessarily mean dangerous by Pojut · · Score: 2, Insightful

    I know of at least one rather large and well-known company that doesn't use OSS because of "security", yet voluntarily continues to use IE6.

    That sort of thing really pisses me off.

    1. Re:Open source doesn't necessarily mean dangerous by Opportunist · · Score: 3, Informative

      Quite the opposite. OSS is often far more secure than its "commercial" counterpart, for the obvious reasons.

      1) No deadline. OSS is usually "done when it's done". Most OSS software I know is in perpetual beta, never reaching what its maker would call a "release state", but offers at least the same level of security and stability (if not better) as its commercial counterpart. Simply because there is no date we have to push something out the door, secure or not, ready or not, we have to make it for christmas (or for the new Windows version).

      2) No need to "sell" the software. You needn't dumb down and strip security so potential customers accept the level of burden security adds to the fold. Security is never free. It always comes at the price of overhead. When you have two software tools available, customers will pick the one that is more "accessible". Which usually also is the less secure one. Because security often adds layers of additional overhead (either to you, the user, slowing you down and requiring you to enter passwords or access things in a certain way, maybe even with additional tools instead of from "inside" the tool you're mainly using, or to the system, meaning your software will run slower).

      3) Peer review. Your code can easily be reviewed by thousands of "hackers" trying to find an easy way into your system, instead of having to poke at decompiled code. If you can read the source, far more people are able to poke and prod at it, resulting in more secure software instead of less, because security holes get found faster and, in turn, fixed faster. By the time you start using the product, a few months after its release, you may rest assured that all the easy to find security holes have been found by hobbyists. With CSS you often need experienced ASM cracks to dig those holes up, resulting in fewer people able to look at those holes and thus a slower patching cycle.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
  5. The other half by maxume · · Score: 2, Funny

    And the other half isn't even tested.

    --
    Nerd rage is the funniest rage.
    1. Re:The other half by Opportunist · · Score: 3, Funny

      Nah, the other half crashed when pitted against the security test suite.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
  6. Well now by Monkeedude1212 · · Score: 4, Informative

    That's extrapolating a bit much, isn't it? And scanning through the article, they don't even name the sample size, just percentages.

    And yes, they mention that its only the stuff that they test, "so imagine what the rest is like". Well - thats it though, if someone is professionally developing with security in mind, they probably know how to test it in office or know somebody who can. Thus - no need to pay this corporation to test something you can do yourself.
    If you are developing with security in mind - but aren't sure exactly what you're looking to protect against - THATS when you go to companies like these.

    This is a pretty much skewed data source (probably a slashvertisement for them, too), and is the only study of its type. Take it with a weeks worth of salt.

  7. Security is no selling point by Opportunist · · Score: 5, Interesting

    It just is not. Actually, quite the opposite: The better your security, the more your potential customer will be put off by it.

    Users do not care about security until it is too late (i.e. until after they got infected), and only then they will bitch and rant and complain how insecure your piece of junk is. If you, otoh, take security serious and implement it sensibly, they will bitch and rant already at install because they hate the hoops to jump through and the obstacles to dodge to make your software "just work".

    Security is the antagonist to comfort. By its very definition. No matter where you look, security always means "additional work". Either to the user, which means overhead to his work, or to the program, which means it will invariably be slower than its competing products.

    Thus security is not only an "unnecessary evil" when selling your product. It is actually hurting you when you try to convince someone to buy your stuff. Your software will be slower due to its security "burden", and it will be less comfortable to the user. The user does not see the glaring security holes when he buys the product. Only after, when the product bites him in the ass because it opened him up to an attack. But by then, he will already have paid for your product. And he will have bought your product instead of the more secure product your competitor offered, because yours was faster and easier to use.

    --
    We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    1. Re:Security is no selling point by characterZer0 · · Score: 3, Insightful

      Protecting against SQL injection attacks, XSS, buffer overflows, and validating user input does not put off users.

      --
      Go green: turn off your refrigerator.
    2. Re:Security is no selling point by ka9dgx · · Score: 3, Informative

      Actually, good security would be a GREAT selling point, if someone actually implemented it.

      Security is the ability to run code without unwanted side effects. Windows, Mac, Linux do not offer a simple way to do this. The closest you can get is either Sandboxie on Windows, AppArmor on Linux, or setting up a VM per program.

      If you offered a way to specify the limits of side effects on an application before and while it runs, you could make a ton of people very happy. I suspect there is some money to be made there as well.

    3. Re:Security is no selling point by Lord+Ender · · Score: 2, Informative

      Security is the antagonist to comfort. [etc. etc. etc.]

      Your entire rant is based on a false premise. In most cases, security actually increases "comfort" or "convenience." It's damn inconvenient to use a system which crashes, misbehaves, and needs to be frequently rebuilt due to security problems. Removing buffer overflow vulnerabilities from your software in no way inconveniences your users.

      Authentication is perhaps the only piece that sometimes is inconvenient. Just typing your username to log in is more convenient than having to type a password. But that's the exception to the rule. And systems which time you out while you're using them, and don't integrate with SSO, are actually not "more secure," they're just badly-implemented. So that's not a trade-off either.

      --
      A slashdotter who didn't build his own computer is like a Jedi who didn't build his own lightsaber.
  8. Security firm says security is an issue by SlappyBastard · · Score: 4, Insightful

    Hmmm . . . there's a word for that . . . XKCD, can you help me?

    http://www.xkcd.com/703/

    --
    I scream. You scream. I assume that means we're both acquainted with the problem. We proceed.
  9. What about commercial open source software by weeble · · Score: 4, Informative

    So lots of comparisons between open source and commercial software; however there is a lot of open source software that is sold, i.e. commercial. In addition it has been shown that most of the code for the Linux kernel was developed by people who were paid to do it by Red Hat, IBM, Intel and others. Does that mean that the Linux Kernel is commercial software.

    May be the article should refer to closed source proprietary and open source software.

    The article reads as if the author does not fully understand the how Open Source software is developed and is just a large advert (a.k.a. press release) for the auditing software.

    --
    Slashdot Beta should die a painful death.
  10. They get paid to find security holes by dcraid · · Score: 2, Interesting

    Will a security firm ever certify that a solution is perfect on the first pass? Not if they want to be invited back for a second.

  11. In other news... by GuruBuckaroo · · Score: 3, Insightful

    More than 90% of all software tested fails to compile the first time. Seriously, that's what security testing is for - finding holes so they can be filled.

    --
    Poor means hoping the toothache goes away.
  12. Not a shocker by ErichTheRed · · Score: 2, Interesting

    Coming from the systems integration side of things, I don't view this as a surprise. Developers are great at writing software, but in my experience they have no idea about how the platform they're deploying it on actually works beyond the API function calls they make. This leads to internal applications that I have to throw back because part of the requirements are, "User must be a member of the Administrators or Power Users group." Most dev guys just don't get that it's very dangerous to give the end user full rights to an Internet-connected Windows box. There's just too many holes in Windows to safely allow it.

    To be fair, there are a lot of reasons for stuff like this...not the least of which is deadlines for deploying "something that works." I've been there on the systems side too...scrambling at the last second to get hardware and operating systems deployed because of a deployment date. There are also a lot of apps coded in C++ and other unmanaged languages that open the system up for all sorts of buffer overrun attacks. Not much you can do there except vigilant code checking.

    I think a little education on both sides of the fence would be useful. Developers should get some kind of training in "systems administration and internals for developers" and systems guys should definitely be educated in what holes are safe to open up on their systems. (That's a big cause of this too -- there's a lot of low-skilled systems admins out there who take the developer's instructions at face value without checking to see if full access is really needed.)

  13. "remediate"? by Voline · · Score: 2, Insightful

    Try "remedy", or does that not sound pseudo-technical enough?

  14. Obsolete? by vlm · · Score: 2, Informative

    The conventional wisdom is that open source is risky.

    Does anyone believe that anymore, other than journalists quoting other journalists and PR people?

    I did some google searching, trying to find when that old FUD campaign started. It seems to not show up much until 1998.

    The 12 year old advertising/FUD campaign is getting kind of tired.

    --
    "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
  15. Re:Code has bugs... so don't trust it. by ka9dgx · · Score: 2, Insightful

    The reason users mess things up is that they have bad tools. There is no simple way to run something in a sandbox.

  16. Comment removed by account_deleted · · Score: 2, Insightful

    Comment removed based on user account deletion

  17. Re:As misleading as 'Show all warnings' by clone53421 · · Score: 3, Insightful

    I've worked with managers and developers that want to eliminate all the warnings in the source.

    There’s a good reason for that, and it’s not as petty as you think.

    Warnings exist to let the programmer know that the actual behaviour might not be what the programmer thought was most intuitive. If it’s implicitly casting a float into an int, you damn well better know what that means and what effect it’s going to have on your code... it means that 1/2 == 0, for starters. Similarly, there’s absolutely no reason why you can’t use (count = 5) as an expression, except that its value is always 5, not true or false as you may have incorrectly thought.

    Lazy, sloppy, or inexperienced programmers are going to fall for these sort of pitfalls. Experienced, careful ones won’t nearly as often. But if you force a lazy, sloppy, and inexperienced programmer to learn why the compiler is giving a warning and eliminate it, he’s going to end up slightly less lazy, more careful, and with a little more experience than he started with, because he’ll hopefully understand the warnings by then and know what the code is actually doing.

    --
    Alexander Peter Kristopeit bought his basement from his mommy for one dollar.
  18. Secure web app design is HARD by einhverfr · · Score: 3, Insightful

    I know you meant it in jest, but there's a serious point there.

    First, having written a number of small utilities, and some larger GUI-based tools, I have to say that web app design is fundamentally more difficult primarily because you have a number of specific challenges in this area that only apply to them.

    Not only do you have to deal with all the usual problems, but you also have to deal with XSRF, XSS, and so forth. This is because you are you have a program which is generating HTML code as output, not merely a nice UI that is the product of the code, and this is happening in a stateless environment so information is somewhat limited in the interaction well beyond what it would be normally. Furthermore, authentication is more difficult in a stateless environment, especially where multi-tier systems are involved.

    Most people don't appreciate how easy these are to mess up and how hard they are to fix when a developer creates a project without adequate thought to security in the first place.....

    --

    LedgerSMB: Open source Accounting/ERP