Slashdot Mirror
Over Half of Software Fails First Security Tests
An anonymous reader writes "Even with all of the emphasis on writing software with security in mind, most software applications remain riddled with security holes, according to a new report released today about the actual security quality of all types of software. Close to 60 percent of the applications tested by application security company Veracode in the past year-and-a-half failed to achieve a successful rating in their first round of testing. And this data is based on software developers who took the time and effort to have their code tested — who knows about the others."
Reader sgtrock pointed out another interesting snippet from the article: "'The conventional wisdom is that open source is risky. But open source was no worse than commercial software upon first submission. That's encouraging,' Oberg says. And it was the quickest to remediate any flaws: 'It took about 30 days to remediate open-source software, and much longer for commercial and internal projects,' he says."
Now they need to test the users.....
"City hall" in German is "Rathaus" Kinda explains a few things......
I thought the only measure of a project was whether it makes the deadline.
As Bruce Schneier has said, trying to bolt on security to an existing product or application can be very difficult and time consuming. Sometimes you even have to redesign things. Designing for security and using secure coding practices from the beginning, however, makes it much, much easier.
I know of at least one rather large and well-known company that doesn't use OSS because of "security", yet voluntarily continues to use IE6.
That sort of thing really pisses me off.
Living With a Nerd
There is no requirements document for security that you can follow and guarantee that your application is secure. You're really trying to anticipate all the ideas other people may have about compromising your code. In general, this is impossible to achieve, so you do the best you can.
And the other half isn't even tested.
Nerd rage is the funniest rage.
That's extrapolating a bit much, isn't it? And scanning through the article, they don't even name the sample size, just percentages.
And yes, they mention that its only the stuff that they test, "so imagine what the rest is like". Well - thats it though, if someone is professionally developing with security in mind, they probably know how to test it in office or know somebody who can. Thus - no need to pay this corporation to test something you can do yourself.
If you are developing with security in mind - but aren't sure exactly what you're looking to protect against - THATS when you go to companies like these.
This is a pretty much skewed data source (probably a slashvertisement for them, too), and is the only study of its type. Take it with a weeks worth of salt.
It just is not. Actually, quite the opposite: The better your security, the more your potential customer will be put off by it.
Users do not care about security until it is too late (i.e. until after they got infected), and only then they will bitch and rant and complain how insecure your piece of junk is. If you, otoh, take security serious and implement it sensibly, they will bitch and rant already at install because they hate the hoops to jump through and the obstacles to dodge to make your software "just work".
Security is the antagonist to comfort. By its very definition. No matter where you look, security always means "additional work". Either to the user, which means overhead to his work, or to the program, which means it will invariably be slower than its competing products.
Thus security is not only an "unnecessary evil" when selling your product. It is actually hurting you when you try to convince someone to buy your stuff. Your software will be slower due to its security "burden", and it will be less comfortable to the user. The user does not see the glaring security holes when he buys the product. Only after, when the product bites him in the ass because it opened him up to an attack. But by then, he will already have paid for your product. And he will have bought your product instead of the more secure product your competitor offered, because yours was faster and easier to use.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Veracode offers the service of finding security flaws in your source. By definition organizations and developers that submit their source to them are going to have more secure software (according to Veracode) when it's released, after it's been certified.
All this shows is that there are developers using a company that specializes in finding security bugs to . . . find security bugs. It's just like using any other debugging tool, you rarely get a clean compile with no bugs on the first try.
Most ignorance is vincible ignorance. We don't know because we don't want to know. --Aldous Huxley
Hmmm . . . there's a word for that . . . XKCD, can you help me?
http://www.xkcd.com/703/
I scream. You scream. I assume that means we're both acquainted with the problem. We proceed.
So lots of comparisons between open source and commercial software; however there is a lot of open source software that is sold, i.e. commercial. In addition it has been shown that most of the code for the Linux kernel was developed by people who were paid to do it by Red Hat, IBM, Intel and others. Does that mean that the Linux Kernel is commercial software.
May be the article should refer to closed source proprietary and open source software.
The article reads as if the author does not fully understand the how Open Source software is developed and is just a large advert (a.k.a. press release) for the auditing software.
Slashdot Beta should die a painful death.
That is 50-50 is good news if the sample was broad enough . Could be interesting to match that numbers with amount of users... could be a lot of those programs that their userbase coincide (or is even lower) with the amount of developers, and see how insecure are programs with more than 100,1000 or even more users (i.e. if the top 20 % of top safe applications have the 80 % or more of users,or the distribution is better than that).
Will a security firm ever certify that a solution is perfect on the first pass? Not if they want to be invited back for a second.
Code has bugs, it always will. You need to reduce the attack surface, why not reduce it all the way down to the kernel of the OS? If you don't need to trust any of the users programs with the security of the whole system, you've solved a lot of problems.
Don't trust the users? Not a good idea. The users are the administrators these days.
Don't trust the internet? Well... it's a communications medium, just a set of tubes.
Don't trust the programs? Great idea!
Sometimes security doesn't matter, esp. with regard to the "internal project" stuff mentioned.
Of course, this is the area that basic utility scripting is used, you and perhaps one or two others are the only ones using it, you already have access to any other system you could get via a cross scripting technique, access to any DBs you'd get with a SQL injection, etc.
Don't blame me, I voted for Kodos
More than 90% of all software tested fails to compile the first time. Seriously, that's what security testing is for - finding holes so they can be filled.
Poor means hoping the toothache goes away.
Coming from the systems integration side of things, I don't view this as a surprise. Developers are great at writing software, but in my experience they have no idea about how the platform they're deploying it on actually works beyond the API function calls they make. This leads to internal applications that I have to throw back because part of the requirements are, "User must be a member of the Administrators or Power Users group." Most dev guys just don't get that it's very dangerous to give the end user full rights to an Internet-connected Windows box. There's just too many holes in Windows to safely allow it.
To be fair, there are a lot of reasons for stuff like this...not the least of which is deadlines for deploying "something that works." I've been there on the systems side too...scrambling at the last second to get hardware and operating systems deployed because of a deployment date. There are also a lot of apps coded in C++ and other unmanaged languages that open the system up for all sorts of buffer overrun attacks. Not much you can do there except vigilant code checking.
I think a little education on both sides of the fence would be useful. Developers should get some kind of training in "systems administration and internals for developers" and systems guys should definitely be educated in what holes are safe to open up on their systems. (That's a big cause of this too -- there's a lot of low-skilled systems admins out there who take the developer's instructions at face value without checking to see if full access is really needed.)
Try "remedy", or does that not sound pseudo-technical enough?
The conventional wisdom is that open source is risky.
Does anyone believe that anymore, other than journalists quoting other journalists and PR people?
I did some google searching, trying to find when that old FUD campaign started. It seems to not show up much until 1998.
The 12 year old advertising/FUD campaign is getting kind of tired.
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
A customer just run their $10k scanner on our software and only found 3 problems. But it turned out the vendor had grabbed every 'security vulnerability' ever reported on any discussion board/mailing list and listed it as a problem. e.g. 'I tried this URL and my computer slowed down a bit. I think it is a denial of service attack.' Took a few days to research and disprove their claims. Meanwhile, how many other such claims is it making? To me, it is analogous to switching on 'Show all warnings' - I've worked with managers and developers that want to eliminate all the warnings in the source. Apart from just rock polishing, I think it distracts them from focusing on the real issues like security and performance. Like any job, you need to have the right tools and know how to use them. We do use Java Findbugs and network scanners. But poor use of any tools only leads to cuts, bruises and visits to the emergency room.
you know you want to
Comment removed based on user account deletion
Obviously, Veracode's tests aren't thorough enough. But it raises the question, "who tests the testing software?"
The conventional wisdom is that open source is risky. But open source was no worse than commercial software upon first submission. That's encouraging
...um, I’d have started with the opposite premise, that open-source is safer. In light of that premise, I think their findings are somewhat discouraging... except,
It took about 30 days to remediate open-source software, and much longer for commercial and internal projects
Now that’s encouraging.
Alexander Peter Kristopeit bought his basement from his mommy for one dollar.
mostly to security consultants. nobody else really cares because it just doesn't matter that much.
This obviously causes security holes.
In its last several releases, everyone's favorite Open Source browser has become an unstable mess of add-ons, plugins, and other hacks that chew up memory like a fat kid with a chocolate-dipped corn dog. In fact, just last week, SecurityFocus released news of a devastating exploit in Firefox 3.5.5 that they blame squarely on its unstable architecture.
From its infancy Firefox has been the product of collaborative effort, unifying code from hackers worldwide. But thanks to the Hayes Law, we see that there is a "sweet spot" to such a development style, and that Firefox has long since left it behind. In the chart below, we can see that the number of Firefox developers has increased exponentially since 2002, and that number will more than double in 2010.
But it's time to be honest: either Firefox, as a modern web browser, will have killer performance on 64-bit, multicore Intel chips or it's not worth downloading and installing. And since, as we have seen in the recent past, that Firefox is actually getting slower with each release, Firefox is certainly a waste of time for anyone who takes their web browsing seriously.
The Hayes Law states that, given a specific type of software project, there is a certain complexity associated with it, and with that complexity an optimal number of developers. It's actually a little more complicated than that, taking into account development model, coding platform, programming language, and code repository platform, but in the end it's easy to plug in the numbers and see where a project's headed.
Against the Hayes Law, Firefox appears to have jumped the shark sometime after the Firefox 2.0 in 2006. The next major release, Firefox 3.0 in 2008, introduced many issues users today complain about: bloat, sloth, instability, and insatiable hunger for memory. Firefox user complaints increased in tandem, all syncing up with the jump in developers. Ergo Firefox's problem: too many cocks in the kitchen.
To further underline this growing problem, Firefox completely falls down in Acid3: Firefox 3.5 scores 93/100, and Firefox 3.6 scores only 87/100. Needless to say, Firefox 4.0 mockups score 0/100. Sadly, this is a continuation of a trend: Firefox took the longest of all browsers to beat Acid2. And don't even think about Acid4. Firefox is collapsing under its own weight.
The core of this problem looms: the number of developers, as seen in the chart above, will only continue to skyrocket for Firefox 3.6 and beyond. By the time Firefox 4.0 is released, sometime in December 2010, the number of developers will be nearly 4,000, almost a full magnitude greater than the optimal 445 or so in 2006. Clearly, Firefox is about to capsize.
So what is to be done? Users can petition the Mozilla Corporation and the Mozilla Foundation to rethink their development model, focus on optimization instead of new features, and perhaps backpedaling on some of the less sensible projects like Mozilla Mobile and the non-standard XUL interface. Concerned individuals should log into Mozilla's Bugzilla and let loose with their bug and crash reports like never before.
Unless Brendan Eich and Mitchell Baker take their heads out of their asses, however, the best course of action is to escape Firefox like rats from a sinking ship. There are other options out there: Apple's small, fast, and efficient Safari, coded by several dozen professional programmers, is currently the best browser for Mac and Windows. The time-honored Internet Explorer continues to embrace and extend Web standards. Other browsers like Chrome, Opera, and Lynx are out there too but aren't for everyone.
In the end, Mozilla Firefox as it stands is a sick browser that is in need of emergency surgery not ready to take on the challenges of Web 2.0 and things like CSS 3, HTML5, and JavaScript 1.9. Unless something happens soon, Firefox will take the entire World Wide Web—and everyone who depends on it—back to the Stone Age of the Internet.
...and I don't give a **** for security. I am working as freelancer. As such there a two possibilities: I calculate correctly including all costs for proper design and tests, or I get the contract. Customers pay ****, customers want ****, customers get ****.
Back when I was in charge of hiring new programmers for a web development shop, the very first thing I'd do when I got a resume would be to load up the applicant's personal website, if he had one.
No, I didn't look at the aesthetics of the site. I didn't care about the cleanliness of the HTML. The implementation language and web framework didn't matter. I had more important things on my mind: I would find a form, and type hello world' -- ; SHOW TABLES. If the site misbehaved, I'd toss the resume in the trash and adamantly refuse to reconsider it.
Management thought I was nuts --- these were guys with degrees! They came with great recommendations! And they're cheap! What does one bug matter? But with SQL injection being the now #2 security vulnerability, who's whining now?
Attention to correctness is the bedrock trait of a good developer. Everything else comes second; security is just one property of correct code.
"Conventional wisdom" depends on who you ask. The convention wisdom I've heard is that OSS is actually more secure. More eyes, etc. The flip side of his analysis is that while OSS was no more vulnerable than closed source it was also no less vulnerable, which would suggest the closed source model is equally capable of producing secure code.
You can check out the full report online from the Veracode.com website (requires registration).
We disclose the sample size in the appendix (1591 applications).
You can test the quality of code you are developing yourself with a simple source code scanner, but testing third party code is a little more challenging. I don't know too many significant applications that are entirely created in house, with no dependency on third party libraries.
Disclaimer: I work for Veracode and was a coauthor of the study.
If you take a look at the full report (registration required), you'll see that the application pool from which the report was drawn was 47% Java, 31% C/C++ (on Windows, Red Hat Linux, and Solaris), and 22% .NET. Other data is provided (industry, supplier type) to help frame the terms of the application pool from which the data was drawn. We acknowledge the inherent selection bias (the applications in the report come from our customers) in the methodology section.
Disclaimer: I work for Veracode and was a co-author of the report.
Over Half of Software Fails First Security Tests. Well that good, now i want the second half to be tested to.
Even though the "conventional wisdom" is that the science of programming has entirely changed to consider security issues from end to end, in reality this does not appear to be the case at all.
I think this is a very interesting and valuable insight. The people doing the talking have completely sold everyone on a vision in which the coder keeps security in mind from the get-go, but the people doing the, uh ... doing ... are doing things the way they have always done them, and tacking on the security piece after the fact.
Is it that programmers in general simply believe that buyers are unreasonably paranoid? Or is it that planning for security throughout the process is too costly and time-consuming?
---don't make me break out my red pen.
FYI, the system-wide registry hive (HKEY_LOCAL_MACHINE, or HKLM) is stored in a different file than the per-user registry hives (HKEY_CURRENT_USER, or HKCU, is stored under the user profile directory).
The complaint that each registry hive is all in one file is still somewhat valid (I kind of like having a centralized tool for changing configuration settings, which is a lot easier when you have fewer files containing the settings) but only on Win9x was the entire registry stored in a single location.
Additionally, individual registry keys have their own ACLs. It's actually much finer-grained security than the Unix-like way of doing things. The defaults are that stuff in HKLM is world-readable and Admin-writable, and keys within HKCU are read/write by you (and Admin) only, but it's all configurable.
There's no place I could be, since I've found Serenity...
I know you meant it in jest, but there's a serious point there.
First, having written a number of small utilities, and some larger GUI-based tools, I have to say that web app design is fundamentally more difficult primarily because you have a number of specific challenges in this area that only apply to them.
Not only do you have to deal with all the usual problems, but you also have to deal with XSRF, XSS, and so forth. This is because you are you have a program which is generating HTML code as output, not merely a nice UI that is the product of the code, and this is happening in a stateless environment so information is somewhat limited in the interaction well beyond what it would be normally. Furthermore, authentication is more difficult in a stateless environment, especially where multi-tier systems are involved.
Most people don't appreciate how easy these are to mess up and how hard they are to fix when a developer creates a project without adequate thought to security in the first place.....
LedgerSMB: Open source Accounting/ERP
It seems to me that a through test would initially flunk pretty close to 100% of all software. So this means Veracode is too lenient by about 40%.
Last, but not least :
4) Open source development has access to lots of readily available building blocks, for free. If a commercial developer want to add some feature to its software, they have to either pay a 3rd party developer (with the whole IP right labyrinth associated with that, in addition to the cost), to go "Not-Invented-Here" syndrome and write it themselves (and we know just how much secure it can be to re-invent the wheel, rather than getting a tested and proved version), or just plain skip the feature.
Security is increase because :
a) re-writing the thing on several time is a waste of resource, better spend the same time perfecting what is already here.
There's a lot of application which implement some kind of HTML parser for layout presentation. It's often buggy and has sometime been exploited (I remember ICQ's code for displaying ads, for example).
The open source world has Gecko and WebKit, which have been perfected for years and thus contain a lot less vulnerabilities.
b) the components are freely available and easy to get as much as they are easy to update.
The Linux world has seen some major security problems in the past. Fixing it was a snap. Simply patch the library. Every project depending on it can freely get an upgrade. Distro simply install a patched version and every software depending on it auto-magically benefits it.
Windows world share a lot less code. When the "bug in image decompressor" (I think it was WMF... not sure), one had to track every last software which might use its own copy of the library. Less shared code : more difficult to fix.
c) Ready-to-use blocks for security.
Whereas small closed source project might resort to half-assed security modules, most opensource project simpy count on SSH-Tunnels (front-end for torrent downloader, for example) or on GNUTLS / NSS+NSPR.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
The conventional wisdom is that open source is risky.
No. The conventional wisdom is, that open source is much much safer!
Who wrote this? Some PHBtard?
Any sufficiently advanced intelligence is indistinguishable from stupidity.