Slashdot Mirror
Study Confirms the Government Produces the Buggiest Software
Sparrowvsrevolution writes in with a link to a Forbes story about the lackluster code produced by government agencies."Humans aren't very good at writing secure code. But they're worst at it when they're paid to do it for the U.S. government, according to a study that will be presented at the Black Hat Europe security conference in Amsterdam later this week. Chris Wysopal, chief technology officer of bug-hunting firm Veracode plans to give a talk breaking down a vulnerability analysis of 9,910 software applications over the second half of 2010 and 2011. Government-built applications came out far worse than those created by the commercial software industry or the finance industry. Only 16% of government web applications were secure by OWASP standards, compared with 24% of finance industry software and 28% of commercial software. By SANS standards, only 18% of government apps passed, compared with 28% of finance industry apps and 34% of commercial software. Wysopal and others blame the difference on a lack of accountability of federal contract developers, who aren't held to security standards and are even paid extra to fix their bugs after creating them."
And you thought it was Microsoft...
Duh.
I once took an excursion to Reddit, and later HN. Unlimited up/down voting sucks when dealing with a hive-mind.
Unfortunately, all the outsourcing going on in the Government (because it's easier to get money for a contract than to hire a developer on a permanent basis) is what's really killing the code here. Most outsourcing firms have a "throw the code over the wall" attitude, and spend more time deflecting blame for bugs than trying to fix them. I can't think of a business where there's less accountability than Government contracting, except possibly foreclosure management....
I work for a government agency and I can swear this to be the absolute truth. I believe the reason to be a lot of politicking in management and not enough actual IT experience. No one wants to step on toes or else it might come back to bite you later when you need funds for a project so when user X asks for feature Y in software Z and there is no way it can be implemented without hacking together a mess of SQL query strings that may or may not work, well then you do it, because if you don't do it. User X may at one point be on a committee that can divert funds from your server or software upgrade budget.
I got here through a series of tubes
This program has performed an illegal operation and must be shut down.
If the problem persists, contact the program vendor.
I was a software tester for the DoD and can confirm the stupidity here. (I can't really talk about the exact program but I can tell you with 100% certainty that it was mission critical.) We were contracted to run massive amounts of automated testing on the latest build of the software I was working on. Upon finding bugs, we needed to do regression testing... to decide if we would fix them in the latest build, because if they were present in previous versions we were under no obligation to do so unless specifically paid to do so.
There are industry-common metrics for good code.
With its focus on long-term outcomes, big budgets, and relatively stable personnel it seems to me non-outsourced government work would tend to produce better code.
Part of the government wrote the code for the space shuttle, the most bug-free program ever written. Seriously, look it up, that code is amazing.
The problem with these specific problems isn't with government but with improper requirements and possibly graft. These are much easier to fix on a local level than bad code in my not so humble opinion.
If video games influenced behavior the Pac Man generation would be eating pills and running away from their problems.
Obvious loop hole...
"and are even paid extra to fix their bugs after creating them."
"If any question why we died, Tell them because our fathers lied."
The rules for government aquisition don't help. As there isn't any usefull formal metric for software quality, it normaly must settle with the cheapest competitor whoever it is, however it works.
Rethinking email
Reminds me of this Dilbert comic
That institutional insecurity, says Alan Paller, researcher director of the SANS Institute, is the result of a private contractor system that actually rewards insecure coding. âoeThe consequences for private sector software writers who write insecure code in commercial software is high costs for patching along with substantial embarrassment for their companies and job insecurity for them,â he says. âoeIn contrast, the consequences for private sector software writers who write insecure code for the government is contract add-ons to fix the problem, and more revenue for their companies and job security for them.â
Palm trees and 8
Spokesman of the German Home Office (BMI, in charge of the "Federal Trojan Horse" exposed by the CCC) at the Federal Press Conference 2011-10-12.
My first job was doing software for the federal government (mainly tools for tracking government property and assets and the nightmare of paperwork associated with them). It was _horrible_. It was a well paying job, great benefits, very relaxed (but political) environment .. but the atrocities committed to the art of software development combined with the painfully slow pace were unbearable.
Everything was always caught up in red tape. Requirements were always wrong, outdated, or both. In a lot of cases there was no clear time frame or end plan for software .. that shit would get figured out when/if the project was finished. And projects were often arbitrarily cancelled. Stuff that was finished and deployed often went unused for various reasons.
There was also an anti-change culture. Anything new was met with extreme resistance. Also there was this feeling that anything that improved our efficiency would decrease our staff. It wasn't entirely unfounded. Stuff was scheduled to take a certain amount of time. If it took less time, it wasn't like there was more work to fill in the holes.
And the code. Wow. I think it's part because the federal government has a lot of co-op/student programs .. and part because most programmers that care about software quality got the hell out of there (like I did) .. but the code that came out of my department was.. terrifying. Thankfully these were internal apps. Database queries involving multiple tables can be complicated.. but it's easy to put the same data in multiple tables! So just create multiple tables with the various data sets you need! Suggest a database view (at least half way to an ok solution) .. the DBA (yes, they had a DBA, a professional database guy, and he allowed this!) won't allow it (he won't reject it, he just offers to "look into it", which if you don't know is office politics for "nope!").
Ok I'm gonna stop and let my blood pressure come back down...
We're a heck of a lot better at writing secure code than the other animals
OK. So government contractor produce the shittiest code, due to a lack of accountability. However, the 34% rating for commercial software is absolutely horrible and inexcusable. Commercial software is almost twice as good, but twice as good shit is still shit.
In some other countries, government employees are smart and work hard. In South Korea for instance the gov't software all run on IE6 activeX plugins and are rock-solid.
pretty good for government work.
do you know what's lacking in the scenario you outlined, here, let me post again for those who weren't paying attention:
-long term outcomes = by the the time they're done, shit can the whole mess.
-big budgets = everyone is fat and happy, $800 wrenches, and $1500 toilet seats for everyone.
-relatively stable personnel = people's ass rooted to chairs. it's so fucking stable, you'll be hard pressed to find a pulse, much less a sense of urgency. ...it's not about finding the one thing that a government employee did well. One can always find a few examples of something done well.
But even in those cases, that are so rare, where the government did something well, it always falls on it's face when you factor in "how long did it take" (always wayyy too long), and "at what cost" (the costs are always ass raping with a drumstick. sideways.)
Wow, I just found out that I'm better at something than the government.
Actually, the Government had just about nothing to do with the Space Shuttle code.
The group that did it was founded by IBM and has been passed around to a number of other vendors (I believe they have ended up at LockMart).
I'm not sure if this supports or discourages your point.
myke
Mimetics Inc. Twitter
1. Much of government is custom software. In the private world less so. Not that there aren't exceptions in either case, but my bank didn't write their own custom software for finances. In government it's almost always build over buy. It's much harder for the government to change policies to fit software when much of what they are writing software for is dictated by legislation.
2. Much of government software is written last minute to meet the demands of the people we've elected that in turn force government agencies to create something from nothing, usually without proper funding and usually with unrealistic deadlines.
3. Much of government software is written by inexperienced people. Contractors and government employees are rehashed from project to project even as technology changes.
I've worked public and private for 15 years now in tech and have seen it all. DoD, Federal, and State projects from both sides of the contract/public servant side. A lot of government software is written in locations with smaller workforces leading to hiring people that are just the best you can get, now what you should get. The deadlines for government projects are almost always unrealistic. The powers that be, and I mean the legislature at the state level and agency heads in Federal, and the commanders/Washington in DoD work, don't feel like there's a ROI on almost any project, it's just stuff they "have" to do, so they don't take into account doing it right. They shoestring a budget, or don't even have a budget, and use whatever resources they can find to get things done.
Most projects aren't even contracted out completely. Many are sure. But I'd say more are a mixture of public workforce and contract or just done but the public servants at hand already. And yes, the contracted out ones are usually the worse IMO because the reason they got the contract is they "knew" the right person and it's a milking of taxpayer money. I've taken over for two projects completely outsourced to very large multi-national contracting firms whose names everyone would recognize. Both were over 70 million contracts. And both were complete crap. The systems were disgusting. We didn't even get printed binders for taking over the maintenance on either. We got some word docs in a network folder, the documents created "after" development was completed. A hodgepodge of technologies and some really bad code. For 70+ million you'd think you'd at least get a Tech Writer on the project and some bound color copies from Kinkos. Nope.
Remember, if the government were to hire it's own developers to make government systems that would be communism and if those developers were allowed to unionize and put their foot down to halt a known bad release until it was correct and secure, that would be ultra-communism
Snowden and Manning are heroes.
I don't know about the US government, but I see things with my local government like: You need a degree in computer science to build a webpage. CS grads are generally not skilled in producing such works, so you end up either picking from the limited group of people who are, or more likely you choose someone who is not a good fit for the job and end up with poor results.
Part of the government wrote the code for the space shuttle, the most bug-free program ever written. Seriously, look it up, that code is amazing.
That's it! Get rid of those nasty high level languages and get back to the bare metal with assembly. None of this new fangled junky stuff.
Kids these days. Never learn anything from their betters.
Faster! Faster! Faster would be better!
Other contributors have mentioned good reasons for government code being less secure. I'm going to give another interpretation: sample bias.
How do you get a representative sample of industry code or government code? It's just not possible.
Ah yes, blame the contractors for everything. Ignore the fact that not that long ago, one of the highest ranking federal IT officers came out and basically confessed that most federal IT projects fail because most of the federal government doesn't even remotely handle requirements gathering properly. That's a nice way of saying they have a lot of money and don't know what they want to do with it, but damned if they're not going to blow it anyway before firmly figuring out what they need.
There is a useful formal metric, maintenance cost. The lowest price including both development and maintenance for the entire anticipated lifespan of the product should be used, not just the development rate. Unfortunantly, that's too rational, odds are the not yet approved metric for software quality will be something like 'whatever uses more RAM.'
Government web applications are generally intended to Provide public information. 16% secure
Finance industry web applications are intended to transmit money the fact. 24% secure. The fact that this is less than twice as secure as web application that are generally informational only is frightening as heck.
Commercial applications are a mix of the two and come out to 28%. Strange.
Of course now the government is switching to agile/scrum (as opposed to the prior methodology of OMFGRAD) en masse so that requirements are gathered on the fly/after the fact and collected on sticky notes and discussed for 10 minutes a day. Because hell, if you can't get good requirements might as well have a methodology that minimizes the need for them.
Of course, considering almost all government software is dictated by business logic and legislation and often rely on existing legacy systems that can't be easily changed, I don't think it's exactly wise. I gag every time the cafe-latte sipping PM's gush about switching over toe scrum on another project so I can spend twice as long building software because my requirements are even worse now. But hey, it has a catchy name, it must be good for government work. We're all so grown up now.
It's not like a can get a high level requirement that I need to capture user information and go build a user screen in the government world. Every freaking little detail is going to be exacted upon on a user screen with rules and laws (and legacy systems) dictating what I can and can't do what is and isn't there and how it interacts with other systems. It's not that agile/scrum is always bad. It's just a square peg in a round hole of current government in most cases.
On a more serious note, I wonder if this is related to all the disparate contractors actually writing the programs?
At least not 100%. You can blame the many headed beast they have to answer to. With every dept. head feeling they have to justify their existence by exerting power in the form of conflicting demands. Also add to this rule sets for compliance that are decentralized and NOT overseen by people who know how to program (generally) but instead know how to be gov't administrators. So compliance will often mean having internal conflicts as to what an application can do.
This is really about government controls run a muck.
from management and the customers. Unit testing, regression testing, QA, code reviews, proper design, refactoring once problems are found. These are all discouraged because "the customer wants it now" or "it's not in the contract". Also as others have mentioned, the hiring of essentially coding mercenaries (contractors) that do not stay with the company long-term and have no accountability or interest in long-term quality. Plus, it's cheaper (on paper at least) to hire a contractor at an hourly wage than an actual full-time software engineer.
This is partially due to the Cost + Fixed Fee model which encourages companies to reinvent the wheel instead of developing and using stable, tested code bases. Spending more money grows your revenue and your company, being efficient and spending less money limits the growth potential of your company. The second major reason is they are married to outdated development practices like the Waterfall model and generally answering new/potentially better ideas with "No, that's not the way its done."
Why is it such a hurdle to get software certified for government use when the government can't even write merely competent software?
Well, duh. Thanks to the rules (lobbied for by we-all-know-who), most government contracts are cost-plus: the contractor's paid the bid plus any additional costs that come up after the bid. With those terms the best approach is to low-ball the bid to insure you win it, do a shoddy job and then bill any time needed to fix the problems as an additional cost.
The fix is obvious: eliminate cost-plus. Make the standard terms fixed-cost: the contractor's paid the amount bid to deliver the goods as specified in the bid, and if they don't meet spec the contractor has to eat any costs needed to correct the deficiency. The only costs that're paid beyond the bid are those for changes the government requested after the bid was placed, and those costs are fixed before the changes are started. But the lobbyists for the big government contractors will fight that tooth and nail.
And to think that 90% of the software created by the government is actually created by commercial companies.. so the study is actually bogus...
Maybe the government just needs to add, "Must conform to X standard." in their contracts. If no standard is specified in the contract the contractor has no obligation to conform. This is an oversight by the government, not the contractor. What we need are knowledgeable people writing the contracts, not some bureaucrats buddy who donated some money and only knows how to write Hello World to a console..
/me uninstalls SELinux.
There are a couple of assumptions here which are actually validated. One is that all government code is outsourced, actually its not.
The fundamental problem here is that government agencies especially can't tell shit from suger in terms of the quality of their software.
Industry can't tell them either, I know from experience that Government already spends enormous sums of money on processes which are meant to make software better, guess what? It doesn't work.
Government agencies (and staff promotions) are measured by inputs rather than outputs, ie how much they spend rather than what they produce, until this changes government software will be shit.
Veracode isn't offering to fix "buggy" code, they're selling "cyber terror" fixes.
Conceived by unaccountable committees. No independent cost/benefit analysis, no human factors or usability analysis. No testing prior to rollout. Spotty or no technical support. No GAAP accounting to see if the software saved money or accomplished stated goals. No money back guarantee and many of the developers leave the company every four years.
Please do not read this sig. Thank you.
It's likely that the percentage of outsourced projects tracks the prevalence of security problems. Certainly, the government has a very high level of outsourced vs in-house development. I think that financial institutions also tend to largely outsource (especially customer-facing) development.
None of those percentages 16-28% sound very good to me.
The reason the govt is the worst is what others mentioned--too many cooks spoiling the broth.
The Department of Homeland Security consists of 22 separate Agencies that report to 88 different Congressional committees.That last sentence is a problem statement in my opinion.
Our (Dutch) police have a new computer system that is so bad the consensus is now to abandon it and start again from scratch. It has cost lots of hundreds of millions of euros.
-- Cheers!
This isn't a study.
This is a press release declaring that everyone who is not already their client has a desperate need for Veracode's services. No different than when Norton sends out a "study" that shows how terribly dangerous the internet is or how much malware exists for smartphones.
This just sounds like they're angling to get themselves some more government business. And you know, kudos for them.
It breaks my pluginses, my precious!
Goverment are faget asshole too busy sucking gay faget cock to write good codes. We need to get rid of goverment and set up constitutional anarchy and send all the fagets away to France or some other faget country.
You'd be surprised how much software from all business models is written by queer folk! Microsoft actually lobbies the state of Washington for gender-neutral marriage so that they can poach more gay programmers. Google does the same. Your OS, browser and phone were probably designed by fagets. The field of computer science was founded by Alan Turing, an internationally infamous faget. Face it dude, queers are too smart and useful, you'll never get rid of us.
Contractors don't care about that stuff. They're there to build something as quickly as possible that meets the minimum spec so they get paid. Bugs just mean future work.
-- "So they told me that using the download page to download something was not something they anticipated." - Bill Gates
since when? buggy = insecure, I might agree with, but the fact that gov code did not score high on some arbitrary web security standards (not the ones that the contractors were being paid to implement, does not mean the code was buggy.
Governments are prone to several problems that cause serious problems with program quality. Speaking as a government programmer, starting with the biggest problem:
1. Consultants. Anytime you have someone external come in and build the code, they don't know anything about the business. They build whatever the spec is (hopefully). In my experience in government, specs are usually very bad at explaining what the users actually need. You need to understand the business in question pretty well to do that. Someone who actually understands how Environmental Inspection & Enforcement is done will be able to write a better program to do it then some guy who is just reading what a word doc says to build, because the first person has the knowledge to know when the spec is wrong.
And that's on a good day. Then you get the consultants who use crappy obsolete technology to throw stuff out quickly, hoping to get more money to fix it later. It won't integrate with anything else, because other consultants did that. When it needs changes because of legislative changes to the business put in by the politicians, nobody is going to know how to change it. It's an expensive and ineffecient model.
Whenever you hear about a $300 million system that didn't work, the odds are good a lot of consultants were involved.
2. Scope creep. Governments are infamous for this. They say we're going to do X. Then another branch jumps in later, now we're doing Y. Then another one. Oh, then there is a department merger and we need it to also work for some other department. Then there's an election and the priorities all changed. Good luck keeping up with that. It's made even worse if you're trying to do the project as one giant release that's all things to all people.
What governments need to do in order to deal with this inevitable problem is split projects up into phases and deliver smaller pieces. It's a lot better to get the first piece out there and in use in a relatively short timeframe then it is to try and build the entire mega-solution at once.
3. It's easy for government employees to become insular, because government is different from the rest of the industry. It's a trap to fall into where you don't keep up with what's new and changing just because you don't particularly need to. Given enough time, skills can become lax and obsolete. It's something that can be dealt with, but employees have to be encouraged to keep learning.
-- "So they told me that using the download page to download something was not something they anticipated." - Bill Gates
where the p[people don't realize the 'government' isn't one group, one set of funds, or one set of guideline.
Oh wait? they sell a product? and don't release the details of said study? Maybe they do understand why a general grab at the generic label 'Government' makes for a misleading results.
The Kruger Dunning explains most post on
No it isn't. Please stop spreading your crap on everyone's toast.
The Kruger Dunning explains most post on
That speaks volumes to your CS grads....
The Kruger Dunning explains most post on
and are even paid extra to fix their bugs after creating them.
this is the standard procedure when you, the client, approve work without kicking the tires. obviously there are some exceptional situations, and some legalese is required. if you don't want to pay for bugs to get fixed, either don't approve them or get a warranty. nobody writes bug-free code, that's a myth inherent in non-programmers. but on the whole, if you buy something without warranty, you get what you get. that's the whole point of charging extra for warranties.
if i'm a landscaper and i plant the trees you want me to plant, and later you find out they give you allergies, i'm not pulling the trees out for free. nor will i take shit off you for not knowing you're allergic. there are other analogies you could use to try to disprove me. my analogy doesn't prove me correct, but i believe i am and it helps illustrate why.
insensitive clod overlords obligatory xkcd car analogy russian reversals whoosh pedant fanbois ftfy in 3...2...1..PROFIT
Concludes, not confirms. Studies do not confirm anything (unless done by Netcraft).
"Ignorance more frequently begets confidence than does knowledge"
- Charles Darwin
That speaks volumes to your CS grads....
You mean how they aren't all expert web designers (distinct from web developers, i.e. programmers)? I have a computer science degree, and web design is far from a strength for me. I can create a web page that someone gives me a drawing for, but creating that drawing isn't something that all, or even most, computer science graduates are good at.
The slight difference in percentages really surprised me. Somehow I expected better of the private sector.
I have seen quite a few "Government" built software applications. 40% of the time it is a single person or kid just out of college with no or not much experience that works in that group and was asked to create it because they have programming experience on their resume. Another 40% of the time it is someone who bid low on it and is frantically posting to forums and groups asking for help with their work because they really don't know what they are doing. With these first two you can also add in that the application was created in DOS and now runs in a emulator on a winME system.
It looks like the remaining 15-20% may actually know what they are doing.
Being in the defense industry, it matters how the software works. It has to perform well, be easy to use, and THEN costs are considered. Producing bad software also means no future contracts. The government doesn't throw contracts to anyone who says they can make something cheaply, they have to show competence, produce a cost schedule, and a detailed architecture plan. The government has lots of standards for software, depending on the area the software covers. The software industry also has standards that the government uses.
Bottom line, the government picks contracts based on performance, quality, and cost. Stop stereotyping contractors into a big group of slackers that produce crap.
That's not true. There are poor developers in contracting, but most of the time the failure is within the communication between the people who will be the end-users and the people who will be implementing products. Developers can't anticipate every use of a software.
Brian Fundakowski Feldman
Because as anyone who's ever had to engage the government on any outsourcing or code maintenance deal knows that the government's list of security 'requirements', application testing criteria, security and encryption demands, documentation standards, SLA's and standards compliance stretches from here to the fucking moon making regulatory and audit compliance insanely expensive and almost guaranteed to fail.
I would be quite sad to be a CS graduate that came out of the program skilled in a specific technology as a result of the schooling. Passing knowledge gained through the demonstration of theory, sure, but that is far from expert knowledge of an application-specific domain. That only comes with experience, and you don't want to be wasting your precious school time with that.
Except, once you have experience, you are sought after in hopes you can duplicate what you've done in the past. You are no longer a CS graduate, you are someone who has done great things with technology X, who may or may not be a CS graduate on the side.
To seek a CS graduate, specifically, for a job of application, says that you are looking for a bright mind that is new to the game that you can mould to your specific technology stack. Nothing wrong with that. We continually need new blood in the game so that they can become experts in the future. However, they are not skilled with your specific technology yet. You are going to have trouble if you expect them to hit the ground running.
So, the problem with government hiring, through my observation, is that they often seek a CS graduate when they really want an expert. It should come as no surprise that you are going to find poor results if you ask for the wrong thing in the first place.
I know that Slashdot has just copied the article title, but it seems incorrect:
- The article only seems to discuss security: this is only one class of bug.
- Surely a bug is a mismatch between the requirements and the implementation. If certain security criteria are not required, then it is not a bug if they are not met !
I suggest the title should be more like "Study Confirms The Government Produces The Least Secure Software".
Have you ever seen a mostly-technical person try to design things? Obviously, if they have a clue about UI design they can at least follow web conventions for structure, but....they don't know what isn't ugly:
Example: Illinois Department of Financial & Professional Regulation
Part of the government wrote the code for the space shuttle, the most bug-free program ever written. Seriously, look it up, that code is amazing.
And probably the most expensive software per line of code ever written (that delivered, of course).
Most software shouldn't be that good (at least until we know how to prove software automatically). Better to cure malaria than make perfect timeclock software for the park police.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)