Slashdot Mirror
More Than Coding Errors Behind Bad Software
An anonymous reader writes "SANS' just-released list of the Top 15 most dangerous programming errors obscures the real problem with software development today, argues InfoWeek's Alex Wolfe. In More Than Coding Mistakes At Fault In Bad Software, he lays the blame on PC developers (read: Microsoft) who kicked the time-honored waterfall model to the curb and replaced it not with object-oriented or agile development but with a 'modus operandi of cramming in as many features as possible, and then fixing problems in beta.' He argues that youthful programmers don't know about error-catching and lack a sense of history, suggesting they read Fred Brooks' 'The Mythical Man-Month,' and Gerald Weinberg's 'The Psychology of Computer Programming.'"
I work at Microsoft. We use agile development and almost everybody I know here has read the Mythical Man Month. Get your facts straight before taking cheap shots in story submissions. Thanks.
Most of the teams I've had contact with inside the tools group at MS (in the last four years or so) use SCRUM.
I don't know how widespread that is in other divisions (say the MSN/Live folks or the Microsoft.com teams) but that clever comment in the submission is nothing more than an ignorant cheap shot.
Don't be so twitterish and make up crap about Microsoft. Get your facts straight or you just come across as an idiot.
Web2.0: I love when people Flickr my cuil and digg my boingboing until my google is reddit and I start to yahoo
I find it more than a little amusing that summary mentions waterfall method being a time honed, excellent example of how all software should be made. Here's the history of the none existent waterfall method has to say about it.. Waterfall Method does not exist!
they cost a shit ton of money is what happened.
A project I was on in 2000ish went as follows:
Steppings A0, A1, A2, and A3 were halted in-fab because someone found a critical bug in simulations.
A4-A7 did not work.
B0-B4 did not work B6 did not work
C0-C4 did not work
B5, B7, C5 sorta worked.
The company folded.
That's what a software mentality working on hardware will get you.
Steppings in CPUs are a little different. Often an earlier stepping was functional enough to start the design cycle for Dell HP, et.al. but not ideal. The later steppings start by fixing the deficiencies, then beyond that are likely cost cutting.
-nB
whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
The most common errors: SQL injection, command injection, cleartext transmission of Sensitive Information, etc.
People make mistakes. Software needs to ship, preferably yesterday.
How much would it cost to have perfect software? I happen to have worked in an industry that requires perfect coding. So I can imagine what it would look like if Microsoft tried it.
The debugger would cost half a million dollar per seat (gdb is free). There would be an entire industry dedicated to analyzing your source code and doing all kinds of proofs, coverage, what-if analysis and other stuff that require Ph.Ds to understand the results.
The industry I'm referring to is the chip industry. Hardware designers code pretty much like software developers (except the languages they use are massively parallel, but apart from that, they use the same basic constructs). Hardware companies can't afford a single mistake because once the chip goes to fab, that's it. No patches like software, no version 1.0.1.
It's just not practical. Let the NSA order special versions of Office that cost 10 times the price and ship three years after the consumer version.
But for me, "good enough" is indeed good enough.
-- FairSoftware.net -- work where geeks are their own boss
I worked within the same space about 10 years ago - I was a sysadmin for a group of asic design jockies as well as the firmware and device driver guys and I'm gonna call you on this...
The hardware designers were under the same sorts of pressures, if not more so, than the software guys and I saw many bugs that would end up in the shipping silicon. The general attitude was always "oh! a bug: well the software guys will just have to work around it."
And as for "no patching", well that's also BS, you can patch silicon, it's just rather messy having to have the factory do it post-fab by cutting traces on die.
So much for perfection!
Andy Warhol got it right / Everybody gets the limelight
Andy Warhol got it wrong / Fifteen minutes is too long.
Anybody who has worked on driver code can tell you that most hardware vendors ship an errata with their chip. Some hardware has rather significant erratas, like the SiI 3112 SATA controller that got installed in pretty much every SATA compatible motherboard back before SATA support was properly integrated into Southbridges. The CMD 640 is another example of a chip with an extensive hardware errata.
I read the internet for the articles.
You remember wrong.
The old stuff was always failing and had a bunch of problems...
You are probably thinking about your old DOS PC. If the floppy disk wasn't corrupted thinks in general worked and it worked well. Why because the programmer could fairly easily test all the cases, and allow security bugs, as a buffer overflow can be prevented as it would take to long for the guy to physically type in the data... And it was one application with essentially the full PC at its whim.
If you tried to do computing with Multi-tasking such as apps like Desqview your stability was greatly reduced. It would make you want to wish for Windows 95. Or remember Windows 3.1 stabilty...
Downloading data via the Modem was a hit or miss activity. I remember getting disconnected trying to download Slackware as there was a combination for the zmodem that passed the hangup code to the modem, forcing me to regzip the file over again to get it.
Just because you were doing simple things back then it wasn't because they were better.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
I work at a company that tests aircraft instruments to FAA regs (DO-178b) and I know exactly what you are talking about, but for different reasons. What we are working on basically amounts to full computers (one project even used the PowerPC G5 processor, Alphanumeric keypad, and 3-color LCD Screen), but because of where they are failures and bugs are not an option (for software that qualifies as DO-178b Level A, if it fails there is a very high probability that people will die).
The testing that we do for hardware and software is way beyond any other industry I've heard of (before reading your post) and took me a bit to get used to.
For those of you interested, we do Requirements base testing, all the functionality of a product is covered by requirements detailing exactly what it will and will not do. Once the development is underway we are brought in as testers and we write and execute tests that cover all the possibilities for each requirement. An example is a requirement that states: "Function A will accept a value between 5 and 20". This simple statement results in a ton of testing:
Value In range and Value out-of-range is tested:
Boundry conditions are tested:
Limits of Value Type is tested (assuming unsigned short):
So 10 tests from a simple statement. Once all tests are written we do "Structural Coverage" tests, we instrument the code and run all the tests we've written and make sure that every line of code is covered in a test.
Only once all that passes is that instrument considered ready to go, that takes a LOT of man-hours to do, even in situations where we can script the running of the tests (some situations are 'black box' testing where we just get the instrument itself and have to manually push the buttons to run any tests... takes a Long time!)
This level of testing is making me take a second look at some of the software I've written before starting this job and wincing... spending a few quick min playing around with a few different values really doesn't cut it for testing anymore in my eyes...
DEMETRIUS: Villain, what hast thou done?
AARON: Villain, I have done thy mother.
Shakespeare invents 'your mom'
You. are. fucking. kidding. me., right?
The sources of complexity have changed, but not significantly increased.
When's the last time your code had to:
The horror you described is for Level C certification. Level B has the requirement of testing each implied else. Meaning for every standalone if statement, you have one test that makes sure it does the right thing when the conditions are true, and a second test which makes sure that thing does not happen when the conditions are not true.
Level A certification requires Modified Condition Decision Coverage (MCDC). The means a test case must exist for every possible logical combination of terms for every conditional expression. These are extensions of the structural coverage requirements for Level C which is only statement coverage, every line has to be exercised at least once by a test case.
This sort of exhaustive testing is done because of a very different operating environment from desktop software. Desktops don't have single bit events. A single bit event occurs when a gamma ray strikes a 0 in RAM and turns it into a 1. As a result some places forbid the use of pointers in their code, since a single bit event could have catastrophic consequences. Every function has to be tested to be able to best withstand random garbage input and fail gracefully since you cannot rely on RAM storing values correctly.
However, the medical industry has similar guidelines for things like X-Ray machines and ventilators, which I believe have a very similar set of best practices codified into a certification process. Depending on the certification level, the cost of testing varies. It is VERY expensive to develop software this way since Test Driven Development doesn't cut it. For these industries the testers need to be independent of the developers meaning developers don't write their own unit tests, which in turn leads to more cumbersome software processes.
I still work in safety critical embedded software, but am eternally thankful I no longer have to worry about certification with a mandated software process. I now write software for boats instead of planes and have started doing model based controls design, test driven development, and in general have tailored the software process to best suit the company. The goal is to produce better software faster, in order to respond to changing business needs, not to fill out checklists to pass audits. The FAA process is the most reliable way to the highest quality software possible; it is not efficient, cheap, or even modern.