Slashdot Mirror
When Bugs Aren't Allowed
Coryoth writes "When you're writing software for an air traffic control system, military avionics software, or an authentication system for the NSA, the delivered code can't afford to have bugs. Praxis High Integrity Systems, who were the feature of a recent IEEE article, write exactly that kind of software. In "Correctness by Construction: A Manifesto for High-Integrity Software" developers from Praxis discuss their development method, explaining how they manage such a low defect rate, and how they can still maintain very high developer productivity rates using a more agile development method than the rigid processes usually associated with high-integrity software development."
When you're writing software for an air traffic control system, military avionics software, or an authentication system for the NSA, the delivered code can't afford to have bugs
I've been in this industry for quite some time and let me be the first to say that I wish I could repeat this sentence with a straight face.
There are a huge number of yeast infections in this county. Probably because we're downriver from the bread factory.
The authors contend that there are two kinds of barriers to the adoption of best practices... First, there is often a cultural mindset or awareness barrier... Second, where the need for improvement is acknowledged and considered achievable, there are usually practical barriers to overcome such as how to acquire the necessary capability or expertise, and how to introduce the changes necessary to make the improvements.
No, the reason so much software is buggy is economics. Proprietary software vendors have to compete against other proprietary software vendors. The winners in this Darwinian struggle are the ones who release buggy software, and keep their customers on the upgrade treadmill. Users don't typically make their decisions about what software to buy based on how buggy it is, and often they can't tell how buggy it is, because they can't try it out without buying it. Some small fraction of users may go out of their way to buy less buggy software, but it's more profitable to ignore those customers.
Find free books.
Luckily, bugs are just fine if you happen to run a company that builds voting machines, such as Diebold. And if you think that elections aren't in the same category as air traffic control, I suggest you take a tour of Iraq. Elections are very important for your continued existance upon the earth.
Electric Monkey Pants
In fact this is the whole point - Praxis manages to deliver software with several orders of magnitude less bugs than is standard in the software industry, but does so in standard industry time frames with developer productivity (over the lifecycle of the project) on par with most non-critical software houses. Praxis does charge more - about 50% above standard software daily rates - but then when you are getting the results in the same time frame with massively less bugs a paying little extra is worth it... you'll likely save money in the support and maintenance cycle!
Jedidiah.
Craft Beer Programming T-shirts
Linux, Firefox, and OpenOffice are some of the best software on the planet. I think is a good practical testament to the OSS philosophy.
And yet they all still suffer from a metric crapload of bugs. Praxis produces software with so few bugs that they are willing to provide a warranty that says they'll fix any bug found within the first 10 years, for free. If their software had the defect rate of Firefox or OpenOffice they'd be bankrupt in short order.
I've always attributed it to people being used to the behavior of MS Windows. And I'm not saying that to start a flamewar. I'm serious. Unreliable avionics systems should be unacceptable, but these days, that doesn't seem to be the case.
Many years ago, I remember reading a quote from an employee at a major aircraft subcontractor along the lines of "If my company paid as much attention to the quality of our work as Microsoft, airplanes would be falling out of the sky on a weekly basis, and people would accept this as normal." I've heard many people, even programmers, claim that bugfree programs are impossible to write. They are not- they just cost far more in time and money than most companies can afford in this commercial climate. When success depends largely on being first to market and bugs and crashes are accepted as a normal fact of life, then they always will be a normal fact of life.
Unfortunately, I think the blame lies at least in large part with the consumer. As long as people put up with programming errors in a $500 software suite that they would never accept in an $80 DVD player, we will continue to have these problems. Unfortunately, too many people still consider computers to be too much black magic that is out side of their (or anyone else's) grasp. Most people have little to know knowledge of how their car works under the hood, but they still believe that the engineer who designed it has enough knowledge to do it without making mistakes and expect the manufacturer to pay for those mistakes when they happen. Why should they believe any differently about the people who write the software they use?
If I don't put anything here, will anyone recognize me anymore?
TFA cites a particular NSA biometric identification program which has "0.00" errors per KSLOC.
Now, this got me thinking. It is completely possible for a biometric identification program to identify two different individuals as the same person (like identical twins), or for it give a false negative identification (dirt on a lense, etc). Is this a bug? The code is perfect: no memory leaks, the thing never halts or crashes or segfaults, all the functions return what they should given what they are.
I think the popular definition of "bug" tends to catch too many fish, in that it seems to include all the behaviors a computer has when the user "didn't expect that output," what a more technical person might call a "misfeature." TFA outlines a working pattern to avoid coding errors, not user interface burps -- like for example, giving a yes/no result for a biometric scan, when in fact it's a question of probabilities and the operator might need to know the probabilities. Such omissions (the end user would call this a 'bug'), are solved thru good QA and beta-testing, but TFA makes no mention of either of these things, and seems to think that good coding is the art of making sure you never dereference a pointer after free()'ing it. It does mention formal specification, but that is only half the job, and alot of problems only become clear when you have the running app infront of you.
Discussion of TFA has its place, but it promises zero-defect programming, which is impossible without working with the users.
Don't blame me, I voted for Baltar.
So, if this toolset and methodology are so good, I have to wonder why it does not get more widespread use? According to their info, it is developed in the 70's and 80's, so that's not new. And why are softwares so buggy and have such a lousy reputation anyway? Not to start a flamewar, but let's just list a few possible "reaons" here:
.... so, are software vendors a bunch of irresponsible kids that need constant monitoring?
1. Why aren't schools teaching this methodoly thoroughly? Why aren't this toolset and programming language taught in school by default? I learned a bit of Ada at school, but that's only part of a comparison between programming language design. So, are schools to be blamed? Or those profs don't know better? Why aren't proper engineering methodologies emphasized?
2. Someone developed a nice methodology, with a nice toolset and programming language, and got greedy and made it too expensive to acquire. Other tools are good enough, and the resulting softwares are acceptable to the market, so, this nice thing never got widespread use.
3. Programmers are asked to do the impossible. We (I include myself here) had to work with customers who don't know what they want, only give very fuzzy requirements (Praxis's customers, from their list, are different kind of animals, and they probably know better than most of the customers we had to work with), and even if we lay out the whole detailed plan in front of them, they still don't know what they want. They will agree to the plan, sign and approve it, and until you have completed the whole system according to the plan, they would ask to redo the whole thing. If a customer dares to ask a civil engineer to add 2 more stories between the 3rd and 4th floor after the custom-built building is finished, guess what would the civil engineer say? Programmers are asked to do this all the time (I know I have been asked to), so are customers to blame? You can't get the system done properly if requirements are shifting all the time.
4. Programmers are a bunch of bozos who know shit about proper engineering. Yeah, I can take the blame, I've been programming for over a decade, and I know how programmers work: methodologies are for pimps! If a bridge engineer can't tell or prove how much load the bridge can take, I'm sure people would tell him/her that s/he has no business in building bridge.
5. Customers of packaged softwares would buy a buggy software to save one buck anyway, why would vendors put extra efforts and costs to make it better? Look at the market, a lot of good softwares didn't survive, and sometimes, the worst of the line prospoered (no naming here!). So people get what they asked for.
6. Customers (even custom-built projects customers) are a bunch of cheap folks, they would go to the least priced, no matter what. Praxis's customers are willing to pay 50% more for quality work, how many of your customers are willing to? We are willing to fix our bugs, free of charge, for the first 10 years too, if our customers are willing to pay 50% than the market rate for quality work. But so far, I've never met one such customer yet. Granted, I don't work in the defense industry. So, don't blame us for lousy work, if customers try to squeeze out every single buck out of it. And in China (and some other countries too), you have to pay a huge amount for kickback too, sometimes, as high as 80% of the project's budget.
7. Software vendors are a bunch of greedy bastards, they put buggy softwares on the market, without having to accept any responsibility (just read your EULA!). Industry problem or government problem? Not enough regulations (for safety, for useability, etc)? Other industries seem to do ok, e.g. medical, civil,
8. The indsutry is developing too fast, people are chasing the coolest, hippiest, most buzzword-sounding technologies. No one gives a shit about "real engineering". And there are simply too much to learn too, in how many industries can you say people are required to master that much technologi
Then their ability to produce bug-free code depends, as usual, on control factors, not on real-world engineering.
In as much as a civil engineer depends on control factors via refusing customers who demand that the building have 6 stories not 4 just one month before construction is due to finish, yes. Real world engineering makes certain demands of the client. Someone who wants to build a treehouse for their kids doesn't consult an architect and a civil engineer, and civil engineers don't take contracts from people who refuse to set out some limits on what they want built, and what they expect of it.
Praxis uses solid engineering. Their "Correct by Construction" approach is solidly grounded in axiomatic mathematics and uses similar sorts of formal calculations and logical and mathematical proofs as you might expect to see from civil, electrical, aerospace, or ny other kind of engineers. Take the time to read sample chapters from the SPARK book to get an idea of exactly what they are doing. There is very definitely quite solid engineering going on.
Craft Beer Programming T-shirts
People differ in ability in every field; the bell curve is real, and only the people who are at the high end of the curve can be considered one of "the best gurus". They will never constitute a large percentage of the group. Ever. Furthermore, there is usually a huge difference in performance between people who are in the top 10% of their field and those who are in the top 0.1% of their field. Most people would consider those in the top 10% as "the best gurus", but really it's only that tiny segment at the very top who deserve the appelation. Even then, you can expect a marked difference between those in the to 0.1% and those in the top 0.01%. Fact of life, folks.