Slashdot Mirror
When Bugs Aren't Allowed
Coryoth writes "When you're writing software for an air traffic control system, military avionics software, or an authentication system for the NSA, the delivered code can't afford to have bugs. Praxis High Integrity Systems, who were the feature of a recent IEEE article, write exactly that kind of software. In "Correctness by Construction: A Manifesto for High-Integrity Software" developers from Praxis discuss their development method, explaining how they manage such a low defect rate, and how they can still maintain very high developer productivity rates using a more agile development method than the rigid processes usually associated with high-integrity software development."
probably helps too :P
You slashdotted stsc.hill.af.mil!
Unpretentious Sydney reviews by unqualified Sydney reviewers
Uh... it's going to be kind of hard for the NSA to do its job without bugs, isn't it?
*rimshot*
When you're writing software for an air traffic control system, military avionics software, or an authentication system for the NSA, the delivered code can't afford to have bugs
I've been in this industry for quite some time and let me be the first to say that I wish I could repeat this sentence with a straight face.
There are a huge number of yeast infections in this county. Probably because we're downriver from the bread factory.
The only method I have seen with almost perfect reliability is where the inputs and outputs are overloaded to handle any datatype and can be proven mathamatically not to crash. I guess a CS degree is still usefull.
The problem is to obtain it you need to write your own libraries and not use ansi or microsoft or any other products as you can not see or trust the source code.
If you can prove through solid design and input and output types that the program wont lose control then your set. Its buffer overflows and flawed design that has not been tested with every concievable input/output that causes most serious bugs in medical and aerospace applications.
However in practice this challenge is a little unpractical when deadlines and interopability with closed source software get in the way.
http://saveie6.com/
Can be proven safe. I wonder what subset of modern OS design could be done in such programming languages.
autopr0n is like, down and stuff.
The authors contend that there are two kinds of barriers to the adoption of best practices... First, there is often a cultural mindset or awareness barrier... Second, where the need for improvement is acknowledged and considered achievable, there are usually practical barriers to overcome such as how to acquire the necessary capability or expertise, and how to introduce the changes necessary to make the improvements.
No, the reason so much software is buggy is economics. Proprietary software vendors have to compete against other proprietary software vendors. The winners in this Darwinian struggle are the ones who release buggy software, and keep their customers on the upgrade treadmill. Users don't typically make their decisions about what software to buy based on how buggy it is, and often they can't tell how buggy it is, because they can't try it out without buying it. Some small fraction of users may go out of their way to buy less buggy software, but it's more profitable to ignore those customers.
Find free books.
Luckily, bugs are just fine if you happen to run a company that builds voting machines, such as Diebold. And if you think that elections aren't in the same category as air traffic control, I suggest you take a tour of Iraq. Elections are very important for your continued existance upon the earth.
Electric Monkey Pants
When you're writing software for an air traffic control system, military avionics software, or an authentication system for the NSA, the delivered code can't afford to have bugs
I've been in this industry for quite some time and let me be the first to say that I wish I could repeat this sentence with a straight face.
That was my first thought, particularly with military avionics. A few years ago they put out a hardware/software update for the ENS system (Enhanced Navigation System) which led to frequent crashing... and it took over a year for them to come out with a message saying that it was a bug and not to waste countless man hours trying to repair it.
It's sort of a new concept, though, as I'd never really seen such problems with traditional avionics systems (non glass-cockpit stuff). I've always attributed it to people being used to the behavior of MS Windows. And I'm not saying that to start a flamewar. I'm serious. Unreliable avionics systems should be unacceptable, but these days, that doesn't seem to be the case.
Ususually when the software and the phrases "life support" or "nuclear weapons" are together in the same sentence.
"I am the king of the Romans, and am superior to rules of grammar!"
-Sigismund, Holy Roman Emperor (1368-1437)
nearly unlimited funding probably helps too
The old technology axiom applies:
High Speed, Low Cost, High Quality.
Pick 2 out of 3.
The theory of relativity doesn't work right in Arkansas.
The Master Money server done by Praxis was done Fixed Price, and with a warranty that says Praxis would fix any bug discovered over the net 10 years -for free-.
How many of you would be willing to place that kind of warranty on YOUR CODE?
dave (who's tried SPARK and liked it a lot, although proofs are much harder than they should be...)
Why can't automatic verifications systems be used for this? You start with an input set and define the output set. Run a program verification system to make sure the outputs are in the output set and don't go out of it?
The inputs or outputs could be infinite but in that case use logical constructs to verify it.
I'm not a researcher or student of this theory. So, maybe someone can illustrate to me why this wouldn't work or be applied to industry?
In the world of software development, there have come to be two defacto models.
1. Get the software out the door ASAP - quite simply, bang out code as fast as possible that meets a loosely defined specification. Then once the product is adopted, parachute help in like no tomorrow to steadily improve the product.
2. Engineer the software - not as a simple as it sounds. This requires that a specification be drawn. A plan be prepared. A team of solid engineers formed and lead by a competent manager. Then, throughout the entire development cycle, test and debug code.
My company does the latter and to do date we have retained 100% of our customers. I'm shocked by the number of developers that approach our company for jobs that don't have the first clue about how to even write a test harness, let alone do any real debugging. Then again, they don't teach much of that stuff in school and it seems that unless your role was specifically in testing at a previous job, that you're not going to have too much experience in that area. Its economics and marketing that put the bugs in software, not computer science.
It's not hard to produce nearly-bugless code when you have both the budget to do proper quality control, and the incentive to do so.
The reason why Windows is not bugless is that they have the budget to properly debug it... but little incentive to do so before launch. The customers will purchase it anyway and gratefully accept bug fixes after the fact. Airports or the military who bought faulty mission-critical software would not be so forgiving.
Freedom: "I won't!"
The main obstacle in writing a decent code is usualy the management - their frequent changes of mind (about what they want - which is usualy different from what is helpful to the users) and their "good enough" and "legacy first" attitude. Overreaching ambition is another problem - one needs to limit himself to fewer things to do them well - and the management pressures usualy run in oposite direction. (Salesmanship bullshit never helps, especialy if it starts to influence the direction of your project.)
I doubt that we will ever figure out - and I suspect that even if we did figure out we couldn't do much about it
Open source model doesn't work for most (99%?) of military and government applications.
So, its wise to spend alot of time discussing methods when you can't just "LET THE WORLD SEE".
Sure the author could throw in a one-liner "Use open source model if you can.", and make you happy, but why bother.
Modesty is one of life's greatest attributes
Oh, even better argument... What if you open source your code and no one looks at it... because no one cares.
Then what?... Yes you do have to learn to write solid code without relying on the rest of the world, it's true.
Modesty is one of life's greatest attributes
That's why systems and platforms like these are written in a tried and true language like JOVIAL.
-- SKYKING, SKYKING, DO NOT ANSWER.
I count less than 400k source code lines among their examples ("SLOC"). Collectively, this is at least an order of magnitude (maybe two or more actually, I don't know) shorter than the really big projects out there. So I guess I have two questions. First, is this rate really good given the size of the projects described? And second, for the huge projects, what sort of bug rates are theoretically achievable?
Linux, Firefox, and OpenOffice are some of the best software on the planet. I think is a good practical testament to the OSS philosophy.
And yet they all still suffer from a metric crapload of bugs. Praxis produces software with so few bugs that they are willing to provide a warranty that says they'll fix any bug found within the first 10 years, for free. If their software had the defect rate of Firefox or OpenOffice they'd be bankrupt in short order.
control structures != Turing complete. You can have loops as long as they have constant maximum bounds. Whatever it happens to be that you mean when you say "Nand is Turing complete" it makes no sense when you actually typed it. "turing-complete (for arithmetic)." makes no sense at all. WTF? Someone failed CS 315.
autopr0n is like, down and stuff.
unlimited risk can be an incentive too.
Professor Middlebrook at caltech was an innovator in an unusual field. Sattelite electronics. Since no repairman was coming they wanted robust electronics. He desigined circuits in which any component could fail as an open or a short and it would remain in spec. You know that's a remarkable achievement if you've ever desinged a circuit before. Notably you can't really do this using SPICE. Speice will tell you what comething does but not how to design it. To do that you need a really good sense of approximations of the mathematical formula a circuit represents to see which components are coupled in which terms. And you need one more trick. The ability to put in a new element bridging any two points and quickly see how it affects the cicuit in the presence of feedback. To do that he invented the "extra element theorem" which allows you to compute this in analytic form from just a couple simple calculations. They still don't teach this in stardard courses yet. You can find it in Vorperians text book, but that's it. If you want to learn it you gotta either go to the original research articles from the 70s.
Some drink at the fountain of knowledge. Others just gargle.
I was at an X windows technical conference many years ago when someone gave a presentation on X with Ada. When the speaker mentioned that it was for an air traffic control application, there was a sharp intake of breath all around the audience, most of whom had flown in for the meeting.
It still doesn't show anything about the quality of the code. I've been on teams that built great systems (like the ones that supply those great maps in google maps) under mil-spec/SEI standards, but the performance and extensibility of that system really was lacking (and those requirements were in the use cases).
Still, knowing some of those guys, they do some quality work.
So Windows is never used in any life or death situations? That is hard to believe.
Code without bugs. Well I never...
Register the editry.
30 LOC is net. You spend the first 45% of a high-reliability project doing the design work, and the last 45% doing the verification. The 10% in the middle is code generation.
These guys seem to be claiming they can reduce redundancy in the design work, and rework in the verification work. They're doing it by using a design-description method that prevents unambiguity (and therefore using a team that is TRAINED to write unambiguous requirements, so their magic language may not be the key), a coding method that avoids unprovable structure (and probably eliminates a lot of other sorts of flexibility), and a verification method that first validates the design and then verifies the code as it's produced (no new value there as everything has to be touched at least once anyway, and if a big bug turns up that causes a lot of code to be redone you have to redo formal verification on those units again; something that's less likely if formal verification is delayed until full-alpha code is demonstrated, having been informally verified along the way).
Their claims of massive error reduction are, at best, anecdotal. Let's see them do this after taking over a half-coded project with minimal design requirements, a hard deadline, and a budget that can be cut by governmental forces at will.
Not that anyone else rtfa, but defects per line of code seems like a bad measurement of how high quality your code is. More lines != more productive.
Then again I've been suspicious of hyped press releases claiming that the government has super efficient ways to write superior code, ever since that mars orbiter crashed due to mistaken units conversion....
"TV is great! Every New Year's I make a resolution to watch more TV." - Ann Coulter
In school, this article was required reading for our class. It addresses the same topic as TFA.
The essential point being there is a trade-off between quality and quantity and each organization/project needs to decide how far they want to lean in either direction.
TFA cites a particular NSA biometric identification program which has "0.00" errors per KSLOC.
Now, this got me thinking. It is completely possible for a biometric identification program to identify two different individuals as the same person (like identical twins), or for it give a false negative identification (dirt on a lense, etc). Is this a bug? The code is perfect: no memory leaks, the thing never halts or crashes or segfaults, all the functions return what they should given what they are.
I think the popular definition of "bug" tends to catch too many fish, in that it seems to include all the behaviors a computer has when the user "didn't expect that output," what a more technical person might call a "misfeature." TFA outlines a working pattern to avoid coding errors, not user interface burps -- like for example, giving a yes/no result for a biometric scan, when in fact it's a question of probabilities and the operator might need to know the probabilities. Such omissions (the end user would call this a 'bug'), are solved thru good QA and beta-testing, but TFA makes no mention of either of these things, and seems to think that good coding is the art of making sure you never dereference a pointer after free()'ing it. It does mention formal specification, but that is only half the job, and alot of problems only become clear when you have the running app infront of you.
Discussion of TFA has its place, but it promises zero-defect programming, which is impossible without working with the users.
Don't blame me, I voted for Baltar.
If someone sent a copy of this to Micro$oft? Would any of them read or comprehend it? It could make a difference in the version after Vista.
Oddly enough it would make perfect sense to some people at MS. The Singularity OS project from MS research uses a lot of the same ideas in development methodology and formalism. Whether Singularity will ever make it out of MS research, or simply remain a curious little side project, is of course an interesting and quite open question. Only time will tell.
For other OSs developed in a similar mold, try Coyotos which, while still getting seriously underway, looks quite promising indeed when it comes to secure and robust OSs.
Jedidiah.
Craft Beer Programming T-shirts
Who was it that said a test system can never be perfect because it must be at least as complex, or more complex than the system being tested?
"No matter where you go, there you are." -- Buckaroo Banzai
Not by anyone intelligent. Anyone that uses a M$ product in a mission critical area is courting catastrophe.
Professional Politicians are not the solution, they ARE the problem.
in practicallity, it does.
/.ers, break it down to an unrealistic and non-practical point. Making your arguement little more then shaky ramblings.
You ned to keep the lines of code measurment in contexts with other issues. Like design, and all the other usual suspects. But that applies to ANY metric you use.
Also, this may help Perl programmers to write script that only does ONE thing per line!
also, just:
assert(1);
won't compile.
Of course you, like many ignorant
The Kruger Dunning explains most post on
Yes, yes, open source projects fix bugs for free. The point is that they can afford to do that, because they have so many volunteers to do the bug-fixes. Praxis doesn't have volunteers, so the only way they can afford to do bug-fixes for free is by having a very low number of bugs.
The site is slashdotted at the moment, so I can't read the article.
A good example of people writing complex but bug-free software under time pressure is the annual ICFP Programming Contest. This contest runs over three days, the tasks are complex enough that you usually need to write 2000 - 3000 lines of code to tackle them, and the very first thing the judges do is to throw corner-cases at the programs in an effort to find bugs. Any incorrect result or crash and you're out of the contest instantly. After that, the winner is generally the highest-performing of the correct programs.
Each year, up to 90% of the entries are eliminated in the first round due to bugs, usually including almost all the programs written in C and C++ and Java. Ocassionally, a C++ program will get through and may do well -- even win, as in 2003 when you didn't actually submit your program but ran it yourself (so it never saw data you didn't have a chance to fix it for). But most of the prize getters year after year seem to use one of three not-yet-mainstream languages:
- Dylan
- Haskell
- OCaml
You can argue about why, and about which of these three is the best, or which of them is more usable by mortals (I pick Dylan), but all of them are very expressive languages with uncluttered code (compared to C++ or Java), completely type-safe, produce fast compiled code, and use garbage collection.
"Windows XP is a third generation product"
You say that like it's a good thing!
The only way Windows will be fixed is to start with a clean sheet of paper, completely re-write the entire code base. There are errors in XP that were unfixed errors in 98. I've seen 2 errors in XP that were identical to ones in 3.1. That is a lack of attention to detail.
Professional Politicians are not the solution, they ARE the problem.
Thanks for stating the obvious. Any Software Engineer with half a soul has the same guidelines.
My motto is: "If you strive for perfection, then the end result will always be better than settling for mediocrity."
"No matter where you go, there you are." -- Buckaroo Banzai
I have some beef to pick with the article: 1. It alleges that CMM5 organizations have about 1 defect/KLOC. Having worked and knowing such organizations, I can anecdotally confirm numbers like these are fiction. CMM5 certification has more to do with greasing palms rather than any absolute defect measurement. 2. A defect rate of 0.04bugs/KLOC is not zero bugs/KLOC. The difference is infinite in magnitude if that single bug is -- kills the user. 3. Low defect rates are more often a product of poor testing, not superior development.
Open source model doesn't work for most (99%?) of military and government applications.
You're wrong. Open source doesn't mean "let the world see". It means "let the people using the code see".
Military people are particularly fond of open (to them) source (or binary objects so simple that a disassembly is readable), just as they're fond of having complete design specs for their artillery. It doesn't mean they tell "Teh Enemy" the source, just as I am under no obligation to disclose the source of modifications I've made to the linux kernel to anyone other than those I give copies of the modified kernel to.
Thinking "Open Source" means "openly downloadable by everyone on the planet" is the #2 mistake I see closed source weenies making. (#1 is thinking open source means anyone on the planet can openly UPload to open source CVS repositories. That is such an idiotic notion, I don't know where to begin with them.)
Anyway the military will completely ignore I"P" laws if it suits them (But hey, I"P" is really bullshit...).
FTA: "For example, C, C++, Structured Query Language (SQL), Ada '83 and Ada '95 have been used. However, such languages are intrinsically unsuitable for deep static analysis and are only ever used for the non-critical parts of the implementation."
What language do they use?
Hey, good point... bring out a case where HUMAN error caused something to go boom. I think if you read the article, you'll see they mean software only, human error is still up to the error between the chair and keyboard.
Want to find other gamers to play board and role playing game
The truth of the matter is, no matter how hard we try, very few people were actually "meant" to code. People don't think like machines. And the few that do are probably working for companies that do high-integrity software that human lives depend on (and Google of course). I think very few people can write complicated bug-free code, and most of the ones that "do" get lucky and get their bugs caught during testing (and don't create more bugs fixing it). Companies are in a hurry. They don't like to devote too much time to QA. Since bugs are an inevitable part of complicated software, it's the companies decision as to how many estimated unknown bugs is considered "stable enough for release". Be it a text interface or some amazing simulation VR system, the more user interactivity and freedom they are given, the harder it will be to create bug-free software. When talking with non-technical folks when they bitch about their computer freaking out, I just tell them "It's easy to make something do what you want it to do. Making it not do what you don't want it to do is a lot more complex". I think the key to minimizing bugs other then proper testing is dividing the program up into as many reasonable simple parts as possible (Yes, I'm a fan of OOP) with strict guidelines for interaction outside of the class/method, and have the overall interactions of each class evaluated by a reasonable number of people so possible problems can be spotted. And I am done ranting.
In undeveloped countries, the consumer controls the market. In capitalist America, the market controls you.
Summary of the article: "We're great. Trust us. Hire us. Pay us lots of money--it may look expensive, but we promise it will be cheaper in the long run. Really."
Comment removed based on user account deletion
Why? It is not just a measure of lines of code per day. A measure of lines of code per day implies a heavy addition non-programming work to ensure that each line does not affect the integrity of the software. So it does make sense, unless you forget that the quality of these 'lines of codes' is nowhere near comparable to the code the average Joe writes.
For he today that sheds his blood with me shall be my brother.
"Yes, yes, open source projects fix bugs for free. The point is that they can afford to do that, because they have so many volunteers to do the bug-fixes."
The Praxis approach starts with the mindset that bugs are bad and shouldn't leave the room. Open-source has much looser requirements. Bugs can be released with the attitude that "a thousand eyes" will catch the mistake. Also unmentioned by OSS advocates is the problems caused during the duration of the bug being unfixed, till those eyes catch it (which isn't guarenteed). Also OSS isn't guarenteed (read the disclaimer, and remember the "/." story awhile back about programmers being held liable for problems with their code. Read the responses) while Praxis is. And last Praxis plays in a field that OSS doesn't. (mission/life critical)
" Open source model doesn't work for most (99%?) of military and government applications."
care to explain what you mean, and why?
I believe that opening it up can be one step in a process towards 0 bugs.
Obviously, if no one looks at it you get no benefit. OTOH just a couple of good programmers with an interest can have a very high return.
The Kruger Dunning explains most post on
Which is precisely the kind of project they avoid, because it's a disaster waiting to happen. Praxis caters to customers who want quality, and are willing to do what it takes to get it.
" If their software had the defect rate of Firefox or OpenOffice they'd be bankrupt in short order."
yes, but you are trying to get orples
Firefox and openoffice are continous works in progress, as opposed to a project with a firm set of goals.
I doubt Praxis could make the same offer if customers changes the requirements, and feature list all the time.
Praxis also has certian requirements about the enviroment in which they work.
If I took a Solaris 9.0 project, and tried to run it on linux 1.0, I don't think Praxis would be liable for the bugs that would occur.
The Kruger Dunning explains most post on
Their claims of massive error reduction are, at best, anecdotal. Let's see them do this after taking over a half-coded project with minimal design requirements, a hard deadline, and a budget that can be cut by governmental forces at will.
Their claims of error reduction are based on the development method and a lot of the important stuff happens very early on, taking over a half finished project that failed to follow such a method is of course not going to work. They can't make existing code bug free, but they can write new code that has vastly less errors than most software. As to hard deadlines and budgets - as far as I am aware Praxis already works with deadlines, and apparently their project for Mastercard was done on a fixed flat fee, so working with fixed or limited budgets doesn't appear to be an issue either.
Jedidiah.
Craft Beer Programming T-shirts
Same with Linux, or any other system not developed under stringent requirements.
Rex is 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
Actually, there was a case (in 2000, I believe) where a destroyer (the ship, not robot) was running a version of windows, got a divide by zero error, and the ship's systems all crashed. It was dead in the water for a few hours.
Yes, found a wiki:
Note, it was running Windows NT 4.0. So yes, in military warships we even us Windows. I also believe that Britain's newest destroyer runs on a version, but I forget which, as I read it a while back.
Want to find other gamers to play board and role playing game
The end result - In a year, no one will remember that you were 6 months late - make a buggy release and in a year EVERYONE will remember the buggy release.
Why I always have time to do it over, and never the time to do it right in the first place
I have mod points and I am not afraid to use them
...they're undocumented features.
I could design and implement a bulletproof voting box over a couple of weeks for probably around $10,000 tops, dropping that to $2000 or so in volume production with a healthy profit included. Diebold seems to have problems with this simple task. They also make ATMs, and I bet the banks don't put up with that kind of shit. The only reasonable explanation is intentional malfeasance.
3 days ago I was given insufficient documentation and no testing information for a new invoice process. I told my Project Manager that the specs were not sufficient, I was told to "Do what you can". Over the last 3 days I have wasted a few hours waiting for people to get out of meetings so I could ask them piddly questions about the new process, double tracking over code as new requirements were "discovered", and dealing with a 3rd party database that isn't designed to handle what the users think they want.
Today I sent 2 sample invoices up the the leasing department. They didn't understand why certain items where showing up twice (due to multiple transactions for an asset on an invoice), so they sent down two test cases. I ran both test cases and they both failed to pass the original (vague) business rules I was given.
End result, I wasted 3 days developing an Invioce no one can use because no one could be bothered to come up with a requirements doc.
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
Most customers are business people. Most business people are idiots. Therefore: most customers are idiots. Given that fact, the only thing that can save you from spending months cranking out a specification/contract that will never be fully read, or pounding out garbage that's a nightmare to maintain is to go Agile. I favor XP, but anything where the process includes delivering the most important features using Test Driven Development, small iterations, acceptance tests, etc. will work. Besides, by the time you get something working in front of them, they'll "reprioritize" to the newest Shiny-Thing and you'll be glad not to have produced the 1/2 ream of dead tree UML (that was out of date on day two of "coding"), or the rat's nest in your source tree (mostly written by the guy who quit two monts ago after his wife divorced him for never being home).
*** Sigs are a stupid waste of bandwidth.
This is very interesting and encouraging work. One possible difficulty I would like to point to is that their approach requires specialized skills that most developers do not have. Also, as pointed out by one comment here on Slashdot, one often has to access third party libraries that are not fully reliable. Finally, in a business environment, one will have difficulty selling a highly academic methodology that requires a radically different skill set (e.g., formal analysis, predicate calculus, etc.) than what is available. Their work may be very effective in high-assurance settings that can be very careful about assembling the right team and defining the right process. I can see it being used in military and avionics applications, as they say. For business application environments, it is probably more practical to take a "best practices" approach, and tackle the problem from a methodological perspective, by adding assurance steps to existing maintstream methodologies, rather than by requiring an enrirely different approach for writing requirements. I have just written a book about this by the way. See "High-Assurance Design" on Amazon, or look for the article by myself and Scott Ambler which should be out soon. - Cliff
Wouldn't UML help with engineering? It's designed of this purpose. You can UML anything, and reports have it that UML makes it easier to find bugs, and make deadlines.
You are completely missing the point of CbyC development. One of the fundamentals is that these apps are RESISTANT TO CHANGE. So yes, MS could make a much more solid OS, IF you ran their, and only their proprietary hardware (as Apple used to/still does), only used the pre-installed applications in the exact manor they were designed to be used, and never drempt of changing anything.
When you are talking about an air traffic control system, you can set a very specific set of requirements. The Air Traffic Control system will never have to open an Excel worksheet, or run Quake 4, or be compatible with hundreds of other vendors tools. The Air Traffic Control system will never have to deal with someone swaping graphics cards and updating drivers. It doesn't have to worry about spyware and root kits. It doesn't have to worry about internet access.
If you want to rag on MS, go for it, but don't think CbyC is the answer. It would only result in an OS that you wouldn't want to use. (As a consumer it would be worthless, but it could be great for imbedded systems)
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
Not anyone like, say, the US Navy, for example:
, 13758,00.html
s ID=2275
http://wired-vig.wired.com/news/technology/0,1282
Or air traffic controllers:
http://www.techworld.com/opsys/news/index.cfm?New
Or nuclear power plants:
http://www.securityfocus.com/news/6767
Regardless of how you rate the intelligence of the parties involved in these little incidents I think you'll find that Windows is very often deployed in mission critical areas.
And yes, often with catastrophic consequences. >)
Have fun,
Nathan 'Nato' Uno
http://web.unos.net/
Bugs are allowed, but software "should" be tested and patched before release.
1 bug per 10K lines of code is pretty good, but by no means remarkable. The teams which actually write avionics software for jets routinely reach levels of no more than 3 defects per 1M lines of code.
"I've seen 2 errors in XP that were identical to ones in 3.1."
Shenanigens. Which ones?
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
The Coyotos project attempts to implement a full OS that can be mathematically proven safe and secure. It's actually quite a fascinating project; reading the mailing lists and watching BitC and Coyotos develop feels a bit like what it must've felt like to watch UNIX and C grow up in the 70s.
It's like all the suits who love to say "failure is not an option," but then we see the occasional failure, but people still say "failure is not an option" because it's the attitude they're trying to convey, not the reality. The right attitude will bring about the right reality, or so management would have you believe.
Re-read the article you linked to on the Navy. The Aegis software is from Lockheed Martin, not M$. The Navy is more than capable of screwing up and making itself look worse in a half-assed cover-up, but they don't use M$ products for system controls.
If you want a better comparison, go to the bank. Windows everywhere in the floor, except at the teller stations. Unix or RPG are there with all the servers on Unix or AIX. M$ is fine for typing letters, but never for actually handling accounts or real money.
Professional Politicians are not the solution, they ARE the problem.
I RTFA to the point where they started putting restrictions on languages of choice and then I stopped. I don't disagree. I just realize the article applies less to my work than I thought. My software doesn't need that level of perfection early in the process. However as per usual, some good stuff to take away from the article at a general level and apply to work. Of course the general stuff we've all heard before. It's just applying it that's the hard part at times!
Like the current security bug in image files. Caused by a 'feature' from windows 3.1 To be honest though if you wrote something the size on Windows perfectly it would still seem like it had bugs to a decent portion of users simply by virtue of the number of different system configurations it had to run on. Add to that the fact that humans *always* make mistatkes I would say that for a large software development project it is essentially impossible to be bug free.
I can't speak for the original poster, but I can't believe that the parent poster and I are the *only* people that believe that LOC is a poor metric.
Measuring lines of code added per day causes deletions and modifications to be considered *bad*. If I add 10 lines today and those 10 lines allow me to delete 20 lines from last year then I have a net productivity of -10 LOC and I have been unproductive.
The argument *could* be advanced that if I'd done it right in the first place the original 20 LOC that I'm deleting should have been the 10 LOC I added today so I *should* be penalized for doing poor work last year. I consider this argument to be shortsighted. What if we're interfacing with a new library? Is that a bad thing from a productivity perspective? What if adding a new feature involves rewriting 10 existing lines and interspersing 10 new ones? Is that less productive than tacking on 20 lines today?
Counting lines added, lines changed, and lines modified as a metric is nearly as bad. The reason these metrics fail, IMHO, is that lines of code have no "average" value. Some lines are more valuable, some are less valuable, some have no meaning in the context of others, some have so much value that they cause others to be obsolete, etc. Grouping them together is like measuring the intelligence of a room of people by adding up the number of people present - meaningless.
Even your point, that a single line of code implies a specific amount of background work ("heavy addition non-programming work") is fallacious, IMHO. Does each feature have equal merit? Equal difficulty? Does each line of code imply exactly the same (or even *about* the same) amount of "heavy addition non-programming work"?
This is certainly not true for me - the difficulty of the code I write varies wildly, both within and between applications. I can assure you that I would likely expend much more effort on each line of code for an optimized backend search engine than I would on the CGI for the web interface that drives it. Is it therefore less productive for me to work on the backend because I expend more effort per line of code?
IMHO, LOC is only useful for publishing statistics, not for measuring meaningful changes in productivity.
Have fun,
Nathan 'Nato' Uno
http://web.unos.net/
If operating systems ran airlines:
UNIX Airways: Everyone brings one piece of the plane along when they
come to the airport. They all go out on the runway and put the plane
together piece by piece, arguing non-stop about what kind of plane they
are suposed to be building.
Mac Airlines: All the airline personnel look and act exactly the same.
Every time you ask questions about details you are gently but firmly told
that you don't need to know, don't want to know, and everything will be
done for you without your ever having to know, so just shut up.
Windows Air: The terminal is pretty and colorful, with friendly stewards,
easy baggage check and boarding and a smooth take off. After about 10
minutes in the air the plane explodes with no warning whatsoever.
Windows NT Air: Just like Windows Air, but costs more, and uses much
bigger planes, and takes out all other planes in a 40 mile radius when it
explodes.
Linux Air: Disgruntled employees of all other OS Airlines, (with UNIX
geeks who finally figured out what kind of plane they were suposed to be
building) decide to start their own airline. They build the planes,
ticket counters, and pave the runways themselves. They charge a small fee
to cover the cost of printing the ticket, but you can also download the
ticket and print it yourself. When you board the plane you are given a
seat, four bolts, a wrench, and a copy of the Seat-HOWTO.html. Once
settled, the fully adjusable seat is very comfotable, the plane leaves
and arrives on time without problems, and the in-flight meal is
wonderful. You try to tell the customers of the other airlines about the
great trip, but all they can say is, "You had to what with the seat?"
with apologies to Doc Searls and Linux Journal.
Professional Politicians are not the solution, they ARE the problem.
Or, more likely from reading the article, Praxis can have a very low number of bugs by very tightly constraining the work that they're willing to do.
Anything built using an open model (Linux, Firefox, OpenOffice, etc) is, by definition, open to broad changes in feature sets and functionality. I seriously doubt that Praxis could have developed anything as feature rich as any of those three tools using the same number of man-hours.
The bug rate is a consequence of the development model. In exchange for a high bug rate you get a high feature return rate. There are always tradeoffs in software development - it's a matter of choosing the right tradeoffs for the situation at hand.
Have fun,
Nathan 'Nato' Uno
http://web.unos.net/
Just to add to mix, the specification "z notation" mentioned in the article can be found here
Or just have a UID 1000.
The more you know, the less you understand.
Hey, I'm just saying... if Star Trek has done just about every planet in the known sky and the one you picked blew up spectacularly in a feature film... that might not be a good omen.
Mmost programmers don't have any source of pride to recheck their work. They just write the crap and assume it will work. A long time back I needed to write a machine language checkerboard memory test for a Honeywell H316 minicomputer. I revised the program several times after inspecting my code and managed to get it to run perfect the first time in only 23 lines of code. Check your code many times and your software won't have problems.
You did see the counts of lines of code in Praxis' projects? The header files alone in Linux have more lines of code than all Praxis ever wrote or ever will write.
The *software* might be from Lockheed Martin, but the Navy deployed it, including the Microsoft components delivered by Lockheed Martin, in a production environment. The Navy deployed it, the Navy's ship stopped working (which I think is part of the definition of "system controls" - controls which determine whether or not the ship is working). No sense in blaming Lockheed Martin for that one...
v a-general-features.htm
And if you think Windows isn't used in banking ATMs, which handle both accounts and "real" money, perhaps you should consult with Google:
http://www.universal.com.sa/english/products-opte
http://msnbc.msn.com/id/3675891/
(and there are many others...)
Have fun,
Nathan 'Nato' Uno
http://web.unos.net/
I'll see your compiler error and raise you a thousand pointless #defines!
Nevermind that '9x and NT are completely different code bases, but hey, keep talking like you know what you're talking about - after all, this is /.
I will give you that the article isn't clear that Lockheed Martin's Aegis system deploys Microsoft components. However, I used to work for Lockheed Martin, specifically *on* the Aegis system components, and I probably shouldn't comment further.
, 00.html
So instead I offer this link to the US Navy deploying Microsoft for critical control systems:
http://www.wired.com/news/technology/0,1282,13987
Trust me, they do it.
Have fun,
Nathan 'Nato' Uno
http://web.unos.net/
Not a single trek joke yet, the geek level is really dropping around here!
People who think they know everything really piss off those of us that actually do.
Am I the only one amused by Praxis exploding due to unsafe practices?
/chuckles to self
-volve
One time I was developing a system used by attorneys to manage revenue contracts for a major networking company. They would threaten to terminate my contract if there was a single bug or say something like "I don't think he's taking his job seriously". My manager described the legal team as "unforgiving".
-c0d3r-
I've been a controller for 13 years and have worked in the automation end of things for almost 4 years now. There is NO SUCH THING as bug-free Air Traffic Control software. The best we can hope for is heterogenous redundancy and non-simultaneous failures. Some engineers seriously think they could replace all those controllers with an intelligent algorhythm. What really scares me is that the more they try, the less engaged the people become and the harder it is for them to fall back to manual procedures when the worst happens.
Everyone used to laugh at how Windows NT could only run for 34 days before it needed a reboot. Some of our systems can't run HALF that long without needing a server switch-over or complete cold-start.
That's how much of the system you control. By system I mean the entire channel, including the hardware, software, input, etc. It's much easier to engineer low defect software when you control all of that. For example if you are developing something that runs only on one particular version of one kind of embedded device, and can only have input given to it in a certian way, and you can gaurentee that it is the only thing running at a given time, and so on.
Much harder if you are making something that has to run on a massive set of arbitrary hardware that can have any number of other, quite possibly buggy, apps running and that can recieve all kinds of bad input through all kinds of different channels.
That's part of the problem I see is that people look at systems that are engineered and controlled by one company, and then think that software that runs on comoddity hardware should be as reliable as something where everything is carefully controlled.
Then their ability to produce bug-free code depends, as usual, on control factors, not on real-world engineering.
think of the bugs!!! Seriously where would most of us geek types be without the bugs. What would you blame the missed deadline on then, huh?
So, if this toolset and methodology are so good, I have to wonder why it does not get more widespread use? According to their info, it is developed in the 70's and 80's, so that's not new. And why are softwares so buggy and have such a lousy reputation anyway? Not to start a flamewar, but let's just list a few possible "reaons" here:
.... so, are software vendors a bunch of irresponsible kids that need constant monitoring?
1. Why aren't schools teaching this methodoly thoroughly? Why aren't this toolset and programming language taught in school by default? I learned a bit of Ada at school, but that's only part of a comparison between programming language design. So, are schools to be blamed? Or those profs don't know better? Why aren't proper engineering methodologies emphasized?
2. Someone developed a nice methodology, with a nice toolset and programming language, and got greedy and made it too expensive to acquire. Other tools are good enough, and the resulting softwares are acceptable to the market, so, this nice thing never got widespread use.
3. Programmers are asked to do the impossible. We (I include myself here) had to work with customers who don't know what they want, only give very fuzzy requirements (Praxis's customers, from their list, are different kind of animals, and they probably know better than most of the customers we had to work with), and even if we lay out the whole detailed plan in front of them, they still don't know what they want. They will agree to the plan, sign and approve it, and until you have completed the whole system according to the plan, they would ask to redo the whole thing. If a customer dares to ask a civil engineer to add 2 more stories between the 3rd and 4th floor after the custom-built building is finished, guess what would the civil engineer say? Programmers are asked to do this all the time (I know I have been asked to), so are customers to blame? You can't get the system done properly if requirements are shifting all the time.
4. Programmers are a bunch of bozos who know shit about proper engineering. Yeah, I can take the blame, I've been programming for over a decade, and I know how programmers work: methodologies are for pimps! If a bridge engineer can't tell or prove how much load the bridge can take, I'm sure people would tell him/her that s/he has no business in building bridge.
5. Customers of packaged softwares would buy a buggy software to save one buck anyway, why would vendors put extra efforts and costs to make it better? Look at the market, a lot of good softwares didn't survive, and sometimes, the worst of the line prospoered (no naming here!). So people get what they asked for.
6. Customers (even custom-built projects customers) are a bunch of cheap folks, they would go to the least priced, no matter what. Praxis's customers are willing to pay 50% more for quality work, how many of your customers are willing to? We are willing to fix our bugs, free of charge, for the first 10 years too, if our customers are willing to pay 50% than the market rate for quality work. But so far, I've never met one such customer yet. Granted, I don't work in the defense industry. So, don't blame us for lousy work, if customers try to squeeze out every single buck out of it. And in China (and some other countries too), you have to pay a huge amount for kickback too, sometimes, as high as 80% of the project's budget.
7. Software vendors are a bunch of greedy bastards, they put buggy softwares on the market, without having to accept any responsibility (just read your EULA!). Industry problem or government problem? Not enough regulations (for safety, for useability, etc)? Other industries seem to do ok, e.g. medical, civil,
8. The indsutry is developing too fast, people are chasing the coolest, hippiest, most buzzword-sounding technologies. No one gives a shit about "real engineering". And there are simply too much to learn too, in how many industries can you say people are required to master that much technologi
Still irrelevant, IMHO. They don't list a complexity for the project(s) that they're claiming 30 LOC/day for and I can't think of a reasonable way for them to do so.
If the next project they do ends up at 20 LOC/day is that one less productive? How would anyone know? How would they publish a relative complexity scale to explain the 33% decrease in LOC "productivity"?
Even if you only look at the final result LOC is a poor metric for the same reason it's a poor metric when measured on a daily basis: lines of code have no intrinsic measurable value. I just don't see that they can be used to measure meaningful changes in productivity.
Have fun,
Nathan 'Nato' Uno
http://web.unos.net/
The only way Windows will be fixed is to start with a clean sheet of paper, completely re-write the entire code base. There are errors in XP that were unfixed errors in 98. I've seen 2 errors in XP that were identical to ones in 3.1. That is a lack of attention to detail.
Your right about the problem being history but it isn't necessarily a "lack of attention". My understanding is that a fair amount of programs take advantage of bugs. Simply fixing them would cause problems for users. No one likes it when programs stop working after a patch.
"If you want to learn it you gotta either go to the original research articles from the 70s."
r ial/nEET.pdf
r ial/slidesAppC.pdf
http://www.edn.com/archives/1995/080395/16df4.htm
"The extra element theorem is used for analog circuitry. The gist of it is that you remove the reactive elements ( or dependant sources ) from a circuit and then put them back in through a process of correction factors."
[n-extra element theorem]
http://ece-www.colorado.edu/~ecen5807/course_mate
[Middlebrook's extra element theorem]
http://ece-www.colorado.edu/~ecen5807/course_mate
In the end, software companies are in it for the profits. They have no lemon laws to respect, they have no trades description act to obey, no ombudsmen to answer to, no consumer rights groups to speak of, no Government-imposed standards certification and virtually no significant competition. Customers are often infinitely patient and completely ignorant of what they should be getting - the machines are like Gods and the software salesmen are their High Priests. To question is to be smote.
Were standards to be mandated - perhaps formal methods for design, OR quality certification of the end result, you would see no real impact on net software costs. Starting costs would go up, but long-term costs would go down.
Nor would you see any serious impact on variety - if anything, there is a greater range of car manufacturer and design today than there was in the 50s and 60s when cars had the unnerving habit of exploding for no apparent reason.
What you'd see is a decline in stupid bugs, a decline in bloat, an increase in modularity, possibly a reduction in latency and a move from upgrades to fix things that SHOULD have worked in the first place to enhancing things that can be relied upon to CONTINUE working fter the patches.
Money would not be made by selling the same product with a different set of defects to the same market, money would be made by always going beyond last year's horizons. The same way most manufacturers, from cars to camping gear to remote control aircraft to air conditioning units to microwave ovens to home stereo manufacturers have all been doing - very successfully - for a very long time.
The IT industry isn't going to change in the foreseeable future, the only way we'll see change in our lifetimes is if it is imposed on the Pointy Haired Bosses. We could easily see 99.9% reliable software, with no additional cost, in our homes in a year, with the lack of constant fixes actually saving money. And that's why it won't happen. Not because the IT corporations are mean, thuggish and ogreish - they are, it just isn't way it won't happen.
It won't happen because they're geared both towards the profit motive and towards the outdated notion that the market is tiny. (That last part was true - in the 1950s, when entire countries might have three or four computers in total, operating in two, maybe three different capacities. You can understand a desire to go after the after-sales service, when there simply isn't anything else left to do.)
Today, Microsoft's Windows resides on 98% of the desktop computers, but because of the support system needed to run the damn things, 98% of the world's population didn't have significant access to one. Ok, putrid green is a lousy colour, but the idea of clockwork near-indestructible laptops that - in theory - could be built to weigh 5 lbs or less and run high-end, intensive applications is beginning to filter through to the brain-dead we call politicians.
You think someone in the middle of Ethiopia who is fluent only in their native tounge is going to want to pay for telephone technical support from someone in India, in order to figure out why their machine keeps locking up?
When computing is truly available to the masses (ie: when even a long-forgotten South American tribe can reasonably gain access to one), the ONLY way it can be remotely practical is if said South American can look forward to a reliable, usable, practical experience where all usage can be inferred from first principles, and where NO software service calls are required to get things to work, ONLY required to get more things for working with.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Nevertheless, regardless, and bethatasitmay, I was siding with the person who expressed the opinion which got modded flamebait. Um, yes, bork bork, I know that an air-traffic-control system would make a poor desktop OS, along with other mission-critical applications such as power-plant-control systems at the plant I used to work at. But, yeah, surely *any* programmer making a desktop OS would at least have a thing or two to learn from the mere *reading* of the specifications from such projects, and perhaps it might even inspire them to make their desktop a more hardy place, maybe even through porting an idea inspired by the mission-critical system, leading to some of that innovation we're all hoping for (even us non-Winodws-users) in Vista.
Why do posts that start with "You're missing the point!!!" always start out in left field and head for the parking lot?
http://en.wikipedia.org/wiki/IEFBR14
/dev/null. It returns immediately when called. It had four or five bug reports filed against it.
IEFBR14 is sort of an executable version of
IBM could of course write a defect-free return statement. All the bugs were requirements drift that Praxis could not have prevented.
This was done as a test of a cheap autopilot. It would never be used in anything but a test. The Captains career is over if his ship collides with anything. He wants an alert human helmsman.
USN Sub sailor 1976 - 1980
Professional Politicians are not the solution, they ARE the problem.
The qualities of an ideal test is a framework similar to ACID for databases and INVEST for user stories. It describes six verifiable qualities that a test should have. On a related note, this article doesn't make me really that excited. I've been using test driven development with JUnit and NUnit to deliver tens of thousands of lines of code into production with similar defect rates (about two defects found over the course of several years of code being in production). I maintained good productivity rates, delivered code into QA with no defects found, into pre-production with no defects found and finally into production where defects were found only after a great deal of time in operation. The code was not simple either: it was asynchronous, guaranteed delivery, multi-threaded messaging code for enterprise systems. Developers who don't do TDD should be paid about 1/4 as much as developers who do, IMNSHO.
Helping with organizational effectiveness is our job.
Bwah-hah-hah-ha-ha!
Havoc Video
The sentance you refer to compared M$ 3year cycle with crash problems to the Navy's 5 year cycle with more testing, which is when this happened. The Lockheed Martin software was in the test phase, not deployed operationaly.
Professional Politicians are not the solution, they ARE the problem.
Nevermind that '9x and NT are completely different code bases, but hey, keep talking like you know what you're talking about - after all, this is /.
and yet, we still masturbate...
wait... what was the topic again?
Havoc Video
This article couldn't have been more coincidental to my current project: I've been re-reading James Martin's books, "Application Development without Programmers" and "System Design from Provably Correct Constructs", with the goal of selecting a method to program mechanical devices.
Martin's thesis, and remember this was back in the 70's and early 80's, was that the program should be generated from a specification of WHAT the program was to do, rather than trying to translate faulty specifications into code telling the computer HOW to do it. (Trust me, that poor sentence does not come close to describing the clarity of purpose in Martin's books.) Martin proposes that a specification language can be rigid enough to generate provably correct programs by combining a few provably correct structures into provably correct libraries from which to derive provably correct systems.
The definition of the time, HOS (for Higher-Order Software) was actually used by a company called HOS, Inc.(!), and apparently worked pretty well. Many of the constructive ideas were included in OOP and UML, but ideally, if I understand the concept properly, it would be equivalent to generating software mostly from Use-Case analysis. There are similar approaches in MDD and MDA methodologies. I wonder what ever became of the HOS,Inc. and the HOS methods? It looks like they had a handle on round-trip software engineering in the 80's.
OK, why would this be a good thing? Well, for one thing, computational/programmable devices are prolifierating at a tremendous rate, and while we can engineer a new device in 3 to 6 months, the programs for the device take 18 months to 3 years (if they are finished at all). Hardware development has greatly outpaced software development, by some estimations a 100x diference in capacity...yet they are built on the same fundamental logic!
The best argument, IMO, is that since larger systems are logarithmically more complex, and since it is impossible to completely test even intermediately complex systems, it will require provably correct modules or objects to produce dependable systems. If the code is generated from provably correct components, then the system only has to be tested for correct outputs.
Furthermore, code generated from provably correct components can allow machinery and devices to adapt in a provably correct way by rigorously specifying the outputs and changes to the outputs.
Praxis is on a roll. The methodology employed is probably more important than the genius of the programmers. It should get better,though. The most mediocre Engineer today can produce better devices than the brilliant Engineers of 30 years ago using tested off-the-shelf components. IMO, this the direction programming should be taking.
"The mind works quicker than you think!"
In reality, my system is a hell of a lot more effective. Show the bugs a picture of the SPARK product manager and bugs are so utterly freaked out, that they run for the hills rather than residing in the code.
BeauHD. Worst editor since kdawson.
I can't speak for the original poster, but I can't believe that the parent poster and I are the *only* people that believe that LOC is a poor metric.
I won't say that it always is, but I think it often is. LOC measurements can't take into consideration situations like those where your productivity as defined by LOC is only 1/3 of your cubicle-mate's, but the code that you spent three times as long on will save 10 other coders from having to write an assload of additional code, simply because you took the time to think things through and came up with a better design. In this situation, how does LOC square with actual productivity? In addition, with fewer LOC you also have probably reduced the potential for failure that all that other code would have introduced.
Please stand clear of the doors, por favor mantenganse alejado de las puertas
But seriously, that's cool. Any 'net resources on this for us software types who'd like to think we can solder two wires together without burning down the house?
They're allowed to have one!
Congratulations to slashdot for actually addressing the issue of safety-critical software. Folks, this is a completely different world than, say, the world of games or google searches -- where 80% of the output is *expected* to be crap. And the right language for the job is Ada.
I watch Brit Hume on Fox News
I was told that NAND is Turing-complete by someone smarter than I. I realized that NAND itself has no program flow, thus it can't be turing complete all by itself.
It is possible to build any logic circuit using nothing but NAND gates. That includes processors and even entire computers.
It is also true that area and speed are generally interchangeable in logic design. One can do some rather amazing tasks with really small circuits so long as your are not in any hurry to see the results.
However, I submit that the smallest "touring complete" processor is at least 3 NAND gates.
2 nand gates = 1 SR flop, the smallest storage element buildable from NAND
1 nand for computation.
With only 1 bit, it's well short of the infinite storage requirement. But then, all constructable devices fail that one.
Doesn't the underlying operating also play a big part in the reliability of the software? Even if your code is correct, can you say the same for what you run the code on, or what you compiled/interpreted it with? Then there is the hardware.
It reminds me of many medical devices that have software running on Windows internally. I remember having LASIK done and the device looked like it was run by a PC with a Windows app. I did not turn out blind.
Since they also give SLOC and whole-lifecycle-SLOC/day metrics, we can calculate the "sizes" of the projects...
CDIS: 42.5 person years
SHOLIS: 10.5 person years
MULTOS CA: 9.8 person years
A: 9.7 person years
NSA: 8.6 person months
So we're not talking about "tiny" projects here...
Then their ability to produce bug-free code depends, as usual, on control factors, not on real-world engineering.
In as much as a civil engineer depends on control factors via refusing customers who demand that the building have 6 stories not 4 just one month before construction is due to finish, yes. Real world engineering makes certain demands of the client. Someone who wants to build a treehouse for their kids doesn't consult an architect and a civil engineer, and civil engineers don't take contracts from people who refuse to set out some limits on what they want built, and what they expect of it.
Praxis uses solid engineering. Their "Correct by Construction" approach is solidly grounded in axiomatic mathematics and uses similar sorts of formal calculations and logical and mathematical proofs as you might expect to see from civil, electrical, aerospace, or ny other kind of engineers. Take the time to read sample chapters from the SPARK book to get an idea of exactly what they are doing. There is very definitely quite solid engineering going on.
Craft Beer Programming T-shirts
As a computer science student, I would like to know why we aren't taught this in school. My professors have told me that software verification is a nearly impossible task which is too complicated for any real system, but this article claims otherwise. Why aren't we being taught that we should be responsible for writing reliable code and how to do it? Why haven't I heard of Z or other methods for writing provable specifications? My professors can't even write a decent specifcation. And supposedly this is one of the best Computer Science programs in the US. If students aren't even taught that it is possible to write near bug free software, it's no wonder that software is so unreliable.
And where's the description of GNU/Linux Air?
[Fuck Beta]
o0t!
I think Linux, Firefox and Openoffice are great, and having a lot of eyes look upon source code may make all bugs shallow, but it still leaves finding bugs to chance rather than systematically preventing them.
These people produce code with defect rates better than CMM level 5 (most companies aren't even at CMM level 2). Firefox, OpenOffice and Linux aren't anywhere near zero-defects.
Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book
There are errors in XP that were unfixed errors in 98. I've seen 2 errors in XP that were identical to ones in 3.1. That is a lack of attention to detail.
You say this like it was a problem. But what were the errors?
Things like: "boot disk failure" probably wouldn't change much from DOS 1.0 to WinXP....
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Every industry have interests in valuing byproducts.
In software industry, bugs are a byproduct and, there are ways to sell them as features.
There are compagnies to sell trashcans so trashes dont invade your house. You can produce gaz out of shit.
There are compagnies to sell enough memory, hand hard disk space so you have place to put leaks and fragmentations. You can even buy virus and spam cleaners.
Economically: Wastes have values and you can make money out of it by recycling. No matter if these are hardware or software wastes.
Léa Gris
Screw funding. It's irrelevant.
Screw specifications. Nobody has them anyways.
Give me a clear, predefined spec, and I'll meet it. I'll guarantee bug fixes,too.
But that's not how software evolves.
Despite careful attention, despite voluminous meetings, emails, and specifications, I never get a clear idea what the client needs me to develop until AFTER a prototype has been built.
In fact, I'd wage that there's a quasi-quantum principle at work: You can either work towards the customer's actual needs, or the predefined, agreed upon specification/costs/specifications. Answering either means ignoring the other.
Consider this the Heisenberg Uncertainty principle. The software is half-dead, half-alive. Either it meets the needs of the customer (and associated scope creep, bugs, ets) or the originally defined specification. Releasing the software defines whether the cat is dead or alive.
It seems that:
1) People will commit, in aggressive fashion, that they need something until they get it, at which point, they'll angrily point out all the flaws in it.
2) People don't actually know what they need until they see that what they have isn't it.
3) When you take anything produced because of (1), and then compare that to the feedback produced by (2), you end up with cases where the code is producing a result unexpected in the original design.
These are called bugs.
4) The only intelligent way to proceed with (1) and (2) is to consider software an iterative process, where (1) and (2) combine with (3) and lots of debugging to result in a usable product.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
JML is a formal specification language for writing contracts for your Java programs. (It has nothing to do with UML or XML, other than sharing two letters.)
There are a number of quality research tools that understand JML including a compiler (jmlc), a documentation generator (jmldoc), a unit test generator (jmlunit), and an extended static checker (ESC/Java2), a kind of automated theorem prover, like FindBugs on steroids).
These tools and technology let you do design by contract and contract-based unit testing in Java 1.4 and are used by many universities and companies to help them build quality software. See the ESC/Java2 FAQ for more information.
Disclaimer: I am part of the research community that develops JML and the co-author and maintainer of ESC/Java2.
Joe
Joseph R. Kiniry
http://kind.ucd.ie/~kiniry/
Lecturer
UCD School of Computer Science and Informatics
I would have said no payments for the first six months, then 19.7% after that.
Their secret is slave labor
People sleep peaceably in their beds at night only because rough men stand ready to do violence on their behalf.
The reality about Correctness by Construction is that bugs do make it to the final product and the more complex the application the more bugs there is. In safety-critical applications, highly reliable software is simply not good enough, only because a single bug can be catastrophic. What is needed is 100% defect-free code, guaranteed. In other words, unless the program is guaranteed bug-free, it must not be deployed. Doubt = failure. The only way to achieve 100% reliability in software regardless of the complexity is to abandon the algorithmic model of software construction and to adopt a non-algorithmic, synchronous model.
Yikes! How did a divide by zero error do all that? Surely the application should have just crashed, leaving everything else running. Even if it did die entirely, why couldn't they simply reboot?
At the age of 17 I spent a week in the Praxis offices on "Work Experience" (Americans may think of this as a very short internship), to find out what developing software would be like as a career. This was a major formative event of my life: I thought that developing software sounded good, I really liked using Real Computers (multiuser, multiprocessing systems with powerful operating software, like VMS and SunOS), and the people impressed me greatly. It definitely set me on the path to the career in systems development and administration that I have today.
The person who made the biggest impression on me was the sysadmin. He got his own office-cube instead of having to share, he wore much more casual clothes and had a lot more hair and beard than most of the staff, he got to have big toys (several workstations, a LaserJet IIIsi big enough for an entire office that seemed to be his alone, etc) and he didn't seem to get much hassle from anyone else. This was obviously the job for me.
The sysadmin was obviously rather a BOFH. When I was sat at the UNIX workstation for the first time, and had poked around with basic file-handling commands, I asked "What's the text editor on this system?". He answered "emacs - e m a c s - here's a manual" and picked about 300 sheets of paper off the Laserjet and handed it to me.
I got to play with UNIX (SunOS), VMS, Oracle development environments. I still have the Emacs manual printout somewhere at home - it served me well when I went to University where printing anything out was charged by the sheet!
I'm very glad they're still around.
"For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled"
All components of the beaurocracy in government will use Eminent domain to sieze objects of use, and USUALLY will pay, as that is the principle of Eminent domain.
Food not Bombs is a nice platitude but it breaks down when you notice that the Bombees are usually well fed
Their claims of massive error reduction are, at best, anecdotal. Let's see them do this after taking over a half-coded project with minimal design requirements, a hard deadline, and a budget that can be cut by governmental forces at will.
uh huh. That's like saying, "sure the Golden Gate Bridge is nice and strong, but let's see them build a bridge across the Straights of Gibraltar using only legos and silly string - oh and let's see them have it finished by close of business today."
The point is, a REAL engineer wouldn't take that kind of project, because a REAL engineer would understand that it's not reasonable.
So the question is, why do software engineers accept projects that aren't reasonable? Why do we volunteer for death march projects? How do we change the culture of software so that customers don't feel entitled to demand the impossible, and software companies don't feel compelled to take on the impossible?
Software development is hard. Believe me, I know. But this culture we have created is just making it harder.
"OS would at least have a thing or two to learn from the mere *reading* of the specifications from such projects"
In the same way that an air frame mechanic might gleam some knowledge from watching "The Great Biker Build Off" on the Discovery channel.
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
"Firefox and openoffice are continous works in progress, as opposed to a project with a firm set of goals."
Each release has a firm set of goals though, so that release could be significantly less buggy than is currently the case.
"I doubt Praxis could make the same offer if customers changes the requirements, and feature list all the time."
Neither OO or Firefox have any customers, though: they are free software, and not therefore subject to the financial and competitive pressures that can and do result in buggy commercial projects. Who actually decides what features to add? The development team. Who sets deadlines for implementing these features?The development team. Who decides when they are coded to sufficient quality for public release? The development team. Who therefore is to blame when that code ends up being sub-par? The development team.
"Praxis also has certian requirements about the enviroment in which they work."
All software has specific operating environment requirements.
"If I took a Solaris 9.0 project, and tried to run it on linux 1.0, I don't think Praxis would be liable for the bugs that would occur."
How is this in any way relevant to the fact that both Firefox and OO are buggy? Such bugs manifest themselves on systems that both claim to support, and are not restricted to people who try and put them on (for example) an XBox-360.
I'm not going to change your sheets again, Mr. Hastings.
TFA cites a particular NSA biometric identification program which has "0.00" errors per KSLOC
And what would 0.34 errors look like?
or perhaps they had less than 5 errors in a thousand LOC.
From the IEEE article:
"The process is time-consuming. For the Mondex project, spec-writing took nearly a year, or about 25 percent of the entire development process. That was a long time to go without producing anything that looks like a payoff, concedes Andrew Calvert, Mondex's information technology liaison for the project. "Senior management would say: 'We are 20 percent into the project and we're getting nothing. Why aren't we seeing code? Why aren't we seeing implementation?' " he recalls. "I had to explain that we were investing much more than usual in the initial analysis, and that we wouldn't see anything until 50 percent of the way through." For comparison, in most projects, programmers start writing code before the quarter-way mark."
Isn't suposed that one has to give something working to the user and then correct any mistakes made?
IBM had a major intiative back in the mid-1980s called AD/Cycle which was tied to SAA (System Application Architecture) which was based on these and similar ideas prevalent back then. This is the old "holy grail" and an attempt to fix the waterfall methods of development, which had actually been since the early 1950s with mixed success in delivering software on-time and in-budget.
AD/Cycle involved not only IBM but a number of "AD/Cycle partner" companies like Bachman, and KnowledgeWare. KnowlegeWare's CEO was the former scrambling NFL quarterback Fran Tarkenton. A google of "fran tarkenton knowledgeware" will turn up references to Jim Martin, as well as some interesting things about how the company ended up.
An incredible amount of development money went down the rat-hole chasing the AD/Cycle dream.
The problem turns out to be the difficulty, if not impossibility, of creating rigorous specifications which produce useful results in the face of problems which aren't very well understood at the outset. The less the requirement is for a "black box" with well-defined inputs and outputs the more this is likely to be the case.
Many problems turn out to be wicked that there is a feedback loop between the implementation and the requirements. A classic book from the era was Wicked Problems, Righteous Solutions (http://www.amazon.com/gp/product/013590126X/102-5 477977-4320940?v=glance&n=283155)
Which might be considered one of the old testament texts pointing to today's "agile development" movement.
A non-software example of a wicked problem is city planning in which implementing changes in the road network, housing developments, shopping center location etc. all change the requirements for the same aspects.
Many wicked problems come from "requirements" which often do (or should or must) come from users. Often, the real requirements aren't known until an implementation is given to the users, who then might say, "yup, you implemented exactly what I asked for, but now that I see it, here's what I really want." Or, "Now that we've added this other thing (application, system, business division) why, doesn't this (work more like/interface with/replace/...) that."
Faced with this, a methodology based on "correct" construction from "rigourous" specifications simply moves the problem to debugging the requirements.
Until we do away with the need to change/adapt systems to changing/evolving requirements, which would likely involve eliminating users, this approach will have limited applicability, and will need to stand beside other more widely used incremental development models.
The article comments:
These six principles are not in themselves difficult to apply, and may even appear obvious. However, in the authors' experience, many software development projects fail to deliver against many, if any, of these principles.
I'd argue that this is true, and the main reason that so many software products are so buggy is that most companies reward their developers for making buggy software, and punish them for making quality software.
The latter point is easy to show. I've worked on any number of products where I produced something fairly quickly that worked, and nobody could find any bugs. How was I rewarded? In nearly every case, I was laid off. I'd done the job that I was hired for, they didn't have another project that "matched my talents", and the software was so well documented that if any bugs did appear, others could fix them. So why should they continue to pay me to be nonproductive?
The lesson to developers is obvious: If you want to keep working here, you'll have to make sure that there is still work for you to do. This is not difficult. We all know the results.
It's especially easy to do when you look at the illogical demands that most companies put on programmers. I've been denied all sorts of information about the workings of the tools that I'm required to use, on the grounds that what I'm asking about is proprietary. So I have tools whose behavior isn't fully knowable, and I spend a lot of time looking at output and asking "What could that thing be doing to give such bizarre results?"
Of course, the funniest part of the article was the first principle, of "using a sound, formal notation for all deliverables". In most companies, this means a PowerPoint file.
My general conclusion is that lots of people say that they want quality software, but they intentionally prevent developers from producing it, and they punish developers who do so despite all the barriers. The result shouldn't be at all surprising.
Those who do study history are doomed to stand helplessly by while everyone else repeats it.
Of course, there are arguments that this does not constitute security. I think the concept of free fixes for bugs found by customers works a little better - it keeps all the stakeholders in the loop.
I think the "exclusion of warranty" terms that are common to most EULA's may be more at the heart. The typical consumer who has a problem will eventually run into this, and be forced to spend an insane amount on legal fees to get the EULA invalidated as unconscionable, or (more likely) give up in disgust.
Perhaps there would be an impact from a federal law stating that software makers whose license terms are found unconscionable in court must notify all registered users of the software of that ruling....
//Information does not want to be free; it wants to breed.
I don't see how "open source" = Let the people inside the company see. Sounds pretty damn closed to me.
Have you worked in many companies? Most I have anyone has access to see any code from any project, even Microsoft.
I also disagree w/ your military argument. Alot of projects require security clearance e.g. @ Lockheed Martin. (Where I have also worked).
In a regular software company you can see other peoples stuff, but even then if you have nothing to do with a project as in, you are not even a software devleoper, of course they won't let you see. Its a security risk. The more people that have access to something the more it can just get passed around like your bong does.
Seriously, gov and military projects have security around them, and its tighter than the software development done at K-Mart for a REASON.
My "opinion" about the whole issue stands.
Modesty is one of life's greatest attributes
Quite simple really. The U.S. Gov views it as, "US vs The World". The U.S. Gov doesn't care to share any of our tax-dollars development with "The World". Even if we can get an advantage of more solid code if every missle we made had software on it that was reviewed by arbritrary peers. They don't want every other country in the world using the same software especially if we go to war with them.
Software + Knowledge = Technology = Power. U.S. Gov doesn't want to share the power.
If all a missle is, is software...
Apply this to everything and everything, from a simple router to the space shuttle.
Modesty is one of life's greatest attributes
I'm not knocking the article or shops that follow those practices, since there are life critical applications where it is well justified (such as the ones discussed in the article). However, some here seem to be missing a few points when it comes to the desirability of these techniques for less critical software.
It's worth noting that the article starts out stating that the techniques involved allow production of up to 30 LOC/day. Thirty lines per day! Linux would have required over 493,000 man YEARS (at a reasonable guess +- 100,000 or so) to be where it is today at that rate. Assuming (very) generously that 2000 people could work on it full time without additional slowdowns for management overhead, that would have it ready to go somewhere around 2238 AD
It's simply not reasonable to compare engineering practices where life and limb is at stake to software practices where no such risk exists. It seems inevitable that software development is compared to the efforts to design a highrise or a bridge. What do you bet those cheap consumer devices that use a wall wart (so they don't have to be UL listed) have a higher defect rate?
That's not by any means meant to suggest that no attention should be paid to quality or that some shops don't pay nearly enough to it now. However, it is a very good reason we do NOT want everyone to be CMM5 or equivilant. If they were, we would literally grow old and die waiting for the software we want/need.
I suspect that to achieve less buggy software in a timeframe we can live in and for a cost we can afford will involve advances in modularity and reusability rather than high end development techniques. It will also require a legal and economic structure that don't require so much re-invention of the wheel.
mentioned in the article, you can download the reference manual from here
Check it out: http://www.nsa.gov/careers/students.cfm
I did an internship with State last summer and it was an amazing experience. I'm not sure I'll end up working for the government (given the rising price of living in the DC area and the lower salaries, it's looking less and less likely), but it was a very valuable professional experience in any case.
Check out the State student programs at careers.state.gov, and go for OVERSEAS internships - when you're based overseas as an intern, they give you many more responsibilities and depend a great deal on what you can do. It's a great way to get noticed and, whatever I end up doing, looks great on a resumé.
I know! let's sell a $200 PC to people that live in a country where the average annual income is about $13.50 on credit, then we have a large supply of workers. It may not quite be slavery, but it surely smacks of indentured servitude.
.net apps? write scam emails? blog?
We can probably get start up capital by calling ourselves a non-profit and asking for donations to help the third worlders that we are fixing to exploit.
The only question is, once we have all these new PC owners on the hook what are we gonna pay them to do, code
"Quality" as ambiguous. What I consider "quality" is different from what you consider "quality".
It's not just three, either; it is ALL of the non-functional requirements. At most two of them can be fully optimized in any engineererd system. In practice, the design of a system will be governed by the relative priorities of the non-functional requirements.
The set of non-functional requirements is very large, but you should recognize some of these:
Mental exercise: Choose any two from the above list, then explore how choosing any other item from the list would require a compromise to either of the first two chosen.
So, what "ability"s do you want to have today? ;)
@HbFyo0$k8 tH!$
For the production build, just change that one line to
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
I don't know how could you use Z to validate a specification with your final users unless they have a CS degree.
Use cases are semi-formal as Alistair Cockburn puts it. And that's a real advantage.
A combination of use cases, mock interfaces, prototyping and interpreting user requeriments (based on one's experience) are my best tools.
An updated version with more operating systems:
UNIX Airways: everyone brings one piece of the plane along when they come to the airport. They all go out on the runway and put the plane together piece by piece, arguing non-stop about what kind of plane they are supposed to be building.
Mac Airlines: all the airline personnel look and act exactly the same. Every time you ask questions about details, you are gently but firmly told that you don't need to know, don't want to know, and everything will be done for you without your ever having to know, so just shut up and get on the plane.
Windows 9x Air: The plane leaks gas, doesn't fly straight, and engines fall off during the flight. To fix problems with the avionics, the pilots just land and take off again. However, each plane has the word "internet" painted crudely on the side, and so every flight is full.
Windows XP Air: the terminal is pretty and colorful, with friendly stewards, easy baggage check and boarding and a smooth take-off. After about 10 minutes in the air, the plane explodes with no warning whatsoever.
Windows Vista Air: the executives of Windows Air hold press conferences every three months to talk about their exciting new line of international service, but at every press conference, the list of cities they fly to gets shorter.
Windows Server Air: just like Windows XP Air, but costs more, uses much bigger planes, and takes out all other aircraft within a 40-mile radius when it explodes.
Windows Media Center Edition Air: Just like Windows XP air, except you get to watch the first ten minutes of an inflight movie before the plane explodes.
Linux Air: UNIX Airways employees who finally figure out what kind of plane they are building decide to start their own airline. They build the planes, ticket counters and pave the runways themselves. They charge a small fee to cover the cost of printing the ticket, but you can also download and print the ticket yourself, and otherwise the trip is free. When you board the plane, you are given a seat, four bolts, a wrench and a copy of the Seat-HOWTO.html. If anyone has trouble installing their seat, the other passengers are very happy to help out. Once settled, the fully adjustable seat is very comfortable, the plane leaves and arrives on time without a problem and the in-flight meal is wonderful. You try to tell customers of the other airlines about the great trip, but all they can say is, "You had to do WHAT with the seat?!''
GNU/Linux Air: Captain Stallman refuses to take off because of a panel on the flight deck that is not made out of a transparent material.
Server Linux Air: cargo only, but fast, reliable, and economical, if you can find a pilot who is also a mechanic.
Desktop Linux Air: just like Linux Air, except the flight never leaves because the pilots continually argue over whether the plane should be painted red or blue.
Debian Linux Air: just like Linux Air with pre-installed seats and a self-repairing plane, but their terminal is small and no one can find it.
Gentoo Linux Air: apon arriving at the airport, the stewardess hands you the plans to build an F-22 Raptor.
MacOSX Airlines: employees of Mac Airlines bought planes from Linux Air, and then designed their own style of very comfortable seats and a great terminal. The flight is incredibly smooth and comfortable with great service, you arrive ahead of schedule, and there is a live band on every flight. Tickets are incredibly expensive and you are only allowed to bring baggage if you use suitcases built by MacOSX Airlines.
Again, apologies to Doc Searls and Linux Journal.
For security, the MD5 hash of this message and sig is 09f911029d74e35bd84156c5635688c0.
Well, since my code is mostly GPL, it does not have a binding warranty. But my policy (as with many other open source producers) is to fix bugs at no cost to the user; so in effect, open source software typically makes the same promise. There are probably many reasons why this is so, not least of which is simple pride in workmanship and desire to maintain a reputation.
you had me at #!
Likely 80% of the value of the Praxis method can be achieved without statistically provable languages.
Industry standard UML meets most of the requirements for sound formal notation.
Tooling for UML provides substantial validation of deliverables.
Iterative methods, RUP being my preference, meet the requirement for carrying out and validating the deliverables of small steps. (Note that CMMI still focusses on repeating the process, not validating the deliverables as the requirement.)
Any well structured method with low redundancy meets the requirement to say things only once. Most methods aren't well structured. RUP is pretty good, and can be made solid in this respect with a little attention.
From reasonably well done UML, test cases can be derived systematically that provide substantial coverage. Architects and designers can still create unthinkably complex solutions, but nothing in the article Praxis writes indicates that they have solved that problem in a technical way, but rather in a procedural way. Arguably, the Simplicity rule of XP comes closest to this Praxis practice, but I prefer the second half of the rule (don't add functionality before it is scheduled) to the first (do the simplest thing that could possibly work).
Any decent iterative method deals with the hard things first. RUP explicitly does in the Elaboration phase, for example, while XP has the less well articulated architectural spike and spike processes.
Having run XP and RUP projects and coached organizations around North America on the effective use of RUP, I can say that high developer productivity with low defects is eminently achievable, although not quite to the standard that Praxis achieves. A friend and CTO runs a variant of RUP for his shop and it's rare that defects are found by customers.
Cheers,
Mike Barnard
The reason it is used in the article though is that in the problem space where SPARK is most likely to be used, LOC/Day is still the most common method used to measure productivity (even for product that are developed with different approaches) It is full of problems, but I honestly have not seen a "better" approach that really worked. Function points were big for a while but counting them is more difficult (though honestly probably not more difficult then when the LOC/day crowd starts getting into the Equivilent LOC/Day math used when modifying an existing product). Several people on the thread have scoffed at LOC/day for productivity...I have not really seen anyone else present a viable alternative. The other thing to keep in mind is that there are other measures in place as well on most jobs like this such as defect density, rework, etc. All of these metrics have their problems to but in the end, when you are going to spend millions of dollars writing software, the pointy hair bosses (rightly so) do not want some guy just saying something like "Well, I think it will cost 1.5 million dollars because I submitted an ask slashdot question and that was the average response"....Whether or not such an approach would end up with a better estimate of productivity and final cost is left as an excercise for the reader.
--- Liberty in our Lifetime
I was given a project manager role at Nortel in 2000, and tried to implement some of this. It was virtually impossible; people did not want to (a) change their existing habits or (b) admit their were improvements to be made. While initially management and some on the team liked the ideas, implementing them was virtually impossible. Those who had to address the actual issues simply refused to do what was necessary. They said "yes" in meetings and went right back to the old work.
As a project manager over people in multiple departments and multiple locations, with hiring control over none, I was helpless without the support of executive mgmt, and they of course told me implementing the changes in question was my job. Which it was, but without power, a title is worthless.
Very, very vexing.
Of course, my inability to spell simple words ;-)
in the above post undermines it somewhat
Why is it that software engineers have to write so many dang essays? Maybe if we were as terse and un-expressive as the civil engineers our programs would work better. You ever see a few dozen essays on civil engineering in the bookstore?
I18N == Intergalacticization
Indeed, their technique seems to rely a lot on the sort of "Executable Comments" that act like assertions and other semantic statements. It could throw off their statistics.
I18N == Intergalacticization
They're doing it by using a design-description method that prevents unambiguity
Now that's a method I'd like to see—one which makes it impossible to say anything without a double meaning ;-)
Need to type accents and special characters in Windows? Use FrKeys