Industrial Strength Open Source Code?

dnnrly asks: "I work for a company that writes software for the pharmaceutical industry. We have to work in quite a tight regulatory environment because some of our code ends up in the process of drug testing. Seeing as the FDA are quite picky about making sure that there can be no errors in testing new drugs, our clients have strict rules that we must follow for coding. We have to review all of the code that is written, making sure that everything is traceable to a design specification. Where we use 3rd party software/code we have to make sure that it comes from an ISO9000 source. This is a bit of a problem when we would like to use open source stuff in our code. Projects like log4net and NUnit would be tremendously useful in our code but we're not allowed to use them because they don't tick the right boxes. Now, *I* know that these projects (and others) are incredibly stable just because of the volume of use that they have seen but that isn't enough for some people. How can we certify such software?"

Comments

ISO 9000

[deanj]

No offense intended, but an ISO 9000 source?

ISO 9000 just means the people that "certify" this have all their procedures documented.

I'd be much more interested in a company that delivered software that worked, and stood behind it, rather than a company that puts all it's time and effort into ISO 9000. Every time I hear someone asking for that, they're more interested in being buzzword compliant than anything else.

Man, oh man.... The next thing people will be saying is that the company mission statements actually make a difference.

Re:ISO 9000

[Stradivarius]

I'm sure many of the companies that are certified would probably agree with you. But the fact is that many companies, particularly those selling to government entities, are required to be certified in ISO 9000 or similar programs as a precondition of doing business. So they have to go along, at least to some extent, with the fiction that certification equals quality.

IMHO, certification really means repeatability (do you always follow some established process). Which is certainly important, especially when it comes to very expensive/complex programs, in order to mitigate risk. But it's not the same as quality. One can repeatably implement a lousy process or procedure. One can also be repeatably bad at implementing a good process (you can have the best design process in the world, but a bad designer will still give you a bad product).

I think the whole certification buzz comes about in large part because it's difficult to evaluate the other big factors that impact quality, which are largely human factors - talent, workmanship, management acumen, etc. Because customers have little visibility into those factors, they instead evaluate the factor most amenable to measurement - process compliance.

People with some common sense recognize the certification as a useful but limited tool for evaluating a company. It's the other folks that you seem to have encountered - the ones who think process compliance is the Holy Grail for business success. Those are the ones I try to avoid :-)

Re:ISO 9000

[Monkelectric]

I think ISO 9k is an industry scam. I worked for a company which had no process whatsoever -- literally none. When a new piece of software needed to be written -- someone would walk into your office and say, "I need something to do X" That *WAS* the entire software process. No requirements, no architect, no analysis, no test plan, no documentation, no testing, nothing.

We were 9k certified.

Re:ISO 9000

[Anonymous Coward]

I think this is gradually becoming to apparent to everyone. Just get an ISO 9000 consultant drunk sometime and you'll see that even THEY don't think its worthwhile.

It will never be disclaimed and junked because too many managers have spent too much money on it to ever admit that they've been had. So we're stuck until that entire entire generation of managers dies or retires or until a new, even worse, thing supplants ISO 9000.

Re:ISO 9000

[Sentry21]

Every time I hear someone asking for that, they're more interested in being buzzword compliant than anything else.

Or, in my company's case, is in the medical fields, and thus is required by law to be ISO compliant (as well as CSA-compliant, etc.) in order to do business. Being and staying ISO9000-compliant isn't a cakewalk - you have to have documented procedures for everything you do, and your employees have to know (or know how to find) those procedures. I'd wager that most companies that become compliant do so not because it 'looks good', but because the law requires it of them directly, or because they want to be able to sell to companies with restrictions like the original poster's.

Granted, being compliant doesn't mean that the procedures are useful or easy - a lot of companies fuck their procedures up, developing them by committee, spewing out 20-page procedures for the simplest of tasks, but that has nothing to do with ISO - that's just bad management, and that bad management will reflect in other aspects of the company as well.

Re:ISO 9000

[NateTech]

ISO 9000 is a documentation process, not a quality process. People don't seem to get that, and the marketing spin around it for years has not exactly made that very clear.

One of the engineers at work keeps a photograph of the Firestone plant that produced all the deadly, defective tires for Ford.

The photo shows clearly:
- The plant was shut down and closed.
- The plant has a large "ISO 9000 Certified" sign on the entrance sign.

ISO 9000 just means that you documented your procedures and you can verify and prove that you followed them. It does not tell you whether or not they're smart, safe, profitable, or anything else about your business.

In other words: ISO 9000 forces a company to document what they're doing, but can't save the idiots from doing the wrong things.

Re:ISO 9000

[meiao]

I'd like to see some ISO 14000 code.

"No animal nor tree was injured in the making of this program."

Re:ISO 9000

[ErroneousBee]

until a new, even worse, thing supplants ISO 9000.

"Extreme" methodologies, which are rapidly morphing into "Slash development time, skip the testing" methodologies.

ISO9000 used to be 'Know what you are doing and how to do it' before it became "document everything, including your documentation procedures, especially the procedures for documenting how you document the docuemntation".

Re:ISO 9000

[Anonymous Coward]

ultimitly what the client wants is to have a software developement company responsible for possible mistakes and that those software developers repay possible damages.

The open source free software is often of better quality but it comes with no liabilities and developers do not repay possible damages. I think this is fair enough if you get the peace of code for free. If you want some garantees you should pay for it to the company that decides to provide the maintenance and support of the software in question.

In this particular case one could create a small company ISO:9000 certify it and re-sell the software with required liabilities

Re:ISO 9000

[Lonewolf666]

Scam might be a bit exaggerated, but ISO 9000 is indeed easy to get.

In a small company I worked before (medical field), the documentation and process were not quite as nonexistent as you describe, but they were thin to put it politely. We were ISO 9000 certified, too.

Laff

[Konster]

"Seeing as the FDA are quite picky about making sure that there can be no errors in testing new drugs."

Re:Laff

[ClamIAm]

The testing has to be perfect. The drugs? Not so much.

And I don't understand why the parent is modded troll.

Import it into your own code base, and review it.

[ankhcraft]

Simply import it into your own code base, and then review it as if it was written internally. Basically, learn it inside out, as if you wrote it yourself. If that is not legally sufficient, then the laws need to be rewritten since the lines they would be attempting to delineate would at this point be completely imaginary. It doesn't matter whose head it originates from, what matters is that it is fully reviewed and completely understood to the point where everyone on your team is prepared to stand behind the entire body of code. If that confidence comes from actual understand, it becomes irrelevant who wrote the code in the first place. How would it be any different if, instead, it was code written by somebody who no longer works at the company.

How can we certify such software?"

[thrillseeker]

You need to get the kind of people involved who can statistically show that particilar software approaches lead to less problems. Otherwise, you're stuck with the expense NASA goes to in writing software.

Own it

[keithmo]

Could you, for the purposes of certification, take ownership of the libraries you want to use? By "take ownership" I mean either a) acquire, or b) create the necessary design documents for the libraries, then code review them as you would your own code.

Re:Own it

[Doctor Memory]

That's part of his point, in order to accept the OSS he (or someone on his team) would have to write the design specs, then validate the code against them. Which would basically take as much time and effort as writing it from scratch themselves.

FWIW, I have written FDA-approved software (LIMS), and I know I wouldn't (and didn't) use any third-party stuff in any of my systems if I could write it myself. It takes more time, but we were known as miracle workers at my company, because we actually delivered working systems within a reasonable facsimile of our target date. Pharmaceutical R&D development teams are notorious for eating millions of dollars and spending years chasing state-of-the-art systems that they've convinced some department head they need, only to wind up scrapping the project because it doesn't work or isn't performant. It's like Defense Department work, only with company money. Of course, these are pharmaceutical companies, so it's not like they miss the odd thirty or fifty million...

Mod Parent Up ISO9000 Compliant

[argent]

ISO9000 just means you have documented procedures and you track the steps you take when you follow those procedures.

You shouldn't even need to go to the lengths of merging the code bases.

Document the procedures for auditing and verifying external software to the standards of your internally written software, create an ISO9000 process for tracking those procedures, and follow them.

The starving programmers

[w33t]

This brings to mind a conversation I had the other day.

I think the main reason that (F)OSS is still having trouble competing (despite the widespread acceptance amoungst industry experts) is because of budget.

In this case, the budget to get a piece of software properly certified.

There are many aspects of creating software that require non-technical administrative personell to handle. I don't remember ever hearing about OSA (Open Source Accountants) ;) These people cost money. And it's one thing to freely dedicate your time to devlopment, but it's quite another to freely donate large sums of personal money.

It is truly inpspiring that so many can work together so well towards a common goal, and it is truly stunning to take in the vast amount of software available which is written pretty much completely philanthropically.

But the problem is that few actually get paid to do this, it is done in spare time.

In my conversation I wondered how else software writers could make money, besides the tried and tested subscription and serial-activation systems in widespread use today. How else could programmers, who are doing some of the most mind-bending, skillful crafting of any career, get compensated for their work?

Wouldn't it be great if AMD, Intel, ASUS - or indeed any large electronics manufacturer - would offer up a pool to develop software?

The trick would of course be that the software should of course be standards compliant and thus run even on competitors hardware. But maybe AMD could have a certification that says, "written for and extensively tested on AMD hardware".

Software seems that it should be freely available - it just seems the nature of all information in general - but there is that problem that the programmer needs to make money.

It just seems to me that logically the consumer should buy the hardware - the physical, tangiable thing - and that it should be up to the hardware manufacturers to make hardware as a whole more useful.

Here's the irony in my eyes - right now hardware manufacturers pay Microsoft in order to get a little sticker that says "built for Windows XP".

This doesn't seem right. It seems totally backwards to me.

It reminds me of a quote from "V for Vendetta": "People shouldn't be afraid of their governments, governments should be afraid of their people."

In other words, "Harware manufacturers shouldn't be beholden to the software companies, software companies should be beholden to the hardware."

The fact of the matter is that I just wish that after I pay $2,000 for my nice new system that it would run right out of the box. Macintosh is close - but I would have the added stipulation that the software on any hardware system be standards based so that I don't have a Dell OS, IBM OS, HP OS, Alienware OS...etc.

These are just thoughts, and I understand that there are many counterpoints.

Re:The starving programmers

[shmlco]

Too many open source projects think the project is done when the code is 99% feature complete. We were evaluating content management systems the other day and started looking at FOSS projects. So we're looking at Joomla, features, etc., and it looks good. Then we get to the documentation and it seems like practically every other entry in the User manual's TOC is "todo". Not good.

With a commercial product they can at least afford to document the silly thing.

Re:The starving programmers

[ratboy666]

"There are many aspects of creating software that require non-technical administrative personell to handle. I don't remember ever hearing about OSA (Open Source Accountants) ;) These people cost money. And it's one thing to freely dedicate your time to devlopment, but it's quite another to freely donate large sums of personal money."

Probably because "Accountant" is not a creative process. There are many "Open Source Writers", "Open Source Poets", "Open Source Sculptors", and "Open Source Painters", etc.

"... programmers, who are doing some of the most mind-bending, skillful crafting of any career, get compensated for their work?"

Easy. If I write for myself, its free. If you want to tell me what to do, it costs money. Even if you want me to continue to work on something I did "for free", it'll cost you. I make a very comfortable living as a "programmer", and I am still amazed (every day) that people will pay me for my skills. I would do it for free, but then I would be working on what I wanted.

"Here's the irony in my eyes - right now hardware manufacturers pay Microsoft in order to get a little sticker that says "built for Windows XP".

This doesn't seem right. It seems totally backwards to me."

Wrong. "Hardware" isn't particularly useful (well, I guess you could use it as a boat anchor, a flower pot...

Until software is added...

Software is not "beholden" to hardware. And that's why you see that sticker.

YMMV
Ratboy

Re:The starving programmers

[Sentry21]

I think the main reason that (F)OSS is still having trouble competing (despite the widespread acceptance amoungst industry experts) is because of budget.

In this case, the budget to get a piece of software properly certified.

There are many aspects of creating software that require non-technical administrative personell to handle. I don't remember ever hearing about OSA (Open Source Accountants) ;) These people cost money. And it's one thing to freely dedicate your time to devlopment, but it's quite another to freely donate large sums of personal money.

It is truly inpspiring that so many can work together so well towards a common goal, and it is truly stunning to take in the vast amount of software available which is written pretty much completely philanthropically.

But the problem is that few actually get paid to do this, it is done in spare time.

In fact, a large majority of truly successful open-source projects are backed by a commercial or large not-for-profit entity. Apache, for example, has the Apache Foundation, PHP has Zend, MySQL has MySQL AB, GNOME has Novell, and Firefox has the Mozilla Foundation (and Mozilla Corporation). These organisations pay people for their time to do development work on these projects, because it benefits them directly. Most other open-source projects don't have the backing of such organisations, and thus don't have the time, money, or resources of their larger bretheren.

In my conversation I wondered how else software writers could make money, besides the tried and tested subscription and serial-activation systems in widespread use today. How else could programmers, who are doing some of the most mind-bending, skillful crafting of any career, get compensated for their work?

I'd be hesitant to make a claim like this. There are a lot of far more brilliant, far more skillful people out there doing far more interesting work that would make the average programmer's head spin. Most of the complex work I see programmers doing is implementing complex math (e.g. three-dimensional math, compression algorithms, etc) or mapping complex structures like protiens, DNA, etc., which I guess makes mathematicians, nuclear biologists, etc. the ones that do all the real work. The average nobodies that pound out GTK interfaces to mkisofs aren't doing anything worth praise or fanfare, in the grand scheme of things.

Wouldn't it be great if AMD, Intel, ASUS - or indeed any large electronics manufacturer - would offer up a pool to develop software?

The trick would of course be that the software should of course be standards compliant and thus run even on competitors hardware. But maybe AMD could have a certification that says, "written for and extensively tested on AMD hardware".

Software seems that it should be freely available - it just seems the nature of all information in general - but there is that problem that the programmer needs to make money.

It just seems to me that logically the consumer should buy the hardware - the physical, tangiable thing - and that it should be up to the hardware manufacturers to make hardware as a whole more useful.

Now this sounds like you're suggesting that the entire computing industry should be funded by the companies that make hardware - so the hardware makers become the software makers. The problem with that logic is that that will get you into a monopoly situation. You say 'should of course be standards compliant', but what standards are there, really? The x86 architecture is essentially managed by whoever happens to add a feature that everyone else copies. It's not an ISO or ANSI standard, and so there aren't any standards to base it off of. If Intel was funding people to write software for Intel machines, it would be trivial for them to make it ONLY work on Intel machines - edging AMD out of the market. Microsoft is notorious for doing this (take Lotus Notes a

Re:The starving programmers

[Lodragandraoidh]

I would have the added stipulation that the software on any hardware system be standards based...

Okay - who's standards?

You have to bare in mind interoperability. It is all well and good you want to standardize on 'Docbook' format for your documents - but might be a problem if you attempt to share these documents with your Microsoft Office-centric partners who have no desire to change.

The FDA's One Thing; Your Customers Are Another

[occamboy]

I have a fair bit of experience on the medical device side, not pharma - but they should be somewhat similar.

The FDA has standards for validationg COTS (Commercial Off-The-Shelf Software). ISO9000 is useful, but not necessary for FDA "stuff". For example, I don't think that any OS (Windows, Linux, etc..) is ISO9000 compliant, but your software runs on it, no? How could that be if ISO9000 was mandatory?

It seems to me that, for the FDA to be happy, you'd need to run some sort of reasonable validation of the COTS software, based on how it's being used; write a set of requirements, write a set of tests that prove that those requirements are met, and go to town. It would be good to recognize in the FMEA (Failure Modes and Effecta Analysis) that the COTS software probably has somewhat of a chance of failure - but then, the FDA (at least on the device side) has already announced that you must assume that software will sometimes fail, no matter how it's been tested. Finally, be exacting about audit trail - versions used, how things were built and installed, and so forth.

If your customer insists on third-party code being ISO 9000, despite the FDA not requiring it, then you're SOL. However, that requirement just doesn't make sense, unless they are not using an OS to run their software on (as described earlier).

Good luck!

Re:The FDA's One Thing; Your Customers Are Another

[SIWaters]

The FDA has a standard that's part of SRC 21 for medical device compliance -- 510(k). At its most basic level, it's a documentation process -- you have to document every market requirement, part spec, manufacturing prtocess, QA process, as well as the history of every part that goes into a device including who ordered it, when it was ordered, who it was ordered from, who received it on the loading dock, who accepted it, who did testing against spec, what device it went into, if there was ever a complaint filed on the device, if a part was ever returned for repair, and the disposition of all returned parts, etc, and so and so forth ad infinitum.

In case someone ever gets hurt by the device in use it's possible to figure out who okayed the decision(s) that lead to the alleged defect(s) in the product.

There is a COTS evaluation process, but it's pretty straightforward compared with the process for designing and developing the software that goes into a medical device.

How do I know this? I worked on the development of a software system to automate 510(k) compliance for a medical device manufacturer. I was responsible for translating all of the procedures in the Quality System into functional specs and UI designs. Along the way we learned that most companies do not have automated documentation systems for 510(k) compliance because it's far too complicated and there is no FDA spec for designing and implementing an electronic 510(k) compliance system. There used to be one but it was withdrawn because it was too complex.

What I can say is that the Software Development process covered in our Quality manual requires a level of detail I have never seen in any previous software development management system. Not only must every module and its inputs and outputs be documented, but algorithms must be documented. It must be possible (in case of an FDA audit) to trace customer requirements through to market specs to functional specs to technical specs to code, complete with test scenarios and procedures to ensure that the software sctually does what the design documents say it should do -- and that any deviation from specs discovered during testing are resolved.

The irony of all this is that the computer system we developed has too much information in it and FDA auditors aren't used to auditing the content in electronic systems. So, the system outputs paper reports that are put into file folders for the FDA to review if it's ever necessary.

Sigh.

-- Si

Fork it?

[rolfwind]

I'm not familiar with this software nor their licenses, but can't you just take a build of their software, fork it off as your own build and start treating it as internally made software? The greatest expense would be then certifying that first build.

Re:Fork it?

[Jeff DeMaagd]

I'm thinking that the cost of certifying OSS code might be pretty close to that of just writing new code in the first place. And then there are concerns over whether the software is considered to be redistributed such that the code changes would need to be returned to the public, especially if it is full GPL and whether any commercial code that links to the GPL code would technically need to be released if the software is considered to be redistributed.

Frankly, if the developer has any desires to not produce an open source app/OS/service then I'm not really seeing how using open source is going to be helpful unless they use code licensed by something like the BSD license code.

The starving arguments.

[Anonymous Coward]

"In my conversation I wondered how else software writers could make money, besides the tried and tested subscription and serial-activation systems in widespread use today. How else could programmers, who are doing some of the most mind-bending, skillful crafting of any career, get compensated for their work?"

If "Well, I never would have bought it anyway" is a valid answer towards those who expect monetary compensation. Then "I thought it was free" is a valid answer towards those expecting other forms of compensation.

Why can't you just certify the source code??

[JoeCommodore]

1. We have to review all of the code that is written, making sure that everything is traceable to a design specification.

2. Where we use 3rd party software/code we have to make sure that it comes from an ISO9000 source.

Since it is "open source" wouldn't just #1 apply? I think #2 would only be if you can't review the source, or do you guys get that privledge at that level?

What ISO9000 really means

[hacker]

ISO9000 means one thing:

Our process sucks, but its well-documented.

Re:What ISO9000 really means

[jjohnson]

That's the standard joke, but it exposes the basic point of ISO9000 certification: Your processes suck, but do you even know why?

Every organization struggles with doing what they say/think they're doing. How often have you heard the joke "do we do it the way we're supposed to do it, or the way we actually do it?" At the manufacturer where I worked, this was a common refrain.

The first step in eliminating the gap, and the point of ISO certification, is establishing the discipline of figuring out how things are actually getting done, and sticking to that method. Only then are you actually in a position to evaluate what you're doing and improve it.

I saw an endless stream of vice presidential policy announcements and procedure documents that got ignored on the warehouse floor. What good is an attempt to improve operations when there's a disconnect between the people writing the procedures and those (not) carrying them out?

Re:What ISO9000 really means

[darkmeridian]

As opposed to processes which may or may not suck but which cannot be measured because of the lack of documentation? In defense of ISO 9000, documentation allows reversions and enables organizations to learn from mistakes. I think of ISO 9000 as CVS: the code may be retarded, but CVS allows you to follow the progression of code so you can change it.

get yellow pages

[Anonymous Coward]

"but there is that problem that the programmer needs to make money"
OK
open yellow pages
notice businesses A to Z
notice computers and software used in most of those businesses
pick one, lotta choices, find some business you really could like
get job there
use software and your computer skills to make money in that business, for that business, whatever that is

you have now "made money with software" If you get to program some and you like it, cool. If you have to wrangle some machines around and do cabling, cool. Whatever. You are a computer guy, but maybe you might have to do other things as well-who cares?

there is no need at all to try and be a "pure" software company,or a "pure" programmer 100% of the time, in fact, it's becoming a dinosaur and there is right now 50 million people on the planet learning how to code, with another 100 million (whatever, some gigantic number that is rising fast) waiting in the wings. It is transforming from leet into ordinary. Ho hum stuff. It is not 1980, it is not the time when only a few thousand people knew how to code. And we have a ton of good code out there now to use. It has become diminishing returns really for most projects. How many different ways can you add up numbers, enter text in a paragraph form or arrange graphics on a page anyway? This should be a *clue*.

    The real world has real work to be done, software is a part of it,but don't try to force it to become all of it, especially with the Free and free stuff. Take advantage of it to go do some real work, don't make the software the work. You can use a hammer and saw to work and build stuff, or sit there all day long and polish the hammer and file down the teeth on the saw, admire how pretty it is, tweak the set this way or that by one degree, etc,and wonder how to make money doing that, but it really is better to just go build something with the tools. Occasionaly you may need a little polishing and filing, but for the most part it just might de-evolve into a lot of non productive busywork if you fixate on that.

This isn't as easy as it sounds.

[jd]

I've been told by bosses when working for the DoD that although the code I wanted to use was indeed audited, FIPS-compliant and published by the DoD themselves, it wasn't on an approved list of certified FIPS-compliant software so wasn't acceptable. When I asked how something the DoD wrote and published as an official DoD application could possibly not be acceptable for use by the DoD, I was told that procedures were procedures and had to be followed. The fact that it was GOTS and written by the people who do the certifying was of no consequence.

So, don't imagine that this is going to be an easy one. Open Source projects by IBM might be easier to get past the Great And All-Seeing (but definitely not all-knowing) Pointy Haired Bosses - at least some will be familiar with the phrase "nobody ever got sacked for buying IBM" and may even still believe that. Some of the Apache projects might be workable, provided you use the line of reasoning that since Apache is listed on IBM's website as a project they are working on, it is covered by the "nobody ever got sacked for buying IBM". You don't have to tell them that IBM is only one member of a large consortium, and it might be better not to.

Some projects were connected to IBM and other major corporations but are now independent. Postfix is an example of that. I believe evlog (Enterprise-grade event logging for Linux) is also such a project. Speaking of evlog, I would DEFINITELY suggest using it in any commercial or Government setting. It's not that good and Linux has plenty of other security, but "Enterprise-grade logging" is mandatory in many cases and this provides it. It ticks the right box, even if it doesn't do a whole lot more for you. It's a pure CYA and nothing more.

ISO 9000 (or later) compliance is probably the toughest requirement, as it stipulates documenting the process and activities, where the level of documentation depends on how critical the project is, and Open Source projects have neither that type of documentation or any real concept of criticalness as components are freely reusable. Your best bet is to work through vendors that are themselves ISO 900x certified AND supply either the Open Source OR the links to those projects, then argue that by documenting the use of a project that comes from an ISO 900x certified source, you inherit the certification indirectly. Some bosses will buy that easier than others and depending on the structure of the organization, you may have flexibility on who you present the case to. If so, shop.

NUnit? I doubt it.

[Anonymous Coward]

Logging software is part of the product. Testing software isn't (though it likely is much more important.) I am damn near certain that there is no regulation that would affect the software you use to test your software. (Think about it. If that was the case the FDA would have to certify that Windows was built according to process blah, blah, blah, and that, mercifully, has not happened.)

You're right, it's budget

[Howzer]

But it's not the compliance budget, as you say.

It's the fact that the license cost is zero that works against open source.

Huh?! Yeah, I know, that's hard to understand, but stay with me.

Think about it from the POV of whatever the "head of IT" is called in your company.

What's going to work for him come new job time? What will most qualify him for a "step up" in his career?

In one sentence: the size of the projects he's run. How is that size measured? Budget.

So there is zero incentive for "heads of IT" to go open source. That just reduces the overall size of project budgets, and reduces their standing in the pecking order the next time a sweet job comes up.

"I managed the implementation of a $20million project to replace the finance system." The interviewer will probably be further impressed if that sentence is followed up by "I decided on Big Brand Name X" especially if the interviewer has seen expensive ads for that brand name on billboards in airports.

Be nice to think that people were making technical decisions on the grounds of techincal excellence, or suitability for purpose, but that's an idealistic dream. People make decisions based on "what's best for me", and your head of IT is no different.

Re:You're right, it's budget

[Eivind]

That'd perhaps be true if projects where only measured by cost, and not by benefit. Which would be amazingly stupid. What you're basically saying is the more resources you waste (i.e. higher cost) the more chance of promotion. If any companies work like that, I pity them.

More relevant is the *benefit* of a project. If your project solves a problem worth a million bucks, while costing 100K to run, it's a *MUCH* better project than the 900K project that also saves (or earns) a million.

It's called ROI, and yes, it can be pretty darn hard to fairly estimate. That's no excuse for not even trying though.

Re:You're right, it's budget

[Howzer]

You're right. And companies generally do try, albeit internally, although their methods differ, which is half the point. When a person leaves a company, though, the measure of projects used in interviews (which is what I was talking about) is generally just cost. Candidate 1 Interview Excerpt: "I put in a new X system that cost $20 million." "How successful was it?" "Oh, extremely." Candidate 2 Interview Excerpt: "I put in a new X system that cost $5 million." "How successful was it?" "Oh, extremely." Who do you hire? It doesn't really matter that this is really a criticism of HR -- it's still a significant driver in the slow uptake of OSS, IMO.

Re:You're right, it's budget

[mrchaotica]

Candidate 3 Interview Excerpt: "I put in a new system X that would have cost $20 million, except I used Free Software and did it for only $1 million instead." "How successfull was it?" "Oh, extremely -- and I saved the company $19 million!"

In other words, calculate the cost of doing it the stupid way and then frame the discussion in terms of the amount of money you saved rather than the amount of money you spent.

Re:You're right, it's budget

[Eivind]

But that's just bad marketing of self on behalf of the candidate.

When someone asks what kind of projects you where responsible for, what is stopping you from saying; I was responsible for a large project that earned over $20 million for the company. You yourself has to consciously choose to use the cost of a project as a primary parameter. And that's just dumb.

Re:You're right, it's budget

[Howzer]

Interviewer: "So really, Candidate 3, the budget of the largest actual project you've managed was $1 million?"

Candidate 3: "Yes, but..."

Interviewer: "No, no, thanks, that's quite sufficient." *makes mark on clipboard* "Thanks for coming in. We'll call you."

Again, refusal to see the point I'm making doesn't change the point I'm making!

Of _course_ in an ideal environment all aspects of a person's experience would be taken into account. However, we're dealing with the real world, and the very real "pissing contest" that goes on in these kinds of environments, where people are measured by the _size_ of their last project.

You're saying, I presume, that this doesn't happen, or that this isn't a factor in low OSS takeup? Fair enough. I don't agree, clearly, but you're entitled to that view.

Re:You're right, it's budget

[Anonymous Coward]

When someone asks what kind of projects you where responsible for, what is stopping you from saying; I was responsible for a large project that earned over $20 million for the company.

Interviewer thinks: hmm, why isn't he saying what the budget for this project was? He's hiding something.

You yourself has to consciously choose to use the cost of a project as a primary parameter. And that's just dumb.

If only. Unfortunately, in the real world, interviewers ask standard questions and expect to receive standard answers. Leave out any of the standard information, and the interviewer will assume you're trying to gloss over something that reflects poorly on you. Mention that you managed a project, and conspicuously fail to give fundamental details like what your budget was, and they'll assume the worst.

It's jst a tool.

[StrawberryFrog]

All the code that goes into your program must be certified and follow the coding rules.

But must all the tools? Is your compiler? Is your IDE? I'm not sure that all the supporting infrastructure needs to be held to the same standard as code that goes into the final build. IF a tool finds a flaw, is that flaw somehow not a flaw if the tool is not certified?

The starving programmer? A myth.

[Colin Smith]

It is truly inpspiring that so many can work together so well towards a common goal, and it is truly stunning to take in the vast amount of software available which is written pretty much completely philanthropically.

Philanthropically? Um. no. Free and Open Source software is written because the author needs it. They requre the use of it. By allowing others to use it, the author loses nothing, they are not losing their time because they needed the software in the first place and the effort to copy it is negligible.

Writing Free and Open Source software is just as selfish as any other act. Software which is written with high ideals, for the betterment of man will not succeed because it isn't fulfilling a need.

But the problem is that few actually get paid to do this, it is done in spare time.

Simply not true. A huge amount of free and open source development work is performed by staff developers who are writing software which their organisations need.

Software is called soft because it's maleable, it can be moulded to fit the needs of the users. It's highly unlikely that any piece of software would exactly fit the needs of all users so there is a market out there for the customisation of almost all software, with free and open source software it's particularly easy, there is a market for customisation, service, support.

If you are a developer thinking of creating software for the benefit of mankind or the open source movement... Think again, there's enough abandonware out there already. Do it because you need it.

p.s. Free and Open Source software isn't having any trouble at all competing, it's filling niches left right and centre, though you may not see it filling your niche right now.

Re:The starving programmer? A myth.

[Anonymous Coward]

"Software which is written ... for the betterment of man will not succeed because it isn't fulfilling a need."

because man is already perfect, and hence has no need of betterment, perhaps?

Re:Import it into your own code base, and review i

[Stellian]

Simply import it into your own code base, and then review it as if it was written internally.

I think that's exactly what he's trying to avoid: it is probably cheaper to build your own from scratch, than to integrate, adapt, understand, test and review foreign code. After all, actual coding takes negligible resources (20% ?) when writing mission critical software.
I think what he wants, is to import open source code directly, without extra testing and reviewing, under the premise that it's popularity is a good enough guarantee of quality.
For example, if I need a kernel in my ISO9000 application, it's obvious that the Linux kernel has infinitely better quality than anything I could develop in-house. In fact, it's better than anything all the "certified" software developers in the world, staked together, could produce. And that just goes to show you the real value of certification.

Open Source and ISo 9000

[brunes69]

ISO 9000 mainly revolves around the procedures for developing the software being documented. Now, I may be oversimplifying things, but in the Open Source world, isn't *every* procedure documented, since the whole process is totally transparant?

I mean, you can *see* when the code was changed, you can *see* when bugs were fixed, you can see any QA that was done (if any)...

Shouldn't any Open Source product be able to get ISO 9000 certification, if they do the paperwork? I know that paperwork is *massive*, but if this guy's company wants to get NUnit certified, maybe his company could hire the consultant to do just that?

Re:Open Source and ISO 9000

[Entrope]

No; quite the opposite. Open source development that accepts third-party contributions makes it extraordinarily difficult to meet ISO 9000 process requirements. The standard is not just about documenting the process, but making sure you follow it consistently. When you have workers whose compliance is not audited, you cannot pass the audit. This means that continuously accepting third-party contributions has to be designed into the documented process -- something alien to most ISO 9000 practitioners -- and, in particular, quality controls and testing have to be designed to accomodate that. In practice, you would also want some fraction (at least 5-10%; higher if your process isn't tuned exactly to the open source model you use) of the time spent on the project to be internal process auditing by people who know both ISO 9000 and the documented process.

Re:Open Source and ISO 9000

[Anonymous Coward]

I may be clueless on this but doesn't Open Source just mean that products are licenced under a licence from certain subset of permissive licences? It doesn't require "Bazaar" development model.

Besides, even if it is used, you can specify in your ISO 9000 documents that the contributors (and they are NOT your workers!) are not audited (except, i.e. that they are permited by wichever third party, such as their employers, to contribute), but instead the contributions are audited (similar to any company that works on extraction of raw materials... you don't control the natural processes, but you have the input control and selection instead).

Re:Open Source and ISo 9000

[mabhatter654]

A Distro or individual project could get certified if they put the work into it.. it could even be a selling feature. I'd expect Suse and Red Hat already have something in place to satisfy these requirements as they deal in large enterprise installs already. OSS and ISO really do go hand-in-hand, but hackers tend to like to do things their way... and ISO is all about following instructions. Still, it could be a neat project to provide ISO, Sox, HIPAA, etc. testing to OSS projects in some fashion that was non-disruptive to the fun stuff. It could be something Google Code or Sourceforge add to their hosting solutions.

Re:Open Source and ISO 9000

[charlesnw]

So have a procedure in place that people have to follow in order to submit code. Problem solved. You then have a secondary procedure that does automated testing. Which you should have in place anyway. Like the mozilla project for example.

Re:Import it into your own code base, and review i

[nonsequitor]

ISO9000 is just part of it, and ISO9000 audits are a joke. Right now I'm testing avionics code, tracability of requirements is of course big. To use a 3rd party tool in verification that tool must be verified to the same standard that the project is being verified against. So for Level C code, all 3rd party tools must be verified for DO-178B Level C, and for Level A projects, the tools have to be verified to Level A as well. There are various loopholes as well. To speed things up writing C unit tests for an embedded target, I wrote a test code generator. The reason my generator doesn't need to be certified, is the output still needs to be reviewed by a human. Since I'm only generating code templates, the tool generating that source does not need to be certified. If I were generating all the test code and not reviewing it, it would need to be a 'qualified tool' which means it needs to be verified. I hope that helps.

Do it like a programmer

[npgmr]

Do the necessary unit testing to make sure it fits your requirement.

Open source use not prohibited for FDA approved...

[Fejimush]

I work as a software engineer for a company that manufactures an FDA approved medical device. We are ISO certified as well (for design and manufacturing). There are no restrictions on using open source software for FDA medically approved devices. We use Linux on our embedded single board computers, several of the boost libraries, and many more open source libraries. If you have restrictions using open source in your workplace it is most likely because your ISO certified design process limits open source use. The design process is defined by your company and certfied by TUV auditors for ISO compliance. The actual ISO 9001 documents that specify what is required for an ISO certified medical company is actually very light. Oftentimes companies make additional restrictions upon themselves for their ISO certified design process (usually some dipstick with an MBA and no engineering sense whatsoever, but doesn't realize it). For example, if someone (let's assume the dipstick MBA) at your company stated in your design process (prior to the initial ISO certification) that open source code cannot be used and the ISO auditors certified your design process with that exclusion, then your company is bound to that limitation (even though the FDA or ISO standards don't limit open source use). Bummer eh? Good luck. It's a shame you have to deal with those restrictions.

Re:Open source use not prohibited for FDA approved

[madvenu]

I fully agree with the parent comment.

And quoting from the document 'General Principles of Software Validation; Final Guidance for Industry and FDA Staff' at http://www.fda.gov/cdrh/comp/guidance/938.html#_To c517237928

The Scope section of the document refers to just this type of situation:-

'Where the software is developed by someone other than the device manufacturer (e.g., off-the-shelf software) the software developer may not be directly responsible for compliance with FDA regulations. In that case, the party with regulatory responsibility (i.e., the device manufacturer) needs to assess the adequacy of the off-the-shelf software developer's activities and determine what additional efforts are needed to establish that the software is validated for the device manufacturer's intended use.'

So - it is clear what the FDA wants - it clearly does NOT stop you from using Free/Open Source Software, it just requires you to use quality processes to ensure the software is verified / tested for your intended use - which you SHOULD anyway do, if want to develop INDUSTRIAL STRENGTH software.

--
venu

Cheat as everybody does

[wysiwia]

Nobody ever followed completely all the rules of ISO9000, everybody cheats to some degree even to the level where documents and actual development doesn't have anything in common. Do be ISO9000 compliant all you need to have the necessary docs.

If you want any OSS project be ISO9000 certified all you have to do is employ somebody who creates these docs. He doesn't have to actually develop all he has to do is create all the docs. Yet none of the actual developers have to change anything since what counts are the docs and nothing else. Just keep in mind ISO9000 does not show how the actual development is done, it only shows how it is documented. When you have realized this small but important difference you'll never fail any ISO9000 audit.

O. Wyss

Certifying open-source code

[Quiberon]

• Write a policy
• Implement the policy

  It's like 'how do you certify a new drug'. Taxol is an extract from yew trees; in principle an 'open-source' pharmaceutical, you discover it growing, rather than synthesise it from first principles. It's rumoured to be useful in treating some forms of human cancer.

  It doesn't save everyone; but it does extend the lives of some.

  If you go through the steps of how that got FDA approval, then you'll understand a credible way of arranging approval for "open source code where you cannot control the origin"

  Also, if you're depending on "other people's closed-source code", there will be a service end date beyond which you cannot count on using it. Service End Date for Windows98 has just passed; if you have medical equipment with Windows98 built in, now would be a good time to re-engineer it.

FDA regulations

[Paul Johnson]

I worked in a similar company, and had similar issues. The thing to understand is that the FDA inspectors are not (usually) software literate. Therefore they have to follow a "tick in the box" attitude to the CFR 820 (which if IIRC is the relevant regulation). This is very frustrating because it completely ignores all the real quality issues, and yet if you don't tick the right boxes then you can wind up in court or have your company closed down.

Don't blame the individual inspectors, BTW. They are just doing the job handed to them.

The trick here is to find a way to satisfy the regulations. Get your quality department on side here, because they are the ones who actually have to justify it to the inspector, and its their jobs on the line if they don't succeed.

As for open source, you need to get it considered under the same rules as other "COTS" software. I don't recall the details, but basically you have to come up with convincing evidence that some kind of process is properly followed. ISO9000 is the cheat sheet for proprietary software because its evidence that something of the right sort is done and audited. Open source doesn't have that, so you have to do it yourself.

Start by writing down what a good open source project looks like. Take as much as you can from ISO 9000 here. Do they have coding standards? Show they are adhered to. Configuration management? CVS. Code review? Contribution policy. Defect tracking? Bugzilla, and cite statistics showing that reported defects get fixed. Most large successful open source projects can ace this stuff anyway: if they couldn't they wouldn't be any good.

For requirements you have to get a bit more creative. Is it documented at the user level (e.g. API)? If so then you can call that documentation the requirements document and show that the source complies with it. The fact that this documentation was written after the software is irrelevant from your point of view because you only need to demonstrate that the software is fit for purpose, and that the documentation shows this. Write this argument down and it pretty much covers you.

You might also run a code review of a sample of the software to demonstrate that it meets your normal quality criteria.

Obviously this costs more than simply saying "supplier has ISO9000". But if its cheaper than buying ISO9000 then its still the right thing.

Incidentally, when I was doing this (a couple of years ago) some FDA staffer talked to our Quality Dept saying that open source was random, uncontrolled stuff that should never be allowed near real software. Basically they regurgitated a load of MS FUD. Be ready for this, and be ready for your QA dept to recite it back to you.

Paul.

Re:FDA regulations

[Fnkmaster]

Some creative person might think to set up a company that follows ISO-9000 procedures, take a bunch of Free/Open Source Software projects, manage a particular build or version of them, and offer them to companies as ISO-9000 certified versions of said software. They'd be exactly the same software, only accompanied by detailed documentation/audits of the code. For the service of having hired a few people to audit the software and document the hell out of it, you could charge companies all sorts of money to offer them ISO-9000 certified Open Source solutions.

BTW, I hereby patent this business model so you can't steal my brilliant idea. :)

Problem is documentation

[Kadin2048]

I think the problem that OSS software has is documentation.

Namely, that there isn't any. And I'm not talking about end-user documentation here, I mean process documentation. Specification documents and all that. The kind of stuff that normally gets developed alongside code, in any commercial/industrial development methodology.

There is an unspoken assumption behind the OSS ideals, and this is that the program's source code is the only documentation that anyone should ever need. This, frankly, is not true. It might be fine for code that has an obvious purpose and scope, and for systems software, but it starts to break down when you get into business software. How are you supposed to know from the source code, what the business process is that a particular segment of code is designed to support? You don't. How can you tell if something is a result of bad coding, or an incorrect design? You don't -- unless the same person both understands the code and the processes involved.

While it might be a safe assumption in many OSS projects to have the same person reading the code and analyzing the processes, this just doesn't happen in the real world. You usually have different people (even different groups of people) developing the specifications and processes, and writing the software from those specifications. In many cases, the people developing the specifications don't have the background or knowledge necessary to read the code directly.

I've said this elsewhere, but there's a lot of resistance in the OSS world to writing specifications. I don't know if this is because most of the software is written by programmers in their free time, and these people detest structured methodologies because they have to work with them in their day jobs, or if it's just a consequence of the way OSS is developed, but we're starting to see problems -- it's very hard to merge free software, where the code is the documentation, into a workflow where you have distinct levels of docs, where the code is only the very lowest level of end-product.

It's not if the systems to maintain documentation alongside code don't exist: some sort of Wiki-type interface, which was kept up-to-date against a project's official sources, would do just fine, and go a long ways towards improving the usefulness of OSS. However, there's little motivation for most projects to go to that level of effort.

I really don't have a good solution; I just know that I work in a situation very similar to the OP's, and we don't use any OSS code ... not because we really even care about license issues, but just because it would require more effort to document somebody else's code than it would just to draw up the documentation and have a programmer rewrite it. The former would require running our entire system "in reverse;" it would require a programmer to read the source code of the outside project and write a specification from it, which they're not used to doing. (I can almost see the objections forming right now.)

Re:Problem is documentation

[charlesnw]

Something like this? http://readyset.tigris.org/

self-certifying is very expensive

[rbrewer123]

The problem with doing your own audits of open source projects is that typically the open source project will have lots of extra bells and whistles and generality that your application doesn't need. It may also depend on various libraries that your application otherwise doesn't need. It may be very difficult to untangle the 20% of functionality that you actually need from the rest of the open source package. In the end, it may be easier to just write a "good enough" solution with only 80% of the features of the 20% you wanted from the open source package. If you write it using your internal libraries and coding standards it is probably easier to review and maintain as well, at least at first.

Of course, within another year or so you will have reimplemented all of the 20% of functionality you originally wanted from the open source software, and even more. The open source software will have advanced as well, but now you will be stuck maintaining a large codebase that still doesn't do as much as you could have had if you used the open source codebase originally.

It can be a tough decision.

Re:Import it into your own code base, and review i

[real gumby]

dnnrly: ankhcraft has it right (see below for my credentials for why I say this). Your customers are presumably operating under GMP and/or GLP (the pharmaceutical equivalent of ISO9000 and which is all described in CFR Title 21). They need a basis for your attestation as to the function and performance of the product you provide. Your using vendors claiming to adhere to ISO 9000 is really just a (reasonable!) way for you to not do as much cross-checking on their execution. If you import the software package and review it you can then make whatever performance claims you feel comfortable making -- probably far more robust ones than you could about most of your other outside vendors!

Don't forget that you do have to do all these checks. As a pharma manufacturer I tell the FDA that I rely on CoAs from my vendors, but I rely on them only by getting samples from them, cross-checking their work, and then also cross-checking that on the raw materials that are actually used in manufacturing. And I check during the manufacturing process and after manufacture as well just in case!

But you can't reasonably do all this for, say, a whole O/S. It's just too big and too complicated. This is why you'll see medical systems (or avionics) running on LynxOS or Green Hills' OS rather than standard Linux, ITRON or eCos (though eCos is small enough that you could probably review it yourself too). Regretfully, some are starting to ship with Windows which I am sure has not been subject to the equivalent review.

So why am I so confident in saying this?

• I am currently in the pharma business and running under GMP right now myself so am painfully aware of what it requires.
• I've previously (in previous companies) had plenty of customers who themselves run under ISO 9000 (in telecom, avionics, automotive systems, medical devices, military etc) and so know what they demand of themselves and of their vendors (e.g: downtime requirements of less than 3 minutes per decade)
• I was a cofounder of the first free software (we predated the term "Open Source") business, Cygnus Support, where we had those ISO 9000 customers and satisfied them.
• you're not my company. You'll have to cross-check my suggestions yourself.

In other words I'm probably the only person on the planet who's been under GMP, under software ISO9000 and also been a free software developer.

CFR 21 Part 11

[RUFar]

I work in the Pharma industry as well, and have some experience with validated systems - along with the associated regulations. CFR 21 Part 11 is the pertinent regulation for any and all pharmaceutical data systems - whether used for clinical trials or other cGMP purposes.

In a nutshell, it says nothing about an ISO 9000 requirement - that's SPECIFICALLY your company's policy. I understand why, it's the same deal as Sarbanes-Oxley compliance - they use ISO 9000 standard as the benchmark on which they make their rules, so following ISO 9000 means you'll meet compliance guidelines for process documentation and control on your data systems. It's also used as the standard for physical equipment manufacturing for pharma use, so more than likely one of your Quality guys knows about that, and figured "Hey, it's software manufacturing, and manufacturing means ISO 9000! Done!"

If you're going to use open-source software, you'll need to demonstrate to Quality that you can meet the guidelines for CFR 21 Part 11. I apologize for missing the name of the person who posted saying "act like it's your code" - but they're right. The key is good change management practice. If you use it and the company that makes it doesn't meet your Quality standards, then you need to take the source, review it, compile it yourself, test it extensively with the other applications you'll use on the same infrastructure.

And, every time you upgrade, you'll need to diff the source, document it, and compile and test again. The real deal is that it's your own Quality group setting the standard here, the FDA just wants you to show you have control, and have taken every precaution to avoid negative impact. Good luck!

Mod parent informative

[Eivind Eklund]

It's got the perfect goods for this particular discussion.

Standards for Safety Critical Software

[systemeng]

I don't have FDA experience but for industrial machinery, chemical plant safety systems, and the like, the standard followed is IEC 61508. Unlike ISO 9000, IEC61508 has various hard requirements for failure probabilities and measuring said probabilities. It also has specs like the highest possible reliability than can be assumed in engineering calculations for a device and what has to be done to mitigate failures. It occupies 7 binders on my shelf and provides the basis for how to develop systems that have a verifiably low chance of killing people. A fascinating intro to IEC 61508 is presened in the online edition of Embedded Systems Programming http://www.embedded.com//showArticle.jhtml?article ID=19201765

How about commercial vendors

[savio13]

You could probably find a commercial software company that supports the open source software. For example, you can get support for many Apache products (HTTPD, Tomcat, Geronimo, Axis) from several vendors (HP, IBM, Red Hat, SourceLabs, SpikeSource, etc). And I remember seeing something about SourceLabs 'Certified Software', so maybe try them? S

Re:Fork it? The Perfect leach.

[gd23ka]

Huh? What's wrong with a corporation to "give back" once in a blue moon
even if it's only with funding and supplying the man power for the certification of a
certain piece of OSS?

Why not guide your boss's thinking down that avenue and let the corp,
which is probably benefiting from OSS with a couple of hundred Linux servers,
give a little something back to "the community" ??!