Industrial Strength Open Source Code?

dnnrly asks: "I work for a company that writes software for the pharmaceutical industry. We have to work in quite a tight regulatory environment because some of our code ends up in the process of drug testing. Seeing as the FDA are quite picky about making sure that there can be no errors in testing new drugs, our clients have strict rules that we must follow for coding. We have to review all of the code that is written, making sure that everything is traceable to a design specification. Where we use 3rd party software/code we have to make sure that it comes from an ISO9000 source. This is a bit of a problem when we would like to use open source stuff in our code. Projects like log4net and NUnit would be tremendously useful in our code but we're not allowed to use them because they don't tick the right boxes. Now, *I* know that these projects (and others) are incredibly stable just because of the volume of use that they have seen but that isn't enough for some people. How can we certify such software?"

Comments

ISO 9000

[deanj]

No offense intended, but an ISO 9000 source?

ISO 9000 just means the people that "certify" this have all their procedures documented.

I'd be much more interested in a company that delivered software that worked, and stood behind it, rather than a company that puts all it's time and effort into ISO 9000. Every time I hear someone asking for that, they're more interested in being buzzword compliant than anything else.

Man, oh man.... The next thing people will be saying is that the company mission statements actually make a difference.

Re:ISO 9000

[Stradivarius]

I'm sure many of the companies that are certified would probably agree with you. But the fact is that many companies, particularly those selling to government entities, are required to be certified in ISO 9000 or similar programs as a precondition of doing business. So they have to go along, at least to some extent, with the fiction that certification equals quality.

IMHO, certification really means repeatability (do you always follow some established process). Which is certainly important, especially when it comes to very expensive/complex programs, in order to mitigate risk. But it's not the same as quality. One can repeatably implement a lousy process or procedure. One can also be repeatably bad at implementing a good process (you can have the best design process in the world, but a bad designer will still give you a bad product).

I think the whole certification buzz comes about in large part because it's difficult to evaluate the other big factors that impact quality, which are largely human factors - talent, workmanship, management acumen, etc. Because customers have little visibility into those factors, they instead evaluate the factor most amenable to measurement - process compliance.

People with some common sense recognize the certification as a useful but limited tool for evaluating a company. It's the other folks that you seem to have encountered - the ones who think process compliance is the Holy Grail for business success. Those are the ones I try to avoid :-)

Re:ISO 9000

[Monkelectric]

I think ISO 9k is an industry scam. I worked for a company which had no process whatsoever -- literally none. When a new piece of software needed to be written -- someone would walk into your office and say, "I need something to do X" That *WAS* the entire software process. No requirements, no architect, no analysis, no test plan, no documentation, no testing, nothing.

We were 9k certified.

Re:ISO 9000

[NateTech]

ISO 9000 is a documentation process, not a quality process. People don't seem to get that, and the marketing spin around it for years has not exactly made that very clear.

One of the engineers at work keeps a photograph of the Firestone plant that produced all the deadly, defective tires for Ford.

The photo shows clearly:
- The plant was shut down and closed.
- The plant has a large "ISO 9000 Certified" sign on the entrance sign.

ISO 9000 just means that you documented your procedures and you can verify and prove that you followed them. It does not tell you whether or not they're smart, safe, profitable, or anything else about your business.

In other words: ISO 9000 forces a company to document what they're doing, but can't save the idiots from doing the wrong things.

Import it into your own code base, and review it.

[ankhcraft]

Simply import it into your own code base, and then review it as if it was written internally. Basically, learn it inside out, as if you wrote it yourself. If that is not legally sufficient, then the laws need to be rewritten since the lines they would be attempting to delineate would at this point be completely imaginary. It doesn't matter whose head it originates from, what matters is that it is fully reviewed and completely understood to the point where everyone on your team is prepared to stand behind the entire body of code. If that confidence comes from actual understand, it becomes irrelevant who wrote the code in the first place. How would it be any different if, instead, it was code written by somebody who no longer works at the company.

Mod Parent Up ISO9000 Compliant

[argent]

ISO9000 just means you have documented procedures and you track the steps you take when you follow those procedures.

You shouldn't even need to go to the lengths of merging the code bases.

Document the procedures for auditing and verifying external software to the standards of your internally written software, create an ISO9000 process for tracking those procedures, and follow them.

The starving programmers

[w33t]

This brings to mind a conversation I had the other day.

I think the main reason that (F)OSS is still having trouble competing (despite the widespread acceptance amoungst industry experts) is because of budget.

In this case, the budget to get a piece of software properly certified.

There are many aspects of creating software that require non-technical administrative personell to handle. I don't remember ever hearing about OSA (Open Source Accountants) ;) These people cost money. And it's one thing to freely dedicate your time to devlopment, but it's quite another to freely donate large sums of personal money.

It is truly inpspiring that so many can work together so well towards a common goal, and it is truly stunning to take in the vast amount of software available which is written pretty much completely philanthropically.

But the problem is that few actually get paid to do this, it is done in spare time.

In my conversation I wondered how else software writers could make money, besides the tried and tested subscription and serial-activation systems in widespread use today. How else could programmers, who are doing some of the most mind-bending, skillful crafting of any career, get compensated for their work?

Wouldn't it be great if AMD, Intel, ASUS - or indeed any large electronics manufacturer - would offer up a pool to develop software?

The trick would of course be that the software should of course be standards compliant and thus run even on competitors hardware. But maybe AMD could have a certification that says, "written for and extensively tested on AMD hardware".

Software seems that it should be freely available - it just seems the nature of all information in general - but there is that problem that the programmer needs to make money.

It just seems to me that logically the consumer should buy the hardware - the physical, tangiable thing - and that it should be up to the hardware manufacturers to make hardware as a whole more useful.

Here's the irony in my eyes - right now hardware manufacturers pay Microsoft in order to get a little sticker that says "built for Windows XP".

This doesn't seem right. It seems totally backwards to me.

It reminds me of a quote from "V for Vendetta": "People shouldn't be afraid of their governments, governments should be afraid of their people."

In other words, "Harware manufacturers shouldn't be beholden to the software companies, software companies should be beholden to the hardware."

The fact of the matter is that I just wish that after I pay $2,000 for my nice new system that it would run right out of the box. Macintosh is close - but I would have the added stipulation that the software on any hardware system be standards based so that I don't have a Dell OS, IBM OS, HP OS, Alienware OS...etc.

These are just thoughts, and I understand that there are many counterpoints.

The FDA's One Thing; Your Customers Are Another

[occamboy]

I have a fair bit of experience on the medical device side, not pharma - but they should be somewhat similar.

The FDA has standards for validationg COTS (Commercial Off-The-Shelf Software). ISO9000 is useful, but not necessary for FDA "stuff". For example, I don't think that any OS (Windows, Linux, etc..) is ISO9000 compliant, but your software runs on it, no? How could that be if ISO9000 was mandatory?

It seems to me that, for the FDA to be happy, you'd need to run some sort of reasonable validation of the COTS software, based on how it's being used; write a set of requirements, write a set of tests that prove that those requirements are met, and go to town. It would be good to recognize in the FMEA (Failure Modes and Effecta Analysis) that the COTS software probably has somewhat of a chance of failure - but then, the FDA (at least on the device side) has already announced that you must assume that software will sometimes fail, no matter how it's been tested. Finally, be exacting about audit trail - versions used, how things were built and installed, and so forth.

If your customer insists on third-party code being ISO 9000, despite the FDA not requiring it, then you're SOL. However, that requirement just doesn't make sense, unless they are not using an OS to run their software on (as described earlier).

Good luck!

What ISO9000 really means

[hacker]

ISO9000 means one thing:

Our process sucks, but its well-documented.

Re:Laff

[ClamIAm]

The testing has to be perfect. The drugs? Not so much.

And I don't understand why the parent is modded troll.

This isn't as easy as it sounds.

[jd]

I've been told by bosses when working for the DoD that although the code I wanted to use was indeed audited, FIPS-compliant and published by the DoD themselves, it wasn't on an approved list of certified FIPS-compliant software so wasn't acceptable. When I asked how something the DoD wrote and published as an official DoD application could possibly not be acceptable for use by the DoD, I was told that procedures were procedures and had to be followed. The fact that it was GOTS and written by the people who do the certifying was of no consequence.

So, don't imagine that this is going to be an easy one. Open Source projects by IBM might be easier to get past the Great And All-Seeing (but definitely not all-knowing) Pointy Haired Bosses - at least some will be familiar with the phrase "nobody ever got sacked for buying IBM" and may even still believe that. Some of the Apache projects might be workable, provided you use the line of reasoning that since Apache is listed on IBM's website as a project they are working on, it is covered by the "nobody ever got sacked for buying IBM". You don't have to tell them that IBM is only one member of a large consortium, and it might be better not to.

Some projects were connected to IBM and other major corporations but are now independent. Postfix is an example of that. I believe evlog (Enterprise-grade event logging for Linux) is also such a project. Speaking of evlog, I would DEFINITELY suggest using it in any commercial or Government setting. It's not that good and Linux has plenty of other security, but "Enterprise-grade logging" is mandatory in many cases and this provides it. It ticks the right box, even if it doesn't do a whole lot more for you. It's a pure CYA and nothing more.

ISO 9000 (or later) compliance is probably the toughest requirement, as it stipulates documenting the process and activities, where the level of documentation depends on how critical the project is, and Open Source projects have neither that type of documentation or any real concept of criticalness as components are freely reusable. Your best bet is to work through vendors that are themselves ISO 900x certified AND supply either the Open Source OR the links to those projects, then argue that by documenting the use of a project that comes from an ISO 900x certified source, you inherit the certification indirectly. Some bosses will buy that easier than others and depending on the structure of the organization, you may have flexibility on who you present the case to. If so, shop.

Re:Import it into your own code base, and review i

[real gumby]

dnnrly: ankhcraft has it right (see below for my credentials for why I say this). Your customers are presumably operating under GMP and/or GLP (the pharmaceutical equivalent of ISO9000 and which is all described in CFR Title 21). They need a basis for your attestation as to the function and performance of the product you provide. Your using vendors claiming to adhere to ISO 9000 is really just a (reasonable!) way for you to not do as much cross-checking on their execution. If you import the software package and review it you can then make whatever performance claims you feel comfortable making -- probably far more robust ones than you could about most of your other outside vendors!

Don't forget that you do have to do all these checks. As a pharma manufacturer I tell the FDA that I rely on CoAs from my vendors, but I rely on them only by getting samples from them, cross-checking their work, and then also cross-checking that on the raw materials that are actually used in manufacturing. And I check during the manufacturing process and after manufacture as well just in case!

But you can't reasonably do all this for, say, a whole O/S. It's just too big and too complicated. This is why you'll see medical systems (or avionics) running on LynxOS or Green Hills' OS rather than standard Linux, ITRON or eCos (though eCos is small enough that you could probably review it yourself too). Regretfully, some are starting to ship with Windows which I am sure has not been subject to the equivalent review.

So why am I so confident in saying this?

• I am currently in the pharma business and running under GMP right now myself so am painfully aware of what it requires.
• I've previously (in previous companies) had plenty of customers who themselves run under ISO 9000 (in telecom, avionics, automotive systems, medical devices, military etc) and so know what they demand of themselves and of their vendors (e.g: downtime requirements of less than 3 minutes per decade)
• I was a cofounder of the first free software (we predated the term "Open Source") business, Cygnus Support, where we had those ISO 9000 customers and satisfied them.
• you're not my company. You'll have to cross-check my suggestions yourself.

In other words I'm probably the only person on the planet who's been under GMP, under software ISO9000 and also been a free software developer.