Slashdot Mirror
Industrial Strength Open Source Code?
dnnrly asks: "I work for a company that writes software for the pharmaceutical industry. We have to work in quite a tight regulatory environment because some of our code ends up in the process of drug testing. Seeing as the FDA are quite picky about making sure that there can be no errors in testing new drugs, our clients have strict rules that we must follow for coding. We have to review all of the code that is written, making sure that everything is traceable to a design specification. Where we use 3rd party software/code we have to make sure that it comes from an ISO9000 source. This is a bit of a problem when we would like to use open source stuff in our code. Projects like log4net and NUnit would be tremendously useful in our code but we're not allowed to use them because they don't tick the right boxes. Now, *I* know that these projects (and others) are incredibly stable just because of the volume of use that they have seen but that isn't enough for some people. How can we certify such software?"
ISO9000 just means you have documented procedures and you track the steps you take when you follow those procedures.
You shouldn't even need to go to the lengths of merging the code bases.
Document the procedures for auditing and verifying external software to the standards of your internally written software, create an ISO9000 process for tracking those procedures, and follow them.
I have a fair bit of experience on the medical device side, not pharma - but they should be somewhat similar.
The FDA has standards for validationg COTS (Commercial Off-The-Shelf Software). ISO9000 is useful, but not necessary for FDA "stuff". For example, I don't think that any OS (Windows, Linux, etc..) is ISO9000 compliant, but your software runs on it, no? How could that be if ISO9000 was mandatory?
It seems to me that, for the FDA to be happy, you'd need to run some sort of reasonable validation of the COTS software, based on how it's being used; write a set of requirements, write a set of tests that prove that those requirements are met, and go to town. It would be good to recognize in the FMEA (Failure Modes and Effecta Analysis) that the COTS software probably has somewhat of a chance of failure - but then, the FDA (at least on the device side) has already announced that you must assume that software will sometimes fail, no matter how it's been tested. Finally, be exacting about audit trail - versions used, how things were built and installed, and so forth.
If your customer insists on third-party code being ISO 9000, despite the FDA not requiring it, then you're SOL. However, that requirement just doesn't make sense, unless they are not using an OS to run their software on (as described earlier).
Good luck!
No; quite the opposite. Open source development that accepts third-party contributions makes it extraordinarily difficult to meet ISO 9000 process requirements. The standard is not just about documenting the process, but making sure you follow it consistently. When you have workers whose compliance is not audited, you cannot pass the audit. This means that continuously accepting third-party contributions has to be designed into the documented process -- something alien to most ISO 9000 practitioners -- and, in particular, quality controls and testing have to be designed to accomodate that. In practice, you would also want some fraction (at least 5-10%; higher if your process isn't tuned exactly to the open source model you use) of the time spent on the project to be internal process auditing by people who know both ISO 9000 and the documented process.
I worked in a similar company, and had similar issues. The thing to understand is that the FDA inspectors are not (usually) software literate. Therefore they have to follow a "tick in the box" attitude to the CFR 820 (which if IIRC is the relevant regulation). This is very frustrating because it completely ignores all the real quality issues, and yet if you don't tick the right boxes then you can wind up in court or have your company closed down.
Don't blame the individual inspectors, BTW. They are just doing the job handed to them.
The trick here is to find a way to satisfy the regulations. Get your quality department on side here, because they are the ones who actually have to justify it to the inspector, and its their jobs on the line if they don't succeed.
As for open source, you need to get it considered under the same rules as other "COTS" software. I don't recall the details, but basically you have to come up with convincing evidence that some kind of process is properly followed. ISO9000 is the cheat sheet for proprietary software because its evidence that something of the right sort is done and audited. Open source doesn't have that, so you have to do it yourself.
Start by writing down what a good open source project looks like. Take as much as you can from ISO 9000 here. Do they have coding standards? Show they are adhered to. Configuration management? CVS. Code review? Contribution policy. Defect tracking? Bugzilla, and cite statistics showing that reported defects get fixed. Most large successful open source projects can ace this stuff anyway: if they couldn't they wouldn't be any good.
For requirements you have to get a bit more creative. Is it documented at the user level (e.g. API)? If so then you can call that documentation the requirements document and show that the source complies with it. The fact that this documentation was written after the software is irrelevant from your point of view because you only need to demonstrate that the software is fit for purpose, and that the documentation shows this. Write this argument down and it pretty much covers you.
You might also run a code review of a sample of the software to demonstrate that it meets your normal quality criteria.
Obviously this costs more than simply saying "supplier has ISO9000". But if its cheaper than buying ISO9000 then its still the right thing.
Incidentally, when I was doing this (a couple of years ago) some FDA staffer talked to our Quality Dept saying that open source was random, uncontrolled stuff that should never be allowed near real software. Basically they regurgitated a load of MS FUD. Be ready for this, and be ready for your QA dept to recite it back to you.
Paul.
You are lost in a twisty maze of little standards, all different.