Slashdot Mirror
Industrial Strength Open Source Code?
dnnrly asks: "I work for a company that writes software for the pharmaceutical industry. We have to work in quite a tight regulatory environment because some of our code ends up in the process of drug testing. Seeing as the FDA are quite picky about making sure that there can be no errors in testing new drugs, our clients have strict rules that we must follow for coding. We have to review all of the code that is written, making sure that everything is traceable to a design specification. Where we use 3rd party software/code we have to make sure that it comes from an ISO9000 source. This is a bit of a problem when we would like to use open source stuff in our code. Projects like log4net and NUnit would be tremendously useful in our code but we're not allowed to use them because they don't tick the right boxes. Now, *I* know that these projects (and others) are incredibly stable just because of the volume of use that they have seen but that isn't enough for some people. How can we certify such software?"
No offense intended, but an ISO 9000 source?
ISO 9000 just means the people that "certify" this have all their procedures documented.
I'd be much more interested in a company that delivered software that worked, and stood behind it, rather than a company that puts all it's time and effort into ISO 9000. Every time I hear someone asking for that, they're more interested in being buzzword compliant than anything else.
Man, oh man.... The next thing people will be saying is that the company mission statements actually make a difference.
"Seeing as the FDA are quite picky about making sure that there can be no errors in testing new drugs."
Simply import it into your own code base, and then review it as if it was written internally. Basically, learn it inside out, as if you wrote it yourself. If that is not legally sufficient, then the laws need to be rewritten since the lines they would be attempting to delineate would at this point be completely imaginary. It doesn't matter whose head it originates from, what matters is that it is fully reviewed and completely understood to the point where everyone on your team is prepared to stand behind the entire body of code. If that confidence comes from actual understand, it becomes irrelevant who wrote the code in the first place. How would it be any different if, instead, it was code written by somebody who no longer works at the company.
...
Could you, for the purposes of certification, take ownership of the libraries you want to use? By "take ownership" I mean either a) acquire, or b) create the necessary design documents for the libraries, then code review them as you would your own code.
ISO9000 just means you have documented procedures and you track the steps you take when you follow those procedures.
You shouldn't even need to go to the lengths of merging the code bases.
Document the procedures for auditing and verifying external software to the standards of your internally written software, create an ISO9000 process for tracking those procedures, and follow them.
This brings to mind a conversation I had the other day.
;) These people cost money. And it's one thing to freely dedicate your time to devlopment, but it's quite another to freely donate large sums of personal money.
I think the main reason that (F)OSS is still having trouble competing (despite the widespread acceptance amoungst industry experts) is because of budget.
In this case, the budget to get a piece of software properly certified.
There are many aspects of creating software that require non-technical administrative personell to handle. I don't remember ever hearing about OSA (Open Source Accountants)
It is truly inpspiring that so many can work together so well towards a common goal, and it is truly stunning to take in the vast amount of software available which is written pretty much completely philanthropically.
But the problem is that few actually get paid to do this, it is done in spare time.
In my conversation I wondered how else software writers could make money, besides the tried and tested subscription and serial-activation systems in widespread use today. How else could programmers, who are doing some of the most mind-bending, skillful crafting of any career, get compensated for their work?
Wouldn't it be great if AMD, Intel, ASUS - or indeed any large electronics manufacturer - would offer up a pool to develop software?
The trick would of course be that the software should of course be standards compliant and thus run even on competitors hardware. But maybe AMD could have a certification that says, "written for and extensively tested on AMD hardware".
Software seems that it should be freely available - it just seems the nature of all information in general - but there is that problem that the programmer needs to make money.
It just seems to me that logically the consumer should buy the hardware - the physical, tangiable thing - and that it should be up to the hardware manufacturers to make hardware as a whole more useful.
Here's the irony in my eyes - right now hardware manufacturers pay Microsoft in order to get a little sticker that says "built for Windows XP".
This doesn't seem right. It seems totally backwards to me.
It reminds me of a quote from "V for Vendetta": "People shouldn't be afraid of their governments, governments should be afraid of their people."
In other words, "Harware manufacturers shouldn't be beholden to the software companies, software companies should be beholden to the hardware."
The fact of the matter is that I just wish that after I pay $2,000 for my nice new system that it would run right out of the box. Macintosh is close - but I would have the added stipulation that the software on any hardware system be standards based so that I don't have a Dell OS, IBM OS, HP OS, Alienware OS...etc.
These are just thoughts, and I understand that there are many counterpoints.
My Computer Music Tutorial Videos
I have a fair bit of experience on the medical device side, not pharma - but they should be somewhat similar.
The FDA has standards for validationg COTS (Commercial Off-The-Shelf Software). ISO9000 is useful, but not necessary for FDA "stuff". For example, I don't think that any OS (Windows, Linux, etc..) is ISO9000 compliant, but your software runs on it, no? How could that be if ISO9000 was mandatory?
It seems to me that, for the FDA to be happy, you'd need to run some sort of reasonable validation of the COTS software, based on how it's being used; write a set of requirements, write a set of tests that prove that those requirements are met, and go to town. It would be good to recognize in the FMEA (Failure Modes and Effecta Analysis) that the COTS software probably has somewhat of a chance of failure - but then, the FDA (at least on the device side) has already announced that you must assume that software will sometimes fail, no matter how it's been tested. Finally, be exacting about audit trail - versions used, how things were built and installed, and so forth.
If your customer insists on third-party code being ISO 9000, despite the FDA not requiring it, then you're SOL. However, that requirement just doesn't make sense, unless they are not using an OS to run their software on (as described earlier).
Good luck!
I'm not familiar with this software nor their licenses, but can't you just take a build of their software, fork it off as your own build and start treating it as internally made software? The greatest expense would be then certifying that first build.
ISO9000 means one thing:
So, don't imagine that this is going to be an easy one. Open Source projects by IBM might be easier to get past the Great And All-Seeing (but definitely not all-knowing) Pointy Haired Bosses - at least some will be familiar with the phrase "nobody ever got sacked for buying IBM" and may even still believe that. Some of the Apache projects might be workable, provided you use the line of reasoning that since Apache is listed on IBM's website as a project they are working on, it is covered by the "nobody ever got sacked for buying IBM". You don't have to tell them that IBM is only one member of a large consortium, and it might be better not to.
Some projects were connected to IBM and other major corporations but are now independent. Postfix is an example of that. I believe evlog (Enterprise-grade event logging for Linux) is also such a project. Speaking of evlog, I would DEFINITELY suggest using it in any commercial or Government setting. It's not that good and Linux has plenty of other security, but "Enterprise-grade logging" is mandatory in many cases and this provides it. It ticks the right box, even if it doesn't do a whole lot more for you. It's a pure CYA and nothing more.
ISO 9000 (or later) compliance is probably the toughest requirement, as it stipulates documenting the process and activities, where the level of documentation depends on how critical the project is, and Open Source projects have neither that type of documentation or any real concept of criticalness as components are freely reusable. Your best bet is to work through vendors that are themselves ISO 900x certified AND supply either the Open Source OR the links to those projects, then argue that by documenting the use of a project that comes from an ISO 900x certified source, you inherit the certification indirectly. Some bosses will buy that easier than others and depending on the structure of the organization, you may have flexibility on who you present the case to. If so, shop.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Philanthropically? Um. no. Free and Open Source software is written because the author needs it. They requre the use of it. By allowing others to use it, the author loses nothing, they are not losing their time because they needed the software in the first place and the effort to copy it is negligible.
Writing Free and Open Source software is just as selfish as any other act. Software which is written with high ideals, for the betterment of man will not succeed because it isn't fulfilling a need.
But the problem is that few actually get paid to do this, it is done in spare time.
Simply not true. A huge amount of free and open source development work is performed by staff developers who are writing software which their organisations need.
Software is called soft because it's maleable, it can be moulded to fit the needs of the users. It's highly unlikely that any piece of software would exactly fit the needs of all users so there is a market out there for the customisation of almost all software, with free and open source software it's particularly easy, there is a market for customisation, service, support.
If you are a developer thinking of creating software for the benefit of mankind or the open source movement... Think again, there's enough abandonware out there already. Do it because you need it.
p.s. Free and Open Source software isn't having any trouble at all competing, it's filling niches left right and centre, though you may not see it filling your niche right now.
Deleted
No; quite the opposite. Open source development that accepts third-party contributions makes it extraordinarily difficult to meet ISO 9000 process requirements. The standard is not just about documenting the process, but making sure you follow it consistently. When you have workers whose compliance is not audited, you cannot pass the audit. This means that continuously accepting third-party contributions has to be designed into the documented process -- something alien to most ISO 9000 practitioners -- and, in particular, quality controls and testing have to be designed to accomodate that. In practice, you would also want some fraction (at least 5-10%; higher if your process isn't tuned exactly to the open source model you use) of the time spent on the project to be internal process auditing by people who know both ISO 9000 and the documented process.
A Distro or individual project could get certified if they put the work into it.. it could even be a selling feature. I'd expect Suse and Red Hat already have something in place to satisfy these requirements as they deal in large enterprise installs already. OSS and ISO really do go hand-in-hand, but hackers tend to like to do things their way... and ISO is all about following instructions. Still, it could be a neat project to provide ISO, Sox, HIPAA, etc. testing to OSS projects in some fashion that was non-disruptive to the fun stuff. It could be something Google Code or Sourceforge add to their hosting solutions.
Candidate 3 Interview Excerpt: "I put in a new system X that would have cost $20 million, except I used Free Software and did it for only $1 million instead." "How successfull was it?" "Oh, extremely -- and I saved the company $19 million!"
In other words, calculate the cost of doing it the stupid way and then frame the discussion in terms of the amount of money you saved rather than the amount of money you spent.
"[Regarding the 'cloud,'] ownership was what made America different than Russia." -- Woz
I worked in a similar company, and had similar issues. The thing to understand is that the FDA inspectors are not (usually) software literate. Therefore they have to follow a "tick in the box" attitude to the CFR 820 (which if IIRC is the relevant regulation). This is very frustrating because it completely ignores all the real quality issues, and yet if you don't tick the right boxes then you can wind up in court or have your company closed down.
Don't blame the individual inspectors, BTW. They are just doing the job handed to them.
The trick here is to find a way to satisfy the regulations. Get your quality department on side here, because they are the ones who actually have to justify it to the inspector, and its their jobs on the line if they don't succeed.
As for open source, you need to get it considered under the same rules as other "COTS" software. I don't recall the details, but basically you have to come up with convincing evidence that some kind of process is properly followed. ISO9000 is the cheat sheet for proprietary software because its evidence that something of the right sort is done and audited. Open source doesn't have that, so you have to do it yourself.
Start by writing down what a good open source project looks like. Take as much as you can from ISO 9000 here. Do they have coding standards? Show they are adhered to. Configuration management? CVS. Code review? Contribution policy. Defect tracking? Bugzilla, and cite statistics showing that reported defects get fixed. Most large successful open source projects can ace this stuff anyway: if they couldn't they wouldn't be any good.
For requirements you have to get a bit more creative. Is it documented at the user level (e.g. API)? If so then you can call that documentation the requirements document and show that the source complies with it. The fact that this documentation was written after the software is irrelevant from your point of view because you only need to demonstrate that the software is fit for purpose, and that the documentation shows this. Write this argument down and it pretty much covers you.
You might also run a code review of a sample of the software to demonstrate that it meets your normal quality criteria.
Obviously this costs more than simply saying "supplier has ISO9000". But if its cheaper than buying ISO9000 then its still the right thing.
Incidentally, when I was doing this (a couple of years ago) some FDA staffer talked to our Quality Dept saying that open source was random, uncontrolled stuff that should never be allowed near real software. Basically they regurgitated a load of MS FUD. Be ready for this, and be ready for your QA dept to recite it back to you.
Paul.
You are lost in a twisty maze of little standards, all different.
I think the problem that OSS software has is documentation.
... not because we really even care about license issues, but just because it would require more effort to document somebody else's code than it would just to draw up the documentation and have a programmer rewrite it. The former would require running our entire system "in reverse;" it would require a programmer to read the source code of the outside project and write a specification from it, which they're not used to doing. (I can almost see the objections forming right now.)
Namely, that there isn't any. And I'm not talking about end-user documentation here, I mean process documentation. Specification documents and all that. The kind of stuff that normally gets developed alongside code, in any commercial/industrial development methodology.
There is an unspoken assumption behind the OSS ideals, and this is that the program's source code is the only documentation that anyone should ever need. This, frankly, is not true. It might be fine for code that has an obvious purpose and scope, and for systems software, but it starts to break down when you get into business software. How are you supposed to know from the source code, what the business process is that a particular segment of code is designed to support? You don't. How can you tell if something is a result of bad coding, or an incorrect design? You don't -- unless the same person both understands the code and the processes involved.
While it might be a safe assumption in many OSS projects to have the same person reading the code and analyzing the processes, this just doesn't happen in the real world. You usually have different people (even different groups of people) developing the specifications and processes, and writing the software from those specifications. In many cases, the people developing the specifications don't have the background or knowledge necessary to read the code directly.
I've said this elsewhere, but there's a lot of resistance in the OSS world to writing specifications. I don't know if this is because most of the software is written by programmers in their free time, and these people detest structured methodologies because they have to work with them in their day jobs, or if it's just a consequence of the way OSS is developed, but we're starting to see problems -- it's very hard to merge free software, where the code is the documentation, into a workflow where you have distinct levels of docs, where the code is only the very lowest level of end-product.
It's not if the systems to maintain documentation alongside code don't exist: some sort of Wiki-type interface, which was kept up-to-date against a project's official sources, would do just fine, and go a long ways towards improving the usefulness of OSS. However, there's little motivation for most projects to go to that level of effort.
I really don't have a good solution; I just know that I work in a situation very similar to the OP's, and we don't use any OSS code
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
Don't forget that you do have to do all these checks. As a pharma manufacturer I tell the FDA that I rely on CoAs from my vendors, but I rely on them only by getting samples from them, cross-checking their work, and then also cross-checking that on the raw materials that are actually used in manufacturing. And I check during the manufacturing process and after manufacture as well just in case!
But you can't reasonably do all this for, say, a whole O/S. It's just too big and too complicated. This is why you'll see medical systems (or avionics) running on LynxOS or Green Hills' OS rather than standard Linux, ITRON or eCos (though eCos is small enough that you could probably review it yourself too). Regretfully, some are starting to ship with Windows which I am sure has not been subject to the equivalent review.
So why am I so confident in saying this?
In other words I'm probably the only person on the planet who's been under GMP, under software ISO9000 and also been a free software developer.