Slashdot Mirror


FDA: Software Failure Behind 24% of Last Year's Medical Device Recalls

chicksdaddy writes "Software failures were behind 24 percent of all the medical device recalls in 2011, according to data from the U.S. Food and Drug Administration's (FDA's) Office of Science and Engineering Laboratories (OSEL). The absence of solid architecture and 'principled engineering practices' in software development affects a wide range of medical devices, with potentially life-threatening consequences, the FDA warned. In response, FDA told Threatpost that it is developing tools to disassemble and test medical device software and locate security problems and weak design."

128 comments

  1. maybe coders need PE powers and rules by Anonymous Coward · · Score: 0

    maybe coders need PE powers and rules.

    Make so they have to sign off and give them to power to say NO to boss about doing a rush job.

    1. Re:maybe coders need PE powers and rules by Anonymous Coward · · Score: 1

      Maybe they need PE liability, testing, and training too.

    2. Re:maybe coders need PE powers and rules by Anonymous Coward · · Score: 0

      Never any mod points when I need them.

    3. Re:maybe coders need PE powers and rules by Anonymous Coward · · Score: 0

      But then they'd have to be like, "Software Engineer Engineers".

      you know, because they call themselves engineers already without all of that. Think of their self esteem.

    4. Re:maybe coders need PE powers and rules by aaronb1138 · · Score: 1

      Programmers have desperately needed the backdrop of legal liability for the last 20 years. Unfortunately, I think the no liability clauses in illegal EULAs will continue unchallenged for quite some time. Whenever I hear the phrase, "tort reform," I always think the opposite of controlling legal liability and instead spreading the misery of lawyers on a variety of fields (mostly software development) until they can perform their jobs to the competency level of McDonald's. My pet peeve though is UI scalability and ADA compliance. How in the hell every software firm has ignored this with no class action lawsuits crushing them is beyond me.

      Our best bet is for the extreme minority of highly skilled and competent programmers to develop AI suitable to writing the software on its own from wrote description. On the way there, we should have a competition between Watson and Wolfram Alpha to see which one can get their MCSE, RHCE, and CCIE certifications first.

  2. What are they doing about the 76% HW failure rate? by sizzzzlerz · · Score: 5, Insightful

    It seems like that should be of even more concern.

  3. But they saved so much money by Anonymous Coward · · Score: 0

    But think about how much money the accountants saved by outsourcing the work? The company saved .005% profit! What could possible could possible go wrong

  4. DMCA? by Maximum+Prophet · · Score: 0

    In response, FDA told Threatpost that it is developing tools to disassemble and test medical device software and locate security problems and weak design

    Do we, the patients, have the right to do this, or does it have to go on behind closed doors and NDAs?

    In a distopian novel, the government would do this so that they could turn off your heart, if you said anything out of turn.

    --
    All ideas^H^H^H^H^Hprocesses in this post are Patent Pending. (as well as the process of patenting all postings)
    1. Re:DMCA? by MadKeithV · · Score: 5, Funny

      In a distopian novel, the government would do this so that they could turn off your heart, if you said anything out of turn.

      It wouldn't work on politicians or lawyers. They don't have hearts.

    2. Re:DMCA? by Anonymous Coward · · Score: 0

      Can you explain to me, honestly, why you would want to?

      The number of people who are:
      1) conversant with the custom hardware;
      2) conversant with the custom software;
      and
      3) conversant with the medical information relevant to the operation to the device;
      Probably already are working either for the FDA, or the device manufacturer.

      Hacking your TiVo is one thing. Hacking your pacemaker or implanted insulin pump or artificial heart is another - you can kill yourself with your ignorance. Unless you're really suggesting that you're going to go out and buy a couple artificial hearts to hack on your desktop?

  5. Backups by SJHillman · · Score: 2

    It would seem they don't do backups either. Every time I've had my hearing aids sent in for repair, I've had to have them reprogrammed from scratch because they never save the settings first.

    I have an appointment to have them repaired again in 3 hours. Any bets on whether its a software issues?

    1. Re:Backups by glueball · · Score: 2

      If the medical company follows a process, and you should hope they do, they will send the equipment back to you in a known good state. Your settings are not part of the known good state even though they are within guidelines. Further, if a new setting is added to the hearing aid, where should they set it? Is setting it to the default compatible to your previous settings?

      It's a feature.

      PS, a company following a process will do the same thing even if it's something like my tractor. Every time John Deere comes to service my tractor, they make sure all safety features are working and emissions are functional, no matter what the service is about.

    2. Re:Backups by SJHillman · · Score: 1

      They have the capability to save the settings most of the time, they just choose not to. Even if the manufacturer doesn't do it, I would expect the audiologist to (in most cases, the audiologist acts as intermediary) without me needing to ask for it. It'd be like them reinstalling Windows if you took your laptop in because the exhaust fan died.

      The more technology improves, the more quality control seems to go down the can. I've had more problems with my current set than any other before it. My last set before these also had some problems. Never had to get any pair before the two most recent repaired in the 19 years I've been wearing them, in spite of the fact that I'm taking much better care of them now than when I was younger. Warranty periods seem to be getting shorter too.

    3. Re:Backups by krakelohm · · Score: 1

      Wouldn't that be a big liability for these manufactures to have and store and safeguard your personal medical information even if it just settings on a device?

      --
      You are all a bunch of idots.
    4. Re:Backups by glueball · · Score: 2

      The more technology improves, the more quality control seems to go down the can.

      As a medical professional, I'd say a company that takes in a device on RMA and returns it to a known good, known tested state, is far superior in "quality control" than a vendor who would individualize each service routine.

      What you're asking for is exactly why personalized medicine is doomed to fail.

    5. Re:Backups by SJHillman · · Score: 1

      What I'm asking for is that my newer hearing aids need to be sent in for repair less often, not more often. All of these new features are great, but I need reliability more than I need bluetooth.

    6. Re:Backups by Anonymous Coward · · Score: 0

      You didn't read his reply correctly.

      Here's the key part:

      As a medical professional, I'd say a company that takes in a device on RMA and returns it to a known good, known tested state, is far superior in "quality control" than a vendor who would individualize each service routine.

      And I totally agree. Doing what you want is prone to failure. Settings could change, could be added, deleted, their effects could be changed. It'd be completely stupid to personalize it.

    7. Re:Backups by sjames · · Score: 1

      IF the update renders the old settings invalid, it's a feature. Otherwise it's just flushing other people's time and money down the crapper so they can shave a few seconds off of their own.

    8. Re:Backups by sjames · · Score: 1

      Since it's already in the device and they can in theory access it, the liability exists now.

  6. Re:What are they doing about the 76% HW failure ra by ILongForDarkness · · Score: 2, Insightful

    In my experience there is way way more software failures. The vendor just sends software updates every couple months. Oh yeah the previous version had a problem where if you did things in the wrong order it would change the patient that the radiation machine was programmed for. Sorry about that but here is the fix. Or worse notices saying their is a problem so telling users to double check all the time until they release a new version ... sometime.

  7. Demand Free Software by betterunixthanunix · · Score: 4, Insightful

    Can someone please remind me why people should be unable to examine the software in their medical devices, software that their lives may depend on? Why these programs are not open to public review?

    Oh wait, I got sidetracked thinking that the point of medical devices is to keep people healthy, rather than to rake in profits for the companies that make them.

    --
    Palm trees and 8
    1. Re:Demand Free Software by Anonymous Coward · · Score: 0

      How would you like it if people started to hack your heart? That sure gives a new meaning to 'heart attack'!

    2. Re:Demand Free Software by MozeeToby · · Score: 5, Insightful

      Hiding the source code is not an effective way to prevent hacking, if it were my Windows box wouldn't need a hardware firewall, a software firewall, 3rd party antivirus software, and regular sweeps initiated from a different OS.

    3. Re:Demand Free Software by Anonymous Coward · · Score: 0

      and how does keeping the code secret prevent this? It makes breaking in transiently harder to do, but also harder to fix and harder to spot issues before they are exploitable. In addition it means that you are reliant on only one source of fixes!

    4. Re:Demand Free Software by Anonymous Coward · · Score: 0

      Why these programs are not open to public review?

      Same reason NO proprietary programs are open to review. Why are medical devices different? If your company's software goes bad, you lose millions, lay people off, and people die from lack of medical care and finances.

    5. Re:Demand Free Software by ongelovigehond · · Score: 2

      Why stop at the software ? For a complete review, the hardware design should be open too.

    6. Re:Demand Free Software by Anonymous Coward · · Score: 0

      Ok, but make sure the free software has no disclaiming of warranty. If you're going to claim it's better you then can't run away from the consequences in case of failure.

    7. Re:Demand Free Software by plover · · Score: 1

      He said "review" the software, not "replace" it. That confusion seems to lie at the heart (no pun intended) of all these discussions. There seems to be fear, whether intentionally spread by the manufacturers or not, that if the software was publicly viewable that it would immediately lead to people hacking the medical devices.

      And in the short term that's likely to be a valid concern, because the software is so badly written* that an ordinary hacker armed with the source COULD physically attack a victim. That situation won't change until device makers start adopting best practices. (* This claim is backed by the existence of TFA: if the software was of better quality, it wouldn't be getting recalled so often.)

      I have an idea. Long ago I heard that the NSA created chips for NASA specifically to protect communications to satellites, keeping mission critical aspects such as attitude control safely protected. Why doesn't the FDA ask the NSA to provide a similar set of chips for medical device makers? Not that the folks at the device labs are idiots, but they're certainly not security specialists. Even something as simple as providing a standard security mask for an FPGA would enable device makers to implement NSA-quality security, hardening their systems to the point where it would take an army of Ross Andersons led by a Bruce Schneier to break even one of them. If the hackers can't access the device because a protocol chip stops them, that problem would be solved.

      By itself it won't fix the quality issues, but it would let us get to the point where hackers would be less of an impediment to reviewing and improving the devices.

      --
      John
    8. Re:Demand Free Software by glueball · · Score: 4, Informative

      The MRI machine I use has a complete circuit diagram along with design notes in a binder set next to the machine. In the US, you get the hardware manual for service. I don't believe the same is true for Europe and I have no idea about the rest of the world.

    9. Re:Demand Free Software by GodfatherofSoul · · Score: 1

      I can tell you one reason based on a conversation I had with 2 doctors. Medical professionals are almost apprehensive about giving "unnecessary" information to patients because it leads to problematic self-diagnosing. Say there's an article in Time about a new disease. They then seen a spike of people coming it with timesmagazinitis, and if patients get fixated on some notion of what they might have, it makes it harder to get accurate information from them; e.g. "I have timesmagazinitis, so the fact that I ate rat poison should be of no concern to my doctor."

      --
      I swear to God...I swear to God! That is NOT how you treat your human!
    10. Re:Demand Free Software by autocannon · · Score: 0

      The open source zealots love these types of comments. It's not a valid solution, and if you allowed yourself to think outside the open source box you're in you might see it too.

      Allowing anyone to view the code means anyone can then modify it. Last time I checked, hospitals and other healthcare facilities were not exactly employing highly skilled and knowlegeable software engineers. Engineers that also have a solid working hardware level knowledge of EVERY medical device in use so that they understand just what happens when they decide to "tweak" the software. You do understand that any facility large enough to justify employing someone with that level of knowledge, and required pay, is going to have dozens of different devices from different manufacturers.

      Allowing end users to tweak the software will also lead to ANY failure of the device to be absolved from fault. It's now the facility's sole problem of whether their equipment functions as intended. No more warranty, possibly no more manufacturer maintenance either.

      And that same equipment can no longer be viewed as a blackbox that functions the same across all facilities. I'm sure the Department of Health would absolutely love that scenario. X ray machines spitting out different levels of radiation for the same procedure because some "expert" engineer decided he would tweak the software of his x ray machine and somehow screwed it up.

      For these, and probably other reasons, medical device software should definitely NOT be open source. That's not to say that the manufacturers aren't being diligent enough in producing good software to begin with, they obviously aren't. But open sourcing just shifts all of the problems to the end users, and healthcare facilities will not, and maybe even can not, accept that responsibility.

    11. Re:Demand Free Software by dark12222000 · · Score: 3, Insightful

      Look up "Code Signing". Then bash your head against your desk three or four times as punishment for the stupidity you typed out above.

    12. Re:Demand Free Software by David+Chappell · · Score: 2

      The open source zealots love these types of comments. It's not a valid solution, and if you allowed yourself to think outside the open source box you're in you might see it too.

      Allowing anyone to view the code means anyone can then modify it.

      As far as I am concerned, "anyone" can modify it all he wants just as long as he doesn't install it in an in-service medical device without proper approval. If we don't yet have legal, organization, and technical means to prevent this from happening, we should.

      We are not talking about opensource software here. We are talking about allowing anyone who wants to to audit a piece of proprietary software.

      I do though think that we should restrict access to the text of our laws. Think what would happen if somebody got a copy and modified the law.

    13. Re:Demand Free Software by Anonymous Coward · · Score: 0

      I work at a cardiac medical device company and we have a lot of patented algorithms that we use in our software. The research department spends years of effort and a lot of money in developing them and getting them patented. So depending on the Medical device, this may not be possible.

    14. Re:Demand Free Software by autocannon · · Score: 1

      taken from wikipedia:

      Code signing can provide several valuable features. The most common use of code signing is to provide security when deploying; in some programming languages, it can also be used to help prevent namespace conflicts. Almost every code signing implementation will provide some sort of digital signature mechanism to verify the identity of the author or build system, and a checksum to verify that the object has not been modified. It can also be used to provide versioning information about an object or to store other meta data about an object.

      Where in that statement does it also convey that the software is defect free? Zero bugs?? Look, I'm not even arguing against malware or malicious intent here. Rather that there are few, perhaps even zero, engineers out there qualified to do software updates for all manner of medical devices from different manufacturers if it were done in house. Downloading some "code signed" version from joe schmo does not protect the hospital from fault if the device then malfunctions.

      That's my biggest gripe. If a device malfunctions, and someone has put ANYTHING other than the manufacturer provided software on it, then the facility has tampered with it and they are then liable for anything that may happen as a result of the malfunction. No hospital would be willing to expose themselves to even more potential lawsuits because of that. Maybe more importantly, no insurance company may be willing to cover a hospital if it were discovered to be doing that.

    15. Re:Demand Free Software by dark12222000 · · Score: 2

      Well, yes. If you modify the product, then it's on you. However, having the code be open source means:

      It can be inspected
      It can be verified
      Patches can be written (and then submitted to the manufacturer)

      The idea is not that the little IT captain at your local hospital is going to rewrite the MRI. It's that he's going to run into an issue, pull up the source code, write up a patch, submit it to the manufacturer, then the manufacturer is going to throw it out, force their engineers to write it again, run it through QA testing, and then issue a patch.

      The manufacturer can't claim the problem isn't solvable (patches can be provided) and they can't claim it doesn't exist (source code can be used as proof). At the same time, assuming they use code signing, only they can modify the machine without voiding the warranty.

      Now, here's where it gets better. Say the manufacturer dies off/stops caring/whatever. Source code is already out there - an independent (certified/insured/over-payed) firm can come in, release their own patches, and still modify the machine (albeit voiding the original warranty while still keeping the hospital from being exposed to damages via contract). Suddenly, you aren't depending on the OEM to provide EOL patches for the entire life of the machine, yet at the same time, the OEM isn't responsible for some dickweed tweaking an MRI machine to kill people. Win/Win for everyone.

      In addition, all the research and technology for that MRI machine is now accessible to others, which then lowers the cost of MRI machines, makes them more available around the Globe, and lowers the cost of ownership. At the same time, the OEM is still making money hand over fist from insuring and babysitting the machine (they now have to compete with independent firms, but they have a major advantage as the OEM). Again, Win/Win for everyone.

    16. Re:Demand Free Software by plover · · Score: 2

      Allowing anyone to view the code means anyone can then modify it.

      As the Mythbusters like to say, "Well, there's your problem!" Your entire argument is based on the extension of this premise to imply that you can then install this modified software on the medical device. But that's not a given at all. You can modify the downloaded copy of the code that you have squirreled away somewhere in /users/autocannon/src, but it doesn't mean you can modify the exact copy of the code that's running on the CPU in your insulin pump.

      It may not even be physically possible. Consider that I can burn GPL (v2) code to an FPGA, then burn the fuse to prevent further modifications to the chip. As long as I distribute the source code with the device, I am free to sell the device, even though I've given you no end-user-accessible way of modifying it. Tivo used a variant of this idea, where they burned a digital signature verification process on their devices which then refused to permit unsigned updates to their code. Called "Tivoization", this practice led directly to the creation of the GPL V3.

      Can you take a medical device apart and replace the ROM with your own modified code? Obviously it's technically possible, but if it's a medical device it will no longer be certified for medical use. No legitimate doctor would prescribe that modified device to a patient (outside of the device maker's controlled studies, of course.)

      --
      John
    17. Re:Demand Free Software by Flatwater · · Score: 2

      I hope that binder doesn't have metal rings:

      http://www.howstuffworks.com/question698.htm

    18. Re:Demand Free Software by Anonymous Coward · · Score: 0

      The idea is not that the little IT captain at your local hospital is going to rewrite the MRI.

      Correct - he won't do that.

      It's that he's going to run into an issue, pull up the source code, write up a patch, submit it to the manufacturer, then the manufacturer is going to throw it out, force their engineers to write it again, run it through QA testing, and then issue a patch.

      Incorrect - he will not do that, either. Do you really think that the "IT manager" at a hospital is going to be qualified to write code to operate these devices? Hint: No. If he's qualified, he's not administering a bunch of windows and linux systems at a hospital, he's working for the manufacturer writing the code that operates these devices already.

      Are you really so daft that you don't understand that there are degree programs worth of information you need to have a firm grasp on to write this type of code? MRI design requires knowledge of physics, materials, anatomy and physiology, and medicine. And you think the IT manager at a hospital is going to just whip up a patch to solve a race condition in the software interlock code that manages a high-energy radiation device? There are requirements he's never even *heard* of, much less understands - how do you justify thinking he's just going to pop open a text editor, hack on the code for a bit, and then submit a working patch to the manufacturer?

      You're high.

      Say the manufacturer dies off/stops caring/whatever. Source code is already out there - an independent (certified/insured/over-payed) firm can come in, release their own patches, and still modify the machine (albeit voiding the original warranty while still keeping the hospital from being exposed to damages via contract)

      If the manufacturer dies, then their assets will be put up for sale and liquidated - the source code doesn't need to be "out there" for this to happen, it's standard procedure during a bankruptcy / liquidation.

      In addition, all the research and technology for that MRI machine is now accessible to others, which then lowers the cost of MRI machines, makes them more available around the Globe,

      Not as much as you'd think, given the different purposes, design, and construction of the various machines. Sure, some of the code would be generally useful, but a lot of it would be specific code to control specific hardware in that specific model / version of device.

    19. Re:Demand Free Software by Anonymous Coward · · Score: 0

      Can someone please remind me why people should be unable to examine the software in their medical devices, software that their lives may depend on? Why these programs are not open to public review?

      Oh wait, I got sidetracked thinking that the point of medical devices is to further the OSS agenda, rather than to rake in profits for the companies that make them.

      There, fixed that for you.

    20. Re:Demand Free Software by neonv · · Score: 1

      Can someone please remind me why people should be unable to examine the software in their medical devices, software that their lives may depend on? Why these programs are not open to public review?

      That's a good idea! We really need people changing the code on their medical devices, that will fix all the problems ...

    21. Re:Demand Free Software by Darinbob · · Score: 1

      As far as the FDA goes I believe they are allowed to inspect actual software and development processes during audits, closed or open source.

      As for public review, many devices do include components from other manufacturers that include non disclosure agreements. Many rely upon trade secrets; ie, they're the only company with a new technique that gives them a competitive advantage. Opening this up to public review means opening up to the competitors. I don't doubt that giants like GE (lots of money, bland or mediocre products) would use this to crush the small guys.

      You need profits or no one builds a product! I don't know of any charities that create medical devices. Many of these companies do try to make better and cheaper products, and they attract developers who care about product quality and affordability. If people want to get rich then making medical devices is the wrong way to go about it.

    22. Re:Demand Free Software by garyrich · · Score: 1

      You don't get the source code to their software. You probably rely on results of an FDA audit of the MRI vendor. The FDA auditor would look at the validation protocols for the software. If they say they are using a "waterfall" development paradigm, they will go through all the documentation for that and look for evidence of proper code review and sign offs. This is the sort of things auditors are trained to do. Theoretically they could audit and review the vendor's source code - in reality there are probably a dozen people at FDA that could make any sense of the code. Those people are working trying to make FDA own software work properly and won't be part of an audit team unless people are dying (and probably not then).

      Precedent says that you can get away with murder if you just rely on COTS software (Commercial off-the-shelf). Your MRI probably has a Windows user interface (shudder...) and may have a proprietary database back end, like Oracle and many other layers of commercial software underneath. FDA has little ability to audit them and no ability to access their source code. Also - installing current vendor patch fixes to Windows or Oracle are usually not done frequently. Patch fixes often trigger elaborate and expensive revalidation protocols to make sure the fix doesn't break something else They would be unlikely to find one if it existed but they are required to document that they spent $$$$ trying, so they will put if off. In some cases even updating anti virus definitions would trigger a revalidation, so they don't get applied either.

      --
      -- your Web browser is Ronald Reagan
    23. Re:Demand Free Software by gorzek · · Score: 1

      Ugh, yeah, I've noticed the age of the Internet has made that worse, too. People erroneously think that medical diagnosis is simply a matter of checking off a bunch of symptoms. "Well, I have 8 out of the 10 symptoms listed for African megalocephalitic herpes! I'm going to die!"

    24. Re:Demand Free Software by autocannon · · Score: 1

      In response to both David and plover, what is the device manufacturer's incentive to publish their proprietary source to allow for public scrutiny? There's 3 consequences to allowing that.

      1. Process must be established to allow reporting of defects, evaluation of the defect, code correction, unit testing, and finally QA testing. (ongoing costs)
      2. Competitors now have full access to all of your device's capabilities. Unscrupulous though it is, they have a competetive advantage as well as the ability to just flat out steal your code to shorten their development time. They keep it black box, you'll be hard pressed to ever prove otherwise.
      3. The public and more importantly your competition are now potentially aware of your defects. This can definitely affect future purchases. Hospitals and other facilities don't buy in a vacuum, they get sales pitches from multiple vendors.

    25. Re:Demand Free Software by Anonymous Coward · · Score: 0

      Because they're full of idiotic shit. I'll give you a perfect example. I have an insulin pump made by Medtronic. The pump has a reservoir of insulin from which it is dispensed into your body. Once the insulin level gets to a user-defined level, the pump triggers and alarm to let the user know it will be time to refill soon. It functions very similarly to the fuel light in your car. The IDIOTIC thing about this pump is that the user is given no further warning. Even when the insulin runs out and the pump stops delivering insulin, no indication is given ... NONE. This has led to more than one trip to the emergency room as blood sugar levels spiked wildly out of control.

      When this flaw is pointed out to Medtronic, they act dumbfounded that it would even be an issue. They suggest setting the user-configurable alarm to something close to the empty point. The problem with that is that you might be somewhere where you can't get more insulin for a couple of hours, or you might be asleep when that alarm goes off. For safety, it needs to warn you a couple of hours before empty. I can't believe people are almost dying because of a stupid programmer.

    26. Re:Demand Free Software by Anonymous Coward · · Score: 0

      I don't do any of that and I don't have any viruses. I've used every version of Windows except Me. If you don't open attachments or use Internet Explorer, and stay patched, you're not likely to suffer much.

    27. Re:Demand Free Software by Anonymous Coward · · Score: 0

      Your algorithms are patented. That's great. So what does that have to do with publishing the source code?

      I'll give you a hint: it allows you to publish the source code. Unpatented trade secrets would be a problem. Patented algorithms are not secret by definition.

    28. Re:Demand Free Software by plover · · Score: 1

      I didn't say this was an incentive for the manufacturers to open their source. (f they're basing it on an open source platform that's covered by GPL and not LGPL, they're bound to, but I don't think any medical device maker wants to take responsibility for all of Linux in their devices.) If open source were mandated, however, you still raise valid issues.

      Regarding your points, #1 is true regardless of the source. In the case of open source, they would have more sources of input from external people who review their code, which means an initial run of bugs to fix, but after that there should be more stability. If this helps them make a safer device, the incentive comes in the form of fewer lawsuits due to these defects being found statically, rather than in a patient.

      #2 is certainly a concern. I hate to suggest software patents ever, for anything, but this might be a case where you'd use one. If opening the source were mandated, both manufacturers' code would be exposed, so proof would be easier.

      #3 This plays out both ways: "Go ahead, examine our software, take it to your cousin who knows computers, he can see that we have nothing to hide. The FDA, the FBI, and crack teams of penetration testing hackers have also been unable to find any flaws. But the guys over at Joe's Discount Insulin Pumps, well, they keep their source closed, and maybe for a reason. Do you really think they found all their bugs? ALL of them? What are they really hiding in there?"

      There's a different outcome possible, too. What if the FDA mandated the software be separate from the hardware? "Use PloverCo's software, it's tested and FDA certified for use on any implantable pump conforming to the ABC-123 standard." Now every pump can be built to a standard interface and run a standard package, eliminating user and doctor confusion. ABC-123 would define the firmware that would load software, and both the manufacturers and FDA would have to sign the code before it would load it.

      OK, so there's a third solution: Instead of just open or closed source, give the code signing key to an independent Medical Device Code Quality Board. They'd have to be licensed software engineers (meaning a licensing requirement would have to be defined.) Have them be bonded, so they risk losing their bond if they're caught leaking a copy of the software. They'd take on insurance in case they get sued if the software fails anyway. And they get paid well, of course. Then nobody, not even the device manufacturer, can install unsigned code without the board's signature on it.

      Just think: it would be expensive, slow, bureaucratic, litigious, and probably completely ineffective at detecting problems. It sounds a perfect government solution!

      --
      John
  8. devs by fluffythedestroyer · · Score: 1

    I told them not to take Microsoft Dev's...

  9. Outsourcing by Billly+Gates · · Score: 1

    What do you want to bet this company outsourced to save .03% profit and used the cheapest overseas bidder? But the analyst got his bonus. Maybe I am being too cynical but many of these medical companies are the greediest folks imaginable and when all software is viewed just as a cost center the MBAs start doing dumb things based on GAAP rules rather than common sense.

    1. Re:Outsourcing by Anonymous Coward · · Score: 0

      I would love to know what evidence you've based this 'insightful' comment on...

    2. Re:Outsourcing by Anonymous Coward · · Score: 1

      You're being too cynical. I work for a medical company and I can assure you that the general awareness is that doing things properly has a lower cost than patching things up afterwards. Our devices depend heavily on software and there are no costs being save there. Not much is outsourced.
      If figure that there are companies that have a different mentality, but I figure that the majority would operate by this rule.

    3. Re:Outsourcing by Anonymous Coward · · Score: 2

      Excellent point. I work for a med device company, too, and know for a fact that the cost of a recall and/or patient lawsuits far outweighs any software development expenses. Cutting corners is not considered and the level of design and qualtiy controls is very high.

      I would suggest that Billy Gates is absolutely being too cynical. //24 years in the med device business

    4. Re:Outsourcing by plover · · Score: 1

      Good info, thanks. It raises a thousand more questions in my mind, though, about your quality processes. I would really like to know if your industry has specific mandated or regulated software quality standards you follow, like an ANSI or ISO standards? Or did your firm develop your quality processes entirely in house? Do you use Test Driven Development? Iterative methodologies, such as Agile? Or do you do big designs up front, and hold formal walkthroughs and reviews of those designs?

      Do you have standard metrics for complexity? Do you have mandated automated unit test coverage of 100% of your code? Do you track defect rates over time? Do you maintain requirements traceability documentation? Do you have mandated code reviews, and if so, by whom? Do you use static code analysis tools, such as pmd, Klocwork, Coverity, lint, or the like? Do you have specific standards regarding exception handling? Dynamic memory allocation? Multi-threading?

      I would love to know what measures you take when something as sensitive as a human life is at stake. The quality of my industry's software pales in comparison, as most of it just handles other people's money. It gets people all nervous, of course, and we take it seriously, but it won't actually kill someone if it crashes.

      --
      John
  10. FDA should develop an open platform like NSA did by WindBourne · · Score: 3, Insightful

    Seriously, the smart thing is to develop an Open platform on Linux, with libraries for equipment to use. Likewise, offer up secured ways of updating the equipment. If FDA was smart, they would talk to NSA.

    --
    I prefer the "u" in honour as it seems to be missing these days.
  11. I'm waiting for the day by tekiegreg · · Score: 1

    My Pacemaker becomes Flash Updateable or "We found a bug in your Pacemaker, come in for a minor procedure to upload the fix."...just DEAR GOD don't make the fixes downloadable via wireless.

    --
    ...in bed
    1. Re:I'm waiting for the day by HiThere · · Score: 1

      It probably already is. But the wireless range is quite short.

      I know my wife goes in regularly for reading, and they read it with a magnetic wand that they place on her skin over the device. I believe that they've occasionally added a patch, though I admit I'm not certain about this. (The doctors are not communicative in this area, and may not be knowledgeable. It's hard to tell. They seem to often just be following a guidebook...not necessarily a bad thing.)

      --

      I think we've pushed this "anyone can grow up to be president" thing too far.
  12. Re:What are they doing about the 76% HW failure ra by MozeeToby · · Score: 1

    Hardware can fail for a much wider variety of reasons; poor maintenance, overuse, physical abuse, one off manufacturing defects, etc. Software failures are caused by an error in design or implementation; they are almost guaranteed to be present in every single instance of the device even if it takes an oddball corner case to set it off.

  13. Re:FDA should develop an open platform like NSA di by Anonymous Coward · · Score: 0

    That's just what we need - medical devices and implants with NSA backdoors in them.

  14. It's called OOPS for a reason by Anonymous Coward · · Score: 0

    Time to rethink the whole object-oriented thingy.

  15. Where does 24% come from? by Anonymous Coward · · Score: 0

    I looked through the report linked in the article and failed to see support for the opening claim of the article:

    "Software failures were behind 24 percent of all the medical device recalls in 2011"

    The closest thing I found was a figure 'fig 5' which has no explanation in the surrounding text. It shows a bar graph that looks close to 24% for 2011, but it is not clear if this is "all medical device recalls" for that year, or if it is just the recalls related to the company that is described in that section (they were being audited by the FDA, and it sounded like they had a pretty bad quality system process).

    Am I missing something obvious in the report? I'm interested to know if this is accurate, because I had always been under the impression that medical device recalls were almost always a result of hardware/manufacturing issues. I've worked as a software engineer at a couple of med tech companies and can't think of a single instance at either company where there was ever a recall as a result of software defects.

    1. Re:Where does 24% come from? by mcmonkey · · Score: 1

      Glad I'm not the only one who noticed this.

      That fig. 5 appears after a section addressing issues involving...

      ..incorrect or missing patient results in a laboratory information system, and incorrect or missing notifications to clinicians that test results were out of range.

      Basically, problems with LIMS or other systems. It's not that the device released the incorrect dose, it's that the record keeping system had an incorrect does history or test result, causing a doctor or nurse to administer an incorrect dose.

      To /.'s credit TFA does say what the summary says. However the reported linked from TFA does not say anything approaching what's in TFA.

      Linked to the wrong report maybe?

  16. Re:What are they doing about the 76% HW failure ra by MimeticLie · · Score: 2

    Those things can cause hardware failure, but they generally don't cause recalls. Remember that we're talking about device recalls, here. The hardware failures are also likely to "be present in every single instance of the device" if they need to fix all of them.

  17. Re:What are they doing about the 76% HW failure ra by Anonymous Coward · · Score: 0

    Just because 100 - 24 = 76 does not mean that all the other recalls were hardware failures. Sometimes, a device functions well, but it's use is difficult or confusing.

    Suppose the device is too heavy for the average nurse to carry, and it can't be used safely. There's no material defect.

  18. Re:FDA should develop an open platform like NSA di by PerfectionLost · · Score: 1

    Riggght. The NSA already has enough trouble telling us how many people they are eves dropping on... let's get them officially into the medical gear too!

  19. Re:FDA should develop an open platform like NSA di by plover · · Score: 1

    That's just what we need - medical devices and implants with NSA backdoors in them.

    It's a matter of trust. I'd trust the NSA not to mess with my medical device far more than I'd trust a random device manufacturer to properly update a medical device over the internet. Only one of these organizations has a well deserved reputation for maintaining secrecy and security. Can the NSA sniff your internet traffic? Undoubtedly. But is someone at the NSA's Panopticon Central actually looking at your data? That's the key: you don't know for sure, and you will never know for sure. That speaks volumes for their security, and is a better reputation than these device manufacturers have.

    Besides, the NSA has much less incentive to mess with your devices than other organizations, such as the FBI, FDA, police, TSA, etc. Just because one organization can doesn't mean the others get to.

    --
    John
  20. the software failed? by Anonymous Coward · · Score: 0

    lets put the blame where it belongs -- on the people that write/test the software. sure, it is nice to blame this ambiguous 'software' thing. that way, no one made a mistake. that damned softwares brokes it.

  21. Accountability and responsibility by Anonymous Coward · · Score: 0

    The problem is that most of the time the vendors are not accountable, i.e. have no penalty, for delivering crap. There is no incentive for them to have a product sit for too long in the q&a or engineering lab.

  22. Simple answer. by Anonymous Coward · · Score: 1

    Capitalism is god. Whether or not the product does as it's advertised is inconsequential as long as the seller gets paid.

    As far as your health? You shouldn't have gotten sick in the first place.

    /there are people who actually believe this bullshit. me, not being one of them.

    1. Re:Simple answer. by Anonymous Coward · · Score: 0

      Simpler answer: 99.999% of the population is completely unqualified to review, modify, or understand the software that controls their medical devices in the first place.

      Of the people who are qualified, most are already employed writing software and building devices for biomedical companies.

      Of the remainder (qualified, but not doing the work professionally), they're so vanishingly small a number, and so unlikely to have one of the medical devices in question, that the cost would be exceptionally prohibitive for them to purchase one to "hack on," and there's the very real risk that they will *break* something and kill themselves (or a family member or friend or loved one) because they don't understand what they're doing as well as their gigantic egos tell them they do.

      So, to turn it around: why should a company that builds these devices go to the trouble of providing people with reams of source code that almost every single one of their customers don't actually use, can't understand, and will never look at? That would just add to the cost of the device (somebody has to assemble all those things into a publishable form), and provide absolutely NO benefit whatsoever to the person who owns the device.

  23. "locate security problems and weak design." eh? by Anonymous Coward · · Score: 2, Insightful

    They might as well just start from scratch then. I used to work for a huge healthcare company and dealt with some of the debacles that these devices cause. "Our device only supports WEP....is that going to be a problem?" Pathetic. Luckily the place I worked was big enough to push them around and do things like force them to implement EAP-TLS, but it was tough going. Then you have all the BS with how the FDA "doesn't allow us to upgrade software without extensive testing", which of course is not entirely true.

    These companies are just like every other medical software vendor...for some reason they feel entitled to produce absolutely terrible products that are 10 years behind the rest of the world. I don't know why the medical industry is like this, or why customers put up with it. The general attitude where I used to work was, "OH NO DON'T UPSET THE POOR VENDORS!!". I was like, "aren't we paying them? tell them to fix their product or go to hell". This was true of desktop software, medical devices...everything. They wanted to bring a new system online for tracking blood donations and the software required "act as part of the operating system" privileges. Really? We're going to update our Group Policy which already gives every user local admin rights to allow that too? Why, exactly, would that be? Especially since it's nothing but a database application. Another application we had on the network would crash when our vulnerability scanner would probe its port. The entire piece of software would just die because it got a packet of a type it wasn't expecting. This wasn't an aggressive scan, this was just a probe looking for open ports. I told the department, we can give you a scan exception, but this is not a problem that's going to go away just because we stop scanning your device.

    I think the entire Medical IT industry has a day of reckoning coming. The unnecessary proprietary requirements, the poor design, the unreasonable legacy OS requirements, the poor security...it's endemic to the industry. There's this attitude that they want to use IT extensively in healthcare, but not change the workflows of providers in any way (including things like requiring strong passwords). It's not a sustainable model.

  24. Class III devices by judoguy · · Score: 1
    I worked at a pacemaker company for a while as a consultant. No way in hell are they going to let you dink with the software.

    Letting me see it as a programmer would be interesting perhaps, but modifiable? Again, no way in hell.

    Everyone who gets an implantable cardiac device is by definition trying to die, and likely will from the problem that indicated an implant in the first place.

    The potential company liabilities are profound to say the least. Particularly with the sorry state of tort liability in the US.

    --
    Peace is easy to achieve, just surrender. Liberty is much harder get/keep.
    1. Re:Class III devices by David+Chappell · · Score: 1

      Everyone who gets an implantable cardiac device is by definition trying to die, and likely will from the problem that indicated an implant in the first place.

      They are taking a calculated risk in the hope of dying later than they would otherwise. Or am I missing something?

  25. Re:FDA should develop an open platform like NSA di by WindBourne · · Score: 2

    So, you think that by TALKING to the NSA, who added a nice security module to Linux, they will put a back door into medical equipment?
    If you are going to make wild assertions, you might want to look up what NSA's mission is: listen to other system AND SECURE OURS.

    NSA has some of the best security ppl on-board. I would trust them to handle my medical devices, before I continue to trust the idiots at MS and those that use Windows.

    --
    I prefer the "u" in honour as it seems to be missing these days.
  26. Weak design?? by ZenDragon · · Score: 2

    I like how everybody likes to blame their problems on "weak design" on the part of the developers. That may very well be true, but who is to say what is weak design and what isn't? As a developer, I often find that when beginning a project that has some characteristics that I have never worked with before, that it is almost impossible to find solid established information as to what actually is "strong design." And by that I mean good patterns and practices for a specific applications that really aren't very common. Sure there are a few good basic/general practices to adhere to, but for the most part its each coder for their own. So I wonder, what the hell the FDA things they are going to contribute to a world of code that they likely cant even comprehend? You can sit on the outside and say something like; I require 6 9's of up time, and such and such SLA, and this square brick fits into this hole; but what is done to make that happen will vary infinitely behind the scenes.

    If they really want to improve the stability and reliability of the code that supports these systems the need to open up the architecture to the entire community instead of keeping everything closed off for patent reasons or whatever. Obviously the companies that make this stuff are more concerned about their profit margin than they are about the safety of the patients that their equipment is treating. That's my opinion on the matter at least.

  27. Re:FDA should develop an open platform like NSA di by WindBourne · · Score: 1

    Listening on communications, is a different issue than securing a box.

    --
    I prefer the "u" in honour as it seems to be missing these days.
  28. Karen Sandler by Anonymous Coward · · Score: 2, Informative

    IP restrictions on medical devices' source code, no peer review or approval structure in place from FDA or health organisations. Complex medical devices that are implanted in humans bodies, e.g. insulin pumps, heart defibrillators etc. run software and operate more and more like computers. Here is a case of Karen Sandler, a woman who asked to see the code for the device she was to be implanted with to verify that is was safe. And what she discovered in the process.

    OSCON 2011: Karen Sandler
    www.youtube.com/watch?v=nFZGpES-St8

    1. Re:Karen Sandler by thomasw_lrd · · Score: 1

      I listened to that talk, it was very eye-opening. Made me want to contact my senators.

    2. Re:Karen Sandler by Anonymous Coward · · Score: 0

      So, why isn't she building an open source medical device company? Assuming:

      1) This issue is of extreme importance to her;
      2) Open Source methodologies will create an *objectively better-functioning* product - less bugs, more secure;
      3) Open Source methodologies will drive down costs, yielding lower prices and savings to consumers & their insurance;

      If Open Source is going to give all these benefits, then using it to build competing devices that are verifiably better and cheaper is a no-brainer: anybody building a company to do this is going to make millions, and be a hero to millions of people around the world.

      If you can't build such a company, please explain why it's impossible - why the only recourse is to use the blunt tools of government (and the threat of jail time and fines) to force other companies to do this? Go into business making a competing device, build a better device, and put them out of business with your better device.

  29. Re:FDA should develop an open platform like NSA di by Aidtopia · · Score: 1

    For many devices, Linux is a non-starter. Linux is not a real-time OS, which is typically a requirement for any life-sustaining device.

  30. Disassembling the code?! by Aidtopia · · Score: 1

    In response, FDA told Threatpost that it is developing tools to disassemble and test medical device software and locate security problems and weak design.

    Why would the FDA have to disassemble the code in the devices? The FDA approval process requires giving the FDA examiners access to the source code, along with design docs, schematics, etc. Why would they need to reverse-engineer a device? Something's not right here.

    1. Re:Disassembling the code?! by Anonymous Coward · · Score: 1

      Because software validation requires that every state of the machine be tested. Disassembling the compiled code would be another tool to achieve this. Having "source code, along with design docs, schematics, etc" is considered "design inputs." The compiled code is the design output (the actual thing doing something), which is what must be tested. Checking the design inputs is also required, but is not considered sufficient.

      This requirement isn't for software. It's for every medical product.

      Fro reference, read 21 CFR 820 and ISO 13485.

  31. Re:FDA should develop an open platform like NSA di by Anonymous Coward · · Score: 0

    You really have no clue what you're talking about. The technology isn't the problem, it's the process and people by which the technology is created that's the problem. I'm am so sick of f'ing platform-fanbois thinking their tools are the answer to every problem. Maybe if you spent a little more time seeking to understand you would get a clue to the big world outside the computer sandbox.

  32. Re:FDA should develop an open platform like NSA di by DeTech · · Score: 2

    Riiight... Ever heard of RTlinux? RTAI maybe ? OSADL? You already find it in a lot of cool life/time critical systems already... esp in the defense world.

  33. When Software Attacks by gumpish · · Score: 2

    Obligatory THERAC-25 mention. Software has killed before:

    http://en.wikipedia.org/wiki/Therac-25

  34. Re:What are they doing about the 76% HW failure ra by tlhIngan · · Score: 2

    Hardware can fail for a much wider variety of reasons; poor maintenance, overuse, physical abuse, one off manufacturing defects, etc. Software failures are caused by an error in design or implementation; they are almost guaranteed to be present in every single instance of the device even if it takes an oddball corner case to set it off.

    For hardware, to combat failures you overdesign it. E.g., if it's powered by the AC line, you make sure the power supply components are overrated for their worst case load (derating of parts makes them last much longer).

    If there's an alarm, you add a microphone and light sensors to determine that if you're in the alarm state, there is an alarm sound playing and the lights are flashing. You build in extra annunciators as well just in case the LED dies. You count 75% battery capacity as "low battery".

    And yes, I've seen those countermeaures actually implemented for a medical device.

    Of course, it also impacts software as now it's even more complex as it has to handle and detect these conditions as well

  35. Re:What are they doing about the 76% HW failure ra by jellomizer · · Score: 1

    I would expect it is probably all from one company *Cough*Siemens*Cough*

    --
    If something is so important that you feel the need to post it on the internet... It probably isn't that important.
  36. Re:What are they doing about the 76% HW failure ra by Anonymous Coward · · Score: 0

    It isn't hardware in the sense most slashdoters think. The vast majority is not electronic. Even less has software. Medical device means anything that is used with patients that isn't a pharmaceutical. Bandaids, catheters, hip implants, scalpels, beds, oxygen masks, and so on.

    Recalls can occur from design defect or manufacturing defect. Design defects can range from just plain bad design to misjudging the conditions something might be used under to printing labels with typos or incorrect color. If the potential for unintended harm exists, it get's recalled. Manufacturing defects range from making something from the wrong material to out of spec parts to screw ups on documentation of a batch.

    Since 99% (yeah, I'm pulling that number out of my ass) of medical devices have no software, the concern is devices with software need to be recalled an order of magnitude more often.

  37. Re:FDA should develop an open platform like NSA di by Anonymous Coward · · Score: 0

    Because most of these devices don't use an OS. The software is more like a calculator than a pc. The smart thing to do is to avoid your suggestion like the plague.

  38. Re:FDA should develop an open platform like NSA di by Anonyme+Connard · · Score: 1

    Ours?
    Is Linux yours?

  39. Shug. Spend more $ and stop hiring from India by Anonymous Coward · · Score: 0

    And watch that % go down.

    And don't tell me I'm pulling that out of my butt when their % is just as much BS.

  40. Heart Terminated Remotely by malloc · · Score: 1

    Karen Sandler has a great talk about how pacemaker-type devices (she has one) are completely closed-source, nobody (including the doctors who install them) cares, and the FDA doesn't/can't do much more than rubber stamp the software. Most of these devices now have unencrypted wireless access.

    sudo kill -9 heartbeat Is a real possibility with these devices.

    --
    ___________________ I want to be free()!
  41. a really important difference by slashmydots · · Score: 1

    Pretty much all PC software, tablet app, or other piece of software has a EULA that says "if this breaks your OS or PC or anything, you can't sue us." With software embedded in medical products, it's FDA law that you can't do that. So, write the equivilant of a Toyota Prius acceleration glitch into your pacemaker and it's lawsuit time and you're going to lose. Maybe they should take some of that 1000%+ profit margin on medical equipment and put it into hiring programmers who actually know what they're doing.

  42. It's no small wonder by Anonymous Coward · · Score: 0

    I worked on a medical device, an IV pump. One that didn't make it to market, thank the heavens.

    What did they use as their OS? Windows CE. Even though it SPECIFICALLY says not to use it in mission critical, medical, or nuclear applications. Right on the box. How in the world could you read that and think "Hey! That doesn't apply to me! In she goes!"

    So. My job was to write code into the bootloader. If CE crashed mid-operation (oh yeah unlikely I know) the bootloader upon reboot would recognize a bolus in progress and continue that process until complete. All in ASCII on the bootloader screen before CE reboots.

    I still have nightmares thinking about this horrible kludge being plugged into someone's veins. I am SO glad this product never saw the light of day. It's the first time my code actually stood a decent chance of killing someone.

    Posting as AC for obvious reasons.

  43. Software Protection Agency by Anonymous Coward · · Score: 0

    Let me guess, the next step is for the FDA to recommend a "solution", such as expanding its charter or creating another agency to dictate software engineering practices, provide licensing for "critical" software and mandate occupational licensing for software developers.

  44. Analog FTW. by antdude · · Score: 1

    I still use analog ones. Currently Oticon 380p, a bone conduction hearing aid. It only has three rotatory settings and three switches. 380p model was released in 1994 and I had three different sets with that model since Oticon doesn't make newer analog models anymore since everything is digital now. It's easy to reconfigure since the settings are the same and my hearing is stable. They just break down physically after about five years. Headband breaks down faster (three years?). :(

    Too bad you can't reprogram without the audiologist (unless you're one). That has to be annoying! :(

    --
    Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
    1. Re:Analog FTW. by SJHillman · · Score: 1

      There's a world of difference in quality between digital and analog, especially if you're like a lot of people and hear better in some frequencies and worse in others (especially higher pitches). However, I never once had an issue with any of my analog hearing aids but the digital ones have to be repaired every 12-24 months.

      My biggest beef with the audiologist is that they won't give me a setting for better-than-normal hearing. I figure as long as I'm paying this much money I might as well use one of the otherwise unused program slots for hearing more sensitive than the normal human range for those times when I am in an extra-quiet environment.

    2. Re:Analog FTW. by antdude · · Score: 1

      Wow, they break that easily? If I want digital, then I will have to get an implants since I have no ear holes and cochlear. :(

      --
      Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
  45. Related talk by Anonymous Coward · · Score: 0

    https://www.youtube.com/watch?v=5XDTQLa3NjE&list=PL98382D6677F8E2D4&index=1&feature=plpp_video

    Karen Sandler gives a talk about this topic, at linuxconf.au 2012.

  46. Where does the report say this? by mcmonkey · · Score: 3, Interesting

    As a developer working for a medical device company, I am very interested in this story.

    However, I am not able to find in the linked report either that "24%" figure or the direct quote from TFA.

    The Agency is also acquiring expertise in areas like "detecting malware inside device designs...(and) reverse engineering certain types of malware to best identify the specific protective practices which manufacturers should be employing," the report reads.

    The word "malware" appears twice in the quoted passage, but not at all in the report. And 24 only appears as a page number or date.

    Am I just not hitting CTRL-F right today?

    1. Re:Where does the report say this? by Anonymous Coward · · Score: 0

      Threatpost appears to have quoted the 24% from this other blog posting at the medical device security center: http://blog.secure-medicine.org/2012/06/fda-recalls-attributed-to-software.html

    2. Re:Where does the report say this? by wienerschnizzel · · Score: 1

      It's on page 22. It's not spelled out but it seems to be close to that number on the chart. Not sure how they got to the 24% though - in the text they cite 13 removal and recall actions. The closest number of total software attributable recalls would then be 3 - which is 23.07% of 13.

      On a side note - is it really that alarming that 3 out of a total of 13 recalled system were due to software error? If anything, it is a surprisingly low number.

    3. Re:Where does the report say this? by mcmonkey · · Score: 1

      From that section on pg 22:

      In collaboration with the inspection team, Ms.Simone identified a trend in customer complaints that the firm had not been aware of. This trend involved incorrect or missing patient results in a laboratory information system, and incorrect or missing notifications to clinicians that test results were out of range.

      Basically, problems with LIMS or other systems. It's not that the device released the incorrect dose, it's that the record keeping system had an incorrect dose history or test result, causing a doctor or nurse to administer an incorrect dose.

      Similar to the issue discussed here:
      http://www.economist.com/blogs/babbage/2012/05/medical-devices

    4. Re:Where does the report say this? by wienerschnizzel · · Score: 1

      True, however 'incorrect or missing patient results' in a record can be as deadly as a high dose of radiation. I'm sure you can imagine a catastrophic scenario yourself.

  47. Re:What are they doing about the 76% HW failure ra by mcmonkey · · Score: 1

    In my experience there is way way more software failures. The vendor just sends software updates every couple months. Oh yeah the previous version had a problem where if you did things in the wrong order it would change the patient that the radiation machine was programmed for. Sorry about that but here is the fix. Or worse notices saying their is a problem so telling users to double check all the time until they release a new version ... sometime.

    What you describe are not software failures. If there's something wrong in the code, it's likely a bug. That the wrong software made it through testing and QC and was released may be a failure of process. But a software failure is an event, not a condition.

    If the software would do the wrong thing is certain conditions occur, but the software is fixed before those conditions are actually encountered, then there was no software failure.

    In my experience there are way way more failures of the development process than software failures. Why would this be so, given that one process failure can lead to multiple software failures? Maybe we (and out consumers) are just lucky.

  48. Re:What are they doing about the 76% HW failure ra by mcmonkey · · Score: 1

    If there's an alarm, you add a microphone and light sensors to determine that if you're in the alarm state, there is an alarm sound playing and the lights are flashing.

    How do you add serial complexity to the system without increases the risk of failure? Now you've got to account for failure of the sound-detection process as well as the sound-producing process.

    Why not, for example, add a second alarm? With 2 alarms, the system fails only if BOTH alarms fail to sound. With the detection loop, you get an error condition if EITHER the alarm or the microphone fails.

  49. Re:What are they doing about the 76% HW failure ra by mcmonkey · · Score: 1

    Just because 100 - 24 = 76 does not mean that all the other recalls were hardware failures. Sometimes, a device functions well, but it's use is difficult or confusing.

    Suppose the device is too heavy for the average nurse to carry, and it can't be used safely. There's no material defect.

    Good point (though I might consider "portal device is too heavy" to be a hardware failure).

    There are so many links in this chain which can require a field action. Hardware, software, wetware. Typo on a label, instructions doctors aren't able to follow, etc.

  50. Re:FDA should develop an open platform like NSA di by Anonymous Coward · · Score: 0

    Well, you are somebody who does not work in the medical equipment industry. Most today, and all that have 'SOFTWARE' failures, have an OS.

  51. Re:What are they doing about the 76% HW failure ra by Anonymous Coward · · Score: 0

    Actually a majority of product recalls are related to labeling issues.

    Keep in mind, we are not talking about actual product failures, those are rare. We are talking about product recalls. A product gets recalled because the company can't adequately _demonstrate_ that the product is safe to use. This often times is related to documentation practices. Loss of product traceability is a huge deal. If you have a product in the field and you cant say exactly when it was made, by who, and with what materials (supplier lot number and date of manufacture for every component of that device) you have to recall it. It may function perfectly but it is still out of compliance. Furthermore, things like 'user dissatisfaction' ie a doctor having to pull to hard to open packaging, can be (in some cases) grounds for recall... (not super likely admittedly, but possible).

    25% due to software problems is amazing to me, though of the 4 medical device companies Ive worked at over the last ~8 years none have had any software components. You are required to do a validation of any software that has anything to do with your product or _quality system_. Some companies go so far as to validate Excel spreadsheets used to calculate process averages (no macros, just a plain old spreadsheet).

    For those of you interested Code of Federal Regulations chapter 21 is where the regs are housed:
    http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/cfrsearch.cfm

  52. Re:FDA should develop an open platform like NSA di by Anonymous Coward · · Score: 0

    Well, you are also somebody who does not work in the Avaiation OR Medical equipment industry. Many of the issues are not the upper code, but the OS code that is so poorly written that it requires constant change. The best thing is to get a fairly solid OS, with a solid library, and then update about once a year.

    Also, as one that has worked on both FAA AND FDA, I can tell you that you have NO FUCKING CLUE of what you talk about. When MS was approached to do DO-178B (which would work for both Aviation and Medical), they said see us in about 3 weeks. When we called back, they were laughing and said not a chance in hell could any of their OS's make. So, when you punk platform-fanbois shut the fuck up and allow the adults in the room to develop. It is idiots like you that push Windows crap and make a disaster not just of business software, but also of aviation and medical world.

    BTW, the reason for approaching MS to have them make their OS secured and decent, was because Airbus wanted to use more Windows in their planes. At this time, as the saying goes, if it ain't boeing, I ain't going. Airbus. What a fucking joke.

  53. Re:FDA should develop an open platform like NSA di by WindBourne · · Score: 1

    Ours? Is Linux yours?

    secure ours, means secure the west's OSs. Linux is just closer to being a secured system, and much easier, than trash would be.

    --
    I prefer the "u" in honour as it seems to be missing these days.
  54. Re:FDA should develop an open platform like NSA di by Anonymous Coward · · Score: 0

    17 years in the industry. And most devices don't use an OS. They used embedded stuff made as simple and robust as possible.

    But your point holds merit. Failures are much more common when there's an OS (and the stack of software that goes with it). Which is why OS's are avoided if at all possible.

  55. Re:FDA should develop an open platform like NSA di by Anonymous Coward · · Score: 2, Informative

    You will find that Linux is everywhere in the aviation and medical arena, except for where DO-178B is required. However, you have bluecat linux that has the same API as Lynx's DO-178B system. In addition, several groups are hard at work on doing DO-178B for Linux.

    In the mean time, there are PLENTY of equipment, mainly those using Windows, in which an open platform makes far more sense. And yes, Linux does have more of a real-time OS, than is windows.

  56. Re:FDA should develop an open platform like NSA di by Anonymous Coward · · Score: 0

    1. I didn't say anything about Windows. I said platform fanbois, which would include anyone who thinks platform X is a solution for everything. 2. Most medical devices, especially implantable ones, don't run on an OS... they run on embedded firmware. God help us all if people like you are programming airplanes and medical equipment.

  57. Re:FDA should develop an open platform like NSA di by Anonymous Coward · · Score: 0

    You obviously have no clue of what you are talking about. There is an increasing amount of medical equipment that comes out of GE, HP, etc. that does NOT use firmware, but uses Windows OS with SOFTWARE in it. The fact that you have no clue about it, only tells me that you are a total idiot windows coder that has no grasp of the situation. BTW, I had years of working on z-80 assembler, 68000's, and then moved up to embedded RTs. What I have seen is that idiots like you moved in and have ZERO clue of how to code your way out of a bag. The industry has been pushed by PHBs to use Windows, which is the WORSE OS possible. Sadly, FDA has plenty of idiots, as does other companies (airbus comes to mind).

    As I said, it is actually better to come up with a secured open OS, make it solid, and add in a number of SOLID open libraries. This will NEVER come from MS. And it will NEVER come from idiots like yourself.

  58. Re:FDA should develop an open platform like NSA di by Anonyme+Connard · · Score: 1

    Linux is the West's OS? Where is it stated in the GPL?
    And secure it against who?

    What's wrong with you, USA?

  59. Don't blame the software by Murdoch5 · · Score: 1

    Are you joking that there trying to blame the software structure and the design? That is so much bull I can smell it from here. A good programmer doesn't need over structered code design and over engineered software solutions. Blame who's really responsible, blame the craptasktic programmers who can't do there job. Programmers who needs strict structure and standards to program need to change professioins.

    The reason code standards are made is simply because there are programmers who need to have there hand held and be treated like a 5 year old.

  60. Re:What are they doing about the 76% HW failure ra by Anonymous Coward · · Score: 0

    Thank you for accpet low price for hi quarity device make. ~China

    Thanking you 1000 times for 5 Rupee/hr firmware coding job. ~India

  61. Re:What are they doing about the 76% HW failure ra by aaronb1138 · · Score: 1

    Derating power supplies and such is great, but microphones and light sensors to check on buzzers and lights is quite backwards. Both can be spoofed by ambient noise or light assuming simple to moderate detection models. Cheap, easy, and reliable would be to just measure the current and voltage drop across critical components. Bonus is that you can use one detection method and likely IC for multiple subsystems and that programming it into the software is trivial compared anything else.

    The other method of hardware protection I prefer is just straight redundant systems where the redundant unit initiates a separate set of alarms once activated. For example, if a buzzer fails, the switchover triggers a backup buzzer plus flashing lights or error messages on display.

  62. Re:FDA should develop an open platform like NSA di by Anonymous Coward · · Score: 0

    Companies like GE, HP, and a few others have walked away from firmware and use WIndows HOPING that a powerful processor will cover RT issues. Then they HOPE that the lower stacks are solid, which NOTHING out of MS ever has been. It is long past time for us to switch to a well-known open stack that is solid, with a large number of solid libraries.

    The real problem is that PHBs have taken over companies like GE and HP who have an MS salesperson attached to the front of their pants and basically fired the engineers to keep them from overriding their horrible choices.

  63. a related problem by s.t.a.l.k.e.r._loner · · Score: 1

    A major related problem is that hospitals and other organizations get stuck using poorly-implemented devices because they are too fucking expensive to replace with something else. The expense in and of itself isn't totally unreasonable, because manufacturers of medical devices need to comply with a great deal of regulations and safety testing. However, that doesn't help the users any.

    The hospital I work at, for example, bought a software suite years ago for our pharmacy and financial departments to use, and about a year ago it was rolled out to all other departments in our transition to electronic records. This software is some of the most clunky, ham-fisted crap I've ever seen. We despise the software for a great number of reasons (not the least of which is that it actually takes 2-3x LONGER to do anything than it did with our old paper records), and with the push to get all hospitals and medical offices using electronic records, something better would be most welcome. Unfortunately, with what we paid (and continue to pay) for the software contract, there's no budget to get something different and probably never will be because as far as the board is concerned it does what we need it to do.

  64. Re:What are they doing about the 76% HW failure ra by glodime · · Score: 1

    Where are you getting a 76% failure rate from hardware?

    Medical devises can be recalled for reasons other than hardware failure.

    76% would be the ratio of recalls not due to Software failures to total failures (though Mcmonkey points out that the 24% reported is not found in the underlying report).

    A failure rate would be the ratio of the number of failures to total deployments or uses of medical devices.

  65. Re:What are they doing about the 76% HW failure ra by sjames · · Score: 1

    How do you add serial complexity to the system without increases the risk of failure? Now you've got to account for failure of the sound-detection process as well as the sound-producing process.

    Actually, you don't. All you have to know is that one of them isn't working. The goal isn't to have the thing run for as long as it possibly can before failing silently, it's to make sure it doesn't fail silently. If either the alarm OR the microphone fail, the device is no longer dependable so you enter the unit failed state and signal that condition through LEDs and behavior..

  66. Re:What are they doing about the 76% HW failure ra by mcmonkey · · Score: 1

    I think I see. It's not about eliminated or reducing modes of failure, it's reducing modes of silent failure.

  67. Re:What are they doing about the 76% HW failure ra by sjames · · Score: 1

    Exactly!

  68. Overly Simplistic by kaufmanr · · Score: 1

    Looks like there were at least a couple of other medical device developers in this discussion, but for the rest of the readers here suggesting "open source", it's just not as simple of a solution as that. There are no standard hardware or software platforms that could be used across the board for medical devices, or should there be. The hardware and software needs of a glucometer are completely separate of those for a CAT scanner. The term "Medical Device" is much too broad and applies to many disparate systems to think in that fashion.

    On another note, it is worthwhile to remember than some of the medical devices that are in use in the field (and generate these complaints) are probably quite old. Yes, there have been failures associated with software, hardware, or human factors issues, however complaints associated with devices that have been in constant use in the field for 20+ years comprise at least a portion of this. Have these devices met their goal of reliability? Absolutely. Do these devices meet the current regulator's expectations for new devices (reference IEC 60601 series for basic safety, EMC, software design, usability, home use, etc)? Not necessarily. Is the expectation that medical device companies recall these devices so that hospitals are forced to buy a newer updated version of these devices? Risk analysis (see ISO 14971) wouldn't necessarily agree with this (depending on the failure modes) as an instant recall or end of support or not selling the consumable (if there is one) would result in the hospital's (or long-term care facility, or home user) inability to use the device until it can be replaced. Yeah, there could be ways to avoid or minimize this, but I'd be willing to wager there are millions of "old" devices (20+ years old) in active use in the field and there would be a definite impact to cost and a disruption to many users if this were the approach.

    Final thought: Not surprised the idea of Linux comes up on Slashdot as a solution. Linux has been discussed by many development teams that I've been a part of. However, the 'ol Validation question always comes up. Not that Linux has never or will never be used on a device, but the cost of validating it becomes a much larger question (and cost) than either designing your own simple OS built specifically for its purpose, or purchasing an off-the-shelf "validated" OS. Many devices don't even really need an OS due to the device's simplicity (in the sense of what the average Slashdotter would think of as an OS), so the idea of Linux for everything is a bit off. Note: When I say Validation, I mean meeting the expectation of the FDA 21 CFR 820 as well as ISO 13485. There are more specific requirements depending on the device type/classification, but those are the starting points.