Slashdot Mirror


FDA: Software Failure Behind 24% of Last Year's Medical Device Recalls

chicksdaddy writes "Software failures were behind 24 percent of all the medical device recalls in 2011, according to data from the U.S. Food and Drug Administration's (FDA's) Office of Science and Engineering Laboratories (OSEL). The absence of solid architecture and 'principled engineering practices' in software development affects a wide range of medical devices, with potentially life-threatening consequences, the FDA warned. In response, FDA told Threatpost that it is developing tools to disassemble and test medical device software and locate security problems and weak design."

27 of 128 comments (clear)

  1. What are they doing about the 76% HW failure rate? by sizzzzlerz · · Score: 5, Insightful

    It seems like that should be of even more concern.

  2. Backups by SJHillman · · Score: 2

    It would seem they don't do backups either. Every time I've had my hearing aids sent in for repair, I've had to have them reprogrammed from scratch because they never save the settings first.

    I have an appointment to have them repaired again in 3 hours. Any bets on whether its a software issues?

    1. Re:Backups by glueball · · Score: 2

      If the medical company follows a process, and you should hope they do, they will send the equipment back to you in a known good state. Your settings are not part of the known good state even though they are within guidelines. Further, if a new setting is added to the hearing aid, where should they set it? Is setting it to the default compatible to your previous settings?

      It's a feature.

      PS, a company following a process will do the same thing even if it's something like my tractor. Every time John Deere comes to service my tractor, they make sure all safety features are working and emissions are functional, no matter what the service is about.

    2. Re:Backups by glueball · · Score: 2

      The more technology improves, the more quality control seems to go down the can.

      As a medical professional, I'd say a company that takes in a device on RMA and returns it to a known good, known tested state, is far superior in "quality control" than a vendor who would individualize each service routine.

      What you're asking for is exactly why personalized medicine is doomed to fail.

  3. Re:What are they doing about the 76% HW failure ra by ILongForDarkness · · Score: 2, Insightful

    In my experience there is way way more software failures. The vendor just sends software updates every couple months. Oh yeah the previous version had a problem where if you did things in the wrong order it would change the patient that the radiation machine was programmed for. Sorry about that but here is the fix. Or worse notices saying their is a problem so telling users to double check all the time until they release a new version ... sometime.

  4. Demand Free Software by betterunixthanunix · · Score: 4, Insightful

    Can someone please remind me why people should be unable to examine the software in their medical devices, software that their lives may depend on? Why these programs are not open to public review?

    Oh wait, I got sidetracked thinking that the point of medical devices is to keep people healthy, rather than to rake in profits for the companies that make them.

    --
    Palm trees and 8
    1. Re:Demand Free Software by MozeeToby · · Score: 5, Insightful

      Hiding the source code is not an effective way to prevent hacking, if it were my Windows box wouldn't need a hardware firewall, a software firewall, 3rd party antivirus software, and regular sweeps initiated from a different OS.

    2. Re:Demand Free Software by ongelovigehond · · Score: 2

      Why stop at the software ? For a complete review, the hardware design should be open too.

    3. Re:Demand Free Software by glueball · · Score: 4, Informative

      The MRI machine I use has a complete circuit diagram along with design notes in a binder set next to the machine. In the US, you get the hardware manual for service. I don't believe the same is true for Europe and I have no idea about the rest of the world.

    4. Re:Demand Free Software by dark12222000 · · Score: 3, Insightful

      Look up "Code Signing". Then bash your head against your desk three or four times as punishment for the stupidity you typed out above.

    5. Re:Demand Free Software by David+Chappell · · Score: 2

      The open source zealots love these types of comments. It's not a valid solution, and if you allowed yourself to think outside the open source box you're in you might see it too.

      Allowing anyone to view the code means anyone can then modify it.

      As far as I am concerned, "anyone" can modify it all he wants just as long as he doesn't install it in an in-service medical device without proper approval. If we don't yet have legal, organization, and technical means to prevent this from happening, we should.

      We are not talking about opensource software here. We are talking about allowing anyone who wants to to audit a piece of proprietary software.

      I do though think that we should restrict access to the text of our laws. Think what would happen if somebody got a copy and modified the law.

    6. Re:Demand Free Software by dark12222000 · · Score: 2

      Well, yes. If you modify the product, then it's on you. However, having the code be open source means:

      It can be inspected
      It can be verified
      Patches can be written (and then submitted to the manufacturer)

      The idea is not that the little IT captain at your local hospital is going to rewrite the MRI. It's that he's going to run into an issue, pull up the source code, write up a patch, submit it to the manufacturer, then the manufacturer is going to throw it out, force their engineers to write it again, run it through QA testing, and then issue a patch.

      The manufacturer can't claim the problem isn't solvable (patches can be provided) and they can't claim it doesn't exist (source code can be used as proof). At the same time, assuming they use code signing, only they can modify the machine without voiding the warranty.

      Now, here's where it gets better. Say the manufacturer dies off/stops caring/whatever. Source code is already out there - an independent (certified/insured/over-payed) firm can come in, release their own patches, and still modify the machine (albeit voiding the original warranty while still keeping the hospital from being exposed to damages via contract). Suddenly, you aren't depending on the OEM to provide EOL patches for the entire life of the machine, yet at the same time, the OEM isn't responsible for some dickweed tweaking an MRI machine to kill people. Win/Win for everyone.

      In addition, all the research and technology for that MRI machine is now accessible to others, which then lowers the cost of MRI machines, makes them more available around the Globe, and lowers the cost of ownership. At the same time, the OEM is still making money hand over fist from insuring and babysitting the machine (they now have to compete with independent firms, but they have a major advantage as the OEM). Again, Win/Win for everyone.

    7. Re:Demand Free Software by plover · · Score: 2

      Allowing anyone to view the code means anyone can then modify it.

      As the Mythbusters like to say, "Well, there's your problem!" Your entire argument is based on the extension of this premise to imply that you can then install this modified software on the medical device. But that's not a given at all. You can modify the downloaded copy of the code that you have squirreled away somewhere in /users/autocannon/src, but it doesn't mean you can modify the exact copy of the code that's running on the CPU in your insulin pump.

      It may not even be physically possible. Consider that I can burn GPL (v2) code to an FPGA, then burn the fuse to prevent further modifications to the chip. As long as I distribute the source code with the device, I am free to sell the device, even though I've given you no end-user-accessible way of modifying it. Tivo used a variant of this idea, where they burned a digital signature verification process on their devices which then refused to permit unsigned updates to their code. Called "Tivoization", this practice led directly to the creation of the GPL V3.

      Can you take a medical device apart and replace the ROM with your own modified code? Obviously it's technically possible, but if it's a medical device it will no longer be certified for medical use. No legitimate doctor would prescribe that modified device to a patient (outside of the device maker's controlled studies, of course.)

      --
      John
    8. Re:Demand Free Software by Flatwater · · Score: 2

      I hope that binder doesn't have metal rings:

      http://www.howstuffworks.com/question698.htm

  5. FDA should develop an open platform like NSA did by WindBourne · · Score: 3, Insightful

    Seriously, the smart thing is to develop an Open platform on Linux, with libraries for equipment to use. Likewise, offer up secured ways of updating the equipment. If FDA was smart, they would talk to NSA.

    --
    I prefer the "u" in honour as it seems to be missing these days.
  6. Re:What are they doing about the 76% HW failure ra by MimeticLie · · Score: 2

    Those things can cause hardware failure, but they generally don't cause recalls. Remember that we're talking about device recalls, here. The hardware failures are also likely to "be present in every single instance of the device" if they need to fix all of them.

  7. Re:DMCA? by MadKeithV · · Score: 5, Funny

    In a distopian novel, the government would do this so that they could turn off your heart, if you said anything out of turn.

    It wouldn't work on politicians or lawyers. They don't have hearts.

  8. Re:Outsourcing by Anonymous Coward · · Score: 2

    Excellent point. I work for a med device company, too, and know for a fact that the cost of a recall and/or patient lawsuits far outweighs any software development expenses. Cutting corners is not considered and the level of design and qualtiy controls is very high.

    I would suggest that Billy Gates is absolutely being too cynical. //24 years in the med device business

  9. "locate security problems and weak design." eh? by Anonymous Coward · · Score: 2, Insightful

    They might as well just start from scratch then. I used to work for a huge healthcare company and dealt with some of the debacles that these devices cause. "Our device only supports WEP....is that going to be a problem?" Pathetic. Luckily the place I worked was big enough to push them around and do things like force them to implement EAP-TLS, but it was tough going. Then you have all the BS with how the FDA "doesn't allow us to upgrade software without extensive testing", which of course is not entirely true.

    These companies are just like every other medical software vendor...for some reason they feel entitled to produce absolutely terrible products that are 10 years behind the rest of the world. I don't know why the medical industry is like this, or why customers put up with it. The general attitude where I used to work was, "OH NO DON'T UPSET THE POOR VENDORS!!". I was like, "aren't we paying them? tell them to fix their product or go to hell". This was true of desktop software, medical devices...everything. They wanted to bring a new system online for tracking blood donations and the software required "act as part of the operating system" privileges. Really? We're going to update our Group Policy which already gives every user local admin rights to allow that too? Why, exactly, would that be? Especially since it's nothing but a database application. Another application we had on the network would crash when our vulnerability scanner would probe its port. The entire piece of software would just die because it got a packet of a type it wasn't expecting. This wasn't an aggressive scan, this was just a probe looking for open ports. I told the department, we can give you a scan exception, but this is not a problem that's going to go away just because we stop scanning your device.

    I think the entire Medical IT industry has a day of reckoning coming. The unnecessary proprietary requirements, the poor design, the unreasonable legacy OS requirements, the poor security...it's endemic to the industry. There's this attitude that they want to use IT extensively in healthcare, but not change the workflows of providers in any way (including things like requiring strong passwords). It's not a sustainable model.

  10. Re:FDA should develop an open platform like NSA di by WindBourne · · Score: 2

    So, you think that by TALKING to the NSA, who added a nice security module to Linux, they will put a back door into medical equipment?
    If you are going to make wild assertions, you might want to look up what NSA's mission is: listen to other system AND SECURE OURS.

    NSA has some of the best security ppl on-board. I would trust them to handle my medical devices, before I continue to trust the idiots at MS and those that use Windows.

    --
    I prefer the "u" in honour as it seems to be missing these days.
  11. Weak design?? by ZenDragon · · Score: 2

    I like how everybody likes to blame their problems on "weak design" on the part of the developers. That may very well be true, but who is to say what is weak design and what isn't? As a developer, I often find that when beginning a project that has some characteristics that I have never worked with before, that it is almost impossible to find solid established information as to what actually is "strong design." And by that I mean good patterns and practices for a specific applications that really aren't very common. Sure there are a few good basic/general practices to adhere to, but for the most part its each coder for their own. So I wonder, what the hell the FDA things they are going to contribute to a world of code that they likely cant even comprehend? You can sit on the outside and say something like; I require 6 9's of up time, and such and such SLA, and this square brick fits into this hole; but what is done to make that happen will vary infinitely behind the scenes.

    If they really want to improve the stability and reliability of the code that supports these systems the need to open up the architecture to the entire community instead of keeping everything closed off for patent reasons or whatever. Obviously the companies that make this stuff are more concerned about their profit margin than they are about the safety of the patients that their equipment is treating. That's my opinion on the matter at least.

  12. Karen Sandler by Anonymous Coward · · Score: 2, Informative

    IP restrictions on medical devices' source code, no peer review or approval structure in place from FDA or health organisations. Complex medical devices that are implanted in humans bodies, e.g. insulin pumps, heart defibrillators etc. run software and operate more and more like computers. Here is a case of Karen Sandler, a woman who asked to see the code for the device she was to be implanted with to verify that is was safe. And what she discovered in the process.

    OSCON 2011: Karen Sandler
    www.youtube.com/watch?v=nFZGpES-St8

  13. Re:FDA should develop an open platform like NSA di by DeTech · · Score: 2

    Riiight... Ever heard of RTlinux? RTAI maybe ? OSADL? You already find it in a lot of cool life/time critical systems already... esp in the defense world.

  14. When Software Attacks by gumpish · · Score: 2

    Obligatory THERAC-25 mention. Software has killed before:

    http://en.wikipedia.org/wiki/Therac-25

  15. Re:What are they doing about the 76% HW failure ra by tlhIngan · · Score: 2

    Hardware can fail for a much wider variety of reasons; poor maintenance, overuse, physical abuse, one off manufacturing defects, etc. Software failures are caused by an error in design or implementation; they are almost guaranteed to be present in every single instance of the device even if it takes an oddball corner case to set it off.

    For hardware, to combat failures you overdesign it. E.g., if it's powered by the AC line, you make sure the power supply components are overrated for their worst case load (derating of parts makes them last much longer).

    If there's an alarm, you add a microphone and light sensors to determine that if you're in the alarm state, there is an alarm sound playing and the lights are flashing. You build in extra annunciators as well just in case the LED dies. You count 75% battery capacity as "low battery".

    And yes, I've seen those countermeaures actually implemented for a medical device.

    Of course, it also impacts software as now it's even more complex as it has to handle and detect these conditions as well

  16. Where does the report say this? by mcmonkey · · Score: 3, Interesting

    As a developer working for a medical device company, I am very interested in this story.

    However, I am not able to find in the linked report either that "24%" figure or the direct quote from TFA.

    The Agency is also acquiring expertise in areas like "detecting malware inside device designs...(and) reverse engineering certain types of malware to best identify the specific protective practices which manufacturers should be employing," the report reads.

    The word "malware" appears twice in the quoted passage, but not at all in the report. And 24 only appears as a page number or date.

    Am I just not hitting CTRL-F right today?

  17. Re:FDA should develop an open platform like NSA di by Anonymous Coward · · Score: 2, Informative

    You will find that Linux is everywhere in the aviation and medical arena, except for where DO-178B is required. However, you have bluecat linux that has the same API as Lynx's DO-178B system. In addition, several groups are hard at work on doing DO-178B for Linux.

    In the mean time, there are PLENTY of equipment, mainly those using Windows, in which an open platform makes far more sense. And yes, Linux does have more of a real-time OS, than is windows.