Slashdot Mirror


Fed-Up Hospitals Defy Windows Patching Rules

bingbong writes "According to Network World: 'Amid growing worries that Windows-based medical systems will endanger patients if Microsoft-issued security patches are not applied, hospitals are rebelling against restrictions from device manufacturers that have delayed or prevented such updates. Device makers such as GE Medical Systems, Philips Medical Systems and Agfa say it typically takes months to test Microsoft patches because they could break the medical systems to which they're applied. In some instances, vendors won't authorize patch updates at all.' This is the typical patch vs. crash problem. Unfortunately, the stakes here could be human lives."

24 of 705 comments (clear)

  1. Stop playing solitaire on my dialysis machine by Anonymous Coward · · Score: 5, Insightful

    Why is hospital equipment running windows? Anyone that knows anything about embedded systems with high quality requirements know that you stay away from large OSes. Even Linux is avoided unless you need tcp/ip and if you don't then its better to have a small maybe even off the shelf OS. The Key is to limit the testing requirements and limit changes, which are goofy to test a life support system just to have the latest and greatest IE 6 or 7 that you shouldn't even, have hooked to a wide-open Internet anyway.

    1. Re:Stop playing solitaire on my dialysis machine by dekemoose · · Score: 5, Insightful

      They are running Windows for the same reason that they are connected to a network, some pinhead PHB somewhere is trying to save a buck. It's probably cheaper for them to develop on a Windows platform rather then on a proper embedded paltform. Just like its cheaper for them to put these devices on a shared network, rather than having them properly firewalled off onto their own secured environment. Follow the $$$.

    2. Re:Stop playing solitaire on my dialysis machine by Short+Circuit · · Score: 5, Interesting

      Part of the problem is that the vendors chose Windows as a development platform.

      I'm a rabid Linux user, but if I were designing equipment that held human lives in its anthropomorphic hands, I'd build it as an entirely atomic OS built from Linux or a BSD variant. And communications would be data-only, over a serial port. No network.

      In high school, a nurse from St Mary's (here in Grand Rapids, MI) was showing us screenshots of their radiation therapy machine. I recognized CDE...she didn't know what version of UNIX it ran, though.

    3. Re:Stop playing solitaire on my dialysis machine by mattOzan · · Score: 5, Insightful
      I find it hard to believe they are talking about life support machinery. No specific piece of equipment is ever mentioned, just the generic "medical devices." I'm thinking they are speaking more of hospital informatics systems, like Stentor and EpicCare. When a doctor can't read a patient's medical chart because the workstation is PWNED, or can't send an X-ray up to surgery because the router's been hijacked, that is definitely a problem; but it is somewhat less of a problem than your ventilator quiting because of a BSOD.

      Sounds like a tech-challenged reporter reporting wide-eyed about crashing "medical devices" which she doesn't really understand.

    4. Re:Stop playing solitaire on my dialysis machine by Tongo · · Score: 5, Interesting

      Speaking of a radiation therapy machine with software bugs.....

      This was posted to /. a while back: An Investigation of the Therac-25 Accidents

    5. Re:Stop playing solitaire on my dialysis machine by MindStalker · · Score: 5, Insightful

      But the point is still the same, you should run a machine with only enough code to do the job. Extra cruft is just risking "bugs" which could cost lives.

    6. Re:Stop playing solitaire on my dialysis machine by Omega1045 · · Score: 5, Informative
      Part of the problem is that the vendors chose Windows as a development platform.

      Uh, no. Do you work in the health care industry? I do as a software developer for a vendor. Don't throw the blame on us. We actually changed to Windows off of other systems because hospitals started putting PCs with Windows into their various departments. The backend for the software I work on actually runs in Unix, and we have hospitals that are thinking of going to NT only, which means we have to try to port our code to it or loose that customer.

      --

      Great ideas often receive violent opposition from mediocre minds. - Albert Einstein

    7. Re:Stop playing solitaire on my dialysis machine by FFFish · · Score: 5, Insightful

      Don't be silly. The system should be based on an OS that is proven hard-core stable and real-time, like QNX, Microware OS-9/9k, etc.

      There are a ton of good OSes out there for specialty applications and, surprise!, most of them don't involve Linux! Linux is not the be-all and end-all of OSes.

      For human-life-critical applications, you should be using something that is demonstrably proven.

      --

      --
      Don't like it? Respond with words, not karma.
    8. Re:Stop playing solitaire on my dialysis machine by Anonymous Coward · · Score: 5, Insightful

      They are not usually using Windows for embedded systems. They are using Windows on workstations.

      I am a clinical doctor who programs in a couple of dozen languages and environments and follow the advancement in software solutions closely. I have been involved with clinical informatics only recently for the past couple of years though. Allow me to explain you some of the realities of the current health industry software. I admit I haven't sat down and structured the text well but I tried to put in as many issues as I could think of the moment.

      The doctors want Windows or Macs. They want a familiar set up compared to what they use at home. It is very difficult to get doctors learn a completely different paradigm. There have been documented cases where nearly all the doctors in certain institutions rose up in arms because the developers thought they knew better and tried to force a solution onto them.

      Most of the current set ups are almost always heterogeneous. We buy software from multiple vendors and bridge them together. This is because there are no completely integrated solutions as yet. GE and a few others are trying to close this gap but it is a VERY difficult one. Hospital information needs are not standard as your usual business information needs. The data processing here is often very simple but the volume and complexity of the data is overwhelming. It is not simple as Customers and Invoices. Clinical Medicine deals a lot with relatively abstract data with complex relationships. Most doctors know these relationships intuitively but there isn't enough published literature for a software developer to draw from. Clinical software is extremely expensive to build since the requirements are hard to establish. A lot of iterations are needed to fit the software to a given practice (This never gets completed usually and people settle for close enough).

      Doctors themselves understand their needs best. A few doctors, while they don't hold CS degrees, practice design patterns or do EJB, do quite well to put together MS Access databases to solve their problems where professional software developers have not yet tread. Many times, they distribute these to their colleagues freely (Open Source if you will). Few even sell them. They may not be the best designed tools but they work. Mac's FileMaker and Linux's Total Rekall? don't exactly come close. Windows tools also have a larger number of books available to learn from.

      Platform and tool costs are trivial, developer costs are not. A study in Human Computer Interactions is very essential here. Rich user interfaces are always preferred. Non-windows platforms don't have sufficiently advanced RAD tools. I really wanted Kylix to succeed. But I don't see any momentum behind it anymore. Veteran's Affairs Hospitals have built a remarkably physician friendly system. They are rightly proud of their constantly iterative development. They used Delphi but now that the system is stable (for user experience stand point) they are looking for other platforms. They looked at .NET. I heard they were trying Java now. Personally I am not sure it is the right choice for the client but we shall see.

      The loss of work hours because the software does not fit the workflow at a given hospital is far far greater than losses due to worms and viruses.

      The software should be as intuitive to use as possible. Should not require reading manuals. Hospitals always look whether the given software will slow the physician down in any way because physician time is very expensive and they rather have them seeing patients and generating revenue.

      There is a case for cross-platform tools at the moment too. It is a case of mobility. Most doctors like to be able to review a patient's case online and advice on the phone when necessary. Many vendors provide web pages and applets for this but they often end up very unergonomic. But since the need is often information retrieval rather than data entry, they are accepted in the absence of the better alt

  2. Why do they need patching? by Anonymous Coward · · Score: 5, Insightful

    Why are they even accessible on the internet? Seems like these should be in a secure private network unlikely to be attacked.

    1. Re:Why do they need patching? by blueZhift · · Score: 5, Informative

      Let me tell you, stuff gets inside hospital networks like nobody's business! The problem is that while the outer firewall is secure, there are all sorts of ways for things to get in via individual workstations. This is especially true since many hospitals, like mine, have standardized on IE. I was literally in the process of patching a Windows 2K based acquisition PC when it got hit with Sasser! Lucky for me the patch just barely beat the infection, so I didn't have to rebuild the machine.

      Because the inside of the hospital network is so insecure, I've actually set up my own firewall around my test and development machines. One solution would be to totally cut off the hospital from the internet, but that wouldn't be very practical and would piss off a lot of doctors to boot!

    2. Re:Why do they need patching? by AKAImBatman · · Score: 5, Insightful

      Why are they even accessible on the internet? Seems like these should be in a secure private network unlikely to be attacked.

      Who said they're on the internet? Consider the following scenario:

      The Hospital PCs are connected to a primary server that backs up all data and managed the PCs.

      The Primary Server has a leased line or occasional dial-up to transfer data to a state-wide backup and update site.

      The backup and update site has firewalled internet access for a VPN to GE, and troubleshooting purposes.

      GE communicates with customers via internet email. One clerk in a backroom opens an attachment with an RPC worm. Within a half-hour the entire chain is compromised.

      Any question on why having a monosystem Windows network is a bad thing? Even ONE Unix server in there would help break the chain.

  3. so? by Anonymous Coward · · Score: 5, Funny

    pshaw! what's a few human lives when network security is at stake?

  4. Why in the hell... by daveschroeder · · Score: 5, Interesting

    ...do they not just put these devices and systems behind something as simple as a $50 hardware NAT firewall, especially for a device that costs hundreds of thousands - or millions - of dollars? (Or better yet, why does the vendor not integrate such protection if they're relying on network-connected Windows systems for device control/interaction?)

    The norm is that these devices may need to connect *out* to something else, but don't necessarily need any inbound connections, so a hardware firewall, or even a host-based software firewall, would work perfectly in most instances; those that do need externally initiated inbound communication can *still* set up the necessary rules to allow such communication to take place. And yes, it is just this simple. (I did RTFA, and noted that some vendors actually recommend this, but that, startlingly, "there have been several instances in which viruses originated from medical instruments straight from the vendors"!)

    1. Re:Why in the hell... by pclminion · · Score: 5, Insightful
      put these devices and systems behind something as simple as a $50 hardware NAT firewall, especially for a device that costs hundreds of thousands - or millions - of dollars?

      How is a firewall going to stop an insider from exploiting the network? Does working in a hospital magically transform a person into a paragon of morality?

    2. Re:Why in the hell... by cammoblammo · · Score: 5, Funny
      "there have been several instances in which viruses originated from medical instruments straight from the vendors"!

      Viruses from medical equipment? Haven't they heard of autoclaving? Sounds like a negligence lawsuit waiting to happen...

      --

      Cogito, ergo sig.

    3. Re:Why in the hell... by Ryan+Stortz · · Score: 5, Informative

      No, most machines (from GE atleast) listen for incoming SSH sessions. This is so it's main tech guys can connect (from Wisconsin) and fix the problem. It saves the Hospitals money, they don't have to call in a field service guy for $150+ an hour. The tech guys can even find a faulty board, order it, have it shipped to the hospital, and have a guy swing by the next day and replace it without alot of wait.

      --
      Bugs are just features that have been fixed.
  5. Doesn't have to be a issue by bs_testability · · Score: 5, Insightful

    Medical machines responsible for human life should never need to be patched. The software was tested at one point and should be controlled to stay at that test point until it is to be retested. For machines running windows this means they should be segregated from other parts of yoru network and should be airgap firewalled from the rest of the world. Intenet worms and email trojans shouldn't be relevant.

  6. Can't say I'd blame Microsoft this time around. by Rude+Turnip · · Score: 5, Insightful

    I'm not a big fan of Microsoft, but I don't think the quality (or lack thereof) of their products is the issue here. I've read from their EULAs that their products are not suited towards critical applications (ie nuke facilities, life support). My point is that although a EULA is not a legally-binding contact, the fact that MS is stating in public Windows shouldn't be used in critical applications should tell you something. The bottom line is that if GE, Philips or Agfa build a medical system, they should be responsible for that product from the software up to the hardware. The fact that *they don't have control* over one of the components in their products (the underlying OS) is negligent, IMO.

    I would get laughed out of court if I tried to blame a critical problem with a report I wrote on my secretary, and the same should happen with these companies if somebody's loved one dies from their irresponsibility.

  7. Re:FDA? by m.h.2 · · Score: 5, Interesting

    Having spent 10 years working in the Medical Device/Biotech domain, I can tell you that the FDA really does govern these things. Unfortunately, their internal understanding of computer systems in general is frighteningly scarce. Essentially, the only body of legislation they have to go by is a small portion of a CFR (Code of Federal Regulations: 21CFR Part11) that was released in 1997, and the enforcement guidance documents that followed it. The Code is extremely ambiguous and realistically lumps "electronic documents" and "electronic signatures" together. The compliance issues resulting from the vague document and its (mis)interpretation and enforcement were enough for me to change industries. My heart goes out to all of the people still battling this.

  8. Security AND Stability by for_usenet · · Score: 5, Informative

    I work with MRI scanners, so I know about these issues very well, and here's an example from my own experience:

    An old colleague of mine got funding to start his own reasearch group, meaning he got his own MRI scanner. He asked me to consult on some software that would extract the data from the console of a Siemens scanner (at the time, the console was based on an OLD version SunOS, whose native compilers did not even conform to standard ANSI C) and send it directly to another computer running software that we use for data analysis. The dialect of C was a little strange, but within a week, I was able to get the software together, and my colleague was able to do the type of experiments he wanted to. And his scanner hummed along. This was back in 2001.

    Fast-forward to the present. His console has since been "upgraded" to Windows XP system, and in the times I've spoken to him, he's had nothing but bad things to say about the stability of the "upgraded" system. And it's not that he had a choice, as support for his previous system was phased out. So now patients, doctors and reasearchers in his group are at the mercy of the moods of an XP system. And mind you - this system is not even on a publicly accessible network. It is on its own dedicated, private network, and its stability still can't be maintained, even by the support staff of the scanner manufacturer.

    When it comes down to it, Windows still does not have the stability (never mind the security issues to cut it in really "mission-critical" situations). Maybe in cases where you need your e-commerce site up, running, and handling 1000s of transaction per second. But NOT when peoples' lives are involved.

  9. Everyone's asking why aren't they firewalled... by foxtrot · · Score: 5, Insightful

    Firewalls won't help. If it runs Windows, some idiot's going to bring in a CD full of pictures from his latest vacation and the CD's going to be infected with MyDoom or (heck, probably and...) Sobig or any number of other nasties. Or it's going to be something he wants to print on the nice laser printer at the office.... there's a hundred ways to get infected just by clueless users.

    Pretty soon, the internal network's either too busy generating random traffic to do anything else-- and even if the Big Iron of the business, the dialysis machines and heart-lung devices and all those wonderful things that better damned well not break work fine, you've still got the terminal the nurse sits in front of that keeps track of when to issue you your shot that keeps you alive spending half its time rebooting because it's got Sasser.

    This is not a problem a firewall can solve, and it's pretty darned big: You can't go throwing software around willy-nilly to solve this problem (even though the real problem is that the users _are_ throwing software around willy-nilly), so you can't just go "oooh! A next-day patch from Microsoft, let's hope their two hours worth of QA before it walked out the door was good enough!".

    -JDF

  10. Re:GE Medical Systems by djh101010 · · Score: 5, Informative

    Sorry, Ryan, but you're not correct. I worked for GEMS for 12 years, in software engineering. There _are_ Windows systems embedded into some of these scanners. Most of them do trivial things and are being phased out in favor of *nix systems, but there _are_ Windows-based medical devices.

    It's quite a quandry. If you don't patch the 'doze boxes, (and if you don't have a firewall...) it's possible that someone could infect that system. The problem is, GE (and obviously the other device manufacturers) test the hell out of that specific OS build and patch set. When Windows Update breaks things (which happens more than never), the system is now in a state which GE didn't test, and may in fact break the functionality of the scanner. At this point, the FE has no choice but to re-load the PC from the GE-supplied media(which doesn't have the latest patch that the hospital just installed).

    The solution? It's pretty simple, stop using Windows in critical situations. I was trying to make that point 10 through 5 years ago there, and was involved in some of the very first Linux tests, prototypes, and production implementations there. The current generation of scanners is mostly linux/intel based, although there is still a lot of SGI/Irix at the top-end where heavy image processing is done. The fix for this problem, is to avoid this problem, and that's really the only sensible approach.

    So, yes, they do have 'doze systems embedded in some of these scanners, but it's getting better. The hospital gets to choose between complying with HIPPA and patching the systems, or installing an unsupported patch which might break the scanner. Not a good place to be in, but then again, people shouldn't be reading their email or surfing the web from the MRI scanner's console, and the hospital _should_ have a firewall blocking the slammer/whichever ports.

  11. Coming soon... by Datoyminaytah · · Score: 5, Funny

    > Unfortunately, the stakes here could be human lives.

    Soon to be made into a movie starring Uma Thurman.

    It's called "Bill Kills".

    --
    assert(birth_date<time-86400)