Slashdot Mirror
Know What Time It Is? Your Medical Device Doesn't
An anonymous reader writes "A man with one clock knows what time it is, goes the old saw, a man with two is never sure. Imagine the confusion, then, experienced by a doctor with dozens. Julian Goldman is an anaesthetist at Massachusetts General Hospital in Boston. After beginning to administer blood-thinning medication during an urgent neurological procedure in 2005, Mr Goldman noticed that the EMR had recorded him checking the level of clotting 22 minutes earlier. As a result, four hospitals in the northeast had their medical devices checked, and found that on average they were off by 24 minutes. The easy solution that devices could have used since 1985? NTP."
Why can't the medical devices be hardcoded to use an NTP server on the hospital's LAN?
First, they will use Windows Active Directory for NTP because someone will say "it's authoritative for the whole network". And their clocks will be off.
Then they will run into config hell, and blaming that for clocks being off - they will load balance the domain controllers. Which is precisely what you're not supposed to do with NTP. And their clocks will be off.
Then, some small but relevant IT subgroup will secede, claiming that they need "real" NTP. "Network Security" folks are typical suspects here. So their clocks won't match the rest of the gear (which is still off, remember?)
If you have poor enough technology discipline that your clocks are 24 minutes off already, you're probably screwed.
I want to delete my account but Slashdot doesn't allow it.
NTP does not require access to public networks. Private time servers, usually GPS sourced via rooftop antennas, are very common.
Can You Say Linux? I Knew That You Could.
You don't need to connect to an outside server. You can easily run your own time source (GPS is really easy these days), or have the devices talk to a single internal server which then securely contacts outward. If they're off, at least they're all on the same time. It's really dangerous if everything is reporting different incorrect times.
The easy solution that devices could have used since 1985? NTP.
Holy hell, what about no? There's a huge reason why hospitals try to keep off networks, especially public ones. Do you really want to connect all the timing devices in a hospital to an outside public server? Because running it yourself does no good, it can just fuck up all the devices in the hospital.
Sometimes the ideas non-thinking geeks come up truly scare me.
No one said the NTP server had to be external. NTP is just a method to keep a bunch of clocks synchronized to an arbitrary master.
Sometimes the stupidity of non-technical and non-thoughtful people scare me.
NTP is just a protocol that you can implement. There are solutions that you can install internally that don't require internet access. Just stand up your own internal NTP server and have your own internal official time (possibly synced to something more authoritative). I agree with your sentiment about keeping sensitive medical equipment disconnected from the internet but with hospitals becoming more and more interconnected and not having their own physical infrastructure to do so, the internet looks like it's probably the best option. Yes, there are way to protect your traffic and all that but I must be pedantic and point out that NTP does not mean you must use the common servers available on the internet.
My work here is dung.
Oh noes!!! My android phone will make me 15 seconds early to any appointment!!!! I must therefore dump it and become and apple fanboy.
Supplies!
hospitals try to keep off networks, especially public ones
The sysadmins try, but fail. Most hospital networks and devices ARE connected to the internet in some way. Doctors want their access from the various desktop machines, of course, but many of the diagnostic machines offer things like "click this button to email the ultrasound pictures". So they do.
I was appalled to learn this a few years ago from a hospital sysadmin here on /. The thing he pointed out is that Doctors are Gods. If they say "This thing can email pictures? Well yeah, hook it up!" then the sysadmin has zero choice. Holes get punched in firewalls that should never have been punched, and the gear gets hooked up.
And because medical devices are certified only to work with a particular operating system at a particular patch level, they don't get upgraded unless the vendor comes out with a new certified patched OS. That means the ultrasound machine sitting on that cart might still be running Windows XP SP 1. It's crazy.
NTP would actually be the least of their worries. That's something they could more easily house internally.
John
Actually a linux server on it's own you can calibrate the system clock to be incredibly accurate. you can calculate the drift of the cmos clock and adjust to get pretty darn accurate.
http://tldp.org/HOWTO/Clock-2.html
Do not look at laser with remaining good eye.
NTP have the problem of discontinuing his UTC timestamp while a leap second occur and NTP do not broadcast the actual UTC-TAI offset (historically because he broadcast UTC directly but this is now more a problem that an advantage). GPS and PTP broadcast (something very closely related to) TAI and a UTC-TAI offset, witch is the right thing to provides the precise actual time without discontinuity.
But all of them, NTP, GPS and PTP, have the problem of not broadcasting the historical leap second table, making the client of those protocols alone unable to safely compute a precise date in the past. I hope next NTP protocol will broadcast TAI, and that NTP, GPS and PTP will be able one day to broadcast the leap second table. I am certain that there is still some reserved bits somewhere in those protocol to make that working.
Better yet: this is not an application that requires microsecond precision--within a minute or two is fine, particularly as long as all the clocks agree with each other. Completely private network with a master NTP server that is updated by hand every week or so should work fine.
This is often a case of poor administration, perhaps more frequently than poor design.
For example, I was recently tasked with reviewing the performance of several hospitals in the diagnosis and treatment of stroke. Under national guidelines (UK) a patient with suspected stroke must have had a CT scan within 30 minutes of arrival at hospital, with blood-thinning treatment administered within 60 minutes (if appropriate).
The problem was that the times on the CT scanners were discrepant by +/- 45 minutes from true time - so the images were tagged with the incorrect time. Further, the CT viewing workstations had times up to 2 hours discrepant. The CT scanners were Windows or Gentoo depending on the manufacturer's preference. Similarly, the CT workstations were windows, and were all bound to the hospital domain.
The time discrepancies made my assessment very difficult - and I had to correct for each individual scanner, and assume that the clocks hadn't drifted over the 6 month period of the audit.
I also found several safety issues because of this - e.g. if it was 1am, and a patient had a CT scan, some workstations would be 2 hours slow, so would read 11 pm on the previous day. These workstations would refuse to load the CT scan because the files were filtered by "WHERE [StudyTime] NOW".
I raised a support issue with the workstation vendor who simply said "These are windows workstations. You should ensure that they are appropriately bound to your domain, and configured to sync with your time server or domain controller". So I called IT to configure this, "No way. These are medical devices, we can't change the configuration - and anyway, what will happen if the clock is fast, and the sync pushes the clock back, so that there are 2 occurrences on the same time. That would cause chaos. Even if the manufacturer supports it, there's no way we'll set it up". Of course, their concern doesn't actually exist, because most time sync algorithms (even on Windows) are clever enough to avoid "double time".
There was similar obstruction with the CT scanners. The vendors simply said - we support and encourage synchronisation with a time server. IT or the radiology administrators simply stonewalled the ideas. They refused even to correct the clocks on teh scanners - so the clocks are still wrong to this day (even more so, due to accumulated drift).
Of course, even if the time can be set right - there is disagreement as to how daylight-saving is managed. Some equipment, esp. older embedded kit isn't daylight-saving aware. Do you set it to Summer time or winter time? In most hospitals I've been in, it's been an inconsistent mixture - often with lots of clock drift added, so you can't actually be sure.
our society is further behind then I've calculated.
Your calculations are right. The problem is your clock is way off.
Why does clock drift happen? It doesn't need to happen. It can totally be avoided. It only happens because the equipment manufacturers design inaccurate clocks to save money.
My quartz LCD watch from 1985 was accurate to within 1 second per year. That would WAY outlast the usefulness of the medical device. There should be no way in the world that device was off by 24 minutes.
Right now, I'm dealing with the same problem in my brand new car. It has a fancy on-board computer with a screen that tells me gas mileage, service info, mp3 and radio interface, etc.... The clock is ridiculously fast (gains 3 minutes a week). My new $20,000 car should have a clock in it at LEAST as accurate as the watch I can get from a happy meal.
There is nothing in the vans.
They just have a list of those who haven't paid the tax and go 'round harrassing people. It's easier and doesn't cost as much as a van full of equipment and high-paid engineers.
Emperor
Clothes
None.
--
BMO