Slashdot Mirror
SFLC Wants To Avoid Death by Code
foregather writes "The Software Freedom Law Center has released some independent research on the safety of software close to our hearts: that inside of implantable medical devices like pacemakers and insulin pumps. It turns out that nobody is minding the store at the regulatory level and patients and doctors are blocked from examining the source code keeping them alive. From the article: 'The Food and Drug Administration (FDA) is responsible for evaluating the risks of new devices and monitoring the safety and efficacy of those currently on market. However, the agency is unlikely to scrutinize the software operating on devices during any phase of the regulatory process unless a model that has already been surgically implanted repeatedly malfunctions or is recalled. ... Despite the crucial importance of these devices and the absence of comprehensive federal oversight, medical device software is considered the exclusive property of its manufacturers, meaning neither patients nor their doctors are permitted to access their IMD's source code or test its security.'"
the software running your pacemaker is probably patented too!
Does a government agency examine the source code which keeps airliners in the air, cars on the road, nuclear plants from blowing up etc etc? If the government is going to evaluate and approve every important piece of code line by line we will pretty soon run out of programmers. But then, chip designs will have to be evaluated too because they can fail as well. Next, mechanical designs, engines, turbines, reactors, better make sure that the government is stocked with experts in all those fields too.
After all, nothing can possibly be safe until it is certified as such by the government. Just ask hundreds of thousands of people who died while the drugs that could have saved them were waiting for the FDA approval. They are pretty safe now.
Negative moral value of force outweighs the positive value of good intentions.
The devices themselves are rigorously tested in clinical trials. If they pass those tests, what more do you want?
Seven puppies were harmed during the making of this post.
Too bad this story can't be combined with this story: http://www.nytimes.com/2010/07/20/health/20docs.html?_r=2
That would save us all a lot of trouble.
One of the July 2010 updates bluescreened my 81-year-old dad.
The hospital backed out the update but they had to reboot him in safe mode and go up the back door.
This seems similar to other highly proprietary hardware platforms that vendors keep locked down, either for market dominance, or for *security*. Breathalyzers, police radar guns, ATMs, hearing aids, etc, etc.
On the other side of things, imagine the scandal of somebody with a pacemaker installed for the purpose of athletic advantage, perhaps at the Olympics. Can you say heart hack? The winning line-up of the hacked-pacemaker 500, by embedded OS of choice:
1. DSL (Damn Small Linux), lightweight, fast, and simple
2. OSX, clean, stable, and reliable
3. Windows, DNF (H_RESULT 0x41414141 HEART_EXPLODED)
'We are trying to prove ourselves wrong as quickly as possible, because only in that way can we find progress.' RPF
I work for a company does full life-cycle development and verification of safety-critical software, the main areas we work in are aircraft instrumentation, smart munitions, and medical equipment (including pacemakers). The amount of testing and verification that goes into these software categories often exceed the development cost, and at every level it is documented and traced. What on earth do Doctors think they will see in the source code? We do verification, peer review, tracing, etc. what would an MD find that a room full of software, system, and QA engineers wouldn't? About the only thing that they would be able to look at and have a hope in understanding is criteria for taking action, and that is in the requirements and should be reviewed at that level, not at the code level.
Next thing they know Pilots will demand the ability to review the code for their cockpit management system and soldiers the ability to review the code for their Anti-Tank rockets!
DEMETRIUS: Villain, what hast thou done?
AARON: Villain, I have done thy mother.
Shakespeare invents 'your mom'
For safety-critical software, there indeed should be a required certification regime for reliability. In the security field there is, for example, the Common Criteria. Security is one aspect of reliability (not the other way around). For too long, we have lived without any way of knowing how much effort has been put into making a system reliable. For a phone app this might not matter, but for a pacemaker it does matter.
If they properly test the device, the everything should be covered.
I think the FDA does need to realize there is a software component. For no other reason then to require a full recertification of the devise every time the firmware changes. The risk I see is that an item gets certified and then bugs get introduced later if future firmware updates.
The FDA should also be notified of any bugs uncovered in existing firmware. Put the responsibility of deciding if an item needs recalled our of the hands of the company. I think there are other measures that can be put in place without requiring manufacturers to open source the code.
With that said, if the FDA did start looking at the source code, that would not be a bad thing.
Im a gamer, not a grammer major. This post is full of spelling and grammer mistakes.
I have no doubt that the same issues that affect critical medical devices also affect automobile "drive-by-wire" systems like the Toyota runaway accelerator problem. Those systems need to be subject to inspection and validation by independent experts in the relevant hardware/software technology. And if there are problems, the hardware and software need to be even more thoroughly inspected.
"patients and doctors are blocked from examining the source code"
huh? are either qualified to do so?
Does a government agency examine...
How about the other entities mentioned in the summary (let alone TFA) -- patients and, more importantly, *doctors*? If not them -- who should review them?
After all, nothing can possibly be safe until it is certified as such by the government. Just ask hundreds of thousands of people who died while the drugs that could have saved them were waiting for the FDA approval. They are pretty safe now.
FDA approval works roughly about as well as "self-regulation" works, since the FDA more or less reviews studies provided by the industry.
Though it's worth noting this is probably at the upper bound of effectiveness of self-regulation, since under the FDA they're actually required to submit something that can convincingly pass for a study in order to receive approval.
Tweet, tweet.
Just ask hundreds of thousands of people who died while the drugs that could have saved them were waiting for the FDA approval.
What is your source for these numbers?
I think you'll find that the experimental protocol at best simply extends the life of the terminally ill patient for some few weeks or months. It is not a miracle cure - it is an investment in the future.
39% of lung cancer cases are diagnosed after the cancer has already metastasized (distant stage). The corresponding 5-year relative lung cancer survival rate [is] 2.15% Lung Cancer Survival Rate Based on Stage
....with the line "She hacked into my heart and crashed me."
// max_int should be enough for anyone
for(i = 0; i < max_int;i++){
sleep(1);
beat_heart();
}
// printf("hi!!!!!\n")
If I have seen further it is by stealing the Intellectual Property of giants.
Sure, go ahead, implant one in your chest.
They'd be an awesome life. Knowing the device in your chest is buggy and will have 'updates' released every time the developer makes a commit to the revision control system. Knowing that your entire life depends on a guy who is doing it because he can shout 'OMG FOSS FOR LIFE FUCK THE MAN I'M SAVING THE WORLD'.
Knowing your life depends on developers who only care about the code they write and how it fits their needs.
You'll have 45 buttons on your pacemaker that let you control all the different ways you can stimulate and control your heart. Most of them will return 'not yet implemented', 3 of them will result in a core dump of pacemakerd, 10 of them a PANIC reboot, another 2 cause it to just go silent and halt, and the developer threw in an Easter egg that makes you piss your pants if you hear a penguin.
If you're lucky, you'll get a group of devs that doesn't have 2 or 3 in it that throw temper tantrums on semi-regular basis and threaten to fork it while not putting any effort into the project.
And to top it ALL off, If you complain to anyone about it, the response you'll get is:
You have the source, fix it yourself.
Let me tell you how quick I would be to jump on that train. To tie my life to someone who really doesn't get affected in anyway when his/her software kills me and has no real reason to put any effort into ensuring it doesn't.
The OSS world still doesn't get why companies avoid OSS software, what the fuck makes you think anyone with a 3rd of a brain wants their life to depend on OSS.
I use OSS constantly, there are some great accomplishments. Large portions of my life depend on OSS, but you will probably never find OSS in controlling any thing that my actual life depends on.
I prefer to live, not prove how awesome OSS isn't for every situation.
OPEN SOURCE IS NOT INHERENTLY BETTER, STOP PRETENDING IT IS. You guys REALLY need some perspective. Or just stop letting timothy have access to post to the front page.
Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
Perhaps someone should explain to them the benefits of open source surgery.
I'm not trolling or flaming at all here, I'm genuinely surprised.
By my quick-and-dirty calculations:
I tend to feel rough after four or five beers. How is it you're drinking five to ten times that *a night* and still around to talk about it lucidly? I'd expect some serious delerium tremens in short order on that track...
Curious,
"What in the name of Fats Waller is that?"
"A four-foot prune."
NEVADA GAMING COMMISSION has the code to slots games so why can't the FDA get the code to med systems?
"...neither patients nor their doctors are permitted to access their IMD's source code or test its security.'"
"Aww Thufir, don't feel badly...everyone gets a heart-plug here..."
Let's hope any vulnerabilities aren't wirelessly-exploitable!
Strat
Progressivism (aka US 'Liberalism'): Ideas so good they need a police/surveillance-state to enforce.
Medical device companies typically outsource hardware for a series of hardware tests. Similar arrangements can be made to test software similar to DO-178B test levels for avionics. This should be a documented process.
The clearance system sounds logical. It is not. It is completely arbitrary. -- John Bolton
My girlfriend has an insulin pump made by Medco. It has to do certain things like, if she has a certain high blood sugar level, give the right amount of insulin dose for the next hour to bring her into a normal range. If she eats, she estimates the amount of carbs she's eaten, enters in a certain dose level, and the pump calculates how much insulin she needs, based on the type of insulin she's using.
It uses a AAA battery. If the battery starts to run low, it beeps. If the battery is almost dead it beeps A LOT to let her know to change batteries.
Now... if she gets low on insulin, it will beep once or twice at approximately 10% left, and a few times at 5% left.
IF the pump runs out of insulin---THE PUMP SAYS NOTHING. No "OMG! NO INSULIN!" messages. NO WILD BEEPING! Nothing! If she somehow runs out of insulin in her sleep, she is well and truly fucked as the pump isn't going to wake her up! Nice of it to let her know when the battery is low (obviously if the battery is dead, it can't beep right?), but not so nice of it NOT to let her know when she's actually out of the life-giving fluid she needs to live.
Has it happened that she's run out of insulin and not noticed because the pump didn't say anything? Yes, it has. Luckily not when she's been asleep, but it could happen.
Good thing she doesn't have to milk a hairless cat to live, huh? Still, a little better design here might make a difference.
In Communist Russia, software opens YOU!
Seriously, get a grip. I don't see any hearts bleeding (pun unintended) about how manufacturers aren't required to freely distribute their manfacturing process so it can be checked (by anyone with a keyboard and an opinion). So the implication that these companies should be categorically required to give up what they may well consider a trade secret because it smells like something you're particularly religious about strikes me as downright un-American.
Becaue they made it a condition of the contract, and had the muscle to do so. Doctors simply need to so the same, if they really think this is a risk.
On one hand, the work I did on medical devices was held to a much higher standard than most other software I have written. Peer reviews with recorded signatures, all kinds of automated testing of the code, etc. On the other hand, I also know that no external authority inspected our code, and there were some firmware bugs with dangerous implications. I also witnessed anger from management when customers of a competitor learned of serious defect in the competitor's product. The company I worked for was seriously committed to keeping information secret, regardless of the impact to the patient. At some point most corporations become incapable of an ethical decision and turn in to a cancer on the host society.
Firstly let me state I don't work for a medical device implant company so don't be alarmed by this question ;)
What should I read if I was interested in testing / auditing software for an IMD, or other mission-critical software? Can someone point to some online resources?
Thanks in advance.
Bad Example.
A pacemaker / infusion pump powers just a handful of the almost
7 billion oxygen-consuming global-warming-contributing talking monkeys.
If one dies due to a software glitch - well, no big loss.
A error in slot machine software can directly impact corporate profits.
Big difference.
Duh, there's a lot of money at stake with a gambling machine, but just people's lives at stake with medical devices.
Sure, pacemakers and insulin pumps may run closed-source software. But there have got to be countless systems running critical aspects of infrastructure or even the military using closed-source software as well. Wasn't the Navy using Windows at some point in its ships?
Seems that the stakes are much higher in the latter although given the pace of medical technology/wetware innovation, having some sort of review or 3rd party testing worked into the approval process now rather than later would be prescient.
Serious, WTF! Why are we still having to dick around with these issues of closed systems that you are prevented from reviewing, especially since they affect people's health directly! This should not require any kind of debate and if these medical devices that are certified by a government entity such as the Food & Drug Administration (FDA) then the manufacturers must be required to publicly disclose the design and software source code to the FDA for their review and additionally for public review since the FDA works on behalf of the people. This is braindead simple but we still have to argue for every little bit of disclosure for government certified devices.
This is the same crap that happened with the speed radar guns and it took many law suits to finally force the manufacturers of these devices behind sold to the local police departments acting on behalf of the government and thus the people to finally release the source code for these devices. Some of the devices were found to have faulty programming in them and their results were proven to be uncertain within the degree that would affect their accuracy enough to be difficult to certify if a person was speeding or the radar gun was acting up.
There's that part in the US Constitution about the government being unable to copyright anything that is produced by it This should be extended to mean that anything that is used by the government or certified for usage by the government should at the very least be officially copyrighted by the authors and though this process should be made available for review by the public, similar to what the patent system does. You submit a patent, and you get government protection for your invention, at the cost of exposing your design to the public and then allowing the public to use and benefit from your invention after the protection period expires.
Same thing should apply here to the software of these medical devices. You submit it for certification you copyright your code officially and you get copyright protection for it so that another company can't just cut-and-paste it into their own system. This makes the code available for public review and people can independently verify that the code won't accidentally kill you if you happen to trigger an unusual set of events that is not going to be dealt with and avoid the problems with "edge cases".
The problem is, thoser "pothead" have a good points. Cyrose, Alcoholic coma, and various other illness (korsakof), not even counting those which at the end of the road try other type of alcohol (rubbing alcohol forn example). Alcohol is more adictive and more destructive for the body than marijuanna. So it stands to reason that you should not ban something which is less addictive and less dangerous than alcohol while selling legally alcohol. The problem in this case is *cultural* and not a health one. Those who smoke weed eevryday are about as rare as those which drink alcohol everyday, but they are certainly much., much more fucntional. You never hear of those which don't smoke everyday, because unless you make a blood sampling, you would not be able to distinguish them from the rest of the population, just like your moderate drinker. The bottom line is, there is no reason whatsoever to forbid marijuanna a quite a lot of good reason to ban alcohol (and I did not even mention road accident).
And before you start to call me a pot head, I despise UTTERLY all chemicals which robs me of my mental potential. That include Pot, alcohol BOTH. But I am not a fucktard wanting to ban either , as they are personal choice. The only things I would impose is that having a road accident with any alcohol in the blood, I don't care which amount, or any drug, should automatically lead to heavy prison, while possessing and consuming either in privacy should not be a problem. Driving is something which can kill uninvolved people therefore it should be heavily punished. The rest is voter rethoric which has no scientific basis.
C. Sagan : A demon haunted world:
http://www.amazon.com/gp/product/0345409469/
visit randi.org
Why do you think the FDA can't do that? They can basically do anything they want, followed by the threat to kick you (the manufacturer) out of the US market and/or shut down your factories if they're in the US.
Have a nice day.
Nobody is going to steal your code. It simply isn't worth it. The saving will be a few thousand dollars in programmer salary, and the risk will be a lawsuit worth millions, which is quite likely to come to light is the competition has to release their source.
And the software simply isn't that complex. You don't want it to be too complex. The users want it to do a simple job.
The issue is the entire device, not the standalone code - the code is not usable outside the device, and I'm pretty sure implantable devices ARE regulated by Gov't. This is a non-issue designed, I am certain, to raise the profile of the organization making the claim in the eyes of those outside the industry.
There are many parallels, but to me the easiest is the on-board computer in a car. Certifying the software in the computer is pointless, since it is so closely tied to the hardware it runs on - so the only useful tests are for the entire software, hardware, and vehicle to be tested as a whole.
Ken
Um... they can and they do.
When the FDA auditor is sitting in your office and asks, "May I see the source code?", it's time to paraphrase Winston Zeddemore: When someone asks you if they can see the source code, you say "YES"!
Lose = not win
Implantable pulse generator firmware isn't written for a standardized platform. These devices all contain highly customized hardware, very complex ASIC's with lots of hardware-assisted power savings functions, sleep timers, bidirectional control registers, etc. and the designs vary greatly from model to model, company to company. Without at least a working understanding of this hardware you will only have a cursory and likely somewhat inaccurate view of what's really going on inside an IPG just from looking at the source code. I'm quite familiar with this, I design automated test systems and test code to validate and perform quality test on IPG's!
The Therac-25 worked pretty well until operators began triggering huge overdoses due to a race condition.
Saying that auditing code is useless when you can just audit the device is rather myopic.
One of my co-workers knows a guy who works on pacemaker software. That transitive guy should not, in my co-worker's words, be trusted with a BASIC interpreter on a rusty TRS-80.
The thing between you and your next heartbeat may be a clever fellow who uses "? :" all the time because "it's faster than if .. else."
This stuff should be publically available.
If you're not familiar with it, Everclear seems like strange stuff. It's not even universally available in the U.S. as some jurisdictions effectively outlaw it.
However, almost nobody (I say "almost" because I'm sure there's one crazy idiot out there) drinks it straight. It's always mixed.
When I was in high school, the fun party drink involved cutting a hole in a watermelon, pouring in a bottle of Everclear, then refrigerating the whole thing for a few hours to let it soak. Then you bring it out at the party, poke in a few straws, and people take turns sucking down the intoxicating slush inside.
I know a man who downs a 12-pack of beer every night, minimum. Often, it's twice that. He consumes a minimum of 2 24-packs every weekend.
In between the beer, he consumes a minimum of 2 gallons of whiskey each week.
At the end of every night he's (in the words of one of his former girlfriends, my sister) "knee-walking drunk".
Yet, he can carry on a lucid conversation almost to the end. And every weekday, he gets up and goes to work where he does a fine job related to the construction industry. His work attendance record is nearly perfect and his job performance is excellent. The guy is entrusted with million-dollar decisions on an almost daily basis and he's never let down his employer.
Physically, he's lean and strong. Much of his work is in the field around major concrete placements and he runs rings around guys 20 years his junior.
I don't know how he does it, either, but this has been his pattern for the last 30 years or so.
On how charitable givers should insist on a post-scarcity copyright and patent policy for the results of anything they fund in whole or in part (from a document I wrote): ... This physical public works paradigm is unfortunately then applied to thinking about most digital public works, and there is a major flaw in the analogy. A bridge does not require much marketing. ... ..."
http://www.pdfernhout.net/on-funding-digital-public-works.html
"For example, where can one go to get a freely modifiable design including CAD files for even a simple health-related appliance like a wheelchair? Or worse, where is the community freely collaborating on improving wheel chair designs? Are a few dozen intentionally-vague patents on wheel chair design the best to be hoped for given the trillions of dollars of investments into public works, including vast amount of money spent on medical research?
Consider again the self-driving cars mentioned earlier which now cruise some streets in small numbers. The software "intelligence" doing the driving was primarily developed by public money given to universities, which generally own the copyrights and patents as the contractors. Obviously there are related scientific publications, but in practice these fail to do justice to the complexity of such systems. The truest physical representation of the knowledge learned by such work is the codebase plus email discussions of it (plus what developers carry in their heads).
We are about to see the emergence of companies licensing that publicly funded software and selling modified versions of such software as proprietary products. There will eventually be hundreds or thousands of paid automotive software engineers working on such software no matter how it is funded, because there will be great value in having such self-driving vehicles given the result of America's horrendous urban planning policies leaving the car as generally the most efficient means of transport in the suburb. The question is, will the results of the work be open for inspection and contribution by the public? Essentially, will those engineers and their employers be "owners" of the software, or will they instead be "stewards" of a larger free and open community development process?
Open source software is typically eventually of much higher quality
http://www.fsf.org/software/reliability.html
and reliability because more eyes look over the code for problems and more voices contribute to adding innovative solutions. About 35,000 Americans are killed every year in driving fatalities, and hundreds of thousands more are seriously injured. Should the software that keeps people safe on roads, and which has already been created primarily with public funds, not also be kept under continuous public scrutiny?
A shorter version of that:
http://www.pdfernhout.net/open-letter-to-grantmakers-and-donors-on-copyright-policy.html
"Foundations, other grantmaking agencies handling public tax-exempt dollars, and charitable donors need to consider the implications for their grantmaking or donation policies if they use a now obsolete charitable model of subsidizing proprietary publishing and proprietary research. In order to improve the effectiveness and collaborativeness of the non-profit sector overall, it is suggested these grantmaking organizations and donors move to requiring grantees to make any resulting copyrighted digital materials freely available on the internet, including free licenses granting the right for others to make and redistribute new derivative works without further permission. It is also suggested patents resulting from charitably subsidized research research also be made freely
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.
Don't forget that those inspectors carry badges and handcuffs. A lot of people don't realize that every time you sign a DHF (Design History File) or DMR (Design Master Record) file, it's like signing a federal affidavit. Sign something untrue, and you risk jail time.
The world is made by those who show up for the job.
I work for a medical device manufacturer. We don't make a life-essential device, but all the laws apply to us as well as the manufacturers that make critical devices. The FDA already has the power to examine a manufacturer's source code. When they come in to perform an inspection, the inspectors have the same powers as federal marshals. They can look at anything - just time and resources are the limiting factors. When a device is submitted for FDA clearance, there is a lot of software documentation that has to be included in the application. Our software section is one of the thicker sections in an application. Depending on the level of concern of the device, a manufacturer has to submit all test results, software detailed design, etc. The stuff we have to do during development here is incredible and we're a minor level of concern.
Regulation requires that all designs be periodically, formally reviewed. It requires that the review includes an independent reviewer and that reviewers are just as (if not more) technically competent than the designer. The FDA may not have the resources to review every line of code, but they do have the resources to look at the documentation from the reviews and to look at the documentation listing the qualifications of the reviewers.
Manufacturers are required to conduct risk assessments for their devices and identify any/all reasonably foreseeable hazards and to mitigate those hazards until they are as low as reasonably practicable or the clinical benefit to the patient outweighs the risk. The risk assessment must be conducted by clinical and technical experts. Each mitigation (or fix or change to a line of code) has to be re-evaluated for risk and possible repercussions to the rest of the device. Testing is also quite rigorous and safety and reliability are the top priorities. Our testing takes months. Changes that affect safety may have to be tested in expensive clinical trials on human subjects and the results resubmitted to the FDA for clearance.
Perhaps by having the public look at source code there will be some bugs found. But I'm sure that the bug has already been considered as part of the manufacturer's risk assessment, and any fixes for that bug will not be fast in coming considering the heavyweight nature of the development process.
--The Programming goddess from Gorflaz
Because in gaming there's something more important on the line than people's lives: Money.
Mods, wtf? stonewallred answered my question. How is that possibly worth a Troll mod? Or is it just that someone out there doesn't like him (her?) in general, and happened to have mod points last night?
Sheesh.
"What in the name of Fats Waller is that?"
"A four-foot prune."
One (creative) reading of this is that there were 3 jugs at the table, only one of which comes with beer in it... :D
NEVADA GAMING COMMISSION has the code to slots games so why can't the FDA get the code to med systems?
Yeah, I'm going to rig a pacemaker, then find a guy with a pacemaker throw a secret combination of quarters into his mouth, jiggle his arm and GET RICH!
The FDA DOES require software to be reviewed, along with full UML, state machine diagrams, etc of whatever piece of software that is likely to have an impact on the patient's life. Even for non life-essential devices like x-ray, machines, CT scanners and the like, where a bug could lead to a wrong diagnostic.
My other signature is a car
The more you weigh. jesus, how did I make such a dumn mistake? Is it Monday or something?
Free Martian Whores!