Slashdot Mirror


Ask Slashdot: How Do You Handle Hardware That Never Gets Software Updates? (hpe.com)

New submitter pgralla writes from a report via HPE: Many devices, designed for both long-term and short-term use, were shortsighted when it came to flexibility. How do you handle the hardware that never gets software updates, such as embedded systems and task-dedicated equipment? The article that pgralla shared provides the example of medical devices running Windows 7. "Many of the current generation, when they were first released, used Windows 7, and the devices still work well enough that they remain in service today," reports HPE. "But Microsoft ended mainstream support for Windows 7 back in January 2015, so the operating system gets updated only with an occasional security patch as part of Microsoft's extended support. In January 2020, that extended support will end as well." Many IoT devices are in a similar boat as they're powered by embedded Linux and are not designed to be updated after they enter service."

Of course, these outdated devices create all sorts of security concerns. "Hackers and their access to knowledge and computing power only go up as the years pass, which means that long-lived, fixed-firmware devices become ever more insecure over time," says Michael Barr, founder of the Barr Group, which provides engineering and consulting services for the embedded systems industry. The WannaCry ransomware hack in 2017 affected not just PCs but also medical devices, and ended up costing businesses $4 billion.

44 of 233 comments (clear)

  1. Easy.... by GerryGilmore · · Score: 5, Insightful

    ....don't buy it.

    I've seen SO many people whining about MS' forced reboots, etc. STOP!
    If there is not a sensible option available, demand that your vendor make a version that can be sensibly updated. Too many purchasing decisions just don't have any sensible criteria. ("Oh, it's built on Win XP and you aren't updating it? OK - scratch!")

    1. Re:Easy.... by Shikaku · · Score: 5, Insightful

      Linux is free. Updates only when told to. Doesn't have telemetrics by default. Never looked back except in VMs.

    2. Re: Easy.... by peragrin · · Score: 3, Interesting

      The issue isn't updates but people who don't apply updates at all.

      Linux and osx let you schrdule them but that says the user is smart enough to do so. 20 years of Windows updates have prove that to be false for 99% of users.

      The forced updates of iOS have proven to be !ore secure than the fragmented updates of Android.

      How often do you update your router? If your up time is over 60 days you are missing updates and are insecure.

      That is the issue. The other issue is designing software to use decraprated apis. Anyone building software using win32

      --
      i thought once I was found, but it was only a dream.
    3. Re: Easy.... by Shikaku · · Score: 2

      Actually my router is also Linux. So weekly, every Sunday night. Cronie, the cron job manager handles it for me, even the rebooting if necessary; with the LTS kernel for minimal changes except bug and security fixes.

    4. Re: Easy.... by fred6666 · · Score: 4, Insightful

      How often do you update your router? If your up time is over 60 days you are missing updates and are insecure.

      I don't know any home/small business router company (TP-Link, Linksys, Netgear, ...) updating routers every 60 days. More like 1-2 times per year, for 1-2 years. And then nothing.

    5. Re:Easy.... by ShanghaiBill · · Score: 5, Insightful

      ....don't buy it.

      Not an option with a patented medical device.

      demand that your vendor make a version that can be sensibly updated.

      Right. Sure. Because companies with millions of customers always do a complete system redesign to satisfy "demands" from one whiner.

    6. Re:Easy.... by Shikaku · · Score: 3, Informative

      https://upload.wikimedia.org/w...

      https://upload.wikimedia.org/w...

      The data says very much otherwise, and there's only legacy software forcing people into Windows nowadays. The only thing garbage here is your attempt.

    7. Re:Easy.... by Desler · · Score: 4, Insightful

      Not really. Many more people died without them and had less than half the life expectancy. I'm pretty sure a person who, for example, needs a patented medical device like a pacemaker just to stay alive won't be very impressed by your statement.

    8. Re:Easy.... by ShanghaiBill · · Score: 4, Insightful

      Society got along just fine for thousands of years prior to the invention of said patented medical device.

      1000 years ago people had half the life expectancy they do today, so I would not say everything was "just fine".

      Do you really think it is okay to let people die so your network can be marginally more secure? This is why people roll their eyes at pedantic nerds.

    9. Re:Easy.... by viperidaenz · · Score: 4, Informative

      A pacemaker corrects irregular heart rhythms, that if left uncorrected may result in a heart attack, resulting in death. Hence a pacemaker can keep someone alive.

      People who have pacemakers usually don't have them implanted for fun. They usually have them implanted as their other option is to die from heart failure.

    10. Re: Easy.... by YukariHirai · · Score: 4, Informative

      The issue isn't updates but people who don't apply updates at all.

      This is exactly the idea behind Microsoft's forced updates: most people are never applying updates, which causes problems, so if the updates get applied without user intervention, problem solved. I don't think they're entirely wrong, but they went about implementing mandatory updates in a kind of brain dead way.

      The forced updates of iOS have proven to be !ore secure than the fragmented updates of Android.

      iOS doesn't have forced updates; it is always up to the user to decide to install updates or not, though Apple do a bit to encourage it. The difference between iOS and Android in terms of updates is that Apple as a matter of course rolls out security updates to every device currently supported (and they are supported for quite some time, contrary to the largely inaccurate stereotype of Apple devices getting thrown out and replaced annually) and new versions of iOS to basically all devices capable of running the new version. With android, it's left up to each hardware manufacturer to provide security updates and new versions for their devices. Many don't bother at all, many others do a couple of security updates and maybe a new version while the device in question is "current" before basically abandoning it. Even if a device is technically capable of running a new version, it's not usually an option to "go over the manufacturer's head" for updates; a build has to be tailored to the model in question, and while the wider open source community does offer some for some devices, it's very much a mixed bag of what's supported, how up-to-date it is, and even how trustworthy the third party is.

    11. Re:Easy.... by tsa · · Score: 2

      That is utter bullshit. 99% of those 'sheep' as you call them have better things to do than scrutinizing firmware. They need a device that does what they need it to do so they take what is available.

      --

      -- Cheers!

    12. Re:Easy.... by AmiMoJo · · Score: 2

      I think we might be on a tipping point where Linux can really replace Windows, even for legacy stuff. WINE has got so good now that there really isn't much you can run on it.

      Level 1 Techs on YouTube are running a series of videos about gaming on Linux right now. The focus is on getting Steam for Windows and associated games working with WINE or with a VM that has a pass-through to the GPU to give near native performance.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    13. Re: Easy.... by AmiMoJo · · Score: 2

      With android, it's left up to each hardware manufacturer to provide security updates and new versions for their devices.

      This is a very persistent myth.

      Since V4 back in 2013 they have been patching security issues via Google Play Services, which is mandatory for Android devices. The current version (Oreo, released last year) includes Project Treble, which allows phone manufacturers to ship updates much more quickly by separating out the hardware layer, which is what was causing most of the delays.

      This is why you don't see vast Android botnets rampaging all over the internet. The OS itself is very secure already, being heavily sandboxed and compartmentalized, and with Google pushing out security fixes and having their own malware scanner running constantly as part of Google Play Services it's proven impossible to mass exploit devices in that way.

      The issues we do see are malware authors using increasingly sophisticated methods to sneak malware into the Play Store (just like they sneak it into the Apple App Store), and trying to profit before Google shuts them down, and apps that are simply deceptive and user-hostile. Part of the trade off for having more freedom on Android is that sort of risk, which is easier to mitigate if you live in the iOS walled garden.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    14. Re:Easy.... by Bert64 · · Score: 2

      Very few people intentionally buy windows either, they receive it when they buy the hardware - same as android.

      --
      http://spamdecoy.net - free throwaway anonymous email - avoid spam!
    15. Re:Easy.... by thegarbz · · Score: 2

      ....don't buy it.

      Hahahahaha

      demand that your vendor make a version that can be sensibly updated

      Aaaahahahahahahaha

      +5 Funny. Now to move on to some insightful discussion that actually makes any kind of sense at all than your idealistic ideas that you or your decision matters. Actually something does matter, your indicision matters and is just likely to get you fired.

    16. Re: Easy.... by guruevi · · Score: 2

      Hence why we have DD- and OpenWRT.

      --
      Custom electronics and digital signage for your business: www.evcircuits.com
    17. Re:Easy.... by Xord · · Score: 3, Informative

      I work in the medical industry and I have never yet seen Linux as the OS used with any major medical equipment, such as CT scanners, X-Ray scanners, MRI, Ultrasound, etc. Linux is not always the answer in the real world unfortunately.

    18. Re:Easy.... by Xord · · Score: 2

      I should probably add that our way of dealing with these horrendously outdated operating systems required for the equipment is to vlan them off from the main network and don't allow internet access.

  2. Don't connect it to the internet by MpVpRb · · Score: 4, Insightful

    Many old tools are computer based

    Some old CNC machines run on MS-DOS and a 286 processor

    As long as the hardware stays alive, they continue to do the job

    If they must be networked, restrict their access to the local net

    1. Re:Don't connect it to the internet by kwalker · · Score: 5, Insightful

      Not just the local net. Restrict their access to only trusted control devices on the local net. It may require putting insecure devices on a network segment that has strict access controls, but when the only other alternative is to discontinue a working device (In situations where that's possible), making a sandbox network isn't all THAT much work.

      --
      Improvise, adapt, and overcome.
    2. Re:Don't connect it to the internet by MightyMartian · · Score: 4, Insightful

      This... so much this. Segregate these devices, limit access via VLANs and firewalls. Yes, it may mean only a handful of other devices and workstations can touch these older devices, but you need to reduce the attack surface as much as possible.

      --
      The world's burning. Moped Jesus spotted on I50. Details at 11.
  3. The manufacturer wants you to buy a new one by Bruce+Perens · · Score: 4, Informative

    I have a number of Rohde and Schwarz FSEB and FSEA spectrum analyzers. These cost at least $80,000 new (I bought them used for a few thousand at most). They come with an old version of windows. I similarly have other electronic test equipment with old Windows or even old Linux which the manufacturer doesn't update any longer. For the Linux-based ones I could hack in a new Linux and make it use the old ABI, forget about Windows.

    But what really clued me in was that the Rohde and Schwarz equipment had a battery soldered on the CPU board, and it was an hour-and-a-half service to get to it. A lot of stuff had to be removed.

    Similarly, my Tektronix 500-series oscilloscopes had two 40-pin DIP Dallas Semiconductor battery-backed memory and clock chips. The batteries in these die and they aren't socketed. When the batteries die, the 'scopes lose their calibration. The company won't give you the program to recalibrate them.

    The manufacturers just want you to buy new ones.

    So, obviously I back SDR-based test equipment that's Open Source. Who needs a company that wants to screw you?

    1. Re:The manufacturer wants you to buy a new one by 50000BTU_barbecue · · Score: 2

      "Tektronix 500-series oscilloscopes had two 40-pin DIP Dallas Semiconductor battery-backed memory and clock chips."

      Um, no they didn't. At best, they had socketed transistors.

      http://w140.com/tekwiki/wiki/5...

      You are perhaps referring to the TM500 series, but even those are long in the tooth.

      http://w140.com/tekwiki/wiki/T...

      --
      Mostly random stuff.
    2. Re:The manufacturer wants you to buy a new one by gordguide · · Score: 5, Interesting

      I'd never buy test equipment that requires a computer connected to be usable. Never, ever.
      That's as bad as my flex radio that I never use for the same reason, garbage. Every time I sit down, I just turn on my old kenwood ts-430 instead.
      If it's a self contained device that requires no network connection, maybe. If there are software updates, they need to be installable offline. Mostly analog is ideal though.

      Sometimes "never" is not an option. One electronic test equipment that revolutionized the industry is the Audio Precision line of Distortion Analyzers. Virtually everyone involved in electronic design, testing or repair owns one, and they are almost hobbyist-priced (a new basic unit can be had for less than $US 10,000). The revolutionary part of AP analyzers is they connect to a PC to do the math.

      Now, somewhat on topic, AP is very good at updating their SW interfaces and older machines can use modern versions of the WinOS. They also are not themselves normally required to be connected to outside networks, provided you use a dedicated PC on the bench and not one used for general computing. So much of the problems are solved using good management practices.

      If you want to be anywhere near current, you need an AP. I don't own one; I send my stuff to another engineer who does to test, but he charges $200/Hr. He has the most advanced unit, somewhere near or north of $US 20K. Plus a Windows PC and a printer if you want output charts, of course. My Distortion Analyzer is adequate (Keithley, a unit of Tektronix, $US 6,000) but only measures to the fifth harmonic.

      It is a standalone device, but unless you want to dig around for an old 70's~80's era machine from HP, Tek, Boonton, a Sound Technology 1700B, etc that pre-date the inexpensive computing power era, the norm these days is software / PC / Appropriate Sound Card for low cost measurement. So now you need, again, a dedicated PC and most hobbyists use the same machine for general computing. But the cost is *way* lower than a standalone machine or an AP.

      If you fudge the numbers, it comes down to a classic standalone machine (they still sell for almost four figures and sometimes a couple of thousand) or software like ARTA and a good sound card, maybe $400 worth of stuff total in addition to a basic working PC of some kind. You can fight with your wallet or just give up and go PC-enabled.

    3. Re:The manufacturer wants you to buy a new one by justthinkit · · Score: 2

      Actually, it is dead easy.

      If we go this way -- better design -- the customer wins and we make less money.

      If we go that way -- planned obsolescence -- the customer loses and we make more money.

      If you don't things are this bad, explain why Apple solders pretty much everything on a $1,000 iphone to the motherboard these days. The answer is...so it can be priced at $1,000.

      --
      I come here for the love
  4. Easy, just leave them be. by CptLoRes · · Score: 4, Informative

    Most dedicated systems like this does not belong on the internet, period. So unless there is some flaw or feature need, don't update and it will still work exactly as it did yesterday. And the day before, and the day before that.

  5. Seriously? Treat it as safety-critical by davecb · · Score: 2

    Mechanical systems that keep, for example, trains from running into one another by tripping their brakes into full on, are well-understood. I took a course on doing the same thing in mixed hardware-software systems, so it's eminently possible.

    The gotcha is you have to keep it really simple and run a validator like spin on it's protocol.

    Most developers can do the spin part, but KISS? Distinctly less likely (;-))

    --
    davecb@spamcop.net
  6. Minimalist firewall by Pinky's+Brain · · Score: 3, Interesting

    Implement a firewall with a small microcontroller with a relatively secure TCP/IP stack (ejip if you don't want to spend money, HCC embedded if you do) and do protocol level sanity checking and filtering of all network inputs.

  7. Obligatory response. (Sorry, everyone.) by Hallux-F-Sinister · · Score: 2, Funny

    How Do You Handle Hardware That Never Gets Software Updates?

    Very carefully. (Buh-DUM-Tshhhh)

    Borrowed from “How do porcupines make love?”

    With apologies.

    --
    Our reign has gone on long enough. Indeed. Summon the meteors.
  8. Exactly. Least privilege. Wireshark if needed by raymorris · · Score: 4, Informative

    A basic principle of security is least privilege. If a piece of outdated equipment needs to send udp packets on port 411 to a monitoring station, you set the firewall to allow it to send udp on port 411 to that particular station, and nothing else. If it doesn't need to take to web servers, you don't let it talk to web servers. You allow it to do only exactly what it needs to do.

    Not sure what your equipment needs to do? You could check the manual, and otherwise open up Wireshark and set the filter to the IP of the equipment. Have a look at what it is sending and receiving. Then set the firewall to allow only exactly what is needed.

    This is also an area where vlans come in very handy. Vlans act like completely separate networks, but they are configured within your switch, so a single 48-port switch can handle a dozen different, totally separate vlans.

    Perhaps different parts of your network should be mostly separate, but you need to allow a little bit of specific communication between two vlans. That's when you plug a router or firewall into both vlans and set it to route only specifically allowed traffic between them. This doesn't even require two network ports - the same port can be in multiple vlans and the router can control traffic between vlans issuing a single cat6 cable. This is called "router on a stick".

    If some of this went over your head, here's the simple version'
    Call someone who has a CCNA Security certification or better (CCNP Security or CCIE Security). Tell them you're thinking about segregating different vlans and using an internal firewall to strictly control internal traffic. They'll get you set up.

  9. Re:Open source by 110010001000 · · Score: 2

    Open source has taken off like wildfire. You are using it right now. You are just too dumb to realize it.

  10. I love the FOSS community's cluelessness. by Anonymous Coward · · Score: 5, Informative

    I use Slackware, along BSD, financially support projects that I use, and have followed the Linux community since Linus was still in college. It always amazes me how clueless the FOSS community is regarding issues such as this.

    Just use Linux...
    That's your fault for using M$..
    etc.

    For regulated systems, especially in pharma manufacturing, you are told what to use, how to use it, when to upgrade it, how to upgrade it, etc. Basically, once the system is certified by the FDA - you don't touch it - PERIOD. You purchase enough compute/control systems when you install it to last you through your production, which could be - 10, 15, 20+ years.

    There is no, well, just upgrade to x - it's not allowed.

    Before some equally clueless libertarian pinhead starts spouting off about 'over regulation' - stop and think for just one second what this system does. It controls the valves, temperatures, mixing, fermenting, refining, etc. of a chemical that people are to ingest. Where the difference between good and bad is measured in ppm, ppb, or even ppt depending on what's being made. Some endocrine chemicals are measured in 1/10ths or 1/100th of a ug!

    Do you really want to apply patches to a system such as this? Doesn't matter that they are 'network', or 'mouse driver', or 'display' - the risk is WAY TOO GREAT to jack around with them.

    Keep in mind that 'upgrades' require a new certification of that system, or depending on what it does, the entire production chain - which could run you a couple 10's of millions dollars.

    So, before starting the typical FOSS rant, please have a clue of what you are talking about, first.

    1. Re:I love the FOSS community's cluelessness. by Gravis+Zero · · Score: 3, Interesting

      Before some equally clueless libertarian pinhead starts spouting off about 'over regulation' - stop and think for just one second what this system does. It controls the valves, temperatures, mixing, fermenting, refining, etc. of a chemical that people are to ingest. Where the difference between good and bad is measured in ppm, ppb, or even ppt depending on what's being made. Some endocrine chemicals are measured in 1/10ths or 1/100th of a ug!

      Sounds like a great argument for mandatory system isolation. Instead of networking directly to the system, the systems should be isolated and only provide a standard interface which a simple computer terminal could interface with. Something like TCP over serial using a variant of X11. When you minimize the attack surface to basic keyboard and mouse input validation then it becomes much easier to build a defensible system.

      --
      Anons need not reply. Questions end with a question mark.
  11. Re:Are we back in high school again? by Immerman · · Score: 2

    >ANY device can be infected with a new exploit whether it's up to date or not. New fully updated equipment is no less of a risk than old out of date equipment.

    Those are two very different statements. Yes, any device can be compromised by a new exploit - that's kind of the point of developing NEW exploits. But an outdated device can be compromised by a massively long list of well-known exploits - making it far more vulnerable. New exploits are generally financially valuable assets horded by those who know of them, and they will usually be rendered useless shortly after they become public knowledge. Fewer people attacking, means lower risks that you'll be attacked.

    I don't know if it's still the case, perhaps the target is no longer as appealing, but I recall that back towards the end of Windows XP's product life, even before 7 came out, the rule of thumb was that a freshly installed copy of (non-updated) XP would be compromised within 20 minutes of being connected to the internet - considerably faster than most people could download the updates necessary to secure it. Not that it was ever 100% secure, but there's a huge difference between going into battle in imperfect armor, and going in wearing nothing but a giant bullseye painted on your chest.

    --
    --- Most topics have many sides worth arguing, allow me to take one opposite you.
  12. Re:Open source by tepples · · Score: 2

    That doesn't help when a particular device from a particular manufacturer contains non-free software, as do the substitute devices from all competing manufacturers.

  13. Re:XP? by tepples · · Score: 2

    Devices running Windows XP are already unsupported. Devices run Windows 7 will be in the same boat as devices running Windows XP come January 2020.

  14. Re:Enforce the law by Immerman · · Score: 2

    >There's no reason to update devices that were never designed to change

    Unless part of their functionality is to withstand attack from attackers whose knowledge is constantly growing. And pickable locks are the only thing on your list that qualifies. And as far as that goes...

    We have pickable locks because an unpickable lock is apparently impossible, at least while being remotely easy to use. And locks evolved a LOT before they reached their current state - which are secure enough to deter crimes of opportunity (i.e. they keep an honest man honest). It takes hours of practice, or moderately expensive purpose-built tools, to get good enough to pick an average modern lock - too much effort for pretty much anyone without premeditated criminal intent.

    Invincibility is too expensive, even where it's possible. Security is all about lowering your risk by increasing the cost and risk to the attacker. And when any idiot who picks up the electronic version of a free set of decade old automated lockpicks from the corner website can walk into your house without even trying, take what they want, and trash the place, secure in the knowledge that they'll almost certainly never be caught - then you don't have any security worth speaking of.

    --
    --- Most topics have many sides worth arguing, allow me to take one opposite you.
  15. Asus updates by Anonymous Coward · · Score: 3, Informative

    How often do you update your router? If your up time is over 60 days you are missing updates and are insecure.

    I don't know any home/small business router company (TP-Link, Linksys, Netgear, ...) updating routers every 60 days. More like 1-2 times per year, for 1-2 years. And then nothing.

    Perhaps you should look into Asus, which often updates at least quarterly, and often monthly:

    * https://www.asus.com/Networking/RTAC68U/HelpDesk_BIOS/
    * https://www.asus.com/microsite/2014/networks/routerfirmware_update/

    And has been doing it for 4+ year-old products. Plus there is third-party code that leverages the GPL stuff that Asus releases:

    * https://asuswrt.lostrealm.ca
    * https://github.com/RMerl/asuswrt-merlin.ng

    1. Re:Asus updates by AmiMoJo · · Score: 2

      My friend's Netgear router is about 6 years old and got an update a few months back for some vulnerability.

      Netgear's stuff is low end crap but at least they do seem to support it for the long term, which actually really surprised me.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
  16. They buy it because it's by raymorris · · Score: 4, Insightful

    They buy it because it's better. It's better than Windows Phone (the first, second, theirs, and fourth attempts), it's better than Symbian, it's better than everything else people have tried. Why is it better? Linux is or reason it's better. Even Microsoft is using more and more Linux now. Is that because Microsoft has a religious zealotry for Linux? No, it's because Linux is better. Better than eating their own dog food.

    >> Legacy software forcing people into Windows nowadays.
    > Yeah, more than a billion people.

    Yeah, legacy software has a LOT of people (companies, really) still stuck on Windows. Your point is?

    1. Re:They buy it because it's by iampiti · · Score: 2

      I disagree that Linux is what makes/might make Android special. For most apps the fact that the device runs Linux is completely irrelevant. Most only use the Android level APIs so they don't care what's underneath.
      In fact Google are developing a new kernel (Magenta) and will likely replace Linux in future Google OS (Chrome OS and Android).
      Of course using Linux was a pretty sensible option when Android was first developed but it might be replaced in the future with few consequences

    2. Re:They buy it because it's by raymorris · · Score: 2

      Do you think there might be any reason Android, a very small company at the time, was able to quickly build better APIs and architecture than Microsoft, who while MUCH larger, had to work around the underlying Windows OS?

      Android Inc spent a few million dollars on development, while Microsoft spent a few billion - roughly a thousand times as much. Android got much better results. You don't think the OS they chose might have had something to do with that?

  17. I was personally very upset when... by tlambert · · Score: 2

    I was personally very upset when Motorola refused to provide me a software update for a device, designed for both long-term and short-term use!

    It was an SN74LS139N Motorola Dual Decoder 2-4 Line Plastic TTL chip.

    How dare they deny me software updates for this chip containing two inverters and four AND gates!

    I don't give a damn that they designed it for embedded use, I should be able to update the software running on it!

    Right?