Slashdot Mirror


World's Largest Chip Maker Will Lose $250M For Not Patching Windows 7 Computers (networkworld.com)

A major virus infection forced the closure of Taiwan Semiconductor Manufacturing Company (TSMC) factories last weekend..." writes Slashdot reader Mark Wilson, noting that it's the largest semiconductor manufacturer in the world, selling chips to Apple, Nvidia, AMD, Qualcomm, and Broadcom, and "responsible for producing iPhone processors."

Now Network World reports: The infection struck on Friday, August 3, and affected a number of unpatched Windows 7 computer systems and fab tools over two days. TSMC said it was all back to normal by Monday, August 6. TSMC did not say it was WannaCry, aka WannaCrypt, in its updates, but reportedly blamed WannaCry in follow-up conference calls with the press.... The company said this incident would cause shipment delays and additional costs estimated at 3 percent of third quarter revenue. The company had previously forecast revenues of $8.45 billion to $8.55 billion for its September quarter. A 3 percent loss would mean $250 million, though actual losses may come out lower than that. Still, that's a painful hit. TSMC also said no customer data was compromised....

TSMC isn't directly to blame here; someone [an infected production tool provided by an unidentified vendor] brought WannaCry into their offices and behind their firewall, but TSMC is still culpable because it left systems unpatched more than a year after WannaCry hit.

108 comments

  1. Yep, that's what you get by Anonymous Coward · · Score: 3, Insightful

    for not patching your systems.

    1. Re:Yep, that's what you get by RandomFactor · · Score: 2

      The problem here us unlikely to be that IT was too lazy to upgrade or unwilling to patch. Quite the opposite is generally the case.

      Vendors that supply process control systems will certify exactly what can and cannot be loaded on these systems including patches. It can take years to get a new patch certified from the vendor. And if you load anything uncertified you are taking on that entire liability hit and lose support and such. That's a career limiting move.

      Oh and Windows 7? Not too bad, There are Windows for Workgroups based systems still running machines out there (probably older to be honest.)

      I know of a situation where a system was infected and left that way but just firewalled off for years because they couldn't even load an AV on it. It seems asinine, and it is, but it is also how things have to be done sometimes.

      Often there are quite limited options (if any) available for what you might select to control a particular industrial machine, so just shopping for a different vendor isn't really an option in this space. Not setting up manufacturing systems isn't an option, those are needed to make widgets and without widgets you have a plant or company and all that depend on it on the street. Remember these machines can and do kill or injure people if things go wrong, if you ignore the vendor, just like with any other negligence that harms employees, the liability to the company is rightly phenomenal.

      Now you can make a case that loading the company patching software and AV on these system is prudent, as are all the other things /.ers do to maintain our own systems, and I'll grant that 99% of the time or more you would be right. But explaining that in front of a jury is not something you want to do on that 100th machine -especially- if you do it without the vendor's and your company's approval. Even just the not particularly uncommon case of a patch breaking some obsolete protocol, or the AV making the system stutter during operation can be terribly costly.

      --
      --- Mercutio was right.
    2. Re:Yep, that's what you get by Anonymous Coward · · Score: 0

      You can picture the scenario. They have some super-niche software running on there, cost of porting it to Win10 (which wants to dial home all the time) or Linux is high, very high. Clever coder likely retired or left 10 years ago. Machine has a big "do not change anything" sign on it and they air-gap it just in case. And this sets them up for a failure like this one.
      Now expand this to other companies that may be holding on to "legacy" stuff for a variety of reasons. Medical device makers, healthcare providers,... bonus: a police dept using Lotus Notes in 2018. Change costs and often people with the right IT skills are not in place as the bean counters "don't see the benefit" of paying them. Until the sh!t hits the fan.
      Microsoft could get off the Win10 disaster horse and allow people to patch Win7 - with good patches, not spyware ones - until the hardware reaches EOL.

    3. Re:Yep, that's what you get by nospam007 · · Score: 1

      "for not patching your systems."

      Perhaps their machines didn't have the chips to upgrade to the latest, greatest Windows version.

      You know, the cobbler's kids are barefoot.

    4. Re:Yep, that's what you get by Anonymous Coward · · Score: 0

      It's also what you get for considering the "inside" of your corporate firewalls to be safe. That whole "safe inside the moat" mindset from the 1990s has been over for a long time. The other thing to realize - wannacry spreads over the network. But it can only do so if you have changed your Windows host firewall defaults. They actually had to actively reduce the security on their systems - likely because some admins said, "wah, I need to be able to do remote administration, wah, wah". That's actually the reason that only some companies got hit by wannacry - it was only the ones were administrators overrode the default firewall settings because they felt "safe" behind their moat. Hint: you don't enable the firewall rules that turn on file and print sharing on Windows systems if you want to remain safe.

    5. Re:Yep, that's what you get by iMadeGhostzilla · · Score: 1

      So what's to do? Would it be possible to have each legacy system run inside a sandbox, VM or VM-lite kind of thing, maybe like Sandboxie for Windows but industrial strength, and you make a copy of the sandboxed image every day. If a virus infects the guest OS, you simply go back a few snapshots. If the virus hasn't wiped or encrypted the application-generated data files, you can restore those from the latest sandbox or snapshot.

      Is there anything obviously missing in this scheme?

    6. Re:Yep, that's what you get by Anonymous Coward · · Score: 0

      Perhaps their machines didn't have the chips to upgrade to the latest, greatest Windows version.

      For God's sake, all that was required was to have Windows 7 updated with a patch that existed months before WannaCry/WannaCrypt actually hit. The patches are free, there's no need to update to another version of Windows to be protected from it.

      This site really is filled with irrational hate and a lack of logic. I have no idea why I bother with it anymore.

    7. Re:Yep, that's what you get by RandomFactor · · Score: 1

      The most likely result of that line of inquiry is going to be "Must be run on vendor supplied hardware" and "Vendor does not certify to run in a VM."

      Also just to make it more fun, taking it offline to do a backup shuts down a production line and must be scheduled once a quarter or once a year.

      Hmmmmm, "what's to do"

      Probably nothing until manufacturing via 3d printing and general purpose robotics becomes competitive with classical manufacturing. Not because they are better or worse, but more because once you are buying something more generic you introduce competition and the vendors can then be differentiated on the basis of things like "IT says these guys provide patches and updates to protect the systems better"

      --
      --- Mercutio was right.
    8. Re:Yep, that's what you get by iMadeGhostzilla · · Score: 1

      I imagine with $250M to lose over two days TSMC could easily say Hey Vendor certify your stuff to run in a VM pronto. Vendor would do it, unlike porting their app to Linux. Would they not, realistically?

      As for VM, can you make a correct VM image backup while the VM is running? Seems to me that could be done in the background without affecting production.

    9. Re:Yep, that's what you get by HiThere · · Score: 1

      The question here is "Who is 'they'?". It's quite likely the configuration is specified by the vendor of the $$EXPENSIVE$$ niche machinery. And they aren't going to change their specs, because, since that machinery is expensive, they don't have any old models to test on. And possibly not anyone currently expert in that particular model. (They're concentrating on the next generation model. "Want to order one? You have have it for beta testing on your production line in only a couple of months.")

      --

      I think we've pushed this "anyone can grow up to be president" thing too far.
    10. Re:Yep, that's what you get by drinkypoo · · Score: 1

      I thought chip fabs worked hand in hand with the producers of the equipment to produce a solution that would work for them. At some point during specification, quoting and the like, the question of OS should have come up, and they should have specified something better than Windows.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    11. Re: Yep, that's what you get by Anonymous Coward · · Score: 0

      Clearly in this case they could patch...

    12. Re:Yep, that's what you get by sjames · · Score: 1

      If it's process control, a VM probably won't do. The software will likely be talking directly to some bit of hardware and any stuttering on the part of a VM passing things through to real hardware would be a problem.

      Best you can do is keep the prosess control machines on an air-gapped LAN and hope it doesn't get cooties if you have to temporarily connect to the outside or connect a laptop fpr updates.

    13. Re:Yep, that's what you get by sjames · · Score: 1

      Given the cost to change vendors, probably not.

    14. Re:Yep, that's what you get by Anonymous Coward · · Score: 0

      Well we keep irrationally being told to use windows and it makes us hateful and angry.

      I got an idea, why don't you and your shop switch to linux? Your whole IT team is probably screaming your ear off to do it. You obviously get a sense of it from being here based on your comments

    15. Re: Yep, that's what you get by Anonymous Coward · · Score: 0

      Using Windows is often an irrational decision made by a businessperson whoâ(TM)s been dined and courted by MS sales staff, or fooled by some nice new shiny. Seen it happen many times.

    16. Re:Yep, that's what you get by Billly+Gates · · Score: 1

      Oh come on. This is slashdot where I see IT professionals proudly say they don't patch with a smile.

      I want to say told ya so.

    17. Re:Yep, that's what you get by cheesybagel · · Score: 1

      Well, that's why I've heard Intel still used VAX/VMS to run their factories until at least recently.

    18. Re:Yep, that's what you get by Hadlock · · Score: 1

      They probably do work hand in hand with them, and then support it for the life of the equipment. The problem is when the lead engineer who wrote 90% of the software, in a poorly documented and even more poorly maintainable design, left the company, or got hit by a bus or reason X. So sure, do you want to change feature X like increase the max RPM of the motor by 15RPM to get rid of some harmonic vibration? Yeah we can do that. The problem comes when he leaves and you can't do something important without his blessing because once you upgrade the machine, you can't roll it back if the software update isn't compatible with your update, because you lost the original drive image for the machine, or you don't have a way to reinstall the software from scratch because the original software is lost at this point.
       
      Yeah all that sounds crazy but I worked at a finance company where we were running an unapproved version of windows and all the servers were clones of clones of clones because nobody had any idea how to install all 114 packages in the right order (I'm not even exaggerating), and nobody had ever documented what was customized in the registry to get it to be cross-compatible with some other software they had integrated with it. We had a team of 125 testing our custom integrations against the software that was delivered to us, but if we ever lost all the backup copies of the clone images, we would be truly fucked as we had tried to build a new server from scratch several times over the course of a month and were not able to do it.
       
      I'm not sure how complex wafer fab machinery is, but if intel hasn't gone from 14 to 10nm in five years, I am guessing it is at least as complex as what we're doing, and once it's installed, you don't freakin' touch it and pray it never breaks.

      --
      moox. for a new generation.
    19. Re:Yep, that's what you get by rtb61 · · Score: 1

      The problem in reality was not that they did not patch their airgapped system, is they breached airgap by allowing hardware in with software installed, bad mistake. You airgap a system, than thieving is airgapped, including new hardware and they way new hardware is airgapped, is it is supplied free of software. The software comes in separately and is scanned and checked and then installed on the new hardware inside of the airgap, common fucking sense, or at least it should have been.

      Airgap requires that new hardware and new software are delivered separately, both checked and then the software is installed on the hardware inside of the airgap by your computer security team. You should need to security patch anything inside of the airgap, in fact each and every security patch also represents a security risk, just the way it is.

      --
      Chaos - everything, everywhere, everywhen
    20. Re:Yep, that's what you get by Anonymous Coward · · Score: 0

      I blame Microsoft for this. If they hadn't been scummy assholes and made Windows 10 an uncontrollable, spyware, marketing platform then more people would be willing to accept it. As it is now, only tech noobs and idiots use Windows 10.

    21. Re: Yep, that's what you get by Anonymous Coward · · Score: 0

      Far more frequently the reasons are ignorance and fear of the unknown. You'd be amazed how little people running the show in many places actually knows about computers. They were brought up on Windows, they were trained on Windows and they absolutely fear anything but Windows.

      When you're in that position, you don't really have to wine and dine anyone.

    22. Re:Yep, that's what you get by Highdude702 · · Score: 1

      VFIO

    23. Re:Yep, that's what you get by sjames · · Score: 1

      VFIO is helpful for performance nd security, but still might cause problems if there are hard deadlines.

    24. Re:Yep, that's what you get by Anonymous Coward · · Score: 0

      Having worked as an equipment engineer in the semiconductor industry for years I'm not surprised this finally happened. It would be so funny if it was an AMAT tool :) Most tool vendors use a vanilla Windows image for all their control computers and add specific software for the appropriate tool. No AV software; I have never seen vendors do anything to control network access. Once these images are made they do not upgrade them. You are not permitted to apply regular patches if you want to maintain warranty or under a service contract. If they ever do patch it would be with a new OS image which is very rare. I recently was refurbishing an older etcher with a WinXP SP2 OS and asked them about updating it to the last service pack and updates. Not supported. Their "solution" was to buy a new $25k computer with Win7. I patched it anyway and it worked fine but it wasn't a production tool and long out of support.

      But these computers are not intended to ever be connected to the internet or usually even an intranet. Ethernet is usually only used to communicate to a PLC. Most of these systems use SECS/GEM for inter-tool communication.

      My guess is someone infected it either by connecting to the internet to allow a service engineer to remotely run the tool (commonly done for service) or from a flash drive. Most equipment engineers are using old laptops that get no IT support from their employers because they are always traveling usually throughout Asia, connecting to questionable hotel Wi-Fi networks. Would be easy to infect something with a flash drive when you need to move logs or config files to/from a tool.

      Honestly I don't know why tool vendors don't just lock down network access completely on their equipment. The PLC communication is all it needs on TCP/IP and that's directly hard wired. If a customer changes the access then any problems are strictly on them.

  2. Wrong headline by Anonymous Coward · · Score: 1

    ``World's Largest Chip Maker Will Lose $250M For Using Known-Vulnerable Operating Software''

    The correct conclusion is that windows just isn't suitable to run multi-billion operations with. As long as you ignore that reality, you leave the door open to other parties to take advantage of that.

    1. Re:Wrong headline by jabberw0k · · Score: 1

      Like in War Games: The only way to Win with Windows, is not to mess with Microsoft.

    2. Re:Wrong headline by snapsnap · · Score: 2

      Depends on the price to switch to a system that isn't so insecure.

      Where I work, we tried switching about fifty servers to Linux, but it failed due to the fact we couldn't find people that knew what they were doing for minimum wage. The two high school drop-outs that work for minimum wage do an OK job with keeping those Windows servers running. Windows is acceptable since our customer SLA is 95% so I think we can have almost five hours of downtime a week. Of course we often exceed that amount of downtime because of Microsoft-created problems, but the lost customers cost less than a Linux expert would cost.

    3. Re:Wrong headline by Anonymous Coward · · Score: 1

      Expert for what? For the most part a linux guy doesn't have to do nearly as much as a windows guy. Windows fellas need to run around like they just crapped themselves 24/7 to keep that big jenga tower of interdependant hack code which is ms windows together.

      Ah you guys know your stuff though. It will all be okay, its not like these systems run operations involving salaries and materials that run into the millions of dollars of cost, nooooooooo, they are just toys that the folks with glasses use, we'll give them minimum wage and let them play around pressing blinking buttons and having fun. Little scamps.

      I don't blame you personally, but what you just described, a windows shop run by underpaid people trying their best, that is more or less the baseline of what I have experienced IT to be. What is sad is that if management et al would just listen to their IT experts instead of dismissing them as kids whose toys cost them too much money they might be able to get operations running properly.

    4. Re: Wrong headline by Anonymous Coward · · Score: 0

      Actually retard they ARE doing it, and have done it for years. All systems have failures, and it sounds like this is an Intel failure and user failure. Meanwhile you jump on the opportunity to blame Microsoft. There's a patch for it and they didn't apply it. Are you saying that Linux just auto patches itself? How would a different OS solve anything? Try thinking through your idioticy.

    5. Re: Wrong headline by reanjr · · Score: 1

      Why didn't you assign the project to one of the high school students? If that's their level of competence, you'd be better off with the sane secure defaults on a Linux. It's a learning project for them, and dirt cheap R&D for you.

    6. Re:Wrong headline by kzwork · · Score: 1

      Depends on the price to switch to a system that isn't so insecure.

      Where I work, we tried switching about fifty servers to Linux, but it failed due to the fact we couldn't find people that knew what they were doing for minimum wage. The two high school drop-outs that work for minimum wage do an OK job with keeping those Windows servers running. Windows is acceptable since our customer SLA is 95% so I think we can have almost five hours of downtime a week. Of course we often exceed that amount of downtime because of Microsoft-created problems, but the lost customers cost less than a Linux expert would cost.

      These guys apparently found people that knew what they were doing for minimum wage and the result is...

    7. Re: Wrong headline by Anonymous Coward · · Score: 0

      Well, for starters, linux patchs are not known to brick machines. Windows updates are notorious for this.

      Linux patchs do not take the host system down and force a reboot disrupting activities. Windows will force reboot and not take the fact that activity may be occuring on the system into account.

      Linux patchs take on the order of a few seconds to minutes as compared to windows patchs which take hours and lock the system during those hours.

      Linux requests permission to apply a patch windows will simply force update the machine regardless of settings going so far as to ignore or reset settings.

      Linux also is not know for failures at nearly the same rate as windows, the quality and stability of code between the two is not comparable as linux wins any stability tests hands down. Stability is the core of keeping your organization running and not having to run around with your pants on fire trying to deal with emergencies. Stability reduces costs, but can also mean the difference between an organization surviving or failing.

      Linux also is not nearly as prone to infection given its legacy network architecture where as windows was developed as an inherently insecure system for the home user.

      So, I've thought through the idiocy of the poster, compared it to your idiocy and found yours is the greater as all of these points I have brought up have been around for several years and are well known where as your point is trying to put blame on the users for not patching and completely ignoring that the institution behind all of this has severe issues which are not being addressed because of how costly and difficult it is.

    8. Re:Wrong headline by kzwork · · Score: 2

      Talking about Windows TCO.

    9. Re:Wrong headline by Anonymous Coward · · Score: 0

      People generally stumble upon Linux during college, unless they have been raised in the right environment. You should try hiring college dropouts as well. In any case, transition to another OS environment is not a trivial job as it involves fulfilling the customer requirements with a new software stack with its own quirks, benefits and issues. That may require lots of searching, testing, scripting, getting familiar with a myriad of configuration formats so that trivial errors can be avoided, and so on.

    10. Re:Wrong headline by drinkypoo · · Score: 1

      Where I work, we tried switching about fifty servers to Linux, but it failed due to the fact we couldn't find people that knew what they were doing for minimum wage.

      You should be paying more than the minimum wage, and if you don't, you deserve what you get for your money. Which is Windows. You should go out of business and let someone competent take your place.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    11. Re:Wrong headline by Anonymous Coward · · Score: 0

      Depends on the price to switch to a system that isn't so insecure.

      If you have to switch, you have already lost. It's been known for years that this software was insecurable, unstable, and generally unsuitable for Real Work. They should have never built on that crap in the first place.

      Anyway, now we know the price of making the wrong choice: 250 million euros.

      Where I work, we tried switching about fifty servers to Linux, but it failed due to the fact we couldn't find people that knew what they were doing for minimum wage.

      You can hire an awful lot of minimum wage people for 250 million dollars.

      but the lost customers cost less than a Linux expert would cost.

      This doesn't sound like a lot of value adding that your company is doing. Which would be reason to never begin being a customer of yours.

    12. Re:Wrong headline by Anonymous Coward · · Score: 0

      Anybody that has ever read the EULA knows that, the really short version:
      We (MS (tm)) guarantees nothing regarding this software, if it works you are lucky. Be sure to not use it for anything critical.

    13. Re:Wrong headline by Anonymous Coward · · Score: 0

      ...two high school drop-outs that work for minimum wage...

      If you pay peanuts, you get monkeys. So your bean counter is happy - until one day when the wheels really come off, revealing the true cost of "doing IT on a string budget." Sounds like a crap place to work, with low ethics.

    14. Re: Wrong headline by Anonymous Coward · · Score: 0

      "linux patchs are not known to brick machines"

      Tell that to the people who got their bios messed up by installing a certain new ubuntu release.
      also, messing up some smbios eeprom on your motherboard is so much easier in linux since you get all the tools to shoot yourself in the foot. but that's not the problem, linux shows strang behaviour on some hardware that never gets fixed... people just have to live with it... oh well it happens on windows as well.
      Thing is, on linux it does tend to mess up your hardware to the point of bricking it, whilst on windows, well, from personal experience, it didn't happen that much.

      try it lmsensors on linux
      or on windows, some program to read out temp and set fan values.

      Joy !!!

    15. Re: Wrong headline by HiThere · · Score: 1

      This is a custom machine configuration. There are lots of custom configured Linux machines that can't be updated. Your desktop is not a valid comparison.

      OTOH, if they can't do something like run it in a VM, then the problem isn't the OS, it's the licensing agreement. Or possibly the design. That said, time sensitive things often can't be run well under virtualization. And are often sensitive to even minor system upgrades. So it could well be a combination of time sensitivity and a CYA licensing agreement.

      --

      I think we've pushed this "anyone can grow up to be president" thing too far.
    16. Re: Wrong headline by Anonymous Coward · · Score: 0

      I cannot speak to that issue as I did not experience it, I did however read about the incident online. I also have never had a windows machine brick on me, I have only read about incidents online.

      In comparison though I have heard exactly 1 story of any issue with a linux update, just 1, and from what I recall it did not actually brick the machine but just mangled the existing installation. Windows has bricked or destroyed the OS on multiple occasions to the point where we barely even register it anymore, the volume of stories about manged or bricked machines via windows update has surged ever since win7 when forced updates became a thing. Re-installing windows is just a regular tuesday because of how unstable it is. Re-installing linux is a rare unicorn of a thing you rarely need to do unless you really hosed it up installing/uninstalling stuff in weird ways (been there, done that).

      Even installation is a very different thing, with windows you almost HAVE to call into a help center and deal with everything where as with linux it just goes through seamlessly and works. I think this is because MS wants pre-loaded windows but never wants to replace a disc or have a user install the operating system so they purposely make it an unpleasant and difficult experience. Ironically windows and linux have swapped sides on this as linux used to have a difficult installation but has now become a roughly 5-10 minute breeze, windows used to have a nice easy install but now it stops, takes forever (1hr+), asks a million questions, advertises to you, reboots repeatedly, gives you DRM bullshit, etc.

      I had fun with lmsensors and psensor on ubuntu when my GPU fans got gummed up from me smoking around the laptop. I ended up having to take the keyboard out and use a qtip to dig out the gunk then compressed air shots to dislodge the dust bunnies. The sensors module worked fine for me though on a high end qosmio laptop. I noticed for those sensors that windows doesn't seem to have a built in package for that and neither does linux, both of them leave it to 3rd party programs to fill in the gap and read/report the temperatures. That doesn't really impress me from either of them as certain statistical data about the machine should be baked into the OS (how are the dics performing, what is the temperature, any failed sectors of ram or disc, etc). Actually I would love it if statistical data was not just part of the operating system, but also a very verbose part of it (exactly how fast is the disc spinning, how much voltage is the GPU drawing, how much voltage is the CPU drawing, how many operations per second is it performing, what is the history of these statistics across time, etc).

    17. Re: Wrong headline by Zontar+The+Mindless · · Score: 1

      Thing is, on linux it does tend to mess up your hardware to the point of bricking it, whilst on windows, well, from personal experience, it didn't happen that much.

      try it lmsensors on linux
      or on windows, some program to read out temp and set fan values.

      I've been running gkrellm on all manner of different hardware for a dozen years, and I've never had it—or any other Linux software—brick a machine. I interpret your assertions to mean that you know even less about what you're doing than I do, and I am by no means what I'd consider an expert.

      --
      Il n'y a pas de Planet B.
    18. Re:Wrong headline by Anonymous Coward · · Score: 0

      Like in War Games: The only way to Win with Windows, is not to mess with Microsoft.

      By "mess with" you mean "use anything that came from," right?

    19. Re:Wrong headline by Billly+Gates · · Score: 1

      Last I checked Linux has vulnerabilities too that any competent administrator would patch. FYI I have seen SuSE services use for hosting phishing sites with the customer not having any idea due to a rootkit.

      Rootkits were invented on Unix. Where do you think the term ROOT came from?

    20. Re:Wrong headline by Anonymous Coward · · Score: 0

      They were running the equivalent of an unpatched Debian 5.x system.

      Doesn't matter what you're running-- if you're not keeping up with security patches, you're screwed.

      Windows (since late XP SP2), properly maintained and patched, is a reasonably secure operating system and can withstand the worst the internet can throw at it.

    21. Re: Wrong headline by Highdude702 · · Score: 1

      Whats wrong with lmsensors?

      it8686-isa-0a40
      Adapter: ISA adapter
      CPU Vcore: +1.31 V (min = +0.00 V, max = +3.06 V)
      +3.3V: +3.33 V (min = +0.00 V, max = +5.05 V)
      +12V: +12.17 V (min = +0.00 V, max = +18.36 V)
      +5V: +5.01 V (min = +0.00 V, max = +7.65 V)
      Vcore SOC: +1.00 V (min = +0.00 V, max = +3.06 V)
      CPU Vddp: +0.92 V (min = +0.00 V, max = +3.06 V)
      DRAM A/B: +1.38 V (min = +0.00 V, max = +3.06 V)
      CPU fan: 5625 RPM (min = 0 RPM)
      SYS1 fan: 0 RPM (min = 0 RPM)
      SYS2 fan: 0 RPM (min = 0 RPM)
      SYS3 fan: 0 RPM (min = 0 RPM)
      CPUOPT fan: 940 RPM (min = 0 RPM)
      System 1: +34.0C (low = +127.0C, high = +127.0C) sensor = thermistor
      Chipset: +44.0C (low = +127.0C, high = +127.0C) sensor = thermistor
      CPU: +35.0C (low = +127.0C, high = +127.0C) sensor = AMD AMDSI
      PCIe X16: +36.0C (low = +127.0C, high = +127.0C) sensor = thermistor
      VRM MOS: +45.0C (low = +0.0C, high = -117.0C) sensor = thermistor
      EC_temp 1: -55.0C (low = +127.0C, high = +127.0C) sensor = thermistor

      it8792-isa-0a60
      Adapter: ISA adapter
      DDR Vtt A/B: +0.67 V (min = +0.00 V, max = +2.78 V)
      Chipset Core: +1.05 V (min = +0.00 V, max = +2.78 V)
      CPU Vdd18: +1.79 V (min = +0.00 V, max = +2.78 V)
      DDR Vpp A/B: +2.52 V (min = +0.00 V, max = +4.63 V)
      3VSB: +3.33 V (min = +0.00 V, max = +5.56 V)
      Vbat: +3.29 V
      SYS5 fan/pump: 0 RPM (min = 0 RPM)
      SYS6 fan/pump: 0 RPM (min = 0 RPM)
      SYS4 fan: 0 RPM (min = 0 RPM)
      PCIe X8: +37.0C (low = +127.0C, high = +127.0C) sensor = thermistor
      EC_temp 2: -55.0C (low = +127.0C, high = +127.0C) sensor = thermistor
      System 2: +38.0C (low = +127.0C, high = +127.0C) sensor = thermistor

    22. Re:Wrong headline by Anonymous Coward · · Score: 0

      Not sure if it still does but the EULA of windows server did contain the line "Do not use this software for mission critical systems".

  3. Catchy headline by Anonymous Coward · · Score: 0

    Sounds much better than: World's Largest Chip Maker Will Lose $250M For using Windows Computers

  4. Re:Happy Friday From The Golden Girls! by Anonymous Coward · · Score: 0

    dude, it's Saturday

  5. We stopped patching Win7 when MSFT changed EULA by Anonymous Coward · · Score: 0

    We stopped patching Win7 when MSFT changed EULA to allow their spying and forced patching. When they stopped providing data about individual patches, we began switching to a different OS.
    We are small and the OS didn't matter thanks to our brilliant CIO who mandated webapps for everything using standard protocols since the company was founded.

    We did have 2 people quit when MS-Outlook wasn't available. They were marketing/sales people with lots of paid outlook addons for CRM. They didn't like the F/LOSS CRM we provided which integrated with our F/LOSS communications server and F/LOSS document management server solutions.

    Now we aren't a Windows shop, patching non-Windows systems is fairly easy, as is remote management. The 2 remaining Windows machines (used in accounting) which might be impacted aren't actually used on the internet and don't have access to other storage. Backups are "pulled", not "pushed".

    When guests visit our company, they are put onto wifi outside our desktop LAN. That wifi is effectively like being on the internet. We don't allow direct wifi access to our internal networks without using a VPN + 2FA.

    I feel bad for larger companies that don't have much choice other than to let vendors onto their Windows networks. Companies that do are playing Russian Roulette. It isn't if, it is when they get a major attack.

    1. Re:We stopped patching Win7 when MSFT changed EULA by gweihir · · Score: 2

      I know of a Fortune 500 company that will move to web-terminals after Win7, exactly because of all these problems. They found that qualifying Win7 and dealing with problems from all the updated and lack of security was more expensive than just making all their stuff (mostly custom applications) web-only in their intranet. There will not be any Win10 except by special permission.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    2. Re:We stopped patching Win7 when MSFT changed EULA by UnknownSoldier · · Score: 1

      What is the size (employee count) of the company by chance?

  6. A word about these computers... by GerryGilmore · · Score: 4, Informative

    It appears that the affected machines were those running process control systems. Because of their VERY finicky nature (and usually being designed to be used on a closed intranet), they almost NEVER apply post-production patches.

    I once worked on a medical device where each and very build installed MUST be a bit-perfect replication of the original. Any new release went through horrific levels of qualification and then IT had to be bit-perfect until the next release.

    The typical "patch Tuesday" crap just cannot work in these environments.

    1. Re:A word about these computers... by gweihir · · Score: 1

      Or in other words, MS Windows is just about the worst OS choice possible for such applications.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    2. Re:A word about these computers... by Anonymous Coward · · Score: 0

      I don't understand why the underlying operating system would effect the software running on it. If the software is correctly programmed then the output should be the same regardless of the patch status of the operating system.

      Can you give an example of how changes to the OS could impact the functionality of the software that's running the medical device?

    3. Re:A word about these computers... by Anonymous Coward · · Score: 0

      Any OS that is patched more often than every few years.

    4. Re:A word about these computers... by GerryGilmore · · Score: 1

      I agree. However, again having worked in the industry, I can tell you that - especially until the last 5-7 years - the overwhelming pressure: from developers who started in DOS and just fell into the Windows world by default, especially during the silly-ass "UNIX wars"; marketers who thought that Windows would dominate the world and - why not?; MS themselves who - to their credit - created a pretty amazing set of developers tools *AND* a single, unified target market.

      When I was working at a SCO UNIX shop, I started our transition to Linux. At least once a day, one of the sales dudes would drop by my office to tell me how much easier their lives would be if I just "flipped a switch" and put us on Windows. The first several times, I patiently explained how our entire infrastructure of development, testing and support was much more than just "flipping a switch". Finally, I just wrote a long email to the entire sales force and management, laid things out and told them that I would not accept any meeting to even discuss it.

    5. Re:A word about these computers... by GerryGilmore · · Score: 2

      I normally don't respond to ACs, but you ask a pretty good question.

      Basically, you don't know, and that's the rub. Let's take as an example the latest set Spectre/Meltdown patches. These are known to affect I/O performance (heavily-syscall-dependent) to a degree anywhere from 5-30%. Given that this is ONE patch, the same basic rules apply in, essentially, what are semi-real-time systems. That is, for each and every patch, you must apply the entire set of QA tests, which takes a lot of time and money. Performing this level of testing for patches that arise sometimes more-than-weekly is a non-starter. Just throwing a patch out there and waiting for customer support calls is NOT an option.

      Again, remember that these systems are designed to be used in a closed, controlled environment. In this case, lax procedures allowed a virus inside and....wellll.

    6. Re:A word about these computers... by thegarbz · · Score: 1

      Because of their VERY finicky nature (and usually being designed to be used on a closed intranet), they almost NEVER apply post-production patches.

      Medical device and process control are two very different systems. Process control systems most definitely do get patched. Not instantly, they go through vendor approval first, but they most definitely do get patched.

    7. Re:A word about these computers... by thegarbz · · Score: 1

      If an attack is targeted the choice of OS is quite irrelevant. This attack however didn't look targeted, but then also ... wannacry. I would wager that the evening janitor they entrusted to set this up in his spare time would have done an even poorer job with a more esoteric OS.

    8. Re:A word about these computers... by HiThere · · Score: 1

      Yes, but...
      The questions are "How many of the model were sold?" and "How long since it's been under active development?" and "What's involved (cost) in keeping an idle system around?" and "How many experts in this particular model does the manufacturer currently employ?".

      I suspect that combining the answers to those questions would yield "The manufacturer will not support ANY changes in the supplied configuration.".

      --

      I think we've pushed this "anyone can grow up to be president" thing too far.
    9. Re:A word about these computers... by Anonymous Coward · · Score: 0

      I agree. However, again having worked in the industry, I can tell you that - especially until the last 5-7 years - the overwhelming pressure: from developers who started in DOS and just fell into the Windows world by default

      We are not talking about hundred dollar DIY PCs here

      I for one, BLAME THE EQUIPMENT MAKERS for being worse than dumbfucks in using Windoze on equipments which cost at least HUNDREDS OF THOUSANDS OF DOLLARS, EACH !!

      A friend of mine was fired from his job (of a firm doing genetic sequencing) for updating Win 7 on one machine --- a simple update screwed up an entire chain of event that they had to throw out the result of a project that had already been running for 6 months, due to the update had totally screwed up the calibration of the system. and the reported loss, I heard, was in the tens of millions

    10. Re: A word about these computers... by Anonymous Coward · · Score: 1

      A classic example, that actually has hit many companies, is DCOM: A Microsoft technology that has something to do with setting up RPC connections (usually from programs written in VB or dot net). It is provided by the OS itself, and uses the OS security setup for authentication.. There have been several patches to fix security holes in it, which at the same time caused client apps to start working differently or not at all. Most shops ended up abandoning the technology and switching to various XML based remoting middlewares instead due to fragile nature of the deployed systems..

    11. Re:A word about these computers... by AmiMoJo · · Score: 1

      About 20 years you could buy these little PCI cards that had some kind of BIOS ROM that prevented permanent changes being made to the hard drive. Writes were redirected to free space, and when the machine rebooted they were discarded.

      They were popular with internet cafes. Hit the reset button and the machine went back to the default state, no matter how many viruses the previous user managed to get infected with.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    12. Re:A word about these computers... by gweihir · · Score: 1

      Ah, yes. I have run into that stupidity as well. Many people just do not understand that maintenance is the majority of the cost in OS usage. Fortunately, our customers are usually migration from some commercial UNIX to Linux, and that is pretty painless. Also RHEL is maintaining old software with security and crash fixes forever, so updates are low-risk.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    13. Re:A word about these computers... by gweihir · · Score: 1

      Sure. Or rather, as long as the attacker has the skills, it is. But would anybody in their right mind do a targeted attack against a company, that could put a $10M price (or higher) on their head without any problems? It is good criminal practice to stay an annoyance and to not become a real threat. Competent criminals understand that.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    14. Re:A word about these computers... by adhdengineer · · Score: 1

      years (2005 ish) ago i worked on a system using Windows XP Embedded and you could set up the system to do this. writes would go to RAM and be discarded on reboot.

    15. Re:A word about these computers... by thegarbz · · Score: 1

      But would anybody in their right mind do a targeted attack against a company, that could put a $10M price (or higher) on their head without any problems?

      Yes, because this is the real world and not some funny action movie staring Steven Segal.

      Corporate espionage and corporate sabotage are a very real thing that happens constantly and sometimes is even state sponsored.

    16. Re:A word about these computers... by gweihir · · Score: 1

      You seem to be the one in the movie...

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    17. Re:A word about these computers... by thegarbz · · Score: 1

      You could be right. After all someone is telling me that something that happens constantly doesn't actually happen. Either I'm in a really poorly written movie, or you're gunning for a republican presidential nomination.

      I declare all of history fake news from this point on wards.

    18. Re:A word about these computers... by gweihir · · Score: 1

      I would tell you your data is flawed, but you are thoroughly caught in your filter-bubble, so that is just a waste of time. You are _incapable_ of seeing what is.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    19. Re:A word about these computers... by thegarbz · · Score: 1

      I would tell you your data is flawed, but you are thoroughly caught in your filter-bubble, so that is just a waste of time. You are _incapable_ of seeing what is.

      Yep like I said, all of history if fake news to you nutters.

  7. Patching would have caused more downtime & los by Anonymous Coward · · Score: 0

    $250M is nothing if this sort of thing happens once or twice a decade considering Microsoft releases regressive and debilitating patches once or twice a year.

  8. Save a penny, lose a million by gweihir · · Score: 1

    The classical effect of mindless bean-counters that do not understand risk-management at all. Pathetic. And, since further up you usually find the same bean-counters, those that messed up massively here will likely not even be fired.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    1. Re:Save a penny, lose a million by Anonymous Coward · · Score: 0

      If you save a couple hundred million pennies for every million you lose, you're still coming out ahead. Usually the problem is that nobody notices the saved pennies, while the time you lose a million makes national (or in this case global) news. So the bias is for bean-counters to err too much on the side of safety, not the other way as you're implying.

    2. Re:Save a penny, lose a million by MikeMo · · Score: 1

      You actually have no clue as to why these systems weren't upgraded. You just assume it was the bean counters.

    3. Re:Save a penny, lose a million by gweihir · · Score: 1

      You assume I criticize them not patching. You are wrong.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  9. Re:Patching would have caused more downtime & by gweihir · · Score: 1

    The screw-up here is using an OS that cannot be professionally operated...

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  10. Not lost by Kohath · · Score: 3, Informative

    Just delayed until the next quarter.
    Also, lower revenues are not money "lost".
    Also, a newer story says it's $170 M, (2% of revenue), not $250M: https://digitimes.com/news/a20...

    But it wouldn't be a modern news story without a bunch of exaggeration and misunderstood info, would it? The important thing isn't the correct facts, the important thing is to point and laugh at someone's misfortune. Because news...

  11. Microsoft will use this by xack · · Score: 1

    As anti Windows 7 propaganda. All the while Windows 10 is getting worse. I did a clean install of 1803 in a VM today and it came with a dozen pay to win games pre-installed on the start menu and $kype. This was on the pro version as well. The security risks of using Windows 7 outweigh the time wasted de-bloating Windows 10. Intel is even making new motherboards to support Windows 7.

    1. Re:Microsoft will use this by Tough+Love · · Score: 1

      The security risks of using Windows 7 outweigh the time wasted de-bloating Windows 10.

      What about the privacy risk of Windows 10, and the fact that it is still riddled with vulnerabilities? Just stop abusing yourself and install Linux. If you absolutely must run Windows then run it under KVM. I hear tell that Windows on KVM is actually more efficient than Windows running on the metal, perhaps because of more efficient file system and block device handling.

      --
      When all you have is a hammer, every problem starts to look like a thumb.
    2. Re:Microsoft will use this by Anonymous Coward · · Score: 0

      I haven't updated my Windows 7 box since they started trying to shove Win10 onto it. Windows Update is now a malware vector. Once trust is gone, it's *gone*.

      I only play games on it once in a while, though, so if it ever does get infected, there's nothing to steal and nothing much to be lost on it. I went over to Mac. I don't like Apple, and every time Apple makes a boneheaded decision that makes me think of moving back to Windows, Microsoft makes yet another boneheaded decision that keeps me on Macs. (if the software I need ever gets ported to Linux, I'll try it, until then I'm just a ball in a game of Pong)

  12. Oh man, poor TSMC! by DontBeAMoran · · Score: 1

    If I had lost $250 million, I would WannaCry too!

    --
    #DeleteFacebook
  13. Re:Patching would have caused more downtime & by DontBeAMoran · · Score: 1

    ... like MS-DOS.

    --
    #DeleteFacebook
  14. Not to blame? Wrong! by Anonymous Coward · · Score: 0

    From the article: "TSMC isn't directly to blame here". Quite the opposite, they are. Either they should have patched (and they did not), or if they intentionally decided not to patch (which is understandable for some process control systems) they failed to implement protocols for bringing other systems into the environment. Heads you lose, tails you lose. I would hope the C-level execs that approved this entire process get their heads handed to them, but they will probably fire some low level techie who had no real control over anything.

  15. The problem is not banning Windows by Tough+Love · · Score: 1

    Google learned this lesson and banned Windows from inside their network, a Windows machine can now be connected to the network only with VP approval. Other organizations are perhaps more stupid.

    Windows is also banned from the world's financial systems after the LSE fiasco. But US Navy is too stupid to ban Windows even after towing that missile cruiser back to port. It should be illegal to use Windows in medical devices, until it does become illegal it should should be a lucrative income source for ambulance chasers.

    --
    When all you have is a hammer, every problem starts to look like a thumb.
    1. Re:The problem is not banning Windows by that+this+is+not+und · · Score: 1

      even after towing that missile cruiser [wikipedia.org] back to port.

      That case is ancient. It's Windows 4.0 old. It's Rex Ballard advocacy old. It's tired and anybody with a clue remembers people citing it ten years ago when it was already extremely outdated and old.

    2. Re:The problem is not banning Windows by Tough+Love · · Score: 1

      even after towing that missile cruiser [wikipedia.org] back to port.

      That case is ancient.

      Of course it is, but nothing changed after that, that has to tell you something.

      The LSE fiasco is not ancient, Windows is still banned from the world financial system. Not to mention the top 500 list. Islands of sanity. We need more.

      --
      When all you have is a hammer, every problem starts to look like a thumb.
    3. Re:The problem is not banning Windows by that+this+is+not+und · · Score: 1

      Nothing has changed since Windows NT 4.0?

      Maybe in your world.

    4. Re:The problem is not banning Windows by Tough+Love · · Score: 1

      Nothing of substance has changed in the Microsoft world. Especially, attitude has not changed, you are living proof of it. And for your information, not a lot has changed in the Windows kernel since Windows NT either, but I would not expect a random Microsoft troll such as yourself to know a whole lot about that. Linux on the other hand changed radically (while preserving external interface stability) in that same period.

      One thing in particular has not changed about Microsoft and its products: they remain a clear and present danger to corporate security, and even to democracy.

      --
      When all you have is a hammer, every problem starts to look like a thumb.
    5. Re:The problem is not banning Windows by Anonymous Coward · · Score: 0

      Wow, that last sentence. You are certifiably insane. Please, seek help.

    6. Re:The problem is not banning Windows by Tough+Love · · Score: 0

      See, Microsoft has not changed a bit and Microsofties are still the same, except maybe more bitter now. Thanks for the demonstration.

      --
      When all you have is a hammer, every problem starts to look like a thumb.
  16. Microsoft causes Chip Maker to lose $250M by najajomo · · Score: 0

    Headline corrected for accuracy :]

  17. Was $250 million more than the cost of updating? by sl149q · · Score: 1

    Given the size and numbers, is $250 million more or less than the cost of keeping their infrastructure up to date?

    And even after this costly mistake by a vendor, just keeping their systems tightly locked down and having much better controls over who or what gets plugged into their network may be far cheaper than updating everything.

    Given that they were back up and running quickly, it does appear that they have everything locked down and backed up. I expect they knew what the risks where and are and will update their procedures appropriately.

  18. Re:Not to blame? Wrong! by HiThere · · Score: 1

    That's a good point. Especially the " they failed to implement protocols for bringing other systems into the environment.", as there are many reasons why they may not have been able to patch the system.

    --

    I think we've pushed this "anyone can grow up to be president" thing too far.
  19. Re:Patching would have caused more downtime & by Billly+Gates · · Score: 1

    Right because Linux is so perfect and secure and never has been hacked before or needs patching.

  20. Linux by Anonymous Coward · · Score: 0

    This is what I don't get - people keep using Windows even though it sucks. Every reason I always hear is just people being set in their ways and now want to change. And they keep getting hacked. Insanity.

  21. No, YOU don't understand fab economics by Nova+Express · · Score: 1

    You never want to take a wafer fabrication plant offline for unscheduled maintenance, because having a line down costs you $1 to $10 million an hour while you're down. Worse, if you take it down for anything but regularly scheduled maintenance, you have to re-qualify the tool, which can take weeks.

    And if you have to take all your etch tools, or all your metal deposition tools, or all your steppers down, because they all run on the same version of Windows 7, then you're burning through tens of millions of dollars worth of opportunity cost while the tools are getting patched and requaled to make sure that none of the hundreds, if not thousands, or process parameters were changed due to the upgrade.

    In that environment, not patching is the economically logical choice.

    --
    Lawrence Person (lawrencepersonh@gmailh.com (remove all "h"s to mail)

    http://www.lawrenceperson.com/

    1. Re:No, YOU don't understand fab economics by gweihir · · Score: 1

      You assume I criticize them not patching. That is not correct.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    2. Re:No, YOU don't understand fab economics by Anonymous Coward · · Score: 0

      I am wondering how the software/hardware combination are so fragile on these systems.

      Is the software directly controlling the machine through the PC?
      I would have thought there would be specialised hardware that would handle all the critical parts of the system, and the software on the PC just giving it high level commands.

    3. Re:No, YOU don't understand fab economics by gweihir · · Score: 1

      This is probably some Windows front-end monitoring and configuration software for some SCADA or SCADA like systems. Since Windows has a tendency to break on updates, they probably just isolated the network and there was some report that an outside supplier brought the malware in by being sloppy.

      The sane thing would of course be to put some hardened OS with low-risk patching on these machines (e.g. a hardened RHEL) and still have them on an isolated network. Would now likely also have been the cheaper thing. But you find this fragile Windows crap even on, say $10M MRI machines and the like, where it does not contribute in any meaningful way to the overall cost and doing it better would not be an issue. The problem is machine and automation designers without a clue about security and OSes.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  22. Re:Patching would have caused more downtime & by gweihir · · Score: 1

    You are saying "Linux", I did not. One advantage of Linux is that it usually does not break on update, though. That is, before systemd. But there are other alternatives.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  23. Why use a toy designed for a teen in production? by Anonymous Coward · · Score: 0

    Windows lost its way a long time ago. Why is _ANYONE_ using it for process control? (apart from stupidity)

  24. It's only a license by Anonymous Coward · · Score: 0

    The US Navy got a Microsoft extension for an expired Windows that the Navy has deployed. Remember that we do not own Windows. We only paid to use Windows as a license. It's up to Microsoft to update and repair their Operating System.
    I think you should really think about upgrading to Linux. If you need time to migrate your Windows App/Software to Linux then run your Windows App/Software in Linux. You would not of had this problem if your IT used Linux instead of Windows.

    Microsoft sues everybody like Oracle. If I were you that lost $Millions of dollars then I would do what Microsoft likes to do. Sue them, ie Microsoft.

  25. Re:Patching would have caused more downtime & by Billly+Gates · · Score: 1

    ?? Have you seriously ever ran a distro without updating? No distro in existence can still function after 2 updates. It always requires a re-install because it lacks an ABI driver model which every other OS has for decades now.

  26. Re:Patching would have caused more downtime & by gweihir · · Score: 1

    I have automatic updates every 3 days enabled on some of my servers. No problems in about 12 years now. Were does this stupidity about "no distro can function after 2 updates" come from? Are you utterly clueless what you are talking about?

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.