Slashdot Mirror


Hackers Modify Water Treatment Parameters By Accident (softpedia.com)

An anonymous reader writes: Verizon's RISK security team has revealed details on a data breach they investigated where some hackers (previously tied to hacktivism campaigns) breached a payments application from an unnamed water treatment and supply company [PDF, page 38], and also escalated their access to reach SCADA equipment responsible for the water treatment process. The hackers modified water treatment chemical levels four different times. The cause of this intrusion seems to be bad network design, since all equipment was interconnected with each other in a star network design, and the payments app contained an INI file with the administrative password for the central router, from where the hackers reached the water treatment SCADA equipment. Of course, the hackers had no clue what they were modifying. Nobody got poisoned or sick in the end.

139 comments

  1. By "accident" by fustakrakich · · Score: 0

    That's a good one. Serious CYA

    --
    “He’s not deformed, he’s just drunk!”
    1. Re: By "accident" by Anonymous Coward · · Score: 1

      No one got sick in the end. In the middle, though -- Hoo boy!

  2. now that's some bad network design by softnewsit · · Score: 1

    Who designed that network? Marty McFly in the 80s?

    --
    Go away!
    1. Re:now that's some bad network design by Anonymous Coward · · Score: 1

      This is heavy.

    2. Re:now that's some bad network design by U2xhc2hkb3QgU3Vja3M · · Score: 1

      something something Earth's gravitational pull something something...

    3. Re:now that's some bad network design by Anonymous Coward · · Score: 0

      Great Scott!

    4. Re:now that's some bad network design by Anonymous Coward · · Score: 0

      Verizon

  3. And the worst of it? by wardrich86 · · Score: 5, Insightful

    If somebody had have died or gotten sick, the hacking party would be the ones to get in shit, not the asshat that put the admin password in a text file...

    1. Re:And the worst of it? by Anonymous Coward · · Score: 2, Insightful

      I just don't get this. I feel bad putting the admin password in a file on our demo VM that runs on a local workstation.

      I can't imagine sleeping at night putting it on an actual system somewhere.

    2. Re:And the worst of it? by Coisiche · · Score: 1, Troll

      I was nodding in agreement to that and then a thought suddenly struck me... what if something like this was left deliberately weak so that a part of the population could be disposed of, should it become necessary, and then hackers are the convenient scapegoat for blame in the eyes of everyone else. Especially if the hackers were associated with parties that the monied interests find inconvenient.

      I know they say never attribute to malice that which can be explained by incompetence but maybe sometimes it is malice.

    3. Re:And the worst of it? by Pascoea · · Score: 4, Funny

      Come on. Give them a little credit, it was an INI file, not a TXT file. They probably even named it this_isnt_the_network_password.ini

    4. Re:And the worst of it? by tnk1 · · Score: 5, Insightful

      If somebody had have died or gotten sick, the hacking party would be the ones to get in shit, not the asshat that put the admin password in a text file...

      They both should get in deep shit for it. Yes, the asshole who left the admin password in a text file should get fired.

      However, you should be able to leave an admin password posted on a banner on a 24 hr news station and a good person wouldn't use the password to get in and fuck with a water treatment plant. That's like saying that anyone who leaves their door unlocked deserves to have their house broken into and accidentally burned down while people are trying to steal shit.

      So, yeah, the both hackers and the admin should be dealt with severely. This isn't an either/or situation.

    5. Re:And the worst of it? by Anonymous Coward · · Score: 0

      I highly doubt a utility would allow that. They would lose all of their customers!

    6. Re:And the worst of it? by TheCarp · · Score: 1

      > However, you should be able to leave an admin password posted on a banner on a 24 hr news
      > station and a good person wouldn't use the password to get in and fuck with a water treatment plant.

      You are not wrong but, there is a question of how much of a risk you are taking. Yes nobody SHOULD do it, but, since you know there are some number of people who WILL do things they should not, maybe the person in charge bears some responsibility for not taking more precautions than the honor system?

      Its one thing to say "Nobody should steal my gold coin even if I leave it on my front steps for all to see" your right, nobody should.... but if somebody does....the whole community isn't being put in danger by your lack of caution.

      That said, I wouldn't be so quick to blame the admin or even the guy who set it up, you don't know what constraints he was under or even what training or tools he had available. This could very well have been screwed all to hell before he even got told to make it work.

      --
      "I opened my eyes, and everything went dark again"
    7. Re:And the worst of it? by rbrander · · Score: 1

      You're being silly. Water treatment plants are incapable of producing water that would kill, well, anybody, probably.

      Yes, there's enough chlorine (or other disinfectants) in the plant to kill people; but you could open the valves to their greatest extent without jumping the chlorine content up from the usual part-per-million to more than a couple of parts per million...that is, still way less chlorine than your average municipal pool needs to combat all those filthy kids.

      I suppose there's somebody so allergic and frail of health that such water could kill them, but I don't think killing one person out of a million is the kind of genocide you have in mind.

    8. Re:And the worst of it? by Ungrounded+Lightning · · Score: 3, Insightful

      ... you could open the valves to their greatest extent without jumping the chlorine content up from the usual part-per-million to more than a couple of parts per million...that is, still way less chlorine than your average municipal pool needs to combat all those filthy kids.

      But what if the bad guys CLOSE the valves? Then live pathogens go straight from the water source into the no-longer-purified water supply. Several million customers are exposed. Many are sickened. Some take permanent damage. Some die. Even after the issue is fixed the whole water system needs decontamination. And the whole set of cities fed by the plant are disrupted (which is what they're really after).

      It gets even nastier if the bad guys up the ante by dumping a bit of some particularly virulent bugs upstream of the intakes, during the period where they won't be killed off by the shut-down disinfectant injection.

      They use chlorine because its a heck of a lot less damaging to people than the things it is used to kill off.

      --
      Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
    9. Re:And the worst of it? by Anonymous Coward · · Score: 0

      I look at it this way.

      There are always two levels of defense. You defend against people and animals.

      People can be trusted to not do things, even if you leave the temptation there.

      Animals will always take the temptation, which is why you need to guard against things.

      If the animals get in because you didn't lock the door, you screwed up, but if the animals attacked or killed someone, you still put the animals down.

    10. Re:And the worst of it? by Noah+Haders · · Score: 1

      It's a shame these hackers didn't attack the Flint water treatment plant two years ago, they could have turned the water treatment back on and saved everyone from lead poisoning.

    11. Re:And the worst of it? by TheCarp · · Score: 1

      Right but if the house is your chicken coup, and the family is starving because there are no chickens anymore.... whether it was animals who came in through the unlocked door or bad people, either way, youre children are still not eating.

      --
      "I opened my eyes, and everything went dark again"
    12. Re:And the worst of it? by NatasRevol · · Score: 2

      Management very rarely properly estimates damage assessments from IT.

      --
      There are two types of people in the world: Those who crave closure
    13. Re:And the worst of it? by phantomfive · · Score: 3, Informative

      They had their water treatment plant connected to the internet. That's like putting a banner with the root password, plus leaving the door open with a sign that says, "PLEASE COME IN."

      The incompetence here went very deep. If only the NSA were doing something useful like trying to defend this stuff against foreign hostile hackers, instead of trying to spy on citizens.

      --
      "First they came for the slanderers and i said nothing."
    14. Re:And the worst of it? by Anonymous Coward · · Score: 0

      Ok, even I think that's a touch paranoid. It looks like a standard case of developer incompetence and/or being rushed into production without time to do the right thing.

      Also, much as the idiot designer is partly to blame, in this case I would also portion some blame to the hackers. Just because someone left keys in the lock does not mean it's ok to wander in and move the furniture.

    15. Re:And the worst of it? by Anonymous Coward · · Score: 0

      You're being silly. Water treatment plants are incapable of producing water that would kill, well, anybody, probably.

      A treatment plat will have all kinds of chemicals, not just the ones that are added to the water, but also several that should not be added to the water.

      Camelford water pollution incident

    16. Re:And the worst of it? by Zak3056 · · Score: 1

      We must never forget St. Mary's and Three Waters! All good citizens should remember the Articles of Allegiance, and that Chancellor Sutler only wants to keep us safe from degenerates.

      --
      What part of "shall not be infringed" is so hard to understand?
    17. Re:And the worst of it? by thegarbz · · Score: 1

      Water treatment plants are incapable of producing water that would kill, well, anybody, probably.

      You may want to consider what the first two words of that sentence is and what the purpose of these plants are.
      Also you may want to look into what exactly is in your water supply upstream of water treatment.

      Water treatment and sewage treatment may not be the same thing, but only due to the level of contamination, not due to a difference in the insane variety of bugs and diseases contained within either liquid. Remember, drink beer because cows are shitting in your water right now.

    18. Re:And the worst of it? by flopsquad · · Score: 2

      Um. Can I give you "-1: I Don't Want Anybody to See This" ?

      --
      Nothing posted to /. has ever been legal advice, including this.
    19. Re:And the worst of it? by Anonymous Coward · · Score: 0

      Well, the asshat didn't poison anybody... the hackers did.

    20. Re:And the worst of it? by Monoman · · Score: 2

      Or if the IT guy/department protested but was told to "do it anyway". Get that stuff in writing folks!

      --
      Keep the Classic Slashdot.
    21. Re:And the worst of it? by TheCarp · · Score: 1

      "Yes, we understand your concerns but sales already told them we could make this work."

      --
      "I opened my eyes, and everything went dark again"
    22. Re:And the worst of it? by qfman · · Score: 0

      Spanking the hackers is pointless. On the one hand I am not condoning what they did, but on the other hand they should be thanked for exposing such a blatant lack of security on a system the so many people depend on for survival.

      --
      They who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety.
  4. I disagree by liqu1d · · Score: 4, Funny

    I got rather sick when I read that the admin password was in the ini file.

    1. Re:I disagree by MobyDisk · · Score: 4, Funny

      Yeah, they should have put the admin password in an XML file!

    2. Re:I disagree by U2xhc2hkb3QgU3Vja3M · · Score: 1

      Everybody knows that an XML file is not safe enough, it has to be in a TXT file.

    3. Re:I disagree by Anonymous Coward · · Score: 1

      XML is so 2000's. We put our admin passwords and SQL connection strings in JSON configuration files now.

    4. Re:I disagree by ole_timer · · Score: 1

      union rules, it had to be in the ,ini file.

      --
      nothing to see here - move along
    5. Re: I disagree by Anonymous Coward · · Score: 0

      My admin password is on a label on the front of the unit.

    6. Re: I disagree by Anonymous Coward · · Score: 0

      Why keep it in a file? Just store it as a registry key!

    7. Re:I disagree by Anonymous Coward · · Score: 0

      Yeah, but it was veiled in ROT26

    8. Re: I disagree by Anonymous Coward · · Score: 1

      Come to think of it, that's a better place than in a TXT file on the server.

    9. Re:I disagree by Anonymous Coward · · Score: 0

      Just make sure you change the extension to .bin so nobody looks there and you're good. That's how security works, duh; you've got to hide the secrets!

    10. Re:I disagree by U2xhc2hkb3QgU3Vja3M · · Score: 1

      thisisnotreallyatextfilewiththeadminpasswordinsidejustrenamedtobin.bin

    11. Re: I disagree by U2xhc2hkb3QgU3Vja3M · · Score: 1

      That's one way to protect it from Internet hackers. But then you're vulnerable to the most vicious hackers of them all: the cleaning crews!

    12. Re:I disagree by tnk1 · · Score: 2

      XML is so 2000's. We put our admin passwords and SQL connection strings in JSON configuration files now.

      This. You pretty much need to ensure that your hosts are not able to be accessed because there's still the stupid plain text or MD5 hashed password in an unencrypted text file somewhere in order to connect your app to your database.

      Not that encryption would matter. If someone breaks into a host that has a public key for a database server, then someone can use that same public key for access to the database server as long as they were doing it from the host that they just broke into. Actually securing connections where access is done automatically really requires a lot of thought and not just one encrypted file somewhere.

    13. Re: I disagree by NatasRevol · · Score: 1

      Physical access > network access

      --
      There are two types of people in the world: Those who crave closure
    14. Re:I disagree by stackOVFL · · Score: 1

      .ini: PASSWORD = ""

      Hacker: I guess they don't have a password, what will we do? We'll never guess it now!

      :P

    15. Re: I disagree by ls671 · · Score: 1

      No cleaning crew is allowed in our data center.

      --
      Everything I write is lies, read between the lines.
    16. Re:I disagree by KermodeBear · · Score: 1

      And make sure to base64 encode it. Nobody will figure that one out.

      --
      Love sees no species.
    17. Re:I disagree by Ksevio · · Score: 1

      See the problem was it was written on a post-it on the monitor before, but the facility went "paperless" so it had to be put on the machine instead.

    18. Re: I disagree by Anonymous Coward · · Score: 0

      tattoo it in sections across the lower backs of several shift workers in the machine room.

      security levels:

      1) need all the engineers at once or you don't have the whole pword

      2) who the fuck wants to look at all those furry backs?

      And it can be updated: the tramp stamps can be modified as the pword changes ( 3 becomes 8, 1 becomes part of 4, etc.)

    19. Re:I disagree by Trogre · · Score: 1

      At least no one would have been able to read it then.

      --
      "Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
    20. Re: I disagree by Anonymous Coward · · Score: 0

      Are you planning on using your coworkers back for some weird Shamir secret sharing scheme? This sounds like the start of some really bad nerd porn.

    21. Re:I disagree by Anonymous Coward · · Score: 0

      Come one, really? F****ng amateurs! Real programmers use ASN.1 BER.

    22. Re:I disagree by dcooper_db9 · · Score: 2

      Put it in a readme file. Nobody would ever find it there.

      --
      I do not block ads. I do block third party scripts.
  5. cleartext passwords by Anonymous Coward · · Score: 0

    wow... admin password in an INI file for your network's central point... just wow

    1. Re:cleartext passwords by Archangel+Michael · · Score: 1

      1) Never Cleartext Passwords (there should be no exceptions, but we all know of some)
      2) Networks should be isolated from each other ("Should")
      3) IT budgets are often woefully inadequate for proper securty (Cheap Labor/outsourced, low priority upgrade schedules etc,, lowest cost equipment bids).

      Good IT is expensive. Bad IT is costly. Dodged a bullet on this one.

      --
      Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
    2. Re:cleartext passwords by NatasRevol · · Score: 3, Interesting

      4) IT management rarely has any understanding of risks associated with IT designs/constraints. Even when explained to them.

      --
      There are two types of people in the world: Those who crave closure
  6. "and the payments app contained an INI file with t by Anonymous Coward · · Score: 0, Flamebait

    This is simple best informatics practice when to doing the needful are happily.

    - Raj "H1B" Subramanadanajab

  7. haha, INI file by Anonymous Coward · · Score: 0

    Admin passwords in an INI file... no wonder that "hacktivists" breached their network

  8. Shows my main point in Hacktivism! by jellomizer · · Score: 1

    When these groups try to do their attacks, they don't realize what other fallout which may be happening. Is that Bank using the same data center of a hospital you don't know. Is the budget system going to affect other systems?

    Normally the places with the worst security are not that way due to lack of IT Talent, but because the integration of legacy systems is so connected that it becomes a major undertaking to correct.

    1980's Mainframes were expensive computers, most organizations could normally afford one, and they write all kinds of software on them, from managing their billing to controlling the factory. If you remove one component you can cause all others to have the same problem. Trying to fix the infrastructure will cost millions. For organizations that support the public, usually are under tight budget anyways so fixing them is very difficult.

    So those hackers who think they don't hurt anyone. You are wrong.

    --
    If something is so important that you feel the need to post it on the internet... It probably isn't that important.
    1. Re:Shows my main point in Hacktivism! by cfalcon · · Score: 1

      > So those hackers who think they don't hurt anyone. You are wrong.

      Well, that's demonstrably not true- plenty of hackers haven't hurt anyone. More importantly, it perpetuates the downsides of the 80s style hacker mythology- that there's this group of elites who have this power to break in, or not. The problem with this mythology isn't how true it is- it seems to be pretty accurate, and it pretty fun and interesting. The problem is with how people naturally react to it. "Ok, find the bad guys and put them in jail" is the obvious response. Hacks are the rain- you need to build a roof. The fact that there's an endless array of endlessly insecure systems is the problem, and busting the very few bad hackers doesn't solve the problem- it just provides some justice here and there.

    2. Re:Shows my main point in Hacktivism! by gstoddart · · Score: 1

      that there's this group of elites who have this power to break in, or not. The problem with this mythology isn't how true it is- it seems to be pretty accurate, and it pretty fun and interesting

      You know, reading TFA, I refuse to believe "elites" had a damned thing to do with it.

      This absolutely screams of being a place which could have been taken down by a couple of bored script kiddies on their first day out.

      These guys could have been skilled, but the security reads like it was so bad that it defies explanation how they hadn't already been hacked. The security lapses they describe show a pretty fundamental level of clueless.

      --
      Lost at C:>. Found at C.
  9. "Nobody got poisoned or sick in the end." by jeffb+(2.718) · · Score: 4, Insightful

    Problem is, this is a lot more "just the beginning" than "in the end".

    How many such systems do you suppose have been penetrated by folks who do know what they're doing, and are just sitting on their access until the next political party convention, or major sporting event, or...?

    1. Re:"Nobody got poisoned or sick in the end." by Anonymous Coward · · Score: 1, Interesting

      I distill my tap water before drinking it, using one of these.

      That doesn't solve this problem, of course, but it does give me an extra layer of protection against failings of the water treatment process.

      Contrary to strangely-popular belief, distilled water is only barely acidic (thousands of times less acidic than soda pop, slightly less acidic than a banana), and does not leech minerals from your body. It's water. It is perfectly healthy, and it tastes good.

    2. Re:"Nobody got poisoned or sick in the end." by tnk1 · · Score: 1

      Except I have a lot of trouble believing that such events haven't already happened, yet there have been no attacks.

      Someone is always pissed about something. I'd think that if Trump's business was riddled with holes *this would be about the right time to use those holes before he gets more and more out of control.* Yet we see only one anemic reveal from some Anonymous source which clearly was not some elite hacker who had owned The Donald. And this is a guy who has basically admitted to paying off politicians in so many words, so there should be plenty to find.

      I think the real elite hackers are now owned by someone else or they don't have the access that we assume they do. The rest are challengers or script kiddies.

    3. Re:"Nobody got poisoned or sick in the end." by EmeraldBot · · Score: 1

      I distill my tap water before drinking it, using one of these.

      That doesn't solve this problem, of course, but it does give me an extra layer of protection against failings of the water treatment process.

      Contrary to strangely-popular belief, distilled water is only barely acidic (thousands of times less acidic than soda pop, slightly less acidic than a banana), and does not leech minerals from your body. It's water. It is perfectly healthy, and it tastes good.

      God dammit, not this again. No people, distilled water is not safe to drink. It will try to balance out that PH, it will sap minerals and electrolyres from your water, and it will shorten your lifespan.

      Here's a link

      And here's another.

      Distilled water was a health trend in the 70's, right along with the "don't vaccinate because of autism" trend in the 2000's. It's a clever troll if you want to give someone serious health problems or so, if you really find that funny, but as soon as you crack open a high school chemistry textbook it becomes pretty obvious why it's a bad idea. Did you, sir, ever take Chemistry?

      --
      "Set a man a fire, he'll be warm for the rest of the night. Set a man afire, he'll be warm for the rest of his life."
    4. Re:"Nobody got poisoned or sick in the end." by Anonymous Coward · · Score: 0

      Right.

      You realize that most bottled water on the market is either vapor distilled or reverse-osmosis treated, which has the same net consequence of removing all/most of the the disolved ions from the water... making it more clean and -better- at being water.

    5. Re:"Nobody got poisoned or sick in the end." by Jiro · · Score: 1

      If distilled water actually sapped minerals, consider that once it's going into your mouth it would be combining with saliva, which has a certain percentage of dissolved minerals. Going from drinking regular to distilled water would just mean going from lots of minerals+regular to lots of minerals_distilled. Either way the result from adding the saliva is pretty much the same.

    6. Re:"Nobody got poisoned or sick in the end." by Anonymous Coward · · Score: 0

      Mercola == money-grubbing fear-stoking woo-pushing con artist. There's a whole bunch of misinformation in that article, even if the conclusion (don't drink distilled water for an extended period) is correct.

      If you're getting medical advice from that site, you're doing it wrong, in a huge way.

    7. Re:"Nobody got poisoned or sick in the end." by Anonymous Coward · · Score: 0

      Want your electrolytes and minerals back? Eat one bite of broccoli. There is more in that than in a whole gallon of undistilled tap water.

      The problems your links discuss are important for marathon runners, or people who are sick. They sweat out their electrolytes very rapidly, and pure water won't replenish them. That's why the WHO gave a formulation for oral re-hydration...it tasteshorrible, but is intended as medicinal in these circumstances.

    8. Re:"Nobody got poisoned or sick in the end." by qfman · · Score: 0

      Obviously the NSA et el are a worthless waste of money at best and a massive industrial / political espionage scheme at worst. Lets de-fund 99% of what they are doing. It would greatly increase world security, help balance the budget, put a pinch on terrorist recruiting eforts...

      --
      They who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety.
  10. sounds like classic industrial control networks by dot_bull · · Score: 3, Interesting

    I've rarely seen a classic "control system" (HVAC, security, wet and dry lab systems, anything with modems and 9600kbps transmission, ANSI screens, etc) be configured in anything BUT 1980's architecture. These industrial control systems are so old and embedded no one has the money or incentive to remove them and install modern tech. And most of them are archaic, and so incredibly vulnerable it can make a person lose sleep. Think yet another "tip of the iceberg"moment. Think water control, sewage control, electrical control, alarms control, traffic light control. NOT ALL, but the majority are hopelessly insecure and controlled by people who use FAX machines. Anything installed before 2000 or so (the majority) are childlike in design and harbor absolutely no notion of security.

    1. Re:sounds like classic industrial control networks by Anonymous Coward · · Score: 0

      Absolutely correct! I worked a City position in IT before and was appalled at the quality, control and security of SCADA systems in place there. Not to mention, these apps were state of the are at the time circa 2008. Very scary reality here.

    2. Re:sounds like classic industrial control networks by tnk1 · · Score: 1

      While an insecure control system is very undesirable, you should be able to be able to overlay more modern security on top of it in the places where network interconnection is absolutely needed. You should be okay if the only remote access to your 1980's HVAC is through the 2010's firewall and intrusion detection system.

      Where interconnection is not required, this is all fixed instantly by air gaps between the control systems and everything else.

      I am thinking that this is due to incompetence, not the age or lack of security of the control systems.

    3. Re:sounds like classic industrial control networks by dot_bull · · Score: 1

      I agree with your points. However, the challenge of incorporating the corporate networking security protocols is daunting. The Facilities fiefdom disallows anyone but Facilities touching, well, facilities. The service contracts do not allow for anyone but the service personnel to touch the control systems, in most instances. I like the idea of a front end, but unless the service personnel can reach their systems from their 50 year old control systems, they will void the contracts. And as soon as you mention VPNs or encryption or air gapping - you risk losing all support for the critical systems (I'm speaking mostly from experience in supporting production pipelines in labs). What is needed is a full rethink of ownership of these systems, and buy-in and new contracts. And, lotsa new people.

    4. Re:sounds like classic industrial control networks by Anonymous Coward · · Score: 0

      yeah, but you should see all the bonus money management raked in by not wasting money on upgrades

  11. 2.5mil personal info leaked. by Anonymous Coward · · Score: 0

    "They also informed the water treatment company that the hackers had access to over 2.5 million customer personal and financial records and provided technical expertise on how KWC could fix their IT system to prevent similar incidents."

    1. Re:2.5mil personal info leaked. by campuscodi · · Score: 1

      It's probably a company activating in Europe or Asia, where data breaches don't need to be reported to the public, only to law enforcement. Otherwise, we would have heard about this incident earlier.

  12. Holy crap ... by gstoddart · · Score: 4, Insightful

    and the payments app contained an INI file with the administrative password for the central router

    You know, every time I have encountered anything this moronic I've raised bloody hell over it.

    Why the hell would a fscking payment app need the administrative password for the damned router, and what idiot allowed this on their network. On at least three occasions I've said "no way in hell I'm going to put a plaintext password into an INI file, and if you want me to do it you're going to have to send me an email and CC a lot of other people demanding it". (Reading TFA, it wasn't the actual payment app, but they got it off a web server they compromised which had it in an INI file, so bad job in the summary).

    I swear, security is often either non-existent or written by idiots.

    And that's before you even get to the epic stupidity of having your SCADA stuff to your normal network. I've been in places that had SCADA stuff, and NOTHING was on that network which wasn't fully vetted.

    This whole article reads like "what happens when unqualified people run critical systems" -- right down to the fact that they also had access to "2.5 million customer and financial records".

    I'd like to say I'm astonished, but that would imply that I keep being surprised at just how bad companies suck at fairly basic security.

    --
    Lost at C:>. Found at C.
    1. Re: Holy crap ... by bill_mcgonigle · · Score: 2

      >Why the hell would a fscking payment app need the administrative password for the damned router,

      It's such a pain to have different passwords for everything.

      --
      My God, it's Full of Source!
      OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
    2. Re: Holy crap ... by Anonymous Coward · · Score: 0

      Thats what pw apps are for.

  13. SCADA by Anonymous Coward · · Score: 0

    I think most people forget that their SCADA is connected to their network. Seriously. I've rarely seen one of these networks that's NOT connected to the Internet.

  14. Trump-level stupidity by Anonymous Coward · · Score: 0

    Honestly, the article contains even more gems. These KWC guys were geniuses... real geniuses

  15. Responsibility by Anonymous Coward · · Score: 1

    The persons who approved and connected critical infrastructure devices to Internet accessible networks should be hung. Air gap this stuff, people. It doesn't belong on the Internet.

  16. This Is Why We Can't Have Nice Things by Anonymous Coward · · Score: 0

    I'm sorry, but fucking around with water systems isn't really part of saving the world, is it?

    Public Service Announcement to all script-kiddie "hacktivists":

    If you don't know what you're doing once you're inside a live industrial system, FFS DON'T DO IT.

    1. Re:This Is Why We Can't Have Nice Things by mrchaotica · · Score: 1

      I'm sorry, but fucking around with water systems isn't really part of saving the world, is it?

      You never know; some hacker might have been able to help in Flint (where part of the problem was using water from a polluted source, but the other part of the problem was not using enough of some of the chemical treatment).

      --

      "[Regarding the 'cloud,'] ownership was what made America different than Russia." -- Woz

    2. Re:This Is Why We Can't Have Nice Things by tnk1 · · Score: 1

      I'm sorry, but fucking around with water systems isn't really part of saving the world, is it?

      You never know; some hacker might have been able to help in Flint (where part of the problem was using water from a polluted source, but the other part of the problem was not using enough of some of the chemical treatment).

      You know, I find the potential value of a hacker in that system to be so small as to make it near zero.

      Sure, they can hack in. And then what? Do they know what the chemical treatment required for the Flint Water District is? Are they going to re-route the water through a series of tubes away from the lead tubes through the tubes that go to where they make Perrier water? I guess that instead of a V-LAN, they'd configure the water routers to make a new W-LAN for the H20 packets.

      Hackers are good at hacking, not fixing water treatment. Being an environmental engineer is actually a real job description that you need to have a college education for in the field. The next thing you'll be telling me is that hacking into the Department of Transportation will allow hackers to fix the gradient on that one curve on Interstate 81.

    3. Re:This Is Why We Can't Have Nice Things by Anonymous Coward · · Score: 0

      Or perhaps some idiot hacker created the problem by screwing around in a system they knew nothing about?

  17. Airgap by Moof123 · · Score: 4, Insightful

    Equipment of this sort should be air gapped from the wild wild west of the internet. Frankly anything that is safety related (hospital equipment, elevators, and even HVAC systems) should be unreachable without badging into a building. While there are still ways to propagate things in via USB stick, it would keep clowns from pulling this kind of stuff.

    1. Re:Airgap by tnk1 · · Score: 1

      Yes. Air gap and security guards doing searches should stop most of this stuff.

      Some equipment does need network interconnection with the Internet, but the great majority of it does not.

    2. Re:Airgap by rbrander · · Score: 2

      Bingo. Air gap AND the machines that are on that network should have the USB ports filled with epoxy. When updates are needed, the vendor plugs in a special laptop for the purpose.
      It's extremely useful for SCADA to use Wi-Fi, of course; nothing beats being able to haul a tablet right down under the floor where you've just unstuck a valve and then cycle the valve without running up to the console.
      But the Wi-Fi of course needs to be locked down to a specific set of MAC addresses, not just with passwords. SCADA has to have a whole different approach to security than general-purpose computing. Big and distributed as it is, the SCADA system has to be an appliance, like a Blu-Ray player, only able to run the system programs and no others. But all the OS-level protection against that doesn't touch Air Gap; no-connectivity has to be sacred no matter how tempting.

    3. Re:Airgap by Ksevio · · Score: 1

      While I'd give you top marks from a security standpoint, it's not the 1990's anymore and everything is connected. The technology to secure networks at multiple points is well developed and we shouldn't let the fear of incompetence prevent us from having the benefits of connected systems.

    4. Re:Airgap by Artemis3 · · Score: 1

      The mac address thing is largely obsolete now, with most OSes providing mac spoofing by default, and sniffing the connected devices to copy their mac address is the first step before attempting any passwords anyway.

      The thing with scada is most terminals run windows, and its those terminals who are largely targeted.

      Also you don't need to have exposed USB ports anyway, something behind (physical) lock could do for occasional updates.

      --
      Artix
      Your Linux, your init.
    5. Re: Airgap by Anonymous Coward · · Score: 0

      I would really rather you call your co worker who is sitting in front of a hard wired terminal to exercise the valve. Wireless? Really?

    6. Re:Airgap by lgw · · Score: 2

      the SCADA system has to be an appliance, like a Blu-Ray player, only able to run the system programs and no others

      Did you know BluRay players will execute arbitrary Java code off of BluRay discs, as part of normal operation? I'm hoping you didn't. BluRay is specifically designed to allow a disc to damage the function of the device (by invalidating keys needed to play other discs).

      --
      Socialism: a lie told by totalitarians and believed by fools.
  18. The mystery of water in City of Flint? by jacekm · · Score: 0

    Is this the reason for Flint water fiasco?

    1. Re:The mystery of water in City of Flint? by ShaunC · · Score: 1

      Not unless Rick Snyder figured out how to run nessus...

      --
      Thanks to the War on Drugs, it's easier to buy meth than it is to buy cold medicine!
  19. Now we pin each death in flint MI on them by Joe_Dragon · · Score: 1

    Now we pin each death in flint MI on them.

    Better plead down to life.

  20. Re: sounds like classic industrial control network by bill_mcgonigle · · Score: 1

    >no one has the money or incentive

    Sounds like the incentives are approaching very quickly.

    --
    My God, it's Full of Source!
    OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
  21. Is this the incident? by Bruce66423 · · Score: 1
    1. Re:Is this the incident? by rbrander · · Score: 1

      Nope, looks like that was an accident with manual chlorination, not computerized at all.

  22. This is an "accident?" by westlake · · Score: 1
    Geek logic sometimes escapes me.

    So tell ell me why screwing with the process controls in a chemical plant counts only as an "accident."

    and also escalated their access to reach SCADA equipment responsible for the water treatment process. The hackers modified water treatment chemical levels four different times

    1. Re:This is an "accident?" by Anonymous Coward · · Score: 0

      probably because levels were random, and didn't show any intentional attempt at poisoning the water supply...you know like... hey guys...look what i've found.... two weeks alter it gets boring and you stop coming back because you can't figure out what PLC means....

  23. All the security stuff is off-topic by rbrander · · Score: 2

    There's only one security that counts with a SCADA system: air gap. Plant-controlling systems must not talk to any other network.
    I recently retired from a much-larger utility and we did struggle with the human factor. The plant guys heard all the lectures from their design consultants that put in the system and the IT people who checked the design over. They understand that they must not interconnect. ...and then a year or two later you find them trying to quietly slip two network cards into the same machine so they don't have to change chairs to go from corporate-network-with-Internet-access to SCADA.
    Emotionally, it's hard to believe anybody would *want* to break in; it's not like there's money to be made. Hollywood-movie scenarios where "hackers take over" are ludicrous; every device in the plant has an "On/Off/Auto" switch where only "Auto" leaves SCADA in control at all; the most junior operator could run around the plant hitting those switches in five minutes, restoring manual control. (Then we'd have to bring in a dozen folks with cell phones to run the plant manually; no sweat).
    And as I posted above, it's not like you can kill anybody with a water treatment plant: the worst water you could put out would either be untreated (please boil water) or absolute max chlorine the system could insert (still less than a swimming pool).
    It's going to be the same as "safety"; you can pound safety lectures into people's heads all day, but it seems to take a generation or two for the message to really sink in; hard hats and visibility vests were strenuously avoided as well. We're just going to have to make it a standard, like safety standards: firing for disobedience, regardless of whether anything went wrong.

    1. Re:All the security stuff is off-topic by thegarbz · · Score: 2

      There's only one security that counts with a SCADA system: air gap. Plant-controlling systems must not talk to any other network.

      And you instantly fail all sorts of control, maintenance, reliability analysis, regulatory requirements for data, optimisation, etc tasks as a result.
      Admittedly a water treatment plant is simple and probably should get away with air-gaps, but the words air-gap are the first words that everyone utters when they talk about control systems. This causes two problems.

      1. Air-gaps need to be breached to enable a whole world of optimisation and value improving abilities in control systems these days. An air-gapped plant will be a plant shutdown for financial reasons permanently, though not by hackers.

      2. well I'll quote you for number 2:

      and then a year or two later you find them trying to quietly slip two network cards into the same machine

      To quote the Goldblum "Life finds a way". There is far better security in designing a system with a well thought through network configuration that offers the complete set of capabilities a plant requires than to assume someone incomitent in network security will not figure out a way to do what they want to do.

      It reminds me of a question I had from a plant in Australia where they discovered an operator had plugged a 3G modem into the control system and was watching youtube on panel. They asked how I would train the operators not to do that. My answer was: "Our operators don't do that not because of some training but because we gave them a computer to surf the internet on and watch youtube videos without leaving the board." Don't cut them off and they won't try bypassing things.

      Air-gaps are a dangerous form of security. It's security by network engineer, not security as a culture.

    2. Re:All the security stuff is off-topic by Anonymous Coward · · Score: 0

      > We're just going to have to make it a standard, like safety standards: firing for disobedience, regardless of whether anything went wrong.

      That's the stick, you also need a carrot.

      Those guys who wanted internet access, they should have had another PC right there, maybe even on the same monitor/keyboard/mouse with a KVM switch. Good security makes the path of least resistance the most secure path. Shitty security makes people want to circumvent it because it is an obstacle to getting their jobs done.

      Shitty security is easy for the security people to design, good security takes a lot more work for those individuals but the end result is *everyone else* actually wanting to follow good security practices.

    3. Re:All the security stuff is off-topic by Anonymous Coward · · Score: 0

      >it's not like you can kill anybody with a water treatment plant: the worst water you could put out would either be untreated (please boil water)
      All you have to do is drop something bad in the water supply and have your hackers turn off the treatment and have the monitoring report fake values. By the time anyone catches on to give a boil alert, the contamination has already sickened people. Maybe only a few elderly & children died, but it's enough to terrorize everyone across the nation.

    4. Re:All the security stuff is off-topic by Anonymous Coward · · Score: 0

      If the SCADA is rooted, then damage can be caused while all indicators are showing normal operations.

      It could potentially take up to days before a manual quality test is done and the problem is detected. Use some clever evading techniques, like figure out the sampling schedule to have the chlorine be pumped in a few hours before and up until minutes after the test, and you could stretch it out to months before detection. That's a lot of potential damage.

      It's also already been done to the Iranians and their centrifuges, over an air-gap, so it isn't that far-fetched.

  24. Killing off the population. by Ungrounded+Lightning · · Score: 0

    ... what if something like this was left deliberately weak so that a part of the population could be disposed of, should it become necessary, and then hackers are the convenient scapegoat for blame in the eyes of everyone else.

    They don't need to use a water plant for that. They've got Obamacare.

    (bud-a-boom TISH!)

    --
    Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
  25. Nobody got poisoned or sick in Flint either by mi · · Score: 1

    Nobody got poisoned or sick in the end.

    How do we know, what the mid- and long-term effects will be? There is no one obviously poisoned by tap water in Flint, Michigan either.

    Should we apply the same spin to people responsible for that, as the submitter applied to hackers because he sympathizes with them?

    --
    In Soviet Washington the swamp drains you.
    1. Re:Nobody got poisoned or sick in Flint either by Anonymous Coward · · Score: 0

      You're kidding, right? Those +/-10,000 kids who sucked up lead for over a year aren't permanently poisoned? Those 10 people dead from water-borne Legionella aren't really dead?

      WTFuckingF?

    2. Re:Nobody got poisoned or sick in Flint either by mi · · Score: 1

      Those +/-10,000 kids who sucked up lead for over a year aren't permanently poisoned?

      Nobody in Flint has symptoms. And there being no symptoms is what led the submitter to acquit the hackers as well.

      --
      In Soviet Washington the swamp drains you.
    3. Re:Nobody got poisoned or sick in Flint either by Anonymous Coward · · Score: 0

      You STUPID FUCKER.

      DEATH is a pretty good symptom, would you agree? And 10 are dead from water-borne Legionella.

      Note that THERE IS NO SAFE LEVEL OF LEAD IN THE HUMAN BODY. Children exposed to lead are doomed to irreversible neurological damage, including lowered intelligence and a propensity for violence. This is why people fought for decades to get lead out of paint, old buildings, auto fuel, etc. Tens of thousands of people, including children, were exposed to extreme levels of lead in their water for over a year. That DAMAGE IS DONE.

      The symptoms are there; we just haven't done enough forensic medical investigation on the victims to quantify them (except the dead, of course, death being both final and hard to ignore), first because there hasn't been time and second because, let's face it, such a large longitudinal study is very expensive and since it only affected poor and brown-skinned folks the power structure won't care as long as this is quickly removed from the front pages and buried.

      You are either stupid, intentionally obtuse, or a dyed-in-the-wool, reality-denying neoconservative.

      Do, please, tell us which.

    4. Re:Nobody got poisoned or sick in Flint either by mi · · Score: 1

      You STUPID FUCKER.

      Stop shouting, Illiberal asshole... You aren't on an anti-Trump rally complaining about "not being heard".

      And 10 are dead from water-borne Legionella.

      They did not die on the first day the pipes got contaminated, did they? Not even the first month. Back then, somebody wishing to be sympathetic to the people involved in the mess could've said, the same thing: "Nobody got poisoned or sick in the end".

      Today, somebody sympathetic towards the hackers, is making the same mistake making the same claim. Some of the chemicals involved in water treatment are nasty and messing with their levels may make the tap water poisonous as well. Whether anybody was, in fact, affected may not even be known, because, if it is the well off and White skinned folks, they are unlikely to attract much attention from media or government regulators. But to make the above claim was bogus — and that's my point.

      Children exposed to lead are doomed to irreversible neurological damage, including lowered intelligence and a propensity for violence

      Yes, yes, sure. All according to government "scientists", who also claimed for decades, that fat and cholesterol are bad for you...

      dyed-in-the-wool, reality-denying neoconservative.

      Che Guevara much? Please, don't hate...

      --
      In Soviet Washington the swamp drains you.
    5. Re:Nobody got poisoned or sick in Flint either by Anonymous Coward · · Score: 0

      no you really are a stupid fucker here.
      the effects of lead are well known and have been for decades.

      what next, defending cigarettes?
      go the fuck away

  26. Better call Walter O'Brien by HumanWiki · · Score: 1

    and team.. (embarrassed to admit I watch this show..)

  27. CRIMINAL BEHAVIOR by Anonymous Coward · · Score: 0

    In addition to possibly compromising a public water utility,
    "the hackers had access to over 2.5 million customer personal and financial records"

    Did Verizon report this to the EPA?
    What possible excuse is there for not releasing of the name of this utility?
    Considering the current situation in Flint, this is crazy.

    1. Re:CRIMINAL BEHAVIOR by guruevi · · Score: 1

      I think this was white hat hacking, where a pen testing company got hired and asked "what does this widget on this public app/website do" and modified treatment parameters.

      --
      Custom electronics and digital signage for your business: www.evcircuits.com
  28. Your hands were on the wheel. by westlake · · Score: 2

    If somebody had have died or gotten sick, the hacking party would be the ones to get in shit, not the asshat that put the admin password in a text file...

    The rules are no different than if you and your gang of adolescent thrill seekers climbed over the fence or found an unlocked gate and began flipping exposed switches or opening valves just for the hell of it.

  29. Who was the dipshit engineer? by Lumpy · · Score: 1

    There is ZERO..... Z E R O reason for the payment system to have any connection to the SCADA system.

    Whoever is the manager of that plant and the engineer of that SCADA system and it's network need to be put in jail for their incompetence.

    --
    Do not look at laser with remaining good eye.
    1. Re:Who was the dipshit engineer? by Ksevio · · Score: 1

      You mean ZERO is the amount of thought you put into it? I can think of reasons easily - they're a small company with a couple machines. Both are on their internal network (like the vast majority of companies) because there are people that need to access both of them from their desk and it's easier this way.

      If we start jailing people for incompetence, we're going to need a lot more jails.

    2. Re:Who was the dipshit engineer? by Lumpy · · Score: 1

      Considering I worked in the field for 10 years and have extensive experience with it? A lot of thought went into it... What is your background with water filtration and SCADA systems?

      I thought so.

      Small company with a couple of machines? as a water treatment plant? if the place cant afford another $499 pc to set on someone's desk to keep the systems safe they need to shut down due to even more gross negligence.

      --
      Do not look at laser with remaining good eye.
    3. Re:Who was the dipshit engineer? by Ksevio · · Score: 1

      Really you've worked in the field for 10 years and couldn't come up with a single reason that you'd want to connect to a system remotely? Why spend $499 for a redundant machine for each employee when you can install proper network security instead?

    4. Re:Who was the dipshit engineer? by Anonymous Coward · · Score: 0

      You can blame him all you want, but I doubt he was paid anywhere close to enough to be worth risking going to jail for his code, and I doubt they paid anyone to check the system past the usual "it compiles, let's ship it". You get what you pay for.

    5. Re:Who was the dipshit engineer? by Lumpy · · Score: 1

      Not one that is SANE. only those that are utterly insane and want to make it insecure would want to connect remotely.
      I see you dont understand a thing about security and know nothing at all about SCADA.

      --
      Do not look at laser with remaining good eye.
    6. Re:Who was the dipshit engineer? by Ksevio · · Score: 1

      I work with systems that are sensitive AND connected to the Internet (through proper firewalls). It's possible to do, even a SCADA system can do it, but I see you don't understand a thing about securing a connected system so it'd probably be insane for you to attempt it.

  30. Just why does this equipment by Anonymous Coward · · Score: 0

    Have to be connected to the WAN to bill consumers for the set up.
    Cut the wire this is such a bad idea we made it 200 years without it connected we can go another 200 with ease.

    1. Re:Just why does this equipment by Anonymous Coward · · Score: 0

      You may be the only person here who actually understands the problem.
      There is no good reason to put everything on the internet.
      The typical explanation is cost savings.
      And those savings usually involve a reduction in manpower.
      So you end up with soaring unemployment and millions of people faced with identity theft.
      The standard response scapegoats the poor network administrator. "He was a jerk".
      OK - so who is vetting all these 100's of jerks? Doesn't anyone understand the inherent weakness of the system?
      How many times does this need to happen until we realize that critical infrastructure systems cannot be made safe if they are connected to the internet.

  31. What happens without regular audits by evolutionary · · Score: 1

    This is what happens when systems are left on automatic pilot. I think the technical term is "oops". Fortunately there was yet another system (also left on automatic pilot), that gave people a clue. We all know old systems did things that were just plain stupid (clear text passwords?!?). We've all see and/or done it, but the real problem was there was no review of the system looking for such things. We keep forgetting that security (and reliability) is a process, NOT a technology. And with water treatment, periodic reviews of such system should be mandated by law.

    --
    "Imagination is more important than knowledge" - Einstein
  32. SCADA garbage by Anonymous Coward · · Score: 0

    Air gapped SCADA systems are for LUDDITES! Modern app appers only use apped apps apping apps, NOT LUDDITE SCADA!

  33. Who gets fired? by phorm · · Score: 1

    "Yes, the asshole who left the admin password in a text file should get fired"

    Assuming, of course, that there were other options and that the application didn't work in such a way as that a password in a text file wasn't required. In the latter case, the blame would be on shitty programmers - often a third party - and not the sysadmin who did the best with what he had.

    As somebody who has often had plenty of similar WTF moments with crappy software design, I would not be surprised. In the end though, if one needs to connect to a remote system with credentials saved locally, there's not really a magic bullet for this. Sure, the password could have been encrypted, or it could have been hashed or perhaps key pairs used, but if the system is hacked and owned, even measures like that are really only going to delay the inevitable.

  34. Hang'em high by Anonymous Coward · · Score: 0

    Hackers, therefore guilty

  35. Built by the lowest bidder by Anonymous Coward · · Score: 0

    Oh let's be *agile*....oh let's *offshore*....and you'll get the needful alright....better/faster/cheaper -- choose 2..an admin password to a router in an INI file? Are you effing kidding me?? Just because you can code that doesn't make you smart....at all.

  36. Re:"and the payments app contained an INI file wit by Anonymous Coward · · Score: 0

    I think Slashraj or Sridot has started moderating here because every remark that is hard on india or h1b or offshoring has been modded -1 lately.

  37. Grow up! by thoughtspace · · Score: 1

    Hacking is really no less puerile than going around and smashing windows.

  38. "Of Course"? by cmholm · · Score: 1

    "Of course, the hackers had no clue what they were modifying."

    The report discussed the intruders having little apparent knowledge of what they were doing. The anonymous reader assumes this to mean that the intruders didn't know they were screwing with a water treatment SCADA system.

    I think it just as likely that they had figured out they had tapped into a process control system, and were figuring out how to manipulate the system... driving by Braille.

    The RISK report report authors could have summarized the situation by reaching back to the prophetic words of Simon & Garfunkel: Clowns to left of me, jokers to the right, Here I am, stuck in the middle with you.

    --
    Luke, help me take this mask off ... Just for once, let me butterfly kiss you with my own eyes.
    1. Re:"Of Course"? by Mal-2 · · Score: 1

      Clowns to left of me, jokers to the right, Here I am, stuck in the middle with you.

      That's Stealers Wheel.

      --
      How is the Riemann zeta function like Trump rallies? Both have an endless number of trivial zeros.
  39. So thats how they they are explaining Flint by Anonymous Coward · · Score: 0

    The old hackers did it , what happened did they get bored of 'cos terrorists' or 'wont any one think of the children?' classics!

  40. Nobody got poisoned or sick in the end... by cybersquid · · Score: 1

    ...that we know of.

  41. Nobody got poisoned or sick in the end. by eric_harris_76 · · Score: 1

    Glad to hear it. One December I got sick in the end. At one point, the length of my digestive tract, measured in time, became 20 minutes from the time I ate or drank anything to the time I sat on the toilet.

    --
    There's no time like the present. Well, the past used to be.