Slashdot Mirror


How a Port Misconfiguration Exposed Critical Infrastructure Data (helpnetsecurity.com)

An anonymous reader writes: Attacks hitting companies' electrical systems are possible, especially when information that provides insight into those systems' weak points is freely accessible online. If you think that such a thing is unlikely, you probably haven't yet heard about the most recent discovery made by UpGuard researchers: an open port used for rsync server synchronization has left the network of Power Quality Engineering (PQE) wide open to malicious attackers. They managed to access and exfiltrate 205 GB of data from PQE's servers, up until the moment when the company secured its systems two days later after being notified of the problem.

49 comments

  1. Why not use DFS for windows shares? by Joe_Dragon · · Score: 1

    Why not use DFS for windows shares?

    1. Re:Why not use DFS for windows shares? by Anonymous Coward · · Score: 2, Insightful

      You can configure DFS to provide unencrypted, unauthenticated access, so it meets all of PQE's requirements.

    2. Re:Why not use DFS for windows shares? by Sindar+By+Choice · · Score: 0

      Funny...

  2. If a single port misconfiguration... by Anonymous Coward · · Score: 2, Interesting

    If a single port misconfiguration puts your data security at risk, you are doing it wrong. That's all folks!

    1. Re:If a single port misconfiguration... by _Sharp'r_ · · Score: 4, Insightful

      They setup a server with a service configured to allow connecting on a default port and giving unencrypted/passwordless access to the entire file system.

      Yes, this is the definition of "doing it wrong".

      Any even minimal attempt to secure the server and service via OS hardening and/or taking the 2 minutes to configure rsync/rsyncd to use ssh as a transport would've prevented this issue. As rsyncd has used ssh by default for a while now, either they deliberately turned off all safeguards, or else they are running a _really_ old version of *nix which they haven't kept updated.

      --
      The party of stupid and the party of evil get together and do something both stupid and evil, then call it bipartisan.
    2. Re:If a single port misconfiguration... by _Sharp'r_ · · Score: 4, Interesting

      At the risk of replying to myself, I just went and looked and rsync has had using ssh as the default config for 13 years now...

      --
      The party of stupid and the party of evil get together and do something both stupid and evil, then call it bipartisan.
    3. Re:If a single port misconfiguration... by bravecanadian · · Score: 2

      You can say that about practically any security breach.

      Mistakes happen and the issue is that you'll never be 100%, particularly in a big, complex organization.

      If you're on defense, you lose. Therefore it is important to treat security as risk mitigation and assume you're going to fail so that you can detect and react to breaches.. hopefully before you lose your 200GB of data.

      Unfortunately, most companies don't treat security as an important function and under fund and undermine it.

    4. Re:If a single port misconfiguration... by sjames · · Score: 3, Funny

      All to frequently it starts when some upper manager who was once trapped in a room for 3 hours because he was baffled by the door knob is unable to access a perfectly standard resource. So, in order to avoid further confusing him, all security has to be removed...

    5. Re:If a single port misconfiguration... by Billly+Gates · · Score: 1

      At the risk of replying to myself, I just went and looked and rsync has had using ssh as the default config for 13 years now...

      PHB: It works. Leave it alone

    6. Re:If a single port misconfiguration... by Anonymous Coward · · Score: 0

      Somebody was troubleshooting something. They got the standard advice from the internets: turn off your firewall, antivirus, and security software and try it again. They did. Whatever they were testing worked. So they left it all off.

    7. Re:If a single port misconfiguration... by Neuronwelder · · Score: 1

      The same configuration for 13 years?! Wow!

    8. Re:If a single port misconfiguration... by Anonymous Coward · · Score: 0

      >If a single port misconfiguration puts your data security at risk, you are doing it wrong. That's all folks!
      Is your point that multiple layers of security would have caught this single-point failure early, or that there were insufficient safeguards?
      You can't protect against *determined* efforts by the sysadmins themselves to undermine security by leaving rsync wide open.
      Any firewall, intrusion detection system, egress filtering, etc. system in place would have been circumvented by the sysadmin to get it to work the way they wanted.
      They are doing it wrong, but not merely because someone made an accidental misconfiguration, but because the people involved have no awareness of security whatsoever.

    9. Re:If a single port misconfiguration... by Trogre · · Score: 1

      The link in your sig is dead.

      --
      "Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
    10. Re: If a single port misconfiguration... by Anonymous Coward · · Score: 0

      A place I worked, the edict came down from on high that all permanent marker pens were to be handed in. Turned out the big boss had tried, apparently without success, to wipe the boardroom whiteboard.

  3. SCIF by Infiniti2000 · · Score: 1

    After exposing the Sensitive Compartmented Information Facility (SCIF) plans, I'm betting that's the last government contract they work on.

    1. Re:SCIF by Mr+D+from+63 · · Score: 2

      After exposing the Sensitive Compartmented Information Facility (SCIF) plans, I'm betting that's the last government contract they work on.

      It seems to only show a black box room location from one facility. This is not necessarily info that is required to be secured. In reading the article, most of the information is pretty useless for any kind of attacker on the actual power systems, some of it is business sensitive. Its a security hole they need to plug in general, and certainly a good example of carelessness, but the article makes this out to be a much more significant breach that it is.

    2. Re:SCIF by aaarrrgggh · · Score: 1

      We are in the same field as them; we have contracts for non-disclosure that would ruin us if disclosed. While much of the information is quite mundane, the timing of access to information could have a few of our clients suing us quite quickly.

      While I get things being mis-configured in a small shop, it amazes me how lazy of a solution many companies have. (That reminds me... I should get a pen-test audit done...)

    3. Re:SCIF by Mr+D+from+63 · · Score: 1

      Thanks for the insight. Some companies do have a lazy solution, some have more robust ones yet overlook something simple or don't stay on top of change.

  4. Pen testing is good by steveo777 · · Score: 3, Interesting

    Pay someone to do even a light check of your network. You never know.

    Something very similar happened at an old employer. We did network and voice support for an auto dealer. Every month their long distance and international bills were unjustifiably enormous, but they didn't tell US about it, preferring to bitch at the phone company directly (company was horribly run, really). At some point or another they finally got fed up and told us they didn't want international calls to go out (this was the first thing we heard about the problem) and I turned it off for all but a select set of phone numbers. Over the next few months we get requests to turn off LD on all these extensions and back on. The boss is getting paid so when I get a bit angry about all the stupid switching around he doesn't want me to ask. We started looking into it anyways, and it turned out that one of their headless phone numbers was basically an open relay. The system had been set up by engineers that were long gone, so we just closed off the relay.

    However, someone noticed this and used the relay to call around... for about a year. Got thousands of dollars in free calling.

    --
    This sig isn't original enough, it's time to come up with something witty...
  5. Seems like by bobstreo · · Score: 1

    A comedy of stupidity.

    Did they even have a firewall?

    Who does the reviews and port scans for security changes?

    Who reviews the security postures of applications and services on the internal network.

    Power Quality Engineering, well one out of three, maybe.

    Here's a hint for configuring your security posture. Start with denying any connectivity in either direction. Adjust as needed.

    1. Re:Seems like by aaarrrgggh · · Score: 1

      A decade ago we did have "remote access" via ssh/sftp to the file server. We were caught off guard with one employee needing to work at a client site for a couple months. Terrible practice, but we only had ~8 people at the time and were saving up for the real firewall...

      We did at least use certificate login though, I guess. Not much of a salvation.

      But, anything in the last 5-8 years is really, really stupid. I understand how it happens, but it is still stupid as hell.

  6. Seems like a good reason to... by Gravis+Zero · · Score: 2

    move as many systems as possible off the power grid. Blackouts need not cripple our civilizations.

    --
    Anons need not reply. Questions end with a question mark.
    1. Re:Seems like a good reason to... by Sindar+By+Choice · · Score: 0

      Blackouts need not cripple our civilizations.

      Unless you're playing the Ottomans.

    2. Re:Seems like a good reason to... by Mr+D+from+63 · · Score: 1

      move as many systems as possible off the power grid. Blackouts need not cripple our civilizations.

      This wasn't a grid system. This was a business LAN. Of course the headline is crafted in such a way as to hope someone is confused and assumes actual electrical equipment/systems were involved.

    3. Re: Seems like a good reason to... by Anonymous Coward · · Score: 0

      So we assume the rest of their security is great and there's no way for outsiders to control the grid over the internet? How about all the other utility companies?

  7. Why don't they fly creimer over? by Anonymous Coward · · Score: 0

    He's a miracle worker that will fix the problem WHILE you're explaining it to him, and then he'll clean your closet, finish three months early, and write fifteen eBooks about it!

    1. Re:Why don't they fly creimer over? by Anonymous Coward · · Score: 0

      You sound bitter, lollipop.

    2. Re: Why don't they fly creimer over? by __aaclcg7560 · · Score: 1

      All while losing 13 pounds in 3 months.

      Nothing worse than a fat guy consistently losing weight.

    3. Re: Why don't they fly creimer over? by Anonymous Coward · · Score: 0

      The antibiotics cleared that up.

  8. Fake News for Fake Nerds by Anonymous Coward · · Score: 0

    This is Slashdot. Nobody knows what a port is.

    TFA is shit. Tits or it didn't fucking happen.

    1. Re:Fake News for Fake Nerds by Anonymous Coward · · Score: 0

      What is a port?

  9. I'm sorry for being blunt but "a port misconfiguration" should not even be theoretically possible on mission critical devices. They should not be accessible directly (and indirectly as well) from the world wide net.

    1. Re:IMO by Anonymous Coward · · Score: 0

      Back in 2003, Disney released the video game Tron 2.0, in which one of the levels involved bypassing a mission critical firewall by draining the battery of a PDA.

      Today in 2017, I own an IoT firewall which fails open if I port scan it. This future sucks.

    2. Re:IMO by Mr+D+from+63 · · Score: 1

      I'm sorry for being blunt but "a port misconfiguration" should not even be theoretically possible on mission critical devices. They should not be accessible directly (and indirectly as well) from the world wide net.

      This was a business LAN folder access, not a mission critical device or system.

    3. Re:IMO by PPH · · Score: 1

      This was a business LAN folder access, not a mission critical device or system.

      This

      But utilities and industrial businesses are notoriously paranoid about 'terrorists' (a.k.a. the general public) gaining access to configuration details of their systems. A good part of this is due to to the Dept of Homeland Security throwing money and resources around after 9/11 at poorly understood problems. Never mind that you can discover pretty much everything you need to know about a power grid using Google Maps imagery. Or just driving around, looking at power lines.

      --
      Have gnu, will travel.
  10. researchers or data thieves? by Anonymous Coward · · Score: 0

    Why these upguard "researchers" download 205G of data :D that's way more than proof of concept.

  11. Just doing it vs. doing it right by ErichTheRed · · Score: 3, Insightful

    Mistakes happen, and this was a BIG one, but I'm of the opinion that the pendulum is too far over in the "get it done now" realm and needs to swing a little back toward the middle. I'm in systems integration/engineering, and there is a relentless push for even established companies to be fully buzzword compliant, move towards DevOps, and basically remove any barrier whatsoever to putting stuff out there the second a developer checks in a code change. A lot of this is good, but unless your developers are accomplished system admins who actually know how things work under their layer of code, the Ops people do need to work with them to ensure they're not doing anything crazy or taking shortcuts.

    The whole pendulum-swing thing is being driven by web startups and the ever-raging battle beyond Devs creating Apps! and Ops people maintaining all that LUDDITE infrastructure. When the Ops people were allowed to fully drift over to being the Department of No, the Apps! developers just walked around them and built their Apps! in the public cloud with a credit card. At the same time, all the web startups were starting up with zero legacy anything, so everything was built to that cloud model and it's felt that Devs don't need to know about infrastructure anymore.

    I have one foot in both worlds (agile cloudy stuff and infrastructure-centered stuff) and I feel that items like this are going to pop up more often given the constant push to move move move or else. Stuff like putting confidential customer data on a public S3 bucket, leaving a key system fully exposed to the Internet because your cloud's IaaS platform doesn't stop you, etc. Once the Second Dotcom Bubble pops, I'm hoping we take the good stuff that DevOps gives us and apply a sane schedule to it, letting people who actually understand the nuts and bolts of infrastructure, ports and protocols weigh in also.

    1. Re:Just doing it vs. doing it right by Neuronwelder · · Score: 1

      You're right. "Get it done now" This is why we have defective cars catching fire. Buildings with no sprinklers killing people. And no low center or gravity, multi chamber, thicker metal, train cars that carry oil. The tragic oil spill in the Gulf of Mexico that killed those people on the platform because they didn't want to take time to put a safety valve on.. etc.

  12. Arrested by Billly+Gates · · Score: 1

    I am surprised the researcher wasn't arrested as this is what corporations typically do these days according to what I read here

    1. Re:Arrested by link-error · · Score: 1

          "Researcher".... downloaded 205GB of data? Agreed, they should be arrested. I can see downloading 1 or 2 files and getting a listing or something, but 205GB is well beyond documenting a vulnerability.

      --
      -Unresolved symbol? Byte me!
    2. Re:Arrested by mysidia · · Score: 1

      I don't agree.... Researcher finds some Open public Rsync server publishing files on the internet. Initiates a bulk download to fetch everything being made available so they can take a look at what this is. 205GB is a drop in the bucket these days --- it's an insignificant amount of space assuming a decently fast Gigabit+ network connection and relatively modern PC (My new PC has two 4000GB hard drives in it, so 205 GB would be about 3% of my storage).

  13. How is this a port misconfiguration by Nieriko · · Score: 1

    Isn't it more like a service misconfiguration ?

    Either the port should be closed in the firewall or if needed should be left open like it was, using an IP whitelist (which could be implemented at the firewall). Rsync should have been set up to use authentication in any scenario.

  14. Rsync without SSH? by Trogre · · Score: 2

    I just don't understand how anyone in 2017 could use rsync without going through an SSH channel, preferably with keys if it's an automated process.

    Anyone care to explain?

    --
    "Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
    1. Re:Rsync without SSH? by Anonymous Coward · · Score: 0

      Here are two scenarios for doing rsync without SSH:
      1) I have an air gapped network only accessible to people authorized to access the equipment. My company policy states that if I use passwords or encryption keys they must be changed every 90 days. Thus configuring anonymous read-only rsync without SSH becomes easier with no threat to the server itself. Later, after I leave the company, someone decides that the air-gap slows response time and decides to connect the control network to the corporate Intranet (which is itself attached to the Internet).
      2) I have a secure control network that is only accessible through a firewall and an IDS from my company Intranet. My corporate policy precludes the use of SSH because using SSH would encrypt traffic seen by the firewall and IDS, thus preventing their protective measures from activating. Thus all traffic that passes through those devices must be unencrypted. Later, after I leave the company, someone decides that the control network needs to also be accessible from the Internet to expedite issue resolution.

  15. Idiots by fnj · · Score: 1

    Bwahaha, these monkeys set up rsyncd naked on port 873? And all these stupid commenters are bleating about firewalls when using rsync with ssh is perfectly secure.