Slashdot Mirror

← Back to Stories (view on slashdot.org)

Evaluating Or Testing Utility SCADA Security?

Posted by Soulskill on from the i-hear-stuxnet-has-a-nice-admin-suite dept.

EncryptedBit writes "I am a local elected official involved in bringing new water and waste water treatment plants online in a small town. The new plants will incorporate SCADA, which can be used to change operational aspects at the plants, up to forcing a shutdown or changing operational parameters. Can any Slashdotters recommend ways to make sure it is secure? Any testing recommendations? The operational engineers are oblivious to security and SCADA is a new factor, so this concerns me. Any pointers would be appreciated."

37 of 227 comments (clear)

  1. Don't put it on the Internet! by Anonymous Coward · · Score: 5, Insightful

    Seriously keep it on it's own separate network.

    1. Re:Don't put it on the Internet! by mattt79 · · Score: 2, Interesting

      ++ Absolutely correct!! SCADA should be considered a Level II control system, and subject to the same degree of security as the devices that are actually controlling the valves!

    2. Re:Don't put it on the Internet! by mail2345 · · Score: 2, Funny

      But I want to be able to blame "hackers" when I mess up.

    3. Re:Don't put it on the Internet! by j35ter · · Score: 4, Insightful

      Just keep away from Simatic and Wonderware! Better yet, keep away from ANY kind of winXP implementation. Oh yeah, don't forget to *physically* lock all USB ports and external media. Of course, you should make sure that there is no network connectivity with the outer world. And I mean none! No multihomed machines, no VLAN's etc. And, of course, if you see all alarms going off at once...run :)

      --
      Delta-Mike November Bravo Tango
    4. Re:Don't put it on the Internet! by Bigjeff5 · · Score: 4, Insightful

      Good safe practice for separating a process control network from the internet is something like: internet > corporate network > buffer network > process network. Completely separating it is not advisable, because it can actually make it harder to administer and protect (updates, antivirus, etc). It's an option though if you are diligent with sneakernet updates and whatnot.

      The network your SCADA system runs on should never, ever have direct access to your corporate network or the internet, your buffer network should never have direct access to the internet, and your corporate network should never have direct access to your process network. Be stingy about what you allow through the firewalls at each layer.

      Basically when you need SCADA data outside the process network, you send it to the buffer network, and from there it is accessed from the corporate network. All controls should only be managed from the SCADA network (i.e. don't set something up so that it can be managed from the corporate network).

      Separation is key. As others have said, SCADA networks need a lot of open access to the various systems they control in order to function efficiently, so within the network things have to be practically wide open. The only real option is to protect yourself with layers to make it nearly impossible for anything you don't want to access the system.

      --
      Security is mostly a superstition... Avoiding danger is no safer in the long run than outright exposure. - Helen Keller
    5. Re:Don't put it on the Internet! by Anonymous Coward · · Score: 5, Insightful

      As a security expert who has audited SCADA systems, I must amend the above remark. If you ask them, "I'm here for security, is it connected to the Internet?", they will say "no". If you instead ask them, because you are concerned, "in an emergency, how an administrator can access it remotely", then will tell you the series of systems that will allow you to connect in- firewalls, vpns, and usually a last hop Citrix remote desktop session to the SCADA software itself, which is often Siemens, and is run in a VM. When you tell them to take it off the Internet, they will put your request in change control, and find a way to get rid of you, or if you're a politician, to buy you out. Usually the people running these things have no ability to think adversarially, so they consider something that is several levels removed from the Internet to not be Internet connected, even though it is. They may even tell you that it has its own leased network, run over Frame Relay leased from a telco, which is quite common. This is also internet connected, as the ISP can get pwned, and the frame relay stuff has a management network that is on the ISP's LAN. I've done security for a Fortune 50 ISP as well.

      The short answer is, every SCADA system in the Americas is Internet connected, and no one has the balls to tell them to stop. They will only hire people to audit them who will put on kid gloves and play by their rules, and they refuse to take advice from their vendors, who they pay to be compliant. A security consultant lecturing a SCADA client on security measures is like a temp secretary lecturing a CEO on spelling. It's an event that always ends in a raised eyebrow and a prompt firing.

      Every nuclear reactor in the united states is internet connected. I've seen them. I'm certain. Being a whitehat pen tester these days is like being a turned out whore- you know you're not helping anyone but it's too late to go back.

      Posted anonymously for obvious reasons.

    6. Re:Don't put it on the Internet! by Anonymous Coward · · Score: 5, Insightful

      I almost forgot. Don't ask them, "is your network secure", because they will say yes, even though there's no such thing as a secure network. The appropriate question is, "how defensible is your network" and "what is your dwell time", i.e., what do you have in place to stop intrusions and how long do hackers usually last on your network. But the best question to start with is "what's your incident response budget", which they will brag about, i.e. how much money they spend on getting hackers removed when they're owned. Start there, get them to tell stories about the last time they got hacked. Then you don't have to listen when they start telling you about how they're "secure".

    7. Re:Don't put it on the Internet! by j35ter · · Score: 2, Insightful

      IMHO the standard multi-layered network security approach does not always satisfy the security needs of a process management system.

      you really don't want your windows machines to update themselves: NO NO NO NEVER!!!
      Just imagine the operators' dumb face when WinXP goes into a restart during some critical operation, this could seriously ruin a plant managers Tuesday!

      One important aspect is the internal security, people are lazy, and if you allow the to use a USB drive somewhere (management demands reports in electronic form!) you can be sure that someday some admin will need an aspirin and a new job!

      The best approach is to have a 2 tiered network: PLC's and SCADA ... and basta!

      --
      Delta-Mike November Bravo Tango
    8. Re:Don't put it on the Internet! by noidentity · · Score: 2, Insightful

      Usually the people running these things have no ability to think adversarially, so they consider something that is several levels removed from the Internet to not be Internet connected, even though it is.

      Indeed, everything is connected to the Internet. It's always just an Ethernet cable away. Send the right commands over a phone line, and that cable will be connected for you.

    9. Re:Don't put it on the Internet! by KUHurdler · · Score: 2, Insightful

      Seriously, the original poster needs to hire someone that is an industry expert in Utility Communications. Disclaimer: I work for this division, but we do this stuff every day and it's not the kind of thing you want to try doing on your own. In the worst instances, there are no second chances.

      --
      Fix Your Own TV - RiddledTV.com Avoid the Landfill
    10. Re:Don't put it on the Internet! by John+Hasler · · Score: 2, Interesting

      > Except we all want cheaper stuff. And that means using the lowest bidder.

      It means using the lowest qualified bidder. Do you think you'd get better quality from the highest bidder?

      --
      Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
    11. Re:Don't put it on the Internet! by j35ter · · Score: 2, Funny

      Now *that's* what I call bareback :)

      --
      Delta-Mike November Bravo Tango
    12. Re:Don't put it on the Internet! by Anonymous Coward · · Score: 5, Interesting

      I've also done SCADA system security on the water plants of nuclear reactors, and can confirm that all the ones I've seen have been connected to the Internet. One time I saw a Junxion box and a AP just plugged into the core switch for the control network. It wasn't that crazy given that the Junxion box had its power supply in the manager's office and you can't get within miles of the place without having rifles shoved in your face, but it was still pretty surprising to see it.

      Another site uses default passwords for everything and they have a dial-up modem which drops you right into a login prompt on one of the control hosts. You have to call them to get them to plug it in first, though, so they haven't had any problems. Unlike in Hackers, they don't plug it in for any schmuck who asks; you have to give a CAC ID and it has to match the schedule maintenance roster, otherwise the FBI gets called.

      The really important stuff isn't really under control of a computer though, it's all in some PLC somewhere and there's only one guy who understands the control logic anyway. I'm not too worried about someone breaking into those networks. If anyone tried to do anything bad, it's much more likely that they're just going to break something unintentionally while learning how the system works and trigger an investigation, not create a meltdown.

    13. Re:Don't put it on the Internet! by darkpixel2k · · Score: 2, Funny

      But but but.... I have the god given, constitution granted inalienable right to play Farmville, have my desktop covered with widgets and surf the web while at work! By god, I'm an 'merican!! What the hell do you expect me to do while I'm at work, WORK?!? I don't get paid to stare at those blinking pixels and dial thingamabobs all damn day.

      I think you're confusing us "'mericans" (people residing on the American landmass) with a uniquely retarded subset of those people called 'Government Employees'.

      --
      There's no place like ::1 (I've completed my transition to IPv6)
    14. Re:Don't put it on the Internet! by crossmr · · Score: 4, Informative

      The short answer is, every SCADA system in the Americas is Internet connected, and no one has the balls to tell them to stop

      That's incorrect.
      I used to build SCADA systems and we often included a separate "work terminal" that was connected to the corporate network for workers to access anything outside they needed. It was not connected to SCADA and the SCADA system was not connected to the main corporate network or the internet.

    15. Re:Don't put it on the Internet! by denobug · · Score: 5, Informative

      Wonderware InTouch happens to be one of the most popular flavor of local supervisory system platform. There are very few supervisory system NOT implemented with Windows platform. Even DCS nowadays runs on them as well.

    16. Re:Don't put it on the Internet! by phantomcircuit · · Score: 2, Interesting

      http://www.blackandveatch.com/Markets/Telecommunications/Utility_Automation/Default.aspx I stopped reading there.

    17. Re:Don't put it on the Internet! by thegarbz · · Score: 2, Insightful

      Typical slashdot answer. When most of the worlds popular SCADA implementations are run on a windows based system, and many which are not currently are slowly moving that way, what are you left with? Keep it on it's own network completely isolated depending on the plant may also run afoul of laws requiring off-site historical databases. It'll also anger ops engineers no end when they can't easily aggregate and manipulate the historian data on there computers which gives you another problem, do you let an external party directly onto the process control network, and allow them to bring and install the software they need?

      My advice for the OP, talk to the manufacturer. Many SCADA implementations I have seen from reputable companies come with a very hardened network design. - Only ever allow one way communication from the control network outside. - Have a computer on the outside historize the data so other's have limited access to it. - On the process network itself ensure the boxes are under lock and key. If the operators have access to a physical mouse and keyboard then make sure they can't leave the application. No USB sticks, nothing! - For shits and giggles and time of need put a kill switch on the firewall that lets the data out of the control network.

    18. Re:Don't put it on the Internet! by anchovy_chekov · · Score: 2, Interesting

      As someone who used to deal with SCADA systems all the time, I shudder whenever I hear the words "security" and "SCADA" in the same sentence.
      Back in the bad old days - when SCADA was driven by OPC - you had to turn off security just to get things like DCOM to work properly. It was scary and wrong, but you had very little choice. Talking to friends in the industry it doesn't seem to have gotten any better.
      The real problem stems from engineering departments losing political clout within business that are primarily engineering concerns. The rise of the IT department, with its alignment closer to the management end of things (as opposed to the end that does the work) meant that engineers had to compromise their networks to fit in with corporate policy.. hence we end up with SCADA systems on the corporate network and other such craziness.
      Once I was working in a factory as a consultant and I noticed a set of blue cables running behind I beams. I made a mention about them and next thing the head engineer whisked me around the corner, told me to be quiet.. and the explained that that was the engineers "own" network.. one they had to run to keep the factory going and that they would appreciate it if I kept my mouth shut in future.
      That kind of madness is why we're where we are now, with far too much critical infrastructure available over the internet. And I would heartily recommend the model that was presented to me.. keep it on its own network.. and don't tell anyone from IT.

    19. Re:Don't put it on the Internet! by thegarbz · · Score: 2, Interesting

      Did you miss the obvious problem with your one statement? "In an emergency". I'm not sure what you define as remote administrations but those same type of terminals we have are read only, and yes you need to jump through all those hoops. But "In an emergency" events have occurred several times on the site. The ability for experts located in another country (wonders of the multinational enterprises) to help debug live has saved us from turning our site into a big hole in the ground several times. The number of times we've suffered a direct internet attack on the other hand doesn't need a hand to be counted, and if it did, careful network and SCADA integration goes a long way.

      People resist taking away their ability to remote view a control system because at times it can be damn bloody useful. Alternatively we could just move back to 5-15psi pneumatic loops. Can't hack that.

    20. Re:Don't put it on the Internet! by Mysteray · · Score: 2, Insightful

      While there are some truly compelling advantages to KISS/dumb systems, there's usually no reason that the system has to have a painful interface. The UIs for medical equipment, for example, are highly polished and tested for ease-of-use. It's just IMHO dangerously stupid to put plug-n-play networking, USB, or Wifi on a drug infusion pump.

      If plug A fits into socket B, somebody's going to plug it in. Drop some infected USB drives in the parking lot, somebody's going to stick one in a USB port behind your firewall. Have an open USB port, somebody's going to charge their MP3 player from it. If it has a web browser and connectivity, somebody's going to surf with it.

      Power-grid-like critical systems need to export their data from a DVD burner, not over the network. This can happen even several times a day. If this causes problems due to the latency it introduces into some spreadsheet-based workflows, then the system needs to be fixed. It's horribly broken if desktop office applications have been allowed to creep into the control loop!

      Just my 2c, I don't expect everyone to agree with it.

  2. From what I understand by Runefox · · Score: 4, Interesting

    There isn't much to do with SCADA regarding security - The systems themselves are inherently insecure, the extent of it reaching only so far as default passwords that are scarcely ever changed and the requirement to have a compatible console. If you're connecting these devices to the internet in any way, then you're opening yourself up for a world of hurt. The best security is physical security, with no link to the outside world except in closed, site-to-site communications. I'm by no means an expert, but having heard experts speak about the subject and with some limited experience of my own, there really doesn't seem to be any better way the way things are.

    --
    Screw the rules, I have green hair!
    1. Re:From what I understand by Da_Biz · · Score: 3, Informative

      The systems I work on feed data to our SCADA systems. The entire network is completely walled off from the Internet, and even connectivity to our internal (non-operations) network is mediated by extremely secure bastion hosts.

      I can understand that there may be a need for some access (e.g., system pages an operator to send a warning or emergency message), especially as this is a small town. Keep these sorts of connections absolutely to a minimum, and wrap several layers of security around it.

    2. Re:From what I understand by postbigbang · · Score: 2, Insightful

      "Bastion hosts" are an oxymoron. Every device on a network needs the best self-protection set at the highest possible standard. Layers of security are only incrementally effective. It takes only one bot to bring you down. Firewalls, although they sound impressive, are ineffective when users plug in infected flash drives, or other media. You have to distrust every device on your network without exception-- even routers and switches can be broken into using fuzzing techniques. Continually hack yourself, ethically, using the latest techniques. Then do it again. Be afraid, be very afraid. If you're not afraid, then I'm deeply afraid.

      --
      ---- Teach Peace. It's Cheaper Than War.
    3. Re:From what I understand by NewbieProgrammerMan · · Score: 4, Insightful

      There isn't much to do with SCADA regarding security - The systems themselves are inherently insecure...

      As somebody that worked at a SCADA software company for a few years, and saw (1) the skill level of the core development team and (2) what customers did with our systems, I heartily endorse this viewpoint.

      --
      [b.belong('us') for b in bases if b.owner() == 'you']
  3. Pointers by PiAndWhippedCream · · Score: 2, Funny

    0x00000047, 0x000008C4, and 0x00000A08 All hold awful vulnerabilities, make sure no one has access to them.

  4. Do NOT connect to the Internet! by RedLeg · · Score: 2, Informative

    It's simple......

    Do NOT, under any circumstances, connect the SCADA systems, including workstations which can control or monitor them, to anything which touches or has access to the Internet. Make SURE that your control and monitor workstations have current AV in place. Do NOT connect them to the net to update the AV, figure out how to do it with sneakernet.

    Further, make SURE you use RFC 1918 addressing for the SCADA systems so that they are not readily routable to the 'net.

    Map the interfaces, and have a AAA (Authentication, Authorization and Accountability) strategy for each. Log EVERYTHING.

    If you use a carrier to link remote sites into a WAN, make them prove to you that their pipes are clean and secure.

    Have Fun......

    Red...

    1. Re:Do NOT connect to the Internet! by Jah-Wren+Ryel · · Score: 2, Interesting

      Do NOT, under any circumstances, connect the SCADA systems, including workstations which can control or monitor them, to anything which touches or has access to the Internet.

      When that's not possible due to management pressure, there are options that are better than just giving in and connecting systems up to the internet.

      The simplest of such options is a "data diode" -- its a device that physically only permits data to flow in one direction. For example, optical network connections have a transmit fibre and a receive fibre. A data diode would physically connect just one fibre.

      Implementing a data diode - say to run your monitoring software on an internet connected PC so as to send status updates via SMS to engineers' phones - can take some effort in order to get all the necessary software to work in the one-way environment. But it is a way to get data out of your SCADA system without having to worry about malicious attacks coming in on the same connection. At worst your monitoring system gets fuxxored, but the SCADA stuff continues to run unmolested.

      Here's one data diode product with an emphasis on SCADA, it was just the first one that came up in google, there are many such products out there:

      http://www.datadiode.eu/products/scada

      --
      When information is power, privacy is freedom.
    2. Re:Do NOT connect to the Internet! by jeff4747 · · Score: 2, Interesting

      Make SURE that your control and monitor workstations have current AV in place. Do NOT connect them to the net to update the AV, figure out how to do it with sneakernet.

      Oblig XKCD

      If you're putting AV on it, you're doing it wrong.

  5. Attacking and Defending the Grid by andyverbunt · · Score: 2, Insightful

    I recently went to a Belgian OWASP meeting where Justin Searle talked about "Attacking and Defending the Grid".
    This guy knows what he's talking about. Among other things, he also mentioned SCADA vulnerabilities.

    I recommend you contact him or his company for professional advice (I am in no way affiliated with him or his company - just thought you might be interested)

    If you google for the subject of this reply in combination with "OWASP", you'll find more info about the talk.

  6. VPN by AmericanInKiev · · Score: 2, Informative

    I'm working with an international firm on Scada - we use a VPN to provide a secure private network.

  7. SCADA and Security are not yet integrated by PureFiction · · Score: 3, Insightful

    SCADA systems are not designed, implemented, or operated with network and application level security concerns in mind.
      (Usually. The exceptions know who they are :)

    Your compensating control is physical security to limit access to SCADA elements and programming. It costs more, but you have no sane alternative.

    And before you get too cocky about that restricted air gap, consider Stuxnet turning such a strength into a weakness for exploit. At some point SCADA systems will be security conscious; that day is not today...

    1. Re:SCADA and Security are not yet integrated by ScrewMaster · · Score: 2, Insightful

      And before you get too cocky about that restricted air gap

      This is the fundamental problem right here. The kind of mind that thinks airgap and be done with it is usually the kind of mind that doesn't think about security, and will fall for exactly the likes of Stuxnet. This illusion that everything is fine because there's no internet can often be worse than a well designed remote access system built from the ground up with security in mind. They do exist and the ability to remotely see what is going on can pull you out of the shit, or put you in the shit depending on how it's implemented.

      The fundamental problem is that good security is a process, not a state, and you cannot have good security by any fire-and-forget solution.

      --
      The higher the technology, the sharper that two-edged sword.
  8. Re:Check with Ahmadinijad? by mystik · · Score: 2, Insightful

    Remember Suxtnet? Not too long ago?

    It spread by usb drives, which Gleefully jump the "air gap".

    It's slightly more complicated than simply keeping an air gap, and probably requires the consultation of someone who's had experience securing these types of networks.

    --
    Why aren't you encrypting your e-mail?
  9. Re:Scared yet? by fuzzyfuzzyfungus · · Score: 2, Interesting

    I find the fact that it is being asked commendable: if TFS is to be trusted, an elected official is attempting to learn more about security to make sure he can oversee a project properly. That sounds just ducky to me, and well above the status quo.

    Now, the fact that said official appears to have strong reason to distrust his engineers, and no ready internal supply of expertise, suggests that any script-kiddie with a copy of Telnet and 10 minutes will be able to quite literally put some poor town up shit creek without a paddle once the project goes live; but the fact that an elected official has forseen that possibility and is asking questions down at the geek club to see if there is a way to head it off seems like a good thing...

  10. Re:At least you are aware of the risks. by j35ter · · Score: 2, Informative

    Same thing in steel mills (construction material), seamless pipe manufacturing plants and petrochemical plants in the middle east...oh, and they were just on a different *logical* network than the corporate machines. My former employer's (industrial automation) SOP is to hook up a WEP (yes!!!) encripted AP's on the net so that their specialists had access to the network from everywhere within the plant...

    --
    Delta-Mike November Bravo Tango
  11. I too work with SCADA systems by jake-itguy · · Score: 2, Insightful

    The town I work for has a SCADA system for it water treatment plants and lift stations. A lift station pumps sewage to the wastewater treatment plant. SCADA (at least in my town) has two main components. The first component is a control station which is a Windows XP SP2 PC. A read-only monitoring terminal is also a Windows XP SP2 PC. The second components are several small boxes inside cabinets to which various sensors and radio links are attached. Each lift station, water treatment plant, pump house, etc. has a SCADA cabinet with the small box in it. The sensors are usually RS-232 or RS-483 and connect via RJ45 adapters to the designated ports. A Radio link is in each cabinet. Each SCADA box has an ethernet jack to connect it to a network. The lift stations and pump houses don't have a network connection back to the town's network so those stations are fairly secure.

    When I started at this job the SCADA system was on the same network as the town's PCs. I fixed this by moving SCADA to its own network with no internet access. It took several days and alot of cabling (the remote terminal was the hard part) but I did it. Two weeks later there was a problem and the head water person could no longer remotely access the system to fix it. I compromised and allowed VPN access to the SCADA system. A totally non-Internet accessible SCADA system is impossible. Even if you have someone monitoring the system 24/7 on site so no VPN access is needed, your Frame Relay, T1, or other connectivity options are certainly internet accessible.

    Next we have 9600 baud radio links from each remote station back to the main SCADA control station. I have no way of knowing if the information over these links is encrypted or not. Siemens says the information is encrypted but I can't verify it.

    Siemens also loves XP SP2 and refuses to support us if I install XP SP3, patches, or anti-virus on the system. I can't even turn on the windows XP SP2 firewall. Siemens also seems to love Symantec PCAnywhere. Every PC that has Siemens software has Symantec PCAnywhere installed it. Versions range from 10 to 12. We just had a third SCADA PC installed and it is still XP SP2 and PCAnywhere 12, not even 12.5.

    The best I could do was to physically isolate the SCADA systems to their own network. Allow VPN access only to the control station. I installed RealVNC server on the control station and put a password on it. I setup a laptop with VPN client software and the RealVNC client. So the user connects to the VPN enters a username and password (password changes every 6 months and it is at least 10 characters long, mixcase alphanumeric password), launches the VNC client and enters the VNC password (not the same password as above but uses similar specs).

    I look forward to reading the rest of the posts.