Feds Investigating Water Utility Pump Failure As Possible Cyberattack

SpuriousLogic writes with this quote from CNN: "Federal officials confirmed they are investigating whether a cyber attack may have been responsible for the failure of a water pump at a public water district in Illinois last week. But they cautioned that no conclusions had been reached, and they disputed one cyber security expert's statements that other utilities are vulnerable to a similar attack. Joe Weiss, a noted cyber security expert, disclosed the possible cyber attack on his blog Thursday. Weiss said he had obtained a state government report, dated Nov. 10 and titled 'Public Water District Cyber Intrusion,' which gave details of the alleged cyber attack culminating in the 'burn out of a water pump.' According to Weiss, the report says water district workers noted 'glitches' in the systems for about two months. On Nov. 8, a water district employee noticed problems with the industrial control systems, and a computer repair company checked logs and determined that the computer had been hacked. Weiss said the report says the cyber attacker hacked into the water utility using passwords stolen from a control system vendor and that he had stolen other user names and passwords."

Comments

SCADA vulns

[sl4shd0rk]

SCADA systems were sold en masse under the presumption that they were "secure" because they were not connected to public networks. It will be interesting to see which entities did, or did not, follow their policies. Stuxnet was a USB infection but it was still able to route over the internet to phone home. I'm going to bet that a lot of SCADA networks are implemented to allow egress packets. It will be interesting to see how many SCADA systems are actually "isolated".

Re:SCADA vulns

[Anonymous Coward]

I worked for a Utility in the early 2000's..I was on the post-9/11 security team that had to investigate and close loopholes for that utility. Many sites had interconnected the SCADA systems with the corporate network for GIS information. We were hard pressed to find adequate solutions that would meet the requirements that the federal government set at that time; as the engineering staff didn't want to give up the real-time GIS information they got from the SCADA systems.

Re:SCADA vulns

[mlts]

The ironic thing, there is a secure way to get GIS info out, although it isn't the fastest method. I did this on a lab network that needed to be air-gapped from everything else:

1: Build two machines, each has a NIC, and both have a serial card ($60 from NewEgg for a PCIe to Serial.)

2: Build a custom cable with the RX wires cut, so data only goes one way. I did this so an intruder has no chance of being able to send anything to the box on the secure network, much less create a SLIP or PPP connection.

3: Configure one box on the secure network. It scrapes input from the embedded boxes, formats it (so stuff from one device is marked as such so it can be told apart from a different one and to help keep both machines in sync), then pushes it over the serial device.

4: The other box is configured to passively take what comes over the serial port, un-format it (so stuff from one device goes to one web server, stuff from another device gets E-mailed to an admin, alerts get set if something is wrong, etc.)

The result of this is being able to get reports from the embedded boxes on a real-time basis, but without any way of a remote intruder ever getting on the network. Since the physical serial cable cannot send any data to the machine on the embedded network, it would take a physical attack in order to compromise the boxes.

I'm sure there are faster ways to get data across a cable one-way, but this was ideal, as the data obtained was not much, and the latency of the multiple steps to shoot it to a box, stuff it across a serial pipe, then on the other side, send it where it needs to go was just fine.

AWESOME

[WindBourne]

That is possibly just a kid playing, however, it could be somebody learning. The nice thing is that it has now been detected. Perhaps it is time to push not just security, but to insist that the parts be western or better yet, American made. Seriously, this is infrastructure that should be local to friendly nations. China is hard at work to make sure that they have the ability to import zero food as well as all of their equipment is from local sources. In doing that, they claim national security. Makes sense. But we should be doing the same.

Could be something incredibly simple

[slewfo0t]

As a controls engineer, I program these type of systems all the time. A simple incorrect setting for when the pumps turn on and off (Lead,Lag) could cause this type of problem. It could literally be a new operator that fat fingered a parameter in the SCADA system. To hack these systems requires specific knowledge of exactly what kind of control architecture is in place at the facility and then having the appropriate software to gain access to the control system. Not that this type of hack cannot be done, but it does require specific knowledge. This really sounds like operator error to me.

Re:Perhaps Not All Remote Management Worth The Ris

[Mr. Freeman]

Perhaps it's time that people realize that a lot of things do need to be connected to external networks and that "air gap them" is simply a cop out response equivalent to saying "use a typewriter".

Yes, some things should be air-gaped, nuclear gas centrifuges come to mind. However, many industrial control systems need to report information over the internet. Remote pumping stations, unmanned power distribution centers, etc. Having a lot of data is not simply a convenience. This data allows engineers to troubleshoot failures, predict future failures, and adjust systems for optimum efficiency.

What's really necessary is for some kind of device that will communicate the data to remote places, but refuse to pass any messages from the outside onto the control system. I don't know how difficult this is, but it's certainly harder than "air gap it". On the other hand, this solution actually addresses the problem.

Re:'Been in the water/SCADA industry for 10 years.

[Animats]

What I'm seeing lately are water operators, IT people, and system integrators who are overzealous when it comes to connectivity and all the "neat" things that can be done remotely via technology.

Yes. Read "Access Your Embedded Controller with Ease through a Web Server", from Texas Instruments, which ought to know better. "The designer should also make it as easy as possible to change the settings on a piece of equipment, reconfigure its operation, or fine-tune the system. The more intuitive and explicit that activity is, the more likely the result will be what the operator desires. Losing the instruction manual can seriously impair the user's operation of many systems."

What that paper describes is a family of embedded controllers with a web server in each controller and no security. What's wrong with this picture?