Slashdot Mirror
Communications Protocol Leaves Power Grid Vulnerable
mspohr writes "The NY Times has an interesting story about a pair of researchers who 'discovered that they could freeze, or crash, the software that monitors a [power] substation, thereby blinding control center operators from the power grid.' These two engineers wrote software to test for vulnerabilities in the control systems of electrical power grids which use a protocol called DNP3 to communicate with sub-stations. They first tested an open source implementation of the protocol and didn't find any problems. They were worried that their software test wasn't adequate so they started testing proprietary systems. The broke every single one of the 16 proprietary systems they tested initially and found nine more systems vulnerable in later testing. They were able to install malware and also found firewalls ineffective. The pair reported this to the Department of Homeland Security's Industrial Control Systems Cyber Emergency Response Team, I.C.S.-C.E.R.T. and didn't get much of a response. It's scary that our electrical grid is so vulnerable and there doesn't seem to be much urgency to get it fixed. A few patches have been issued, but who knows if the systems have been updated?"
What are the odds that our best friends already have botnets ready to take our grid down on command?
Excuse me while I got get a few solar panels.
our blinded overlords.
It is not like this is a new issue. Fire all IT managers who were responsible for not doing penetration testing, including the ones at Homeland Security.
If you do NOT hold managers responsible then they are just lifers waiting for their pension!!
I worked for a fellow who'd previously done some work on power grids. He was aware of these problems in 2005 or earlier. I'm pretty sure these problems were also published in the 9/11 comission's report. But I don't think patching holes in power grid controls provides enough theater to keep people scared, so it hasn't been done.
Their first mistake was assuming that the Department of Homeland Security actually cares about homeland security. Department of Homeland Control would be a better, more accurate name.
It's scary that our electrical grid is so vulnerable and there doesn't seem to be much urgency to get it fixed.
Sure - scary to you, scary to me, scary to the old lady down the road.
You know who it's not scary to? The NSA, CIA, and all other clandestine TLAs that profit from allowing harm to come to American citizens.
Remember: the CIA had solid intel about the 9/11/2001 terrorists, but did nothing to stop them; same goes for the Boston Bombers. The more Americans that they can allow to be injured by "terrorists," the fatter their budgets grow.
Stopping terrorist attacks is the last thing anyone in the federal government wants to have happen. THAT is fucking scary.
An enigma, wrapped in a riddle, shrouded in bacon and cheese
Don't electric utilities maintain private communications networks for their critical infrastructure?
Except that open source... oh, I see what you did there....
some karma... and kinda lukewarm about it.
If history is any guide, the managers of these systems are trying to find ways to prosecute the researchers for their actions. It's fairly standard to classify security testing methods as attacks (since that's in effect what they are), and publishing the problems is generally considered telling the "terrorists" how to attack the systems.
But this is about what should be expected for systems that depend on "security by obscurity". And the managers of such systems rarely reward someone who demonstrates how they've failed.
Those who do study history are doomed to stand helplessly by while everyone else repeats it.
If you want to go after someone, it probably should be the vendor that sold the crappy implementation.
I'm not a fan of more government, but since the power grid really goes beyond the company owning it, you should have regulations requiring the testing and remediation of any technical/physical security issues. That takes care of your hypothetical lazy IT Manager, the boss who blocks the good manager because it's expensive and not required, and the company who wants to keep selling equipment.
I had the joy of working on some of the software that runs the bid/sell aspect of "the grid". Let me tell you, it was, in my 20 years of coding, the absolute most convoluted spaghetti I think I will ever witness. The whole code-base I worked on squarely belonged in the Daily WTF - much of it far worse than the stuff you see on that site. If the software that actually runs "the grid" is 10 times better than that, it's still a horrible mess. I quit that job because they wouldn't let you fix/refactor it. As long as it ran, that's what ran. Bug fixes and enhancements were always add-ons. Code was never removed or changed. Insanity.
Not to mention the power systems for the entire DC area are too important to allow any outages no matter how short. Heard about that a few years ago from a fellow contract Engineer trying to replace a dying transformer.
And all the IT departments I deal with are still too arrogant to take any time to understand how power systems work, and that the power supplies in the 'PC' is the weakest link because the manufacturers of the 'PC' are too damn cheap to allow an additional $0.50 and some volume for adequate 'ride-through' capability.
Been there, done both jobs, as overheard many a time at "university" - ain't now new thaing, you can't fix stupid, only deliberate ignorance with a mule calibration 2x4 tool to the cranium of MLM repeatedly until compliance is achieved.
SERIOUSLY, why the heck am I still "working" at a 'utility' company when NOBODY in SLM MLM gives a DAMN about my professional opinion?
http://ics-cert.us-cert.gov/advisories/ICSA-13-291-01
Wait, there was no response to a report about a vulnerability in our energy structure? Gee, I wonder why.. Perhaps they should try submitting the report when the office that will ultimately respond is.. I don't know.. open maybe?
It is not like this is a new issue. Fire all IT managers who were responsible for not doing penetration testing, including the ones at Homeland Security.
If you do NOT hold managers responsible then they are just lifers waiting for their pension!!
Before you loop that noose over the tree branch, perhaps you should check if this report actually reflects the real world.
TFA simply says the tested software from vendors, not real world installations. This software is in actual use, but that doesn't necessarily mean its running naked on the internet. Most often this is run on private circuits, as most of these installations predate the availability of internet. Even when on the internet, most of these installations use VPN between plants and control centers.
Even those foolish enough to put SCADA directly on the net have already been notified by their trade associations (if not the DHS) to start using off the shelf VPN routers immediately, and that happened months ago.
Contrary to the rantings of Slashdot Experts, these places aren't run by total idiots. Nor do they have the luxury of replacing every SCADA controller in their plants. But they do know enough to use common off the shelf technology to provide reasonable level of security, and probably accomplished this a long time ago simply to make management of their network easier.
Sure, you can scan the net and find some SCADA controllers small water pumps in East Podunk Oklahoma. But they don't control big city plants.
Sig Battery depleted. Reverting to safe mode.
Is the problem with the protocol or the implementation of that protocol?
Mr. Crain ran his security test on his open-source DNP3 program and didn't find anything wrong. Frustrated, he tested a third-party vendor’s program to make sure his software was working. The first program he targeted belonged to Triangle MicroWorks, a Raleigh, North Carolina based company that sells source code to large vendors of S.C.A.D.A. systems. It broke instantly.
If the vulnerability is not in an open source implementation but is in third party vendor implementation then it looks like an implementation problem not a protocol problem.
Pen testing? Are you serious? None of this infrastructure testing has had any requirement to do such testing until very recently. The industry is scrambling to drag government kicking and screaming into setting a cold hard edged standard that has to be met minimum for any system to be accepted by government for infrastructure use.
I can crash six different manufacturers modbus TCP over ethernet capable units with the sister of the port 139 BSOD attack against vindos a while back. They all use the same damned TCP/IP stack as drop in and forget 'intellectual' property. Two of them go into wildly erratic behavior with no watchdog protection causing thrashing of the outputs that cause motors or other actuators to go nuts. One loses some function of it's firmware and the rest do the equal of BSOD though one of them reboots and recovers due to a working watchdog.
I have worked directly with the Triangle Microworks Stack, implementing DNP3 in both Master and Slave systems. I am not surprised to find that Mr Crain could crash their stack. However, this is a very overblown response. In order to access the systems running the stack you need physical access to the IP network or RS-485 loop. Then you need to sniff the data, find the target address and know what kind of packet to send. Or run a fuzzer on the network, which would show up as collisions or on the firewall log. Both of these require more sophistication than throwing a chain into a substation!
Well, as for private networks...
Do you remember a few years back when a nuclear plant that was only on a private network was taken over by a virus. (Nothing major happened that time.) This was because in a different building on the network a contractor plugged in his laptop to the private network. I believe that this was by accident. I think he was trying to go on the web. But his laptop had an active infection.
What with wifi becomming increasingly common, I don't think private networks count as security unless they are QUITE strictly controlled.
I think we've pushed this "anyone can grow up to be president" thing too far.
I think you're trying (and failing) to make a pun. "Open Sores" = vulnerable proprietary systems. Lame.
Sure, you can scan the net and find some SCADA controllers small water pumps in East Podunk Oklahoma. But they don't control big city plants.
Well here I am in West Podunk Oklahoma and our water pump is controlled by an Emitrol Sytstems relay board connected to an IMSAI 8080 with RS-232 to our PDP-8. I talk to the pump and post to Slashdot with an LA36 DECWriter. It does lower case but it sure looks funny.
Don't go touching my pump now.
This is my pump.
There are many like it, but this one is mine.
<blink>down the rabbit hole</blink>
Embedded systems (only) get replaced when they break.
Write protect the appliances. It is impossible to remotely modify the code then installing malware should be very difficult. The next trick would be making it impossible to pass executable code to the system's ram.
Even if you couldn't accomplish the second part... the first part is easy and it would mean recovering from any breach with a reboot.
There are ways to secure these systems. But ultimately they're going to have to have limited access from remote users. Security updates and modifications to the software should be done locally. That means a hacker needs to gain physical access to the appliance to compromise it. Then you can keep most attacks out with a good lock.
Some will say this defeats the purpose of these systems. That teh whole thing was supposed to be remotely administered from some central computer command center. How much is that dream worth? Is it really worth all this trouble to not send a technician by every so often to make changes?
Hard code and write lock the appliances. Then sleep like a baby.
I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
"The NY Times has an interesting story about a pair of researchers who 'discovered that they could freeze, or crash, the software that monitors a [power] substation, thereby blinding control center operators from the power grid.' .. It's scary that our electrical grid is so vulnerable and there doesn't seem to be much urgency to get it fixed"
Then don't connect your electrical grid directly to the Internet !!
You must be Captain Obvious! It's a pleasure to meet you, sir!
Ezekiel 23:20
And they don't get replaced when it comes to light they've been broken all along? Sounds almost like the "broken windows" thingy.
Ezekiel 23:20
Whoever is giving access to vital national resources on the Internet should be arrested and shot.
Fata viam invenient.
This problem is often brought about by a LACK of IT involvement. In many operational systems the control system is maintained by a small group with more knowledge of the plant and the vendor package than IT infrastructure. You may be targeting the wrong people.
In any case you're still right. DNP3 is about the most secure of the telemetry protocols, and actually has some basic form of encryption. An attacker shouldn't even be able to get as far as to see or communicate with it.
The researches have shown that the system can be compromised from within the network. This should come as no surprise. In many regards DNP3 is far better than any alternative, many of which do not even offer basic authentication let alone encryption. The critical part is the researchers were effectively sitting at the keyboard of their targeted machine. They shouldn't be able to get remotely that far. They should be separated by isolated networks, firewalls, etc.
Incorrect, the most secure would be secure MDLC by Motorola
I bet 99.99% of this is the SCADA systems. Those are huge open sores in the power distribution network because of the SCADA vulnerabilities. And the geniuses at the power companies thought it was a-ok to hook SCADA into an ethernet network.
If you want to be entertained go read the NERC documents.