Slashdot Mirror


Smart Grid Computers Susceptible To Worm Attack

narramissic writes "Researchers with security consultancy IOActive have created a worm that could quickly spread among Smart Grid devices, small computers connected to the power grid that give customers and power companies better control over the electricity they use. '[The worm] spread from one meter to another and then it changed the text in the LCD screen to say "pwned,"' said Travis Goodspeed, an independent security consultant who worked with the IOActive team. In the hands of a malicious hacker, this code could be used to cut power to Smart Grid devices that use a feature called 'remote disconnect,' which allows power companies to cut a customer's power via the network. The robustness of US power networks has been a hot-button issue after a technical glitch in 2003 caused a cascading power failure in the eastern United States and Canada that affected 55 million people."

32 of 98 comments (clear)

  1. lazy engineering by Anonymous Coward · · Score: 4, Interesting

    I know about these.... they're running windows XP, and are on modems. They call in every now and then to get get updates from the main network.... its' the power grid from the future? More like 1990.

    1. Re:lazy engineering by mangu · · Score: 5, Interesting

      its' the power grid from the future? More like 1990.

      Actually, power systems is a mature technology. The "bible" that every power engineer has is this book, first published in 1955. Notice that the book on sale is the fourth edition, printed in 1982. Nothing is changing very fast in this field.

      The problem that could arise from a large number of Smart Grid computers being pwned is if a worm triggered them off at exactly the same time, this is called a "load rejection" event. It would cause oscillations in the power flow which could end in a blackout but, generally, load rejection is not as bad as generation rejection, which happens when a power plant is cut off.

      Another problem that would cause much more harm to the companies than to users is if the worm instructed power meters to register less power consumption. I see a large black-market arising, if someone figures out how to write this exploit.

    2. Re:lazy engineering by TubeSteak · · Score: 2, Insightful

      Another problem that would cause much more harm to the companies than to users is if the worm instructed power meters to register less power consumption. I see a large black-market arising, if someone figures out how to write this exploit.

      I miss the days when hackers were just doing things for lulz.
      Society would be better off with merry pranksters breaking things because they want a big splash and lots of attention. And usually, the bigger the splash, the sooner the fix.

      Organized criminals, exploiting the same flaws, want secrecy and this is bad for society.

      --
      [Fuck Beta]
      o0t!
    3. Re:lazy engineering by mangu · · Score: 4, Informative

      I miss the days when hackers were just doing things for lulz.

      Problem is old time hackers did things for money, too. Pricing details here:

      In 1971 Steve 'Woz' Wozniak designed a device called the 'Blue Box'. It allowed -- of course illegal -- phone
      calls free of charge by faking the signals used by the phone companies. His friend Steve Jobs instantly realized that there must be a huge market for something that useful. He bought the parts for $40, Woz built the boxes and Jobs sold them to his fellow students at the University of California in Berkeley for $150.

      This well known anecdote is what made me think of the market for an electricity meter hacking device. $150 in 1971 dollars would be about $800 today.

    4. Re:lazy engineering by freaklabs · · Score: 3, Interesting

      And of course you can buy the old Radio Shack auto-dialer and replace the crystal. That turns it into a red-box where you can emulate the DTMF tones that signal coins being dropped into the slot.

    5. Re:lazy engineering by reboot246 · · Score: 3, Insightful

      Good luck finding a working payphone.

  2. Glitch? by Scrameustache · · Score: 4, Insightful

    It wasn't a glitch, it was negligence! Cheap cost cutting measures, enabled by foolish deregulation: Trees were not trimmed around critical power lines, the lines were cut by falling branches, and then a cascading failure spread through the grid.

    --

    You can't take the sky from me...

    1. Re:Glitch? by Scrameustache · · Score: 2, Interesting

      Similarly, the blackout of 1965 was caused by cheap cost cutting measures, enabled by foolish regulation.

      No: The cause of the (1965) failure was human error that happened days before the blackout, when maintenance personnel incorrectly set a protective relay on one of the transmission lines.

      --

      You can't take the sky from me...

    2. Re:Glitch? by Scrameustache · · Score: 2, Informative

      the lines were cut by falling branches

      Apparently I had that bit upside down; it was the power lines swinging low, not branches falling: http://en.wikipedia.org/wiki/Northeast_Blackout_of_2003#Sequence_of_events

      And here's a piece about why regulations are good and should be enforced: http://www.ontariotenants.ca/electricity/articles/2003/ts-03i08.phtml

      --

      You can't take the sky from me...

  3. Asinine by Samschnooks · · Score: 3, Insightful

    Should one of these security bugs be made public, it wouldn't just be dangerous, it would also be expensive, costing utility companies big money as they went back and retrofitted their buggy systems, Pennell said.

    Let me get this straight. Pennell wants the bug to kept undisclosed because it will be too expensive for the utilities to fix. Yet, someone whose clever, maybe those folks who hacked into the grids in other countries, may do it to the utilities here in the US; which will be vulnerable because the bug is "too expensive" to fix. Meaning, that the grid is vulnerable and subject to the damage that everyone is afraid might happen since the bugs exist. I guess if the bugs are kept secret, no one else is capable of discovering them because nobody is as smart as the researchers?

    OooooooKaaaaay. Riiiiiiight.

    1. Re:Asinine by betterunixthanunix · · Score: 2, Informative

      "Um, citation please? Nowhere in the linked article (sorry, I know I wasn't supposed to read it, but I was curious), does it say anything about being expensive to fix. In fact, it says nothing at all about repair cost, which may merely involve a firmware update which could be deployed remotely."

      From TFA:

      "Should one of these security bugs be made public, it wouldn't just be dangerous, it would also be expensive, costing utility companies big money as they went back and retrofitted their buggy systems, Pennell said."

      That would be the last sentence.

      I do not feel sorry for the utilities if they deployed a buggy system that created a national security problem. I would hope that any promise to keep the exploit secret was matched by an immediate effort on the part of the power companies to correct the problem, and to have a third party perform additional testing to discover other possible exploits; clearly, that is not what happened.

      --
      Palm trees and 8
  4. Should I be sad? by stokessd · · Score: 2, Interesting

    This has the potential to suck for the consumer as people could now mess with our power. But after living in several places over the last decade, and being charged $25-$100 to "turn on" my power which is effectively just a change of name on the record at the central office, I can't say I'm shedding a tear for those folks.

    Sheldon

  5. This shows the weakness of anything centralized by cavehobbit · · Score: 4, Interesting

    This demonstrates the weakness of centralized power grids, like big hydro, big nukes, big coal, big solar arrays beaming power down to Earth, Big solar arrays covering the desert, or any other huge centralized 'answer' to our power generation problems. They are all vulnerable to DOS attacks or attacks on central points of weakness like power lines. It takes just one well crafted weapon, whether kinetic, EMP, radiological, chemical-explosive, cyber-viral-worm, etc., to plunge large populations into darkness and chaos.

    Monolithic thinking leads to monolithic engineering, (not to mention monolithic politics), that concentrate your vulnerabilities and limit your flexibility in responding to problems.

    Better to have many smaller, locally distributed sources. They make it far more difficult to attack them. Looks like Edison was right and Westinghouse was wrong. At least partially. Too bad we went with Westinghouse, at least so far as the centralized generator is concerned.

    This is a challenge that evolution, free markets and democracy all respond to with good answers. Authoritarian structures like organized religions, socialism/communism and autocracy in general all respond poorly to.

    This is also a vulnerability of the Internet, with its centralized DNS name servers. I wish I was knowledgeable enough to come up with a solution to that one.

    1. Re:This shows the weakness of anything centralized by doshell · · Score: 3, Informative

      This is also a vulnerability of the Internet, with its centralized DNS name servers. I wish I was knowledgeable enough to come up with a solution to that one.

      The DNS name servers are not centralized. Perhaps you are thinking of the root servers, but those hold only a few records for the TLDs; in order to resolve "slashdot.com", the root servers only know about the ".com" part. Besides, 99% of the queries you make do not ever reach a root server, because you are using your ISP's name server, which does caching. Precisely because it would be unworkable to make every query depend on the DNS servers "above".

      The current problem with the DNS is one of security, but that has nothing to do with it being centralized (indeed I would argue it is easier to secure a centralized system than a decentralized one...)

      --
      Score: i, Imaginary
    2. Re:This shows the weakness of anything centralized by AchilleTalon · · Score: 2, Funny

      So, that's time to change this way to do things. This is the reason I have harnessed a whole flock of squirels to run in a large squirel cage linked to a damn big dynamo which produce enough power to fill my needs. And you know what? It just costs me peanuts!

      --
      Achille Talon
      Hop!
    3. Re:This shows the weakness of anything centralized by Dachannien · · Score: 3, Informative

      One, we have roughly 10,000 power plants of all types in the US.

      Two, transmission losses are roughly 10% (up from 5% 40 years ago, largely due to a failure to improve the transmission grid on par with the increase in load).

      And three, I'm pretty sure the efficiencies being talked about earlier are related to economies of scale. That is, you can build a large power plant at a cost much cheaper per unit of capacity than a corresponding number of small plants.

    4. Re:This shows the weakness of anything centralized by dtmos · · Score: 4, Informative

      Decentralized power generation is a major part of the Smart Grid initiative. See, e.g., the Galvin Electricity Initiative.

      Since power generated in a grid cannot be effectively stored, it must be used when generated. This forces today's utilities into a large control problem, in which consumers' needs (in the form of measurements of line voltage and frequency, sampled throughout the network) are fed back to centralized control points and used to control the output of a relatively small number of generating plants (and current sent along individual transmission lines). Control of this system is moderately well understood, if one accepts that certain heuristics have to be used -- along with occasional human judgement. Considering its complexity, this is one of the great engineering achievements of the 20th Century.

      Decentralized power generation, however, is a completely different type of control problem. With millions of potential generators, the existing control algorithms fail completely; further, as part of the decentralized control algorithm the utility needs to communicate with each power meter (a.k.a. potential generator) in essentially real time, to control any power it may generate.

      Having a meter that bills the customer only for the net of power used and generated is termed "net metering." This exists today, but cannot achieve wide-spread use without better communication with the meters. Utilities like net metering, because they get additional generating capacity without paying for new power plants.

      The Smart Grid, with its communication to individual power meters, effectively enables net metering: Homeowners can generate their own power, use what they need locally, then sell any surplus to the utility for use by others. The meter can inform the utility how much power it is supplying at any time, a number used by the utility to maintain network stability. If the utility has no use for the power at that moment, it can refuse the offer, again by communicating with the meter.

    5. Re:This shows the weakness of anything centralized by Ashriel · · Score: 2, Informative

      I would certainly classify 10% loss as a large percentage of inefficiency; in most companies I've worked with, the minimum acceptable efficiency seems to be 93-95% - granted, I'm not talking power generation and transmission in those cases.

      That is, you can build a large power plant at a cost much cheaper per unit of capacity than a corresponding number of small plants.

      Absolutely. But we're talking initial outlaying of funds here, not maintenance, upkeep, or fueling. More to the point, with generators every few blocks in a city; one or two for every town in the country, the costs are completely removed from private business and placed onto the citizens and/or local government (meaning, again, the citizens). We should be paying for our own power generation directly, rather than paying a large authorized monopoly to charge us for the maintenance and inefficiency of an unnecessary power structure.

      Such a cellular power structure is impervious to large scale failure, and with interconnects, it would become fairly difficult to cause even local power failure.

      No doubt the costs would be greater, but they'd be diffused across the entire population, and the energy itself would be cheaper, since there's no need for profit. Most importantly, I would think that the security and stability of such a system would be more than worth the additional cost.

    6. Re:This shows the weakness of anything centralized by dtmos · · Score: 2, Insightful

      I think you'll find the control problem, whether centralized or distributed, is orders of magnitude more complex than you envision. The hard part isn't the economic part, it's the electrical part: Maintaining a constant amplitude (i.e., voltage), frequency, and phase over a large (both in geographic area and node order) network, with limited ability to transfer power from one point to another, is a very difficult problem -- especially when one has limited control over the applied load, and limited generating capacity. Not to mention all the problems with reactive power due to the uncontrolled nature of the loads (frequently inductive) and the phase delays that occur over distance. Much more information is needed besides "quantity produced vs. consumed."

      The "can not get what is not there" option is amusing to utilities. The more common name for this option is a "brownout" or "blackout" and, even if only local, they typically result in nastygrams sent to relevant regulatory bodies, political officials, and the press. They are therefore to be avoided. From an engineering standpoint this is typically achieved by finding someplace where power is available, and making it available where it is not. This requires a network of transmission lines. The second major headache of utilities today (and related to their first headache, lack of generating capacity to meet their growing loads) is that the transmission system frequently is operating near peak capacity and, during the peak times (usually July afternoons in the US), it is getting more and more difficult to get sufficient electricity to the right places to avoid the "can not get what is not there" option.

  6. Re:If you build it... by hort_wort · · Score: 2, Insightful

    Yeah, I think at this point a hacker going into it is doing a service. Showing the vulnerabilities of a system before it becomes critical to the country in a few years is a good thing.

  7. Re:So if this is in the meter (?) by peragrin · · Score: 2, Interesting

    Bribe just about any good electrician if you want one of those seals. I can put my hands on four of them for upstate NY in less than an hour. (only minor B&E involved as I know where they are stored for one electrician.

    Also if you are good most of those seals can be opened and closed with regular tools. It takes a bit of patience, but is possible.hence why when they really lock you out they use padlocks now. Of course I bet with the right bribe one could get a copy of even those keys as they are most likely keyed the same.

    --
    i thought once I was found, but it was only a dream.
  8. Nothing to see here, move along... by dtmos · · Score: 4, Insightful

    This is non-news.

    There is no single "Smart Grid" device technology. At present there are many proprietary solutions from many different vendors, each using different communication protocols, computer hardware and firmware, and security methods. Each one of these vendors has its products in a very, very small fraction of the utility meters in the nation, most of which, of course, have no Smart technology at all. So the fact that these guys found one architecture vulnerable to a particular stack-overflow attack is bad for the vendor(s) that use it, but not indicative of an approacing nationwide catastrophe.

    Smart Grid system standards are under development, however, and those doing the development are exceedingly aware of the need for high security. The IEEE, for example, recently started a Smart Grid standardization effort, P2030, and the IEEE 802.15.4g Smart Utility Neighborhood Task Group effort is already underway. Since the utilities lose revenue -- potentially all revenue, plus destruction of capital assets -- if their equipment is cracked, they are very much a part of these standard development activities, and security is of constant concern. (There will undoubtedly be an industry consortium tasked with reviewing implementations of these standards.)

    1. Re:Nothing to see here, move along... by betterunixthanunix · · Score: 2, Interesting

      "Since the utilities lose revenue -- potentially all revenue, plus destruction of capital assets -- if their equipment is cracked, they are very much a part of these standard development activities, and security is of constant concern. (There will undoubtedly be an industry consortium tasked with reviewing implementations of these standards.)"

      Ironically, even in the face of lost revenue and destruction of equipment, power companies do not take security as seriously as you would have us all think. In some countries (including the UK, as I recall) the power companies began to deploy meters that required the insertion of a smart card in order to release power, with the idea being that customers could get "prepaid power." As it turns out, many of these systems were vulnerable to replay attacks and clever customers could get free power after purchasing two cards and simply alternating them. The meters would only remember the last nonce used, rather than every nonce; the reason was cost-cutting and an assumption that nobody would actually try alternating a pair of cards.

      I doubt that the companies here in the US will take security any more seriously than those in other countries. The engineers might recommend better security -- assuming they have a background in security engineering -- but the managers will only see that an extra million dollars will be spent to prevent an "obscure" attack that seems like something nobody will ever figure out. That is assuming that the managers even understand what the engineers have told them. Even if the IEEE recommends a secure system, corners will likely be cut that will leave the system vulnerable.

      --
      Palm trees and 8
  9. This won't affect the smart grid... by freaklabs · · Score: 5, Informative

    The attack in question is a side-channel attack that is limited to using a microcontroller with an external 802.15.4 radio that includes an encryption engine. The actual AES-128 algorithm wasn't broken. Instead the vulnerability is that the AES keys are sniffed on the exposed bus when you load the keys into the radio's registers. Contrary to popular belief, you can't take over the nation's smart grid from this attack, and it would be difficult to even take over your neighbor's meter unless you broke into his house. I have more info on my site where I respond to the hack from Travis Goodspeed. The blog post is at http://freaklabs.org/index.php/Blog/Misc/Clearing-the-Air-About-Hacking-Into-The-Smart-Grid.html

    Akiba
    FreakLabs Open Source Zigbee Project
    http://www.freaklabs.org/

    1. Re:This won't affect the smart grid... by smallfries · · Score: 2, Informative

      No it isn't, and if you had read the article instead of the summary before you tried to pump your own blog you would realise that what you describe is not the issue here at all. This is *not* the side-channel attack that your post talks about.

      Here are some basic clues:
      1. It's a worm
      2. It can spread from device to device over the network
      3. No external hardware is required, the exploit is purely software (see points 1 & 2).
      4. Goodspeed is not mentioned in connection with his side-channel attack on AES, but for some theoretical work on possible vulnerabilities.

      You fail.

      --
      Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
    2. Re:This won't affect the smart grid... by freaklabs · · Score: 2, Informative

      Uhhh...I read the article. If it were a pure software exploit. It wouldn't be an expensive issue to fix as mentioned in the last line of the article.
      And if you read my blog article, you'd see that smart meters aren't able to communicate with each other and instead communicate on the utility's backhaul.
      And if you read my blog article, you'd see that Travis Goodspeed posted a blog article of his own detailing a side channel attack on 802.15.4.
      Uhhh...personally, I don't care if you read my blog or not, but since programmers normally try not to repeat themselves, you might want to check out my blog for details on this.

    3. Re:This won't affect the smart grid... by freaklabs · · Score: 2

      Hmmmm...I now realize that posting on Slashdot after I'm dead drunk is not a good thing...oh well...huzzah!!!

  10. Frightening by MobyDisk · · Score: 2, Insightful

    Many of these devices are already deployed and it would be too dangerous to make the bugs known.

    and:

    Should one of these security bugs be made public, it wouldn't just be dangerous, it would also be expensive, costing utility companies big money as they went back and retrofitted their buggy systems, Pennell said.

    I love how they think that not releasing this information makes them safe. This is truly scary: Not like some Internet Explorer exploit on a user's desktop - this is the power grid! Someone is telling us that a remote hacker can take-over the entire power grid, and the companies are not going to stop everything and fix it? Holy crap that's negligent!!!

    It will be a heck of a lot more expensive to NOT fix this, than to fix it.

    (Yeah, I know, "preaching to the choir")

  11. Cause of the 1965 blackout by mangu · · Score: 2, Interesting

    The cause of the (1965) failure was human error

    "Cause" can be defined in several different scopes. When one reads a death certificate, for instance, the cause of death could be listed as a hemorrhage in the brain, or one could say the cause was a bullet, or a drunken brawl which ended in a gun being shot, etc.

    Instead of saying a wrongly set relay was *the* cause, perhaps it would be best to say it was a precipitating factor. If that relay had not been set wrong, there was a large number of factors that could have triggered a similar blackout.

    I guess what the AC called "foolish regulation" was the fact that electricity prices were set by law at such a low level that discouraged investment in the power system. Low investment means, among other things, that technicians will not receive good wages, they will not be motivated enough to pay close attention on what they are doing and will commit mistakes.

    Low investments also mean that companies will not build new power plants and lines. They will try to stretch existing systems to the limit, reaching a point where relatively small failures might cascade to system-wide blackouts.

    Generally, when people bemoan regulation or deregulation they are looking at just one side of the issue. If you regulate, then you must make sure that the regulations will not kill the companies. If you deregulate, make sure to deregulate *everything*, including prices. The problem with what has been called "deregulation" is that removing the regulations that impose quality levels while keeping regulated prices is more or less guaranteed to cause failures in the system.

  12. corruption and 'alignment' by Scrameustache · · Score: 2, Funny

    Bribe just about any good electrician

    Erm... evil, maybe? :)

    --

    You can't take the sky from me...

  13. "Remote disconnect" - implications by Animats · · Score: 4, Interesting

    I hadn't been aware that "remote disconnect" was being incorporated into electric meters. Read this industry analysis of remote disconnect" for background. The "risk items" list doesn't even consider the implications of hostile attack.

    The purpose of "remote disconnect" is to get more control over customers. Utilities are considering using this to enforce collection, and even for prepaid electric service. It's another way to tighten the screws on poor people, like prepaid cellular and paycheck loans.

    There's another feature, current limiting - draw too much current and the power cuts off. The current limit can be set remotely. When someone gets behind on their bill, the power they can use is limited to survival levels until they pay up.

    Vulnerabilities in the remote management system could be a serious problem. Will the keys be kept in a Microsoft system? If you thought it was bad when credit card numbers were stolen, what happens when someone steals the meter key database? The meters have to be physically visited, one at a time, to reset the keys. And who would do that? The meter readers get laid off when this goes in.

  14. Who is really at fault by Orne · · Score: 3, Informative

    Dammit, I'm getting sick and tired of this. Since I was involved in the 2003 blackout investigation for an outside utility company, here's what happened:

    • First Energy (OH) had some lines trip. Because of a race condition in their EMS (Electric Management System), the program never recognized that the lines tripped. Their State Estimator locked up, giving the dispatchers false information. Their redundant backup had the same code, used the same inputs, and got in the same race condition, and there was no watchdog system like Tivoli to measure that the systems were not outputting data.
    • Outside companies who observed odd flows on their systems tried to commuinicate with FE regarding the trippings, but FE said that the trippings were a data error (not recognizing they were real)
    • An hour and a half later, the grid split due to additional overload trippings in FE, and it all went to hell
    • FE executives begin spinning the story so fast you could have generated electricity if you stuck magnets on them
    • In the investigation, they found that too many companies were not adequately protecting their SCADA systems (it was so convenient to put the controls on VPN so you can work on an issue remotely), despite this was not one of the root causes.
    • Six months later, the government issued a report saying every utility was at fault, gave FERC the ability to set industry standards, and gave NERC the ability to fine companies a $1 mil/day for violating those standards.
    • 5+ years later, we're all reacting to these CIP (Critical Infrastructure Protection) standards, which are all poorly defined, everyone's paranoid they may be violating something (which = fine), and so they're all overreacting by clamping down on anything that looks like a SCADA violation.

    I'm tired of all this editorializing that thinks that this stuff is related, but it's not. The root cause was incompetence at FE -- cutting budgets so hard they got rid of tree trimming, failure to communicate properly in emergency situations, and lack of situational awareness -- combined with an over-reaching government that thinks the underlying communcations networks are unsecured. The "technical glitch" was an AIX UNIX machine with poor ICCP error handling, a message queue that failed to empty, and dispatchers that weren't trained how to handle the lack of data. DHS runs one test (Aurora) where they pretend to take over a generator with SCADA, then over-excite it for like an hour before they got it to spark, then suddenly they think the whole grid's at risk so they can get more government funding to justify their existence.