Slashdot Mirror


Staged Hack Causes Generator to Self-Destruct

An anonymous reader writes "It has been revealed that in a U.S. Department of Homeland Security exercise codenamed 'Aurora' conducted in March of this year, researchers were able to cause a power generator to self-destruct remotely via a hack which changed the operating cycle of the generator. 'Government sources said changes are being made to both computer software and physical hardware to protect power generating equipment. And the Nuclear Regulatory Commission said it is conducting inspections to ensure all nuclear plants have made the fix. Industry experts also said the experiment shows large electric systems are vulnerable in ways not previously demonstrated.'"

19 of 258 comments (clear)

  1. this should not be possible by arabagast · · Score: 4, Insightful

    because the automation system controlling the infrastructure is not connected to a public network, like say, the internet - right ?
    If it is, then someone should probably do some quick patching asap.

    --
    Doolittle : ...What is your one purpose in life?
    Bomb no.20 : To explode of course.
    1. Re:this should not be possible by LehiNephi · · Score: 4, Interesting

      It is. It has to be. It would be ideal if you could run isolated networks, but it's impractical. Let's say you run a facility with some gas turbine generators, as in this example. The generator package has to communicate with the control system. The control system has to communicate with the "business" network (for record-keeping, among other reasons), and the business network has to be connected to the internet. There are lots of things you can do to help secure the various levels of the network, e.g. firewalls, vLANs, packet filtering and inspection, intrusion detection and response, etc., but there still is a data path going all the way out from the lowest levels out to the "real world".

      (Our company has also been working with Idaho National Labs on this exact issue, can you tell? The government is taking it pretty seriously)

      There are a few problems. For example, there's a lot of old control gear out there, and if it talks ethernet, it assumes that anything it receives is legitimate. Also, the equipment involved is produced in small enough quantities that there can't be a great deal of effort expended on security features. It's not like Windows, where millions and millions of copies are sold, and lots of people actively look for holes.

      --
      Help find a cure for cancer. Join the [H]orde
    2. Re:this should not be possible by Rosco+P.+Coltrane · · Score: 4, Insightful

      because the automation system controlling the infrastructure is not connected to a public network, like say, the internet - right ?

      You know, the internet isn't the only network out there. The telephone system is another, with wetware acting as clients and servers. For example:

      JOE (technician): *rrring*.. hello?
      JACK (mischievous social engineer): Hey Joe, this is Terry at central control
      JOE: Hi Terry, what can I do for you?
      JACK: I need you to offset the timing on the third generator coil by 20% please.
      JOE: Uh? 20%? That sounds dangerous.
      JACK: It's urgent! the power-grid is not stable, if you don't do this, we'll have New York in the dark!
      JOE: erh.. I really need to talk to my supervisor for this. Who did you say you were?
      JACK: I've already talked to your supervisor. John's gonna be really pissed off if you don't do this!
      JOE: Well ok then. Here goes...
      **KABOOM**

      See? no need for any internet, wetware can be hacked too.

      --
      "A door is what a dog is perpetually on the wrong side of" - Ogden Nash
    3. Re:this should not be possible by kent_eh · · Score: 4, Interesting

      Our company has all our generators (and many other things) remotely controlled, and none of those systems are available to the public internet. We have it all captive on our own infrastructure.
      The local power utility ( I know several of their techs who work on the telemetry gear) also has a remote control system which in entirely on their own infrastructure, and has no interconnection with any system that is accessible from a public network.
      It may not be the absolutely cheapest way to do things, but it's also a lot more secure.
      What's the cost of this sort of failure compared to doing it "right" in the first place?

      --

      ---
      "I can't complain, but sometimes still do..." Joe Walsh
    4. Re:this should not be possible by PlusFiveTroll · · Score: 5, Interesting

      I'd guess most people here have never read about power grid synchronization. Unless your power grid is DC isolated, it shares data telemetry data with other systems in the grid. Any one of these systems getting hacked can put the entire network at risk. There are many ways to damage a generator if you understand what causes it to trip from the system. Delaying the disconnect from the power grid, for even a short amount of time can cause substantial damage.

      http://groups.google.com/group/alt.engineering.electrical/browse_thread/thread/c6a2399745b5413a/dcdf9906b70b85b1%23dcdf9906b70b85b1
      http://www.google.com/search?hl=en&q=power+grid+synchronization+failure&btnG=Search
    5. Re:this should not be possible by evilviper · · Score: 4, Insightful

      They would have to build their own network, unless they are NSA, FBI or AT&T they cannot do that easily.

      What the hell is happening to /.? Has NOBODY here ever heard of a LEASED LINE?

      Call up Verizon or AT&T, tell them you want a T1 from point A to point B. You pay them a few dollars every month, and you have a direct, and fully-private connection from A to B.

      Public networks aren't the only way to communicate.
      --
      Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
  2. Re:Bruce Willis will prevent this from ever happen by Anonymous Coward · · Score: 5, Funny

    Did anyone else immediately think of Live Free or Die Hard when reading this? No, because you're the only one who watched that movie.
  3. Why mention Nuclear? by brucmack · · Score: 4, Insightful

    I don't understand why Nuclear power needed to be singled out. The electrical generators are pretty similar regardless of the fuel source. And if it blows up, it's not going to take the nuclear reactor / coal furnace / (insert steam source here) with it, since they tend to be very well separated from each other.

    1. Re:Why mention Nuclear? by Anonymous Coward · · Score: 4, Insightful

      The parent post is profoundly ignorant of how a modern nuclear reactor works.

  4. Um, WHY was the generator on the internet?!! by jollyreaper · · Score: 4, Informative

    I'm no computer security expert but I do know of the world's most unhackable firewall -- it's called a one inch air gap. Put that gap between the network cable and the NIC and nobody is gaining access.

    Yes, I know power plants will require some net access for web, email, etc. But the office worker network and the command and control computers and network for the generators should have nothing to do with each other! Separate systems, no network connectivity, the plant software should be operating in a vacuum bubble. The rest of the world should not exist for it, no way, no how. Oh, need to install a patch for the software? After being thoroughly tested and vetted on a proofing system, the software is then installed the old-fashioned way, off of CD-ROM's. Now if someone can fuck with the CD-ROM's, THAT I can understand. I can buy the plausibility of the NSA printer hack, even if it was a hoax. (NSA puts a virus on printers heading to Iraq, takes down their network.) The story about the CIA sabotaging software for equipment the Russians were buying to use in their pipelines is true. These are secure systems completely cut off from external contact that were sabotaged by the insertion of compromised components that were not detected. That makes perfect sense.

    It always bothers me when I see movies showing hackers getting in to some place and gaining access to files on servers that should never have a connection to the outside world. Then again, maybe I'm giving the fictional syadmins of the target systems too much credit. Who knows, maybe next week we'll read about some Korean hackers who were able to compromise a Minuteman silo and add it to their botnet.

    --
    Kwisatz Haderach
    Sell the spice to CHOAM
    This Mahdi took Shaddam's Throne
  5. They are connected to the Internet by Isbjorn · · Score: 4, Interesting

    I am the system administrator for a large state government agency. Recently I was essentially forced to connect a Windows XP boiler control system for an electrical generation plant to the Internet, so that the vendor can do remote maintenance. If I hadn't found out about it, it would be connected directly without even a firewall... This system had no anti-virus software, and of course it has a popular remote-control software installed for the vendor's access. The only reason I can sleep at night is that the plant is far away from any populated area, and may be shut down due to other reasons soon. I will be sending this video to a number of people in an email today.

  6. Disconnecting is NOT an option by ExE122 · · Score: 5, Insightful

    These post are getting ridiculous. Too many people are saying "why don't they just disconnect it from the network?" and getting modded as "insightful".

    It's NOT that simple! If they are connected to the network, there is probably a very good reason for it, and not just cause some engineer wants to check his email and download pr0n while listening to the generators hum.

    These generators more than likely are controlled by self-optimizing systems based on a variety of data that is collected. If they're providing power to various remote sites, they need the internet for gathering data from those sites.

    The internet is more than just a public free-for-all, it is the communication medium for many business/mission-critical systems (see LehiNephi's response above). They really just need to have the right security in place to keep it safe.

    --
    Capitalism: When it uses the carrot, it's called democracy. When it uses the stick, it's called fascism.
    1. Re:Disconnecting is NOT an option by makapuf · · Score: 4, Insightful

      s/the internet/a private wan

      why do you need internet (the public one, with no QoS) to have remote access from one point (data collecting / stat computer) to the power plant ?

      Yes, the data have to be collected from somewhere, but why not make a private WAN (or a VPN if best-effort QoS is OK for you) for this ? It's not about playing WoW with your neighbour, it's about remote controlling a nuclear core, so maybe it would make sense.

  7. There are Easier Ways... by xfmr_expert · · Score: 4, Insightful

    There are easier ways to damage the bulk power grid (or local transmission). Pick up a rifle at your nearest sporting goods store. Go to your nearest transmission substation (or even large generating plant). Take a shot at the porcelain on one of the transformer bushings. Kablam! You just removed a few hundred MW (or perhaps more) or generating capacity or transfer capability and caused millions of dollars in damage. If it's a generating station, the cost of lost revenue could drive the total to 70 or 80 million. Actually, I have seen bushings with bullet holes. Obviously not that common, or something would be done about it, but it does happen. It won't always cause an immediate and catastrophic failure, but it certainly can. Especially if one keeps trying... The bigger danger to this nations power grid is lack of investment and a severe brain drain in engineering personnel.

  8. Jumping Generators by torkus · · Score: 4, Interesting

    What a bunch of sad geeks we've become. Instead of crying about how it was connected to the 'net i watched the video.

    I'd like to know what they did to make a multi-ton generator JUMP like that thing did. After a few jumps there were a couple chunks of black stuff flying around. If you watch the "full" video it's clear they cut it at least once if not more. I'm guessing it took them quite a long while to get the generator to "blow up".

    Anyone have thoughts as to how they did it? I'm going to guess they messed with the fuel/air mix or delivery and caused a massive backfire while under/overloading the alternator side. I'd guess for kicks they also forcibly turned off the cooling fans creating an over-temp in the engine. Assuming i'm right and they cut out 95% of the video length that explains it a bit better. The failure seemed two-fold: A failed main-crankshaft seal spewed out white "smoke" (read over-temp coolant) and something up by the valves making black smoke.

    This is probably something you could do to a regular car if you were poking around in the engine management computer.

    --
    You can get rich if you own a politician, but you have to be rich to buy one in the first place.
  9. Re:Don't connect it up by theotherbastard · · Score: 4, Interesting

    Except that would never work with how the power grid is setup. The plants all communicate with Central Control. (I know because I happen to work for an Electric Company) Central Control is a big room with video walls the likes of which you have never seen! (Our main one happens to be the largest video wall in North America) These control centers are (gues what!) controlling how much power goes out across the lines at any given moment. And it has to be carefully controlled otherwise you get a sag or a spike which does all sorts of damage.

    In addition to the Central Control there are Regional Dispatch Offices which have information about the grid as well. These mainly coordinate repair and upgrade efforts. But, they need to know which circuits are hot because people's lives are on the line.

    So, simply isolating the plants would not work. Certainly not in our day and age.

    --
    Buttons aren't toys.
  10. Not possible by dj245 · · Score: 4, Interesting

    As someone who as worked in this position in a power station, let me say that this social engineering attack is not likely. You very quickly learn the names, attitudes, and voices of all the people that frequently call asking for changes to the generators. The number of people calling for these changes is usually a handful, 5 or less. If someone odd calls, we would often ask if another guy we knew was on vacation or sick.

    If someone we never had heard of called asking for something strange, I would have definitely asked to talk to someone I knew at the independent system operator, emergency or not.

    --
    Even those who arrange and design shrubberies are under considerable economic stress at this period in history.
  11. It is mostly bunk by anorlunda · · Score: 5, Insightful

    There is no such thing as an "operating cycle" to change for a generator.

    The generator pictured in the video is not the kind used in large power plants. It appears to be a diesel generator similar to the kind that is used for backup power in many buildings. Backup generators are typically 1 MW or lesss, whereas big power plant generators are 1000 MW or more. It is like comparing a RC controlled model airplane with a 747. Besides being bigger, the 747 and the power plant will have much more elaborate systems to protect things from damage and destruction caused by malfunctioning equipment and/or misbehaving control systems. When there are billions of dollars and /or human lives at stake, one invests more in safeguards such as electromechanical relays, breakers and other non digital gadgets.

    The thing that could cause the generator to jump and destroy itself like in the video is to attempt to synchronize it with the grid out of phase or at the wrong speed. Another post in this thread, "This has happened before computer controls" by Maximum Prophet hit on the correct answer. In small, unattended, backup generators synchronization may be automated by computer, but in large power plants nobody trusts the computer enough to allow this critical operation to be automated. It is still typically done by hand with the aid of old fashioned non-digital equipment. Even if one did mis-synchronize a generator (and it does happen) other protective devices shut things down quickly to limit the scope of damage. And yes, mis-synchronization does happen in real life every once in a while, usually in a brand new installation and usually because the instruments are wired up wrong. The result can be damage sometimes, but I never heard of it destroying a whole plant.

    That is not to say that cyberwar is not a threat, nor to say that it is not good policy to isolate all critical control computer from the net. Again its a matter of money. If you are running a $5 billion power plant, your budget is big enough to hire real people to come and maintain systems rather than using remote diagnostics. Or, if you do want remote diagnostics, you can afford to use leased private lines rather than the internet. Power plants and the power grid can afford gold standard security and they should be required to do it. I don't oppose the security thrust, but I do oppose the hyped up scare tactics designed to panic us into unwise government spending.

    I spent most of my life modeling power plants and their control systems to build operator training simulators. As part of training, we inject myriads of simulated malfunctions. As part of debugging of the models, we get to see just about every detail of the plant and its control and its safeguards working incorrectly before we debug them and make them correct. That gave me and others experiences up to our chinny chin chins about what can go wrong and what the consequences might be.

    I'm afraid that what this is about is another naked grab for government money and using scare tactics to get it. Mr. Joe Weiss in the video works for EPRI. He, and the government committee on critical infrastructure protection, were both singing the song in 1999 that no matter what Y2K bugs might exist, they couldn't do any real harm. Get it? Not that the Y2K bugs didn't exist or would be fixed (at proved to be the case) but that they couldn't do any substantial harm no matter what. Now these same people are saying that a few hacks can cause widespread and catastrophic damage. One can not argue both sides of this issue and keep credibility. If a control system misbehaves, it matters not whether the problem is inadvertent or malevolent. Yet these people pooh pooh the risk of inadvertent bugs yet hype the danger of malevolent ones. It's bunk.

    EPRI wants $100 billion to automate everything in the power grid as a massive research project. Next they'll want another $250 billion to secure it from cyberwar threats. DOE wants a national DOE control center for the

    1. Re:It is mostly bunk by wawannem · · Score: 4, Funny

      whereas big power plant generators are 1000 MW or more.

      Heh... that's it? I once heard of a professor somewhere that was able to build a portable generator, small enough to fit in the rear half of a small car, capable of outputting 1,210 MW...