Slashdot Mirror
Power Grid Insecurities Examined
Joe Barr writes "Chris Gulker has taken a long and careful look at the infrastructure of our power grids and has come to some rather unsettling conclusions." A good read that outlines where the current power grid is at, and suggests some paths for the future that may help avoid future blackouts.
In most states, if you generate your own power (ie solar), you can feed it back to the grid, and the electric companies are required to credit you! Any excess power you have can make you money. Sure, it's an investment up front to move to solar, but it is doable, and some states even offer tax credits.
Says Skroch: "If you have too much security [i.e., no network connections], then the power plant probably won't work."
power plants worked long before the internet was created. no important computer controlling very important things should ever be put on the internet.
The power industry needs to be reinvesting profits in infrastructure (powerlines), not stock dividends. The same companies should have been upgrading their command and control systems to prevent chain reaction blackouts. Am I expected to believe the computer systems that manage the cooling rods in the nearest nuke plant are secure?
Seriously consider the economic impact of the grid failure compared to the recent worm problems. Then think about a nasty combination of the two.
Karma: Censored (mostly affected by decency laws)
Did anyone actually read this garbage before they posted it. This is absolute nonsense. The blackout had _nothing_ to do with computers, much less internet security. The blackout happened because a half-rate utility (First Energy) tried to squeak through an emergency without buying expensive power or shedding load. Period. They operated lines until the sagged into brush. Some small subtransmission and distribution lines had twice rated load. Do the math. That's four times the temperature or over 400C. That had zippo to do with M$ or any bleepin' computer.
...and many of you are liable to freeze (or in southern parts bake) in the dark. If it weren't for BC Hydro selling power to California's PG&E over the common power grid on the west coast it would have been a certainty. Moreover, PG&E DEFAULTED on MILLIONS of dollars owed for said power to BC Hydro--so perhaps the proper term would be BC GAVE California power. Sooo...who uses who's power grid?
Also, before you start singing a round of "Blame Canada" it has been determined to a high degree of certainty by industry experts that the most recent power outage originated in the US (notwithstanding out boneheaded prime minister's impulsive comments on the matter before anything was determined). One thing is for certain--it was the Homer Simpsons on BOTH sides of the border that allowed the outage to propigate to the extent it did (operator error, scheduled outages that left the whole system running at capacity, etc...).
Deregulation has been bungled in its implementation all over the continent, but moreso in the US and particularly in California (well...EVERYTHING involving goverenment in California is royally fscked and has been for the better part of the last decade). The process was always politicised and the fledgling market manipulated by the established players and governments no matter where deregulation happened.
The concept is sound however...creaky old mandated monopolies should be broken up and the system made as open as technically possible to as many potential generation sources as possible. Decades of monopoly (in generation particularly) set us all up for the situation we are in now.
As a result, we presently have a handful of creaky, large utilities running creaky, large power plants with obsolete technology--and newer technology tacked on with duct tape and baling twine with little attention to stability and security. This has nothing to do with what country you are in--it is the situation continent-wide.
I've worked in the industry and have seen it first hand--and this was BEFORE the industry was deregulated (they still had several 1988-era 386s and a 286 in use--in 1996!). The argument then was that competition would compel established players to innovate and become more efficient. NOTHING has changed in these plants since deregulation--they are moving no slower OR faster in bringing new capacity to the grid. Only now demand has reached critical levels as predicted by some years ago. Only the argument has changed. Now instead of being the solution, deregulation is cited as the reason for problems (careless cost cutting rather than being sheltered from competition).
I'm astonished (but not entirely surprised) that since I was last in a power plant that there has been enough integration of critical systems into the general network that blaster-like infections could disrupt operations. Back in the mid 90's where I was, there were two distinct networks with NO connection at all (be it physical or not). If course, the 'net wasn't what it is now either and dozens of on-site employees had to rely on a 56k leased line for outside access.
Hopefully the blackout made everyone feel vulnerable enough to wake up and put at least as much or more into security and stability as they did into y2k compliance...
Hackers controlling the power grid? Utter and total bull.
I work in IT for a major power company. Our control systems have never been hooked to our own network, let alone the Internet, and never will be. How stupid does this guy think we are?
We've been running computerized control systems in nuclear and other types of generation plants for years. We've had computers in substations and control stations monitoring, controlling and reporting status before most industries even knew what to do with them. I saw my first Z-80 processor in a SCADA system shortly after the Z-80 came out. It could talk any of 5 different control protocols and replaced 2 seven-foot racks of hot, high-current RTL and DTL control logic. It was a thing of beauty.
We're not newbs at this. And no way do any of our control systems run Windows. Get real.
Why would we even want to hook up a generating plant or substation to a network just so it can be controlled from anywhere in the world, BY ANYBODY? No way. No how. Nuh-uh. Ain't gonna happen.
We can't even monitor what's happening on the system from the company's own computer network. It's all totally seperate. And for good reason. Who wants a disgruntled employee or just some joker who's bored messing with the system? The only people who can make operational changes to the system are the people actually present at the secured control center or at the generation plants.
We run quarterly modem audits, company-wide, looking for unauthorized lines with modem. We even restrict who gets an analog phone line and whether they can receive calls on that line. Computers attached to the control systems get NO modems. Never ever.
Even our remote monitoring terminals at regional work centers require dedicated connections to the control center and are receive only. The control computers think the remote monitors are printers and only send data, not receive so they can't be hacked from there either.
It's impossible to get to our control system through the Internet. It could probably be done to some degree (perhaps sending a 'breaker open' command to a key substation, if you know which one), but only by hijacking an existing dedicated connection undetected, which is getting harder as we connect stations via fiber optic.
(Often we connect stations by installing the fiber near the high voltage lines on our towers, a security measure in and of itself. Imagine splicing a broken fiber hanging off a helicopter platform while the line 12 feet below you is energized to 350 thousand volts. No, I haven't done it, but I watched it being done and the crew earned every penny.)
If any utility out there has their control systems connected to computers that can be reached via the Internet (or modem for that matter), the persons responsible should be taken out and shot. Then taken to a doctor, stitched back up and shot again. Same for their bosses all the way up to the CEO.
Sorry if I seen a bit testy on this subject, the subject of keeping the control system secure has been drilled into me for more years than I care to remember. Now it's just automatic.
However, on the subject of aging infrastructure, I totally agree. I blame deregulation. Every utility is now trying to cut each other's throat trying to grab customers away from each other. To cut costs (and thus lower their prices to better compete), most if not all utilities have cut their expenses by eliminting maintenance, lengthening replacement schedules and cutting staff, specifically skilled line workers). It's a race to the bottom to see who can provide the cheapest service. And it will probably go on until the whole thing blows up on them. And unfortunately, us as well.
Beta sux! Join the Slashcott! http://hardware.slashdot.org/comments.pl?sid=4760465&cid=46173047
We're starting to see a few problems appear more than once, though.
-
Telecom vulnerability to power failure.
-
"Non-critical" systems that aren't.
-
Cross-connection between business systems and control systems
That's a more realistic picture of what's going on.AT&T was determinedly independent of the power grid in the days of Ma Bell. Every central office ran on 48VDC storage batteries, with backup generators. The backup generators were started once a week, and run for several hours once a month. Once a year, each central office ran for 24 hours cut off from external power.
That was a long time ago, back when AT&T was a regulated monopoly common carrier. In the new, competitive era, that depth of backup can no longer be assumed. Carriers in trouble (WorldCom, Adelphia) tend to cut things like that.
The details aren't in yet, but it's beginning to look as if, during the recent big blackout, some comm links went down very early, so that the fault information that's supposed to divide the grid cleanly into islands didn't get through. Once all the logs have been correlated, it will be clear what happened.
A few weeks ago, CSX, the railroad, had a shutdown due to a virus. Railroad signalling has used "code lines" for decades, for remote control of switches and signals. These are basically serial links over which commands and responses are sent. The safety logic is local, but if you lose a code line, the dispatcher can't throw switches and route trains.
The tendency to centralize train control has resulted in a need to transmit code line signals hundreds or thousands of miles. So they tend to be multiplexed over telecom-like facilities. CSX apparently routed theirs over their in-house general purpose network. The routers in that network were managed by a network management system that ran on Windows. When the Windows machines went down, system management of the routers stopped, and, after a while, this apparently took some key routers down. So a "non-critical" system actually stopped train movements.
It's really convenient to be able to see what the plant is doing from your desktop. Order processing is more efficient if the sales network connects to the factory network. Energy traders need to be able to see what the power plants are doing, and give directions to power dispatchers. These things all create vulnerable paths.