Slashdot Mirror
Network Blackout
An anonymous reader writes "Renesys put together a special report on the effects of the recent blackout on routing and network reachability on the Internet. It includes a cool animation of networks dropping off the internet (presumably as a result of the power outage). It is interesting to see how localized some of the outage was--networks in New York state right up to the Vermont border go dark while everything on the other side of the border is quiet. New York City obviously gets clobbered."
The blaming and finger pointing began almost as quickly as the lights went out. First it was the U.S. and Canada blaming each other for causing this particular blackout, but inevitably the blame conversation turned to larger issues of policy, and how something like this could happen in such a heavily regulated industry.
Some of the finger pointing in the national press has been at deregulation -- if it weren't for deregulation, we would be better able to control and manage the grid. This misguided contention is incorrect in a number of ways.
First, the "deregulation" that has occurred in electricity has primarily been in opening up wholesale markets for power generators and their customers (i.e., utilities), enabling people in Manhattan to continue consuming power (and clamoring now for more regulation) without Con Edison having to build more power plants on the island itself. The existence and growing vitality of wholesale electricity markets has created substantial value in the past decade, through encouraging generation where it is cheapest and sales of power to where it is most needed.
But this limited amount of market liberalization has left the industry in an awkward place. Generation is largely governed by market processes, but transmission and retail distribution remain heavily regulated. The investment decisions of transmission owners and the retail rates that they can charge to their end customers all hinge on rate cases that are decided by state-level regulators. The rates that regulators allow take into account changes in costs, required investments, and the payment to the utility of a rate of return on the assets they own. For much of the past decade this rate of return has been substantially lower than what utilities could earn from doing other things with their money, so they did not invest in building much new transmission capacity or in upgrading existing lines. Nor did a regulatory environment that is a relic from the 1930s, constructed to govern and control local, vertically integrated utilities, either have the incentive or the wherewithal to force the utilities to invest in transmission assets that would carry power to customers in other states.
This lack of investment in the infrastructure that carries the product exchanged in growing, vibrant wholesale electricity markets has become a problem -- not an overnight problem, as those who follow the industry have been concerned about transmission capacity for at least five years. The numbers offered this weekend suggest that electricity volume has increased 30 percent while transmission carrying capacity has increased only 15 percent. This fact illustrates the mismatch between the dynamic markets for wholesale power and the rigid, maladaptive set of state-level regulations and incentives that govern transmission investment decisions.
Markets adapt to changing conditions. The existing electricity regulations do not, and because of that, the transmission infrastructure has not adapted to the increased demand on it from the increasing vibrancy of wholesale electricity markets.
So how do we proceed to ensure that a blackout of this magnitude does not happen again? There are four things that can relieve the strain on the grid. The knee-jerk reaction of many people is "build more wires!" More wires will increase the carrying capacity of the system, and in some cases transmission owners can add lines to existing paths. But this approach faces some serious obstacles -- such construction is expensive and time-consuming. Most importantly, though, getting new lines and towers sited is increasingly difficult, as people and communities object to having such large structures near them or strung overhead.
A second option is to use new technologies, such as high-temperature superconductors and sophisticated computer switching, to upgrade the capacity of the existing power lines. While also expensive, this option gets around the NIMBY issues that accompany the siting of new lines.
A third option is to build
As the link says - http://www.socio.demon.co.uk/mphil/appendix1.html [Quote] The Internet is a web of several thousand computer networks that now extends to just about every region of the world and has 50 to 100 million users. The Internet of today has it's origins in a networking project called ARPAnet which was run by the Advanced Research Projects Agency, a science research body set up in 1957 by the Pentagon. (Hafner & Lyon, 1996, 19) The popular belief is that the military created the ARPAnet, the precursor of today's Internet, so that data held on Pentagon computers could survive a nuclear attack by the Soviet Union. Upon attack, data from computers at the Pentagon and other military installations could be uploaded (sent electronically) to other remote computers not affected by such an attack. [/quote] It's always been my understanding that the Internet would continue working after a Nuclear war, at least that was the plan. If this blackout had effected all of US / Canada like a Nuclear attack would, would any of the Net worked ?
The geolocation of networks is never a precise thing. Networks can be registered in the city of their corporate offices but deployed anywhere in the world (it's how the Internet works--cool, huh?).
There are some interesting, precise tricks that you can play by sending various kinds of packets (usually UDP) and using detailed latency information about each hop of those packets, along with a provider network map, to get closer to the physical location of a particular IP address. We didn't do that for these maps.
Uh, yeah, thanks, we never thought of that.
UPS's run out of juice. Generators run out of fuel. Generators turn out to be less than perfectly maintained and fail after a couple hours. Budgets get trimmed, maintenance gets overlooked, blah blah blah it's never a perfect world. If it was, engineers would be replaced by algorithms.
Companies that are dead serious about power reliability run generator tests every day, and when lightning is detected within miles we automatically start up all the generators and run off them. Yes, we actually go OFF THE GRID every time a thunderstorm rolls in, and in Atlanta that's many times a week. Of course, we've got millions of dollars an hour running through our facility so heads would roll if we weren't this paranoid.
I believe we keep our tanks fueled up for 3 days of continuous service, and we pay a premium to guarantee that when the shit hits the fan, WE get refueled first.
Of course, hardcore multiply-redundant (and *tested*) systems are something that elude the typical IT crew staffed by DeVry grads.
One simple rule for its versus it's