Slashdot Mirror
A Cyber-Attack On an American City
Bruce Perens writes "Just after midnight on Thursday, April 9, unidentified attackers climbed down four manholes in the Northern California city of Morgan Hill and cut eight fiber cables in what appears to have been an organized attack on the electronic infrastructure of an American city. Its implications, though startling, have gone almost un-reported.
So I decided to change that."
We should also consider whether it might be necessary to harden some of the local infrastructure of our communities. The old Bell System used to arrange cables in a ring around a city, so that a cut in any one location could be routed around. It's not clear how much modern telephone companies have continued that practice. It might not have helped in Morgan Hill, as the attackers apparently even disabled an unused cable that could have been used to recover from the broken connections.
Always assume the enemy knows the system. Hardening wouldn't hurt, but redundancy is the most important thing. Hardening a system tends to make it that much more vulnerable to a single insider. Redundancy mitigates this effect. Having such a small group be able cause so much disruption from such a relatively simple act makes it obvious that the city placed way too much on a single point of failure remaining in tact. Have redundant fiber. Have auxiliary wireless setups. Maintain a base of ham volunteers. Multiply your points of failure.
Personally, I think this sort of lax infrastructure security has become endemic. The 'war on terror' rhetoric we were fed for so long has us looking for the next suicide jet-liner attack or what have you, completely distorting any real conception the public had of real-world modern security risks.
I got a catholic block.
Ham radio operators save the day once again... 'nuff said.
Lets not all go blaming terrorist organizations on this one.
My money is on unionized workers facing layoffs or payroll cuts. They would best know how to hurt the system and this sort of sabotage being linked to unions is not exactly unheard of.
"linux is just DOS with a UNIX like syntax" -- Galactic Dominator (944134)
Comment removed based on user account deletion
Define terrorism.
Now define terrorist organization.
If an organized group of people orchestrated this attack in order to bring attention to some goal, wouldn't that make them a terrorist group?
Admittedly, an attack on property is not the same as an attack on people, but yet... to me this seems textbook.
"Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
Jesus. Here come four thousand posts about how awesome hams are, and how Internet over power cables sucks, etc.
My Dad was a ham. Yes, hams are awesome. In their nutty little useful-once-in-a-lifetime, semi-Luddite way.
We love you, hams. We're glad you're out there. But please, seriously, shut the fuck up. On the Internet. Feel free to blather on your radios.
-Peter
Indeed, that's very possible: the contract between the Communication Workers of America and AT&T expired on April 11th.
Q: What does the "B." in Benoit B. Mandelbrot stand for? A: Benoit B. Mandelbrot
Hopefully this catches enough attention to get people to evaluate their area's utilities similar to the blackout across parts of the US and Canada back in 2003. http://en.wikipedia.org/wiki/2003_North_America_blackout
So now a "cyber" attack includes the physical destruction of hardware/infrastructure without any exploitation of any programming logic?
Well, I'd certainlly concede that this could be classified as terrorism but I was refering more to the "ZOMG TALIBAN" kind of terrorists. Modern media interpretation of the word. ;)
"linux is just DOS with a UNIX like syntax" -- Galactic Dominator (944134)
Bruce, the cable cuts were in San Jose and San Carlos. The cable between San Jose and Morgan Hill was cut, but the cut location was in the city of San Jose.
(otherwise, agree with what you said, hopefully wider audience for this will help...)
wouldn't that make them a terrorist group?
I'd presume that some amount of "terror" would need to be created for one to be considered a terrorist. But maybe I'm old-fashioned.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2009/04/11/MNJB170JM7.DTL
If an organized group of people orchestrated this attack in order to bring attention to some goal, wouldn't that make them a terrorist group?
No.
What makes a terrorist group a terrorist group, is that they inflict, you know, terror .
Cutting some cables isn't going to (and, in fact, didn't) send the general populace into a panic.
Yes, it's an inconvenience, but unless they are trying to instill terror in the general populace, they're not terrorists.
Or just regular blackmail:
http://yro.slashdot.org/article.pl?sid=09/04/20/1427259
I assumed these were both the same story at first. But the YRO story was 2005, and this one was a few weeks ago.
There are no trails. There are no trees out here.
We had a similar cyber attack here in columbus, ohio. A disgruntled employee (it is thought) shot the fiber backbone for Time Warner with a .22. I don't believe they ever caught the guy who did this. This one action disrupted the internet for hundreds of companies and thousands of users. It took around 3 days to get the internet back up for everyone.
This was just one fiber cable, imagine if someone had purposely cut lines downtown?
The stuff is very centralized and not well protected.
There needs to be better protection against these sorts of actions, and there needs to be a backup plan in place in case something like this does happen.
This was discussed extensively on the NANOG (North American Network Operators Group) email list.
It appears that the outage affected multiple carriers including ATT and Alternet.
Hosting and Domain name coupons
This sounds like a good old physical attack to me, not a cyber attack. Bashing in someone's computer with a hammer is not the same thing as a infiltrating it with a computer virus/worm/etc.
I doubt many readers here could physically fit into a manhole, much less survive a climb down any sort of non-motorized ladder.
My money is on unionized workers...
I think it was management, upset that so few people wore Hawaiian shirts on casual Friday.
""ZOMG TALIBAN" kind of terrorists. Modern media interpretation of the word. ;)"
Shortly to turn into "ZOMG Wobbly Anarchist Union Menace to be cleansed with fire and legislation" if formerly-gruntled union workers are found to be the cause...
"This post is an artistic work of fiction and falsehood. Only a fool would take anything posted here as fact."
The entire Santa Cruz County area was cut off from all telecommunications outside of Point to Point wireless and Satellite. (Comcast customers aside.) Sprint, AT&T, Verizon, long distance for POTS was all down. TFN's were not able to be dialed by any customers. 911, 611, 411 were not functioning. 'Point-to-point' T1's that were aggregated over DS3's in Hayward, were not functioning for area users. Many of the "redundant" network connections for companies in the Monterey Bay area were completely down. Both legs of their "best practice" 2 provider networks were crippled.
Other than a couple islands of connectivity (namely the Shell Gas station at 41st and Capitola Rd in Capitola, my mother In Law's house, and my Uncle's business) who were lucky enough to only have Satellite service available to them, or were on Comcast, the packets stopped flowing.
Ironically Comcast services inside the Santa Cruz county were still working. Users of Comcast voice wouldn't have noticed (except for the fact that everyone they called went straight to voicemail.)
However, inter CO calling was working (you could
call anyone in the Watsonville-Santa Cruz area if they had a POTS line from a POTS line. Still, corporate communications for nearly everyone in the area (Ag. Brokers, Packers, Pickers, Shippers, Bottlers, etc.) Was down. Commerce came to a halt.
People couldn't get gas at gas stations around the area unless they had cash. Area banks wouldn't let people inside the bank unless you were making a deposit. People couldn't be players in the game of commerce without little pieces of paper. And so once again, cash was king.
More cars sat on the side of the road that day then normal between santa cruz and watsonville. Which begs the question how does the regular joe call for help if the call boxes can't talk to a phone switch?
They might just be really, really, incompetent terrorists? I don't think we have a word mildinconvenienceists.
I am TheRaven on Soylent News
My money is on unionized workers
Well then, ionize the workforce in future.
I am TheRaven on Soylent News
I guess this kinda puts a damper on all the cloud computing hype of late...
When I first saw the way that one worked, I shook my head, and said "You're joking, right?"..
Alas, the answer was no. And the reason that it had been designed as a centralised system (well, ok, there's a 'failover' data centre or two) is (according to the designers) that you'll never lose the main and the redundant connections at the same time.
I seriously hope that they're paying attention to this at the moment. The severing of very few, carefully chosen fibres could quite simply deny a lot of UK hospitals access to their medical records. And if all come on board, then you could deny nearly all hospitals access to the medical records.
This, as can be imagined, would be rather a bad thing...
Bruce makes some good points, but he consistently undercuts himself "information" that is poorly sourced, poorly explained, or just plain wrong.
The question I'm most interested in is why the "internal only" network at Dominican Hospital went down. Bruce doesn't explain this, and I can't find a reference to it elsewhere. I suspect that he just has his facts wrong — Dominican is part of Catholic Healthcare West, and I'd be very surprised if the computers at Dominican didn't rely on servers in a central CHW facility.
That's still a dangerous vulnerability, just like Bruce says it is. But he'd be more persuasive if he checked his facts.
And dude, everybody but you knows that that internet technology research was funded by DARPA. Some DARPA personnel are in the Army, but DARPA has never been part of the Army.
And can we please stop repeating that idiotic myth about the Internet being designed to survive a nuclear attack? It isn't and it wasn't designed to be. The basis of the myth is that early proposals harped on the superior survival characteristic of a decentralized network versus the star topology networks of the time. Not quite the same thing.
The cut location in San Carlos was reported as being at Bing St and Old County Road. That's actually alongside the rail line that runs up the SF Peninsula. There are many fibre optic cables along that right of way. It used to be a Southern Pacific Railroad line, and "Sprint" was originally Southern Pacific Communications.
There aren't that many long haul fibre optic cable routes. Many of them run along rail lines, because the railroad owns the right of way and doesn't need anyone else's permission to run cables. Often you can run cable for miles without crossing a street, which makes installation much simpler.
Its implications, though startling, have gone almost un-reported. So I decided to change that.
DUUUUUUPE
http://tech.slashdot.org/article.pl?sid=09/04/09/2044205
I guess it's kinda reasonable to use the term for an attack on the "cyber" domain (by going after its physical substrate) as well as for attacks that occur within that domain. Either way, it screws up people's access to comms.
I don't think it's reasonable, at least not enough that we should accept it and start using "Cyber Attack" to refer to the target of the attack rather than the means. The reason basically boils down to the opposite of attack, which would be Cyber Defense, and what was mentioned earlier on /., the Pentagon Cyber Command.
If we accept this meaning of Cyber Attack, then that means that an airplane that drops a bomb on an ISP is a "Cyber Attack", while bombing any other form of infrastructure would be a "regular attack". Logically this would also mean that an anti-aircraft gun that is placed near an ISP is a form of "Cyber Defense". Except that isn't logical, it makes no sense. Anti-aircraft defenses should not be under the purview of Cyber Command regardless of where they are located.
No. I insist that the adjective "Cyber" before the word "Attack" should indicate the means, not the target, in the same way that Cyber Defense should mean securing computer networks, not preventing physical assaults that may or may not happen to hit internet infrastructure.
This was nothing more than plain ol' sabotage. It's the same as them destroying a sewage line, except the impact was different. If it was a power line, that too would have cut off many forms of communication, is that a cyber attack? No. It's an attack.
The enemies of Democracy are
Hey hey hey! Hold it right there. I think there is LITTLE that strikes fear into the hearts of most nerds like being cut off from the net. So before you go waving your burning flags and crying alla akbar, maybe you should consider whistling like a 14.4k baud, and putting on that "I make Token Ring Networks" t-shirt.
On slashdot, THAT my friend, is true terror.
If you want to be a real tard, maybe throw in some rhetoric about head to head gaming via Com1!
Moved to http://soylentnews.org/. You are invited to join us too!
The day my employer strips me of my loosely bound electrons is the day I give notice.
Unless of course I'm on the "to be negatively charged" list, in which case:
Sucks to be you, alkali employees!!!
"Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
Thirty years ago Arco Oil and Gas had full data center backup. Where is this thought today in our attention deficit management world?
ARCO did NOT depend on their local Plano TX data center. ARCO had a building prepared in Independence KS on top of pipelines that was an empty data center. They had a contract with IBM to get the next big iron off the production lines. That combined with their backup tapes means quick switch over.
ARCO also never allowed all top executives to travel on the same jet. They flew TWO jets with passengers selected for functional redundancy. Two jets to the same location by the way.
I like the idea, if possible, of local redundancy. Like hospitals have generators.
I would appreciate examples of backup and redundancy today. These quiet things are often unnoticed.
Cheers,
Jim
Sure, you can do things like reducing single-points-of-failure, beefing up security, but you can do this only to a point. At some point, you realize that society is, by nature, cooperative, and if you remove that basic assumption of cooperation, society will fail.
There aren't any exceptions to this. There are just too many possible things that can be destroyed by people who desire a society or civilization to perish.
You can salt fields. The Romans did this thousands of years ago, and the areas they ravaged are, to this day, incapable of meaningful agriculture.
You can poison drinking water. LSD is pretty easy to make cheaply, and a single pound of it thrown into a public water system would cause mass insanity.
This list is infinite: You can destroy power lines, you can cut fiber cables, you can make a bomb out of fertilizer and destroy a building or the Golden Gate Bridge or any of a quintillion other things that are both easily done and highly destructive.
A society is secure when its population are generally happy with it continuing. When a society reaches the point where enough of its population are disenfranchised with it, it will becomes incapable of maintaining the critical infrastructure necessary for a complex civilization. Adding security measures such as multiple points of failure quickly become reasons NOT to fix why anyone would want the civilization to perish in the first place - and thus actually make the civilization LESS secure.
And that's just the simple truth of it. So, if we want to be secure, we need to clear up the reasons why people would want our culture to fail. These include things like
A) Not torturing people.
B) Allowing other countries to be sovereign in their own affairs.
C) Not being overly greedy with our wealth. Exploitation is only good for the short term - it's a long-term destabilizing force and that's bad for everyone.
Really, I don't get it. You get people who swear by our Constitution yet somehow think that torturing is OK. Perhaps they should read the 4th and 5th ammendments? This issue is a deep, dark stain on the freedoms we are otherwise so quick to espouse.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
hospitals have generators.
In addition, resources that should not have failed, like the local hospital's internal computer network, proved to be dependent on external resources, leaving the hospital with a "paper system" for the day.
Hospitals have generators, true. But I know of one hospital that keeps all of it's patient records via remote Windows terminal sessions to a datacenter in the next state.
Not a small hospital either. A huge one. And it sounds like that is the norm.
Windows terminal sessions. Not a remote database for redundancy. Not something that can be cached. A hospital, with complete dependence on a single real-time data link across hundreds of miles. Let that sink in.
"I assumed blithely that there were no elves out there in the darkness"
It's not hard to get private entities to build redundant systems as long as they get paid for it - they're trying to sell reliable service to customers, and many kinds of customers need redundancy, and it's very hard to provide even regular reliability without it. If they had had better geographical diversity down there, then the vandals would have had to cut two different manholes in south county to do the job instead of cutting one down there and one up in the location they vandalized. Post-2001, it _is_ harder for businesses to get information on what redundancy is available, because while they all are much more aware that they need it, the governments have pushed the never-tell-anybody-real-locations paranoia - and realistically, while everybody can tell that the large building downtown with no windows and a faded bell logo on the wall is a telco office, the only way they can tell where fibers are is to look for the "Don't Dig Here - Fiber" signs which don't tell you which ones are critical.
What's hard to get is Right of Way, and governments can sometimes help that but often interfere - highway departments can be really difficult to deal with, compared to railroads which are usually much more helpful because they're in business and you're paying them. It's especially a problem in the area south of San Jose, because the government regulators constrain ex-monopoly-telcos to operating in LATA boundaries, and they're near several LATA boundaries down there (because it used to be mostly empty farmland, and a lot of it has hills that aren't stable enough to put significant housing on, so most of the area is either reservoir watersheds or cattle ranches on one side of the freeway.) It used to be that the only industry down there was one railroad company, some farmers, and biker bars, and it was 30-40 miles from Watsonville up to the San Jose POP, a frequently-flooding river between them and Santa Cruz, and a LATA boundary between them and Monterey. Even so, I found it surprising that one well-placed cable cut was enough - usually there's one direct connection available and if a business customer needs redundancy, you can find them a second connection but it'll cost a lot more because it has to go a lot longer.
But even in northern Silicon Valley and the peninsula, there are a number of areas that don't have as much redundancy as they'd like because the locations where telcos can cross freeways are limited. From a nationwide carrier perspective, things are better - while there are some constraints, like a limited number of railroads and highways crossing the Rockies, and a few major cities that have limited numbers of bridges and tunnels, so cable cuts out west will cost you a bunch of extra milliseconds, but the carriers do have alternate routes, and the growth of Microsoft and the Phoenix-area financial and high-tech data centers has meant that everybody's got extra capacity on the northern and southern routes as well as I-80.
The one other source of right-of-way I'm familiar with was a gas pipeline company that ran lots of fiber along their routes. They had a certain advantage over the rest of the industry, because while Bubba the Backhoe Driver might ignore a "telco fiber - don't dig here" sign, a "Gas Pipeline! Explosive! Flammable! Don't Dig Here or You'll Blow Up and Die" sign generally got its point across better.
Disclaimer: This is entirely my personal opinion, not that of any current or past employer.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
"Ramrod" is an interesting way to refer to me :-)
Although I would like to see more folks become hams, I did not work to eliminate the Morse test just for quantity, but because having Morse on the test didn't make sense for the (then) next century's amateur radio. The survival of Amateur Radio was a goal. Some hams asked me to let it "die with dignity". To heck with them.
I would be happy to see a more intensive technical exam.
As it happens, U.S. ham numbers are around 8000 higher this year than last, but about 20,000 down since 2002. We still have yet to see if we can achieve stability or increase, or if the service is still declining in numbers. Some of us still wonder if we will see it die in our lifetime. That would be really sad.
Bruce Perens.
I am the Network Administrator for an ISP (AS4307) in San Martin, CA (between Morgan Hill and Gilroy) that was directly affected by the cuts.
We are multi-homed by two providers. BOTH providers fiber ran through those SONET rings that were cut. We were COMPLETELY isolated (internet, POTS AND cell) from 2:15am to 10:42pm. Luckily, 90% of our customers are in the Morgan Hill/Gilroy/San Martin/San Jose area, so they were fully aware of what happened.
As a side note, the cuts were actually in San Jose. I live 3 blocks from where the cuts occured (Monterey Hwy and Cottle Rd. for those interested). And it did not just affect Morgan Hill. Some parts of South San Jose were affected, along with Morgan Hill, San Martin, Gilroy, Watsonville, Santa Cruz, and parts of Hollister.
What was interesting was when service was restored, customers who lived out of the area who had not heard of the happenings here, called and told us they thought one of two things:
A) We went out of business
B) Natural disaster (Earthquake was #1 on the list, considering where we are located)
We lost no customers over this fiasco, and are now looking at getting a provider that feeds from completely separate fiber (i.e. from the SOUTH)
Robert Glover
Director of I.S.
South Valley Internet (AS4307)
Pretty easy to take out public safety trunked systems too. All you need is a hammer and some nails.
In my city the repeaters are on telephone poles. Just punch a hole through the feedline. If the repeater designer knew their shit they'll detect the high SWR an shut down the oscillator and amplifiers. But I can tell you, I've seen lots of gear that has no such SWR protection.
You don't even have to go that far. A little conductive grease, or even water in a connector will also reflect lots of RF power back to the emitter.
It is virtually impossible to protect any given communication medium. You must have several independent means of communication.
-- Vernor Vinge, A Deepness In The Sky.
Always we hear that something should be privatized because private industry is more efficient. Yet never does anyone stop to ask whether efficiency is the only concern.
Thus rather than having a reserve in transmission capacity on our electric grid, since deregulation we simply eat farther and farther into former safety margins. Rather than spend the time to set up proper local mirrors of systems, hospital networks collapse when their Internet connection breaks. It's reasoned that the time-integrated cost of safety margins exceeds the price to be paid when failures they would have prevented occurs.
And so far, they're mostly right. We have a little more latitude for technical failures on Earth than the fictional inhabitants of Namqem. But eventually, as we hop and skip blithely into privatization of core systems, we're going to pay a horrible price for it. It's sad how many innocent lives it's going to take, but no one listened to those calling for improved maritime safety until Titanic sank either.
Heading south on 101 from San Jose Thursday morning, I was on my cell as I usually am. It cut out about 4 miles north of my Morgan Hill exit. I thought it was a dead spot. I got to work and realized we had no phones, no internet, no cell if you were a Verizon or AT&T customer. The only link we had was AM radio (KGO), who told us of the outage. We needed an ETA for restoration of service. How? We drove north until we had cell coverage, and called our respective providers. Neither had a clue. We called our spouses outside the DOS area and they said that cables were cut, but still no ETA. Finally we heard on the AM radio they expected to restore service by end of day. We ended up sending our customer service and order entry people home, and the rest of us worked the internal network or paperwork for the day. The phone came back around 4 PM, but the internet and cells never did until the next day. No 911, all the stores and restaurants were pretty much cash only. It was truly eerie. It was front page news for a couple days but has faded from view since. We think it was almost certainly the union, since the first thing the union did was vehemently deny they had anything to do with it.
I hate being bipolar; it's awesome!