Slashdot Mirror
Vulnerability of Telco Switching Equipment
call -151 writes: "Interesting New York times article about the Sept 11th attacks' effect
on the Verizon switches in lower Manhattan. Turns out there
was a problem in that much of the network switching was in one
building and it has taken a while to restore service. Sounds like there
is lots of pondering about the vulnerability of the network,
even when it is distributed across many physical locations.
Of course the attacks are making lots of people rethink their
vulnerabilities, but the estimate is for five years' work before there
could be redundant paths for the lines into their switches in
the one building, with no plans to spend the money to do it.
Maybe someone should send them a few hundred thousand 'self-install'
kits like they do with their DSL service ..."
So they are saying that if you take out a large telephone networks Central Office, people connected to this office will suffer lost connections. Infact some long distance connectivity will suffer as well.
Why does this suprise anyone. Hmmm let me see, if you take out your ISP, all of the sudden you will loose connectivity to the internet unless you pay A LOT of money to have a second line put in. Even then the chance that both of those lines run through some common area is pretty high.
Things are easy to engineer with fully redundancy, what isn't easy is to do it cheaply enough that people will still be willing to pay for it.
This makes perfect sense: the Internet did well because it relies on smart endpoints (computers) and unintelligent routes. The best routing, then, is equal speed routes from and to every endpoint and we see something approaching this with multiple routes connecting small groups of hosts.
The phone company relies on dumb endpoints (phones) and a smart system in the middle. The best (simple) routing solution would be every phone connected by a line to a central switching station. In an urban area, this is exactly what we see- one or two central switching stations or point of failure.
This really shouldn't be any surprise at all.
As a side note, this is also why growth and development has been much faster than on the phone- to change the phone system you have to change one place - but no one will let you, because you might break it for every other customer. On the Internet I can tinker with one or two machines and everyone else is unaffected.
I don't suppose anyone else remembers the
infamous fire in a Bell Canada phone exchange in Toronto. This fire knocked out phones in much of the city for a couple of days as the crews scrambled to fix things. It was interesting trying to do business....
In my company's case, we still had working Internet via ISDN, so we were still able to go about our business. Some cell phones weren't working, however.
You can accomplish anything you set your mind to. The impossible just takes a little longer.
for reasons below.
Might sound like a troll, but here goes. If you would like more specific info on the tech, reply and I'll reply to you.
I work for a large regional telco in Canada.
I routinely work on various switching and transport equipment. I think I'm finally somewhat qualified to post to slashdot on atleast one topic.
Modern telco equipment is VERY expensive. Large transport shelves will range anywhere from $150 000 to $400 000 per shelf (Canadian, transport being the fibre equipment) There will be several of these shelves per Central Office.(found in every neighborhood) Cards to fill these shelves will range from $8000 to $70000. (they burn out WAY more than I like, usually at 3 in the morning) Switching equipment is even more expensive, the prevelent DMS technology from Nortel Networks is per capita is even more expensive. I would imagine their competitors prices are about the same, although don't quote me. You will have several of these shelves per office as well.
As well, any good telco will have spare equipment on hot standby - major components at 1:1 and lesser at maybe 1 to 10 or 1 to 8 depending on manufacturing
Incidently, you also need expensive people to program and maintain the equipment. A good example is a DMS technician who will get paid the same as an excellent UNIX admin. (and rightfully so, the DMS is a convuluted enviroment to work in)
Each Office needs to be built to the highest standards, physical security, enviromental controls, backup battery plant and huge power systems to feed the equipment
Outside Plant, (that being the fibre and copper cable), is expensive as well, and even more expensive to maintain, this is why you see very few redundant routes, possibly only within a city. Often there is only 1 redundant route, in the classic SONET ring configuration, and often both sides of the ring have to terminate in one physical location. (office building collapses, phones don't work)
I don't know anything about the telco in manhatten, but I can imagine the catastrophe of losing a major office. If they were cutting corners on redundandcy, (which thankfully happens very seldom in Canada due to the regulations here) I could see major routing problems.
For those of you who thing telephone networking is like IP routing, it's not even similar. It's a hiearchy, you cut off the head, it suffers. Many companys may only have 1 or 2 hosts (a host being the "CPU" of the network.) This is due to the expensive of running a host. Telco equip manufactures charge an arm and a leg and your first born, and the liscensing is microshod style draconian.
What I'm saying after all that is - if you want total redundancy everywhere, it's going to cost more money for service. I don't know what the competition is like in Manhatten - but if you're not paying much for your cell phone, there might be a reason.
Just a thought. Flame away.
As long as you have lots of wire going back to an endpoint, the endpoint is vulnerable. Most CATV systems have the same weakness, too. About the only thing that isn't as vulnerable to a single point of attack is the power grid at the plant level, and that's because of grid interconnection (there were some interesting power grid-related articles in IEEE Spectrum a few months back). But at the local level, a few substations feed large portions of a city - in my city of 40,000 or so a single squirrel took out a large portion of the town earlier this year. And we have our own generating station here, too.
In any tree-shaped network taking out the trunk takes down all the branches. Verizon is just doing what makes (in the pre-9/11 world) good economic sense in not having full redundancy, with multiple paths. What you might see someday in the not-too-distant future is a few areas (like Wall Street) get second switching stations further uptown, but really the best solution for a business that really never thought about the phone network is a dish pointed to a CLEC that isn't in the same CO as the primary circuits from the ILEC.
If Winstar had remained viable they might well be seeing a big demand spike hit about now as corporate DR people realize their potential weakness.
-- Josh Turiel
"2. Do not eat iPod Shuffle."