Sprint Routers Stolen; NYC Internet Outage Ensues

cbnet2004 writes "This story on eWEEK reports that late Sunday night a number of Sprint's DS-3 network cards were stolen from a Verizon colocation center at 38th St in Manhattan. Some customers apparently have service back but a number remain down -- it could be a while. The latest rumor on this situation is that some fiber optic cables were cut as well; this could put the affected customers out for days more."

Comments

Re:Was it easy? Why was it not major?

[linzeal]

I used to work in datacenters throughout Silicon Valley and let me tell you that unless they have hired some kick ass security guards than shit gets stolen all the time. Usually small stuff like PDAs, or the like. Once I heard of an entire rack being stolen when it was left outside, thank god they were empty. Security for these places should be like fortx knox, and the second the card was removed there should be of been a notification to the current on-site physical security detail. These systems will not work unless interopabrable measures are taken to make sure everyones eyes are wide open.

Re:Poor security or inside job?

[Neil Blender]

thefts at locales like this are often done by people with at least some inside knowledge of the site's security.

I have shopped around for a data center more than once. The people who take you on the tours are so eager for your business (at least nowdays) that they show you just about everything. One company even took me into a place where pretty much all the connectivity in Seattle passes through (a level 3 node or something, I can't recall the name.) This place was secured by two locked doors with no guards and street level access. I have seen plenty more 'security' that would be pretty easy to bypass. If you were a terrorist, it would be pretty damn easy to destroy many of these places.

Physical Security

[Qbans]

It all comes down to physical security in the end. You can have the most secure network, but usually anyone with physical access to the equipment can attack it in several ways. They can obviously steal it, or steal parts of it. Hot swap hard drives are great except when someone can run up to a server in an unsecured server closet and in a few seconds have all of a company's data in their hands. Obviously most hardware vendors also put password "backdoors" (think default Cisco configs) that allow you to override any passwords, or recover passwords from a serial port.

Most people spend way too much time on thinking of attacks from the Internet or employees, but usually don't look at someone who wants to sabotage the equipment. Computer rooms usually contain all of the proprietary data in a company, and most companies don't put that much effort into patrolling computer rooms for people who shouldn't be there. Executives should make sure that physical security is part of the I.T. plan from the beginning and not an afterthought.

I'm assuming in this case it was in a Verizon C.O. which are usually somewhat secure, but something like this could happen anywhere, computer sabotage I think will become more and more common in the future as businesses rely more and more on them.

Joint terrorism task force???

[pdcryan]

being investigated by New York City Police and members of the joint terrorism task force

That's just great. Not that I don't hope they find the crooks to walked off with this stuff - but once the word "terrorism" pops up, all of the sudden I'm thinking Patriot Act.

These thieves might have gotten themselves some kick butt network hardware - but I bet they won't get themselves due process

Re:Joint terrorism task force???

[Cid Highwind]

But seriously, as much as the internet is becoming the the globe's central nervous system, I don't see much problem with a limited suspension of due process rights with a theft of this scale, even if it still just boils down to theft.

I see a problem. Last year the Patriot act was only for infringing the constitutional rights of terrorists. Last week, it was only for terrorists and drug smugglers. Today it's terrorists, drug smugglers, and network card theives. Who's up next for loss of due process? People accused of robbery? Fraud? Speeding? Keeping overdue library books?

If you start denying due process to anyone it erodes the rights of everyone. Now we're seeing that slippery slope in action.

Re:Joint terrorism task force???

[Malor]

So you only get due process if you're suspected of a LITTLE crime? Once you add a few zeroes after the dollar sign, the rules change?

If we take away rights from whoever is unpopular with the government today, then we don't have rights, period. The whole IDEA of rights are to protect you from the government and other citizens. The worst scumbags in the world have rights. In fact, it's probably the scumbags who most SHOULD have them, because they NEED them the most.

The measure of your rights is what you retain when your your government hates you and wants you dead. Rights that you have only when popular aren't rights, but privileges, which are revocable.

Any group of people that values life over freedom is easy to enslave.

Re:Joint terrorism task force???

[wibs]

So you only get due process if you're suspected of a LITTLE crime? Once you add a few zeroes after the dollar sign, the rules change?

That's funny, after watching the way white collar criminals were dealt with (enron, etc), I was under the impression you were only given due process if you had a few extra zeroes after the dollar sign.
And by due process I of course mean a red carpet...

Re:vulnerability

[idiot900]

Everybody is so concerned with security online. It means nothing if somebody can just walk into your building and take your stuff.

Sure it does. Suppose your data is encrypted using your public key, and you keep your private key with you. If your data is worth more than the media it's stored on, you've just averted a catastrophe by keeping it from falling into the wrong hands.

Re:Disgruntled?

[mausmalone]

I'm reminded of an old saying. "If you pay peanuts, you get monkeys." It used to apply to low paying jobs, but it seems to be spreading to any job where you're treated worse than the equipment (i.e. any non-management job). It's more like "you treat people like monkeys, they act like animals."

Re:Poor security or inside job?

[trawg]

Uh oh - I think after saying that publicly, you now ARE a terrorist.

New York Theives... If it aint locked down

[acomj]

In the mid 1990s parked my GTI in the West 12 th with out of state plates. I was staying at a friends young and naive. Only thing I left in the car was the "Matt pack" in the front seat and a small bag of dirty laundry out of site in the hatch. The Mattpack was a 5 pound lead acid battery with some electronics and LEDs on top for charging and current/voltage control.. About the size of a soda can but square and black. The top contained custom electronics job by Matt Kahn electicrical engineer extrodinare.
Who would want this? Its big its heavy and useless to anyone but me (It powered a flash for my camera.)

Stolen.

So were the dirty clothes.

Basically if your not carefull or tie stuff down in NY it will be stolen. I got over it but I still miss New York

Re:New York Theives... If it aint locked down

[nomadic]

See, that's why New Yorkers don't have cars.

Forget co-lo. Go solo

[rjamestaylor]

Or, if you can't afford solo, go managed hosting with a company like Rackspace.

I do.

I've been to one of their data centers. I met the former black ops specialist who's responsible for building them up and locking them down. Take a router? Ha. You can't get in door of the datacenter, much less into the datacenter.

I'll go back to my own equipment when I need my 1000th redundant DB master. Then I think I can afford to build the redundantly powered, redundantly backed up, quadratically backboned, overly secured, continuously manned building that goes around important production servers. Funny how the facility is usually left out of the equation not only of the cost but of the requirements for 24x7 uptime.

I've seen one too many people lingering in the XO co-lo facility on Barranca in Irvine, CA (last time I was there, anyway) reading the ID tags and ip addresses of the servers in adjacent cages. No thanks. I think I can begin to keep out Internet intruders, but physical accessors always have an advantage (cloop.o or not).

But Quietly..

[bigattichouse]

Ahh.. I wonder if they will actually look on the other machines in the data center to make sure the theft wasn't just a cover for loading services/keyloggers/etc on the boxes through the data center... sort of a distraction with the fringe benefit of some sellable hardware.

Re:How they did it

[lucifuge31337]

Dude...we're not talking about "network cards" in a PC. These are DS3 WICs, probably in 7206VXRs or the like. Not only is there no unscrewing of a case (other than the two thumbscrews at either side), but they're fine to yank while powered up.

What's amazing (and it may not be the case, as we don't know all of the details, I'm sure) is that a simple correlation of the start time of the network down event and the sign-in log and security cameras (if any) hasn't been done to ID who did it. These facilities aren't particularly heavily trafficed by people on Sunday evenings, and they usually aren't all that big.

Re:Was it easy? Why was it not major?

[RollingThunder]

It's all about acting like you're supposed to be doing what you're doing. Act like you own the place, and nobody will say a thing to you.

Re: Be on the lookout for...

[fatman1683]

Not our cards. That's an ATM DS-3 card, and Verizon uses only frame relay on the east coast.

Re:Was it easy? Why was it not major?

[the_weasel]

Right.

Now go back in time to when NT Alpha first came out. Where is your magic Linux-based rescue disk now? I remember when the first of those came out.

Just because its trivial now does not mean it was trivial then.