Datacenter Robbed for the Fourth Time in Two Years

mariushm writes "According to the Register, the Chicago-based colocation datacenter C I Host was attacked by armed intruders recently, making it the the fourth time in two years that armed thugs have made off with data. According to a letter C I Host officials sent customers, 'At least two masked intruders entered the suite after cutting into the reinforced walls with a power saw ... During the robbery, C I Host's night manager was repeatedly tazered and struck with a blunt instrument. After violently attacking the manager, the intruders stole equipment belonging to C I Host and its customers.' Aggravating the situation, C I Host representatives took several days to admit the most recent breach, according to several customers who said they lost equipment, all the while reporting the problems as 'router failures'."

Comments

The evil thing here

[Z00L00K]

is that it was reported as "Router Failures" instead of the real cause.

And if they have been robbed before - why not increase the security? Four times? - That's some kind of record. Maybe it's time to check if the localization of the whole thing is incorrect and move it to a better location where it's less likely to suffer from this kind of incident?

Re:The evil thing here

[Brian Gordon]

I agree, the routers _didn't_ fail, that's how the thieves got in in the first place.

Re:The evil thing here

[grommit]

In that forum, it was posted that the hosting provider had posted a job application for somebody willing or able to carry a gun. They were hiring at minimum wage or just above minimum wage if you had experience with guns.

So, they're looking to hire people that carry guns that are willing to accept a job at minimum wage. That should tell you something right there.

Re:The evil thing here

[tylernt]

somebody willing or able to carry a gun.

Illinois is one of the most anti-gun states, and Chicago has even more strict rules on top of that. It's almost as bad as Britain. About the only way you're going to have an armed security guard in Chicago is if he's actually a sworn law enforcement officer or you have *really* tight political connections to those in power.

In a more, uh... "free" state, yes, armed security is a realistic proposition. However such states usually have less violent crime too, so you don't need them as much.

Re:The evil thing here

[jcr]

Illinois is one of the most anti-gun states

Sounds like a great reason to locate a data center in Texas.

-jcr

Re:The evil thing here

[nuzak]

It couldn't have happened to a better bunch of scumbags. In fact I wouldn't be surprised at all if they robbed the datacenter themselves to destroy evidence or just for insurance fraud.

Re:The evil thing here

[tomhudson]

They didn't bust through a wall this time - they forced the lock on the front door of the office "suite", according to a customer who went there the next day to check on his equipment.

The "reinforced walls" exist in the same universe as the "router outage".

There were no employees on duty at the time of the break-in. One employee showed up and got himself tazered, AFTER the door had been forced, in response to an alarm.

This was the 4th break-in in 3 years. That alone is suspicious. Taser? Sure, can't kill a co-worker, right?

Security cameras? There's now a question as to whether they (security cameras owned by CI Host) existed in the first place. The only cameras anyone has seen are a few owned by other businesses in the building ...

All very suspicious.

Obligatory...

[mosel-saar-ruwer]

Don't tase me, bro!

Seriously, though, this sounds like something out of a really bad Hollywood B-Movie.

I didn't know you could do stuff like this in real life.

Re:Obligatory...

[Brian Gordon]

No, it sounds like something out of a ridiculously popular A-movie that makes 100s of millions of dollars. You underestimate the american public's willingness to watch total crap.

inside job

[Anonymous Coward]

smells like an inside job / insurance scam 1st would be an anomaly , 4th time i would be looking very hard at the companies and its staffs finances

Re:inside job

[sir_montag]

It most likely was an inside job. A little while back, I was working for a company that was installing some VoIP phones for CI Host and the list of employee phone numbers kept changing from visit to visit - "Oh that guy? No, he doesn't work here any more."

A friend of mine that used to work there said that "being in jail was a fairly common excuse for missing work there". The employees seemed to hate working there, to put it mildly.

And the cokehead that owned the company loved to fire employees at a moment's notice, left and right. I highly doubt there's any employee loyalty there.

So in short, you've got highly unhappy employees that get fired at an amazing rate, with some seriously negative employee loyalty and they're surprised when stuff gets stolen?

To be fair, there was a router failure

[Anonymous Coward]

It turns out the router was unable to route wherever the thieves had taken it.

Not using them anymore

[flyingfsck]

Hmm, I'm not using them anymore. They had regular power failures in Dallas - claiming 'UPS maintenance'. My home DSL setup is more reliable than their data centre.

Re:Not using them anymore

[MBCook]

Ditto. I switched off them a year and a half or two ago. I don't remember what the final rub was, I think it was reliability (website or email going up or down seemingly randomly?). I've been on Dreamhost since, and been pretty happy (note: referral link at the bottom of my website).

I agree with the other posters. They lied. They obviously have no security (or they are facing an inside job). Four robberies in two years?

I'd switch off 'em real fast if I heard this news. I like Dreamhost but if I heard this about them I'd probably switch off them fast too. How can I trust a hosting company that can't even secure their own premises?

Still in business?

[Enderandrew]

After the first robbery, I'd seriously consider moving my data. If my data is still there after the second robbery, I feel stupid. If my data is still there after the third robbery, I should lose my job. If my data is still there after the fourth robbery, I need to promoted to executive management.

The entire purpose of off-site storage is disaster recovery, and prevention of major disasters like this. Why are these guys still in business?

Re:Still in business?

[Anonymous Coward]

Ok, sure it kinda sucks that the place you store your data was robbed 4 times... but what are the odds it could happen a 5th time?

location, location, location

[hcdejong]

Suddenly, buying an old army bunker complex to house your datacenter doesn't seem that excessive.

Re:location, location, location

[rjamestaylor]

This is why Rackspace is moving to an abandoned shopping mall -- better to protect oneself against aggressors, ravenous zombies.

Fool me once....

[SirLurksAlot]

shame on you. Fool me four or more times shame on me!

Re:The evil thing here - continuation.

[Z00L00K]

"at least two masked intruders entered the suite after cutting into the reinforced walls with a power saw,"

In what way was that wall reinforced? Dual layer of sheetrock? If it was sufficiently reinforced it would have delayed the intruders long enough for the police to get there (unless the police chose to not respond). If I was insuring that company I would drop the insurance dead by now due to lack of sufficient protective measures. If the measures were approved by the insurance company I would recommend all other clients to change insurance company.

Anyway - maybe it's time to weave in copper mesh into the T-shirt of all datacenter employees to protect against tazers.

And notice from a comment to the article that any so called man trap doesn't exist - and the security seems to have been far too relaxed. Just a fine example of how not to do things. A good datacenter is located where almost nobody knows where it is - preferably underground in a nondescript location in the countryside. A set of optical fibers will take care of all the traffic. And very few persons shall have physical access to the hardware. Think about how the military handles their datacenters.

A few possibilities....

[dgatwood]

Well, if they actually care enough to try to prevent these attacks, I can see three solutions, any of which should be highly effective:

Deadly force. If you are being robbed at gunpoint on a regular basis, your employees can legitimately say that they fear for their lives, and thus, purchasing of firearms is legally and morally justifiable. Perhaps a couple of guards posted at the entrance with semiautomatic rifles, plus three or four in appropriately concealed locations within the facility (or more if the facility is large enough). Criminals (armed or not) will think twice before attacking.

Electrical interference. Hook a 230 kV transmission line directly to the rebar in the walls. Anyone who tries to cut their way in will likely spontaneously combust, or at the very least, be knocked several meters. Such an attack won't happen twice.

Oxygen deprivation. You probably already have halon fire extinguishers. Assign everyone emergency oxygen masks and a red button remote. In the event of an attack, press the red button and put on your oxygen mask. Assuming you dump enough halon, it will bond with all the free oxygen in the room, incapacitating or killing the intruders in seconds. Assuming they survive, they should still be unconscious when the police arrive to arrest them.

Re:A few possibilities....

[Tim C]

Armed guards are probably legal, and using the fire suppression system you could probably get away with if you didn't specifically give orders to use it as a weapon, but the electrical booby trap is almost certainly illegal. If nothing else it almost certainly contravenes local health and safety laws.

Re:A few possibilities....

[Z00L00K]

Deadly force. If you are being robbed at gunpoint on a regular basis, your employees can legitimately say that they fear for their lives, and thus, purchasing of firearms is legally and morally justifiable. Perhaps a couple of guards posted at the entrance with semiautomatic rifles, plus three or four in appropriately concealed locations within the facility (or more if the facility is large enough). Criminals (armed or not) will think twice before attacking.

If the criminals REALLY want to get in it's not a good idea to arm the employees anyway. Specially assigned guards maybe, but the employees of a data center - no... Just imagine if someone has a bad day... Shooting out at the boss, servers and everyone else in sight. Or if the criminals know about it they will shoot first and check later.

Electrical interference. Hook a 230 kV transmission line directly to the rebar in the walls. Anyone who tries to cut their way in will likely spontaneously combust, or at the very least, be knocked several meters. Such an attack won't happen twice.

Assuming that it's concrete walls... But it's a good idea until the maintenance guy comes in to drill a new hole for a cable.

Oxygen deprivation. You probably already have halon fire extinguishers. Assign everyone emergency oxygen masks and a red button remote. In the event of an attack, press the red button and put on your oxygen mask. Assuming you dump enough halon, it will bond with all the free oxygen in the room, incapacitating or killing the intruders in seconds. Assuming they survive, they should still be unconscious when the police arrive to arrest them.

Halon use is outlawed, at least in some countries since it has a bad effect on the ozone layer. Carbon Dioxide is almost as good, and has the same effect. Of course - you may use any non-oxidizing gas like pure nitrogen or helium instead. As long as it lowers the oxygen level in the compartment. A much more evil way is to use carbon monoxide instead of carbon dioxide. In this case the survivability is even lower, but if it's released by accident it will be much more nasty.

But it seems that the datacenter hasn't taken action as it should and moved the servers to a different more covert location. The daily operation can remain at the same location, but since the servers aren't there anymore the criminals will have to leave empty-handed. This requires that the persons running the night-shift doesn't know about the real location of the servers unless they also are relocated.

I'm one of the victims....

[Anonymous Coward]

I've used them for years, and was an avid supporter of CI Host even while they were enduring constant negative publicity.. I was initially a client of their shared hosting, then upgraded to a dedicated hosting package, and never had an issue aside from the typical short downtime every now and then.. nothing crazy.. so a startup I was working with put a colocated server with them earlier this year and in around 6 months we endure an outage for numerous days, numerous BS excuses, then one day "Oh yeah by the way your server was actually stolen, and good luck finding the real thieves!" So now we come to find that this has happened 3-4 times in the past 2 years, the detective (and even a worker there I talked with) told me they believed it was an inside job. Obviously I am cancelling all of my accounts and taking my business elsewhere. I will proudly do my best to spread the word and tell EVERYONE I know to NEVER use CI Host for *ANYTHING*

Kudos to CI Host!

[rueger]

'At least two masked intruders entered the suite after cutting into the reinforced walls with a power saw ... During the robbery, C I Host's night manager was repeatedly tazered and struck with a blunt instrument.

Good on them for hiring the disabled! Although perhaps the night manager position is not one suited to someone so deaf that they can't hear a Sawz-All cutting though the wall...

I was one of the victims...

[jimijon]

Last November I had ALL my servers stolen there. Now over all the years I have had servers I have backed up data, upgraded servers, clean installed servers, etc., then that fateful day in November after being stonewalled for days I finally went over to the data center only to find ALL my servers stolen. All my data. Yea I had some offsite backups, etc, but .... stolen. I guess they thieves really liked my XServes. They were nice and shiny in a sea of beige and black. Anyway, they gave me some free hosting after that, so, I said ok.. big mistake... about a month ago two of my three servers were stolen. Thankfully I had them a bit spread at the datacenter. Well, can I sue? What can I do now? Same bs, promising me servers and nothing. Last year I lost a lot of clients,,, granted all but two were mostly very small time hosting accounts. This time I now lost a big client even though I got them back up and running asap... get a server, install configure, read the files from the backup server, etc. Anyway it was truly the worst feeling I have had in many a year. It is definitely bs. And what do the Chicago PD do? Well your guess is as good as mine.. maybe they are out ticketing the thieves truck as their meter runs out. Any lawyers out there that can help?

Re:The evil thing here - continuation.

[Anonymous Coward]

I actually had a server hosted in that very Chicago facility. (I actually got referred to it by clicking a "$75 a month colocation" advertisement link on slashdot)

The datacenter in question is in a terrible neighborhood, and I can't see anyone bothering a truck there in the dead of night.

There was no man trap, and no security of any sort, just a tech guy who let me in and opened the glass datacenter door for me.

I doubt they have a panic button of any sort either.

You disable the one guy on call and there would be no police coming, period.

CI Host Chicago

[Average]

I've actually been in this datacenter. Tried to host some boxes there for a while... and when I finally gave up on their shenanigans, I was not near Chicago, so I just abandoned them there (cheaper than shipping).

First, this datacenter is literally two blocks from what is left of the infamous Cabrini-Green projects. Tough neighborhood, so it's not entirely impossible that it is an outside cracked-up scheme.

There was none of the double-man-trap doors or whatever there. The one staffer was in the back playing a Playstation. The couple of customers in the center exchanged cell numbers, so we could call each other to get let back when we needed to use the toilet.

The Dallas billing people weren't any better. Worst... host... ever.

Your server is down..

[Anonymous Coward]

down at the pawn shop!

Who is going to buy the gear?

[Stu101]

Ok, so they robbed $50,000 of routers and servers.

Where are they going to fence them. The average geek has no need of 16 core Xeons, no matter what game they play. If they were dells, (IF) they are going to have TAG #s and it wouldn't be hard to see Dell doing a trace on em, ie very hot property.

Thirdly, no legit business, at least any I have worked in, would touch (some) state of the art servers at half price, no support, from a questionble source with no history. Same goes for all the cisco kit. Bet they end up abroad.

Cut through RC walls? Sounds fishy to me.

[Overzeetop]

Seriously, cutting through a reinf. concrete wall is not trivial, if it was indeed just that. By code, the minimum thickness of a concrete wall is 6" and most used for loadbearing in anything but the cheapest residential construction are 8". You aren't cutting that with a reciprocating saw (aka Sawzall). Second, reinforced concrete walls are required (in order to be considered "reinforced" by code) to have steel bars equal to 0.0014 x wall area in both directions at a spacing no greater than 18". That typically works out to a 1/2" steel bar at 12" on center or a 5/8" steel bar at 16" o 18" on center both horizontally and vertically.

Now, this is a non-technical publication, so "reinforced" may mean anything - like a 1/2" bar at the top and bottom, and around jambs. Also, this is Chicago, known far and wide for severe corruption in the building inspection process.

Still, anything close to a RC wall is going to require a diamond blade and a gas powered saw for any kind of efficiency at all, and the cut rate is going to be measured in single-digit (or fractional) inches per minute. Most also require a water source for cooling. You'd have to be utterly incompetent not to catch these guys before they got in.

Re:where are the rent a cops at?

[Deadstick]

In the brake room watching tv / playing games?

No, one door to the left in the clutch room.

rj

Re:The evil thing here - continuation.

[billcopc]

Stronger walls, and maybe armed security guards. Heck, we have them up here in Canada and we don't have a tenth of the violent crime problems Chicago has.

couldn't of happened to a nicer company ;)

[Indy1]

As a company that host spammers, and threatens lawsuits (cartoonies) against anti spammers, I can only hope the crooks stole the spam servers as well.

http://www.spamhaus.org/sbl/listings.lasso?isp=cihost.com

Re:And the police were where?

[NormalVisual]

All the more reason for armed guards or even armed employees

Good luck there - this is *Chicago*, remember. They, like many other large cities, much prefer to disarm the populace and then pretend that there's no more gun crime as a result.

Here is a copy of the police reports

[inject_hotmail.com]

I found these links to the report from a post on theregister.co.uk

Report 1 Page 1
Report 1 Page 2
Report 2 Page 1
Report 2 Page 2
Report 3 Page 1
Report 3 Page 2

The guy says that $50,000 worth of stuff was stolen...not only servers, but misc crap like routers, and battery chargers for Black Berry units.

I'd say either look for a new web host startup in the Chicago area in the next year, or a lot of stuff going cheap on Ebay.

The saddest part about this is that the crims clubbed and zapped some innocent guy that would have offered zero resistance. For this, I hope they thieves go to jail for a long time.

Maybe they

[monzsca]

replaced the servers with an IBM BladeCenter.

Advertising for a guard - "bring your own gun"

[tomhudson]

I couldn't make this stuff up. They're advertising for a security guard,

"We are seeking motivated individuals" ... translation: work cheap.

"Prior security experience preferred." ... translation: not really a requirement, but if we can get it at no extra cost ...

"Some College is also preferred." ... if you managed to drop out of college instead of high school, you're more "presentable" to our insurers, who are now royally pissed at us ...

... and if you scroll down to the bottom of the page:

"Armed Hand-gun license/permit and ability to supply own weapon a Huge Plus! : translation: "we're cheap! You're desperate AND stupid! Let's talk!"

Re:Advertising for a guard - "bring your own gun"

[tomhudson]

1. The police think it was an inside job
2. The employee wasn't on the premises when the door lock was jimmied open (no, they didn't "cut through a reinforced wall" - they just forced a door open according to the police report and people who went to visit the site the next day to check on their equipment)
3. Fat chance getting a concealed carry permit in Chicago.

   Illinois is one of the few states that has no provision for the concealed carry of firearms by citizens. Open carry is also illegal, except when hunting. When a firearm is being transported, it must be unloaded and enclosed in a case.

   Only cops and military get CCW permits.

CI Host doesn't want to spend the money on secure facilities, instead replacing that with a rent-a-dumb-warm-body. Dumb, because taking this job w/o a permit is just begging to be thrown in jail, and if you have the permit, you can sure as heck do better than CI Host.

Re:Bringing own gun is a good idea

[tomhudson]

You can't GET a carry permit in Chicago, unless you're a cop or military.

CI Host didn't invest in proper facilities. Contrary to the article summary, the robbery was made by people forcing open the door lock to the office, when nobody was there, and an employee "just happened to show up later in response to the alarm". And the crooks "just happened to have tasers" instead of guns. And the crooks "just happened to steal all the non-existent video surveillance cameras".

And the walls are not "reinforced" - they're plain ordinary office walls. Unless you want to count a new coat of paint as "reinforcement".

Re:The evil thing here - continuation.

[Lord Ender]

if you don't know what a datacenter is, you're probably nowhere near interested in this entire field.

The fact that he is asking at all proves that he is interested in the field.

You must have very poor memory, nuzak, if you can't recall what it was like learning your trade.

When I was in high school, I knew I wanted to work in the computer field, yet I didn't know what a datacenter was. If I had met a pretentious jerk like you, I might not be managing a datacenter, today. Luckily, I met people who answered my questions instead of people like you. I also asked questions on slashdot, and learned quite a lot from this place.

And to answer the original question: A datacenter is a room full of computers with experts monitoring and maintaining them 24/7. It has special air conditioning, wiring, and security. Sometimes people rent servers or space there. Sometimes they contain only the computers of the company that owns the datacenter.

Frankie & The Boss: Planning a snatch and gra

[IonOtter]

Okay Frankie. Youse got da info we need to break inta da place, right?

Yeah boss! Lookie here, on da webpage!

Name: CDC-03
Location: 900 North Franklin, 3rd Floor, Chicago, IL 60610
NPA/NXX: 312/640

Not bad, Frankie...not bad! Uh-oh...what's dis? Dis could hold us up...

No signage, nondescript building

No problemo, boss! See? They gave us a picture!

*snort* An dey call us teeves dumb...