Slashdot Mirror
The Backhoe, The Internet's Natural Enemy
Juha-Matti Laurio writes "Experts say last week's Sprint outage is a reminder that with all the attention paid to computer viruses and the latest Windows security holes, the most vulnerable threads in America's critical infrastructures lie literally beneath our feet. A study issued last month by the Common Ground Alliance, or CGA -- an industry group comprised of utilities and construction companies -- calculated that there were more than 675,000 excavation accidents in 2004 in which underground cables or pipelines were damaged." I estimate that one third of those accidents occured within the 5 block radius surrounding my office.
I would love to see what all these "oops" cost. Fiber optic is not exactly cheap, and it is a little more complicated than just reconnecting the severed ends. And then take network down time etc.
30% Troll, 50% Underrated, 10% Interesting
Score:5, Troll
Always carry a length of fiber-optic cable in your pocket. Should you be shipwrecked and find yourself stranded on a desert island, bury the cable in the sand. A few hours later, a guy driving a backhoe will be along to dig it up. Ask him to rescue you.
So the DHS wants to protect this infrastructure by making the location of such lines protected. Which of course is not going to help the situation because when you call Dig Safe they wont know whats under you. So you run the risk of severing more cables, and you run the risk of injury to the workers. I tip my hats to them.
30% Troll, 50% Underrated, 10% Interesting
Score:5, Troll
That reminds me of when Qwest cut all telephone lines to my home town - including 911. It made the local news, and the police chief and fire chief were both pretty pissed about it. They had to increase police patrols since no one could just call in a crime, fire, or medical emergency.
Fortunately nothing serious happened while 911 was out.
Then Qwest did it again, two days later, on the same line...
Ah, telecom monopolies.
You are in a maze of twisty little relative jumps, all alike.
Information Technologist: 0
Red Neck: 1
ah here a great human error story. House gets blow up because they connected the wrong gas line. Home in Lexington explodes. If a company that has the maps of all of its own gas lines can do this. Think of the possibilities when DHS tries to classify the listing of all the fiber optic in the US
30% Troll, 50% Underrated, 10% Interesting
Score:5, Troll
Based on this article, I'd hazard it's either:
1: Backhoes falling off ships transporting them hitting cables.
2: Submarines with backhoes, no doubt performing black ops at the time.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
At our company, we are trying to figure out how to use cable over telephone pole (business class cable) as a backup in case we get "dug up", which would provide a new level of reliability, but I am sure somewhere out there there is still some unavoidable single point of failure that no amount of money can overcome.
A backhoe driver that accidentally digs up your cable, and then backs into the telephone pole?
ZuluPad, the wiki notepad on crack
According to the article, in 2004, nearly half of the accidents were caused by on-site workers not checking with the proper support numbers for underground cables and/or pipelines.
I wonder just how much those incidents would be reduced if companies were fined a stiff penalty for digging without calling these numbers. The type of astronomical fines/penalties levied against virus writers would seem very appropriate in these cases, given the type of economic damage that can be caused by telecom outages.
I'm glad to see that a national calling center is being established (similar to 911, according to the article). Now, it will be easier for workers to call. But I still think we need the other half: better (financial) incentive to make those calls in the first place. -- Paul
OpenSource.MathCancer.org: open source comp bio
And if you'd read TFA, you'd see that the contracter did call. They were given the go-ahead to dig.
This guy's the limit!
Every time I see a backhoe go by I go into an Elmer Fudd voice and say, "Be wery wery qwiet... I'm hunting fiber"
For some reason the Servers and Networks guys don't think it is funny.
Telcos have alot of dark fibre in the States. Most people assume that's optical fibre...but it's actually moral fibre.
Those of use from the telecom world recoginze
this as "backhoe fade" and ARPA has conducted
considable research on the effect of fiber
optic cable to attrace backhoes in the wild...
ARPA Science Research Funding News Today......
ARPA to Fund Network Reliability Research
Washington, DC -
The Advanced Research Projects Agency of the DoD announced today they are
funding a three-year effort to improve the field reliability of
fiber-optic communications networks. The program is aimed at reducing
network outages from damage to buried fiber optic cables caused by
construction machinery. Many telecommunications outages are caused each
year when machines called "backhoes" dig-up underground fibers, cutting
them and causing massive service disruptions.
This phenomenon is commonly referred to as "backhoe fade" and
the uncanny ability of the construction backhoe to locate buried
cables will be the focus of this effort.
Dr. Zweiback Gimfizel of the Marginalia Institute of Technoplasty
has been designated Principle Investigator on the project and
held a news conference today and described the proposed line of
research.
"We are taking a page from the biologists who discovered
the magnetic organ in the brains of homing pigeon. This
organ senses the earth's magnetic field and allows the
pigeon to track its location.
"In like manner, our research will focus
on identifying the specialized organ structure within
the backhoe that can somehow sense the location of glass
fibers."
"The hope is that if this fiber-seeking mechanism can be
identified, measures can be developed to disguise
telecommunications cables, thereby creating "stealth"
fiber bundles which will not attract the attention of
the rampaging backhoes."
In another unrelated statement today, ARPA announced the creation of the
Remote Autonomous Rodent Program which will work on developing specialized
weapons systems for attacking the underground communications systems of
adversaries. In recent theater actions, modern fiber-optic communications
systems have proven quite resilient to traditional attacks and require
new techniques to disable them.
Dr. Gimback Zweifizel of Hardly Yardwell University was designated
Principle Investigator. In a prepared statement, Dr. Zweifizel noted that
this work program was funded for three years and was to produce a field
demonstration of a working system. Other details of the project are
classified.
What would Groucho do?
In California we are required to notify USA DigAlert before all excavation. DigAlert then notifies all agencies with pipe in the area. Most of the time, they come out and mark, the other times, nobody does.
When nobody comes out an marks, and their line gets hit, it's on them. If it's marked and we hit it, it's on us. Accidents happen. Digging around mismarked and unmarked utilities in a big hole in the ground isn't easy.
Personally I'm more worried about my guys hitting a pressurized gas line than someones precious telco wire. Wire gets fixed in a matter of hours.
The Common Yellow Backhoe
The Common Yellow Backhoe attempting to hide from view.
The Hammer Backhoe evolved to fit particular niches.
One line blog. I hear that they're called Twitters now.
I've actually created my own internet outages with my (now sold) backhoe, twice. Neither of them the obvious. I had a 802.11b feed from a neighbor's house, 1.1 miles away (my hill to his tower). Worked great, almost always. Finally figured out that if I parked my backhoe at --> that end of the back yard, it was enough into the fresnel zone of the wireless link that things got wodgy.
Next time I created backhoe fade, was again in an unexpected way. I'd been trenching along the driveway, after dutifully and carefully marking the underground phone line to the house (that the brain-trust from the phone company decided to run next to my driveway, despite instructions not to). I carefully and successfully avoided the cable, no worries there. Then, when reaching juuuuuuust a bit too far over, I got the backhoe stuck in the muddy ditch along the road. Apparently, in the effort to get un-stuck, I pressed down on the cable, which then stretched over a rock in the trench and broke.
The phone company (eventually) got out there and tried to say I dug it up. I showed 'em exactly what happened - yes, I'd been digging. Yes, the wire was marked. Yes, none of my digging was along the wire's path (all true). The cable had clear marks of a pull over a rock, not a cut from a hoe. Shear vs. tension, obvious from inspection.
Phone company guy didn't want any part of explainations until I (a) bet him that I could dig right (made an X) here and find a big rock with a sharp edge "that you people left in the trench of this improperly installed wire", and (b) pointed out that if he's gonna dig the trench, he's standing in poison ivy while doing so, and I could just go get the backhoe and make it easier for all involved.
He called his boss, explained the high points of the situation (including the poison ivy, which inexplicably a guy in his job didn't recognize without help), and they fixed the cable no charge. But, I bet I'm one of those statistcs in the article.
NSA (not a joke). Here is an article from ZDnet about it. AND this is PRE-9/11. What do you think has happened since? http://news.zdnet.com/2100-9595_22-529826.html
A possible solution would be to embed RFID tags every 3 feet or so inside the conduit, allowing for easier location. Code embedded in tag would give owner, pipe or line type, and depth.
In many states there's only one number to call, not several. Anywhere you live in MA (and a bunch of other NE states), you call "Dig Safe" at 1 888 DIG SAFE, tell them the date and location of the dig, and they make sure all the appropriate companies are contacted.
Here in Georgia, USA, at least, you can make one phone call and have all underground gas, cable, phone, sewer, and electric lines located for you. For free. People come from the various services and stick little flags in the ground over the lines.
I had to do this when I dug up part of my front yard to put in a flower bed.
Your design to a real part online: Big Blue Saw
...That way, when he gets lost, he drops the fiber on the ground, waits ten minutes, then asks the operator of the backhoe cutting the fiber for directions.
Simple, easy, flaw (which I'm sure you've already thought of) -- human error. Like the time the construction worker started digging a hole next to my house right on top of the orange paint mark specifying the location of my phone line.
The funniest thing was the foreman trying to fix the line, since the phone company (thank you, SBC) said they'd take a day or two to get there. He was shocked (literally) to find out that phone lines carry electricity. :-)
Backhoes don't care. Why? I'll relate a similar wire cut story. I was called out to our local airport to fix a problem with one of the airlines ground to air radios. This lets ground crews communicate with the flight crew. It is separate from air traffic control. In one room was the radio. In another room several doors down was a monitor speaker that could hear the conversation. I determined there was nothing wrong with the monitor speaker and nothing wrong with the audio and most likely a broken wire. The room separating the radio and speaker was quite large. The wire was run through the ceiling. I had to lift up many a ceiling tile to trace the wire and find the break.
I found the break. The wire had been cut and tied off. There was barely enough wire to splice the two back together. Once repaired the monitor speaker worked again. I was told later by the airline employees, airport facility workers had redone the ceiling in that one room. To me it appeared the workers found the wire in the way of their job, didn't know or didn't care what it was hooked to and simply cut it and tied it off out of the way.
Backhoe operators probably have the same mentality. They want to get their work done. If they cut a cable, it doesn't affect them. They are just doing their job. To solve this problem I would recommend burying fiber next to gas lines. The fiber should be coated with a material that bursts into flame 30 seconds after it exposed to air or cut. Not only will the backhoe operator cut the cable he'll break the gas line as well. The 30 seconds delay is to build up enough gas for a nice explosion. Sure it'll be a mess, but that's one backhoe operator who won't cut any more fiber.
"You'll get nothing, and you'll like it!"
Do I smell a business opportunity that utilizes a Google Maps plugin?
Well in the jarjon used by telcom techs, a sudden and innexplicable loss of 100% of your signal on your T1 line is called a backhoe fade for a reason ;)
Said in that Australian accent we've come to know and love:
Today I'm going to show you one of the wonders of nature. If you look down in that hole there you can long fibrous tendrils. Those are fiber optic cables and they snake their way through the ground all over the world.
Crikey, it's nice to see them. Usually they stay underground so this is really special. Just look at the size of the hole they make as they burrow through the earth.
Oh look! I didn't expect this. The only known enemy to these folks is coming over to investigate. The backhoe. Look at those nasty pointed teeth. I wouldn't want to get caught by them I'll tell you.
I'll just walk away so I don't disturb him. This could get real exciting any moment.
*growl* *snort* dig dig dig dig
Look at that! This is a real treat. The backhoe is digging up the fiber optic! Look at the way those teeth just dig into the soil and expose those poor buggers. Oh wow, just look at it as it tears those fibers to ribbons.
I know it may seem cruel to stand by and do nothing but this is part of nature. Someone has to eat and someone has to be eaten.
But don't worry mate, those fiber optics grow back real quick. In fact, they grow so quickly there will never be a shortage of them no matter how many get eaten by the backhoe.
We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
Neal Stephenson has a hilarious comment on this in "Mother Earth Mother Board", in his description of a big project to lay fiber optic cable in the Pacific Rim.
Q: Why bother running two widely separated routes [for cable from Point A to Point B] over theMalay Peninsula?
A: Because Thailand, like everywhere else in the world, is full of idiots with backhoes.
Q: Isn't that a pain in the ass?
A: You have no idea.
The man who does not read good books has no advantage over the man who cannot read them. - Mark Twain
At one of my previous jobs, we had popped for the whole menu of auto-whizbang-failover magic. Redundant routers, redundant switches, redundant connections from separate providers. Protected to the nuts against outages.
Imagine our surprise when early the first spring after installing all of this, our connection went down. Both T's out. We were more than a little perplexed - the way the odds were explained to us, God himself would've had to smite most of the southeast US to make this happen.
It turns out that it wasn't God, and there was no smiting involved. Instead, over certain stretches, provider #2 was leasing fiber from provider #1, and one of these stretches ran under the edge of a farmer's field in Georgia. Come spring, the farmer comes out with his backhoe, and... well, you know.
For as long as I was there, we were guaranteed at least a half a day of outage somewhere around the beginning of spring. Every time, the problem was eventually reported to us as "A fiber cut in Georgia..." They never would tell us if it was the same farmer every time.
"Hey, the third matrix movie would have been good except for the plot,story, and acting." --AC
You obviously have NO experience in the construction trade. For that matter I guess you have very little experience with things moving as a result of frost. Things burried several decades ago can easily move several feet
There is NO contractor in the world that would accept the responsiblity and/or liablity of locating utility assets (gas, telecom, water, electric and so forth). Each utility provider will dispatch specially trained and equipped technicians to perform this service. The "locater" must be accurate within certain tolerances or the utility assumes the liablity associated with any distruption/repairs including contractor's equipment that was damaged.
Speaking from experience, I have seen a 60 inch water main broken (locator was wrong), a large telecom cable (something insane like 5,000 pairs) running to a 72 story office building (excavator problem) and countless single line telecom cables (just trying to find the damn things using a shovel but electrical tape works wonders).
The short answer is, you can have all the centralized documentation/maps whatever that you want but no contractor will ever put a shovel into the ground until the utilities come on-site and say "You can dig here but not there".
If VISTA is the answer, you didn't understand the question
Over here in Blighty I've been digging with a little mini backhoe for foundations for a greenhouse and found pipe - rusty iron about two feet down which for an 8 inch main is actually shallow. Put a crack in it, but no leakage fortunately. So we called round and Transco (gas infrastructure) reckoned it was theirs and sent a man out.
A short period of digging later and he came out the hole at some speed looking very pale. The said "pipe" had fins on one end and was delivered 60 years ago by some Germans who failed to stop and advise my grandparents of the delivery.....
You're on drugs. NO utility gives depth info, and I'm not aware of any major one-call center that TAKES depth info from the caller. I know for a fact that we don't.
Depth is useless; in Arizona, for example, snow-plows are required to "call before they plow". Why? Soil erosion.
Here in my state, depth is likewise useless; not as much from erosion as it is from grading. Infrastructure goes in first; landscape happens last. It is QUITE common for a 48" deep line to be 24" from the surface after several years.
And that isn't accounting for things that were discovered when trying to bury the lines in the first place; intersections with other plant means you change height at that location. Hitting Bedrock... means you change height.
> had the major fiber carrier in the area... assured him it was 6' down
Not likely. The moron sent to locate the cable may have mentioned the depth in passing, but I work with these "major carriers" and their locators every day, and there is no way in hell they'd say "you're fine to use your backhoe directly on top of my wire up to 5 feet 11 inches deep". Most "Major Carriers", on a long-haul line, will physically PUT A BODY on-site during the dig to enforce the protection of their cables by hand digging over it. If it's an issue, or a very high-value asset, they'll even go so far as to hand-expose it, themselves. They do not, ever, say "sure! Just dig right on top of it".
Ever.
What I'd suggest is that you ask your contractor friend to define what he means by "assure". And as you do this, remember that he's getting sued for being at fault... he won't do anything to deflect responsibility, at all... he certainly won't exaggerate what was said, for certain.
help me i've cloned myself and can't remember which one I am
Is that what that was? I thought we'd found Jimmy Hoffa! I guess we can tell CSI:Backhoe that there's no hurry.
One line blog. I hear that they're called Twitters now.
Things like this happen. It's regrettable, and can be avoided. But not with the current system. I've been in the business of placing underground cable for almost ten years and have seen many such cuts. The main reason? Laziness. Yep good ole plain laziness. We ALWAYS see a cable that we are going to cross by HAND before by power tool. As long as we know it is there. Many of the cuts that our company experience are caused by lack of locate marks. You simply can't avoid what you don't know about. And sometimes it comes from trusting the marks to much. Minnesota law gives us two feet from the mark. In other words, if we dig AT LEAST two feet from any mark on the ground we are not liable for the cut. The problem with this is that locating devices are notorious for being out of adjustment or simply being used in an incorrect fashion. There are companies that get paid to locate for the big telcos and cable companies. They often schedule themselves so tightly that they rush the job and locate very sloppily. I've personally seen marks up to twenty feet off. And you have to understand it from our perspective. Time is money. If it takes longer to find the cable than to simply cut and fix it, then it comes down to which will cost the least. If the marks are more than that two feet off, it doesn't cost us a dime. We try to never do this, but there have been times when we felt it needed to drive a point home to the locators that we need acurrate marks. AS stated earlier.we try to never do this. I have personally looked for one cable for a full day before giving up. Most of the time it comes down to cooperation between the construction company and the locators. We have in the last few years started to locate many things ourselves in an attempt to speed up and make the process less painful. So don't automatically blame the contractor. It may just not be their fault.
Transport Canada (who were responsible for Airports at the time) used to call them "cable finders".