Building a TCP/ IP Network Over Dark Fiber?

1101z asks: "Well I work for a public access station in a city where a second cable/phone/internet company has moved just started operating. Part of there deal with the city was to let us have (for free) dark fiber links between several location in the city and our studio, so that we would be able to cablecast live from those locations. As the computer guy I would like to be able to interconnect computer networks that already exist at several of those locations, when we are not using the fiber for cablecast. The question is what is the cheapest way to build a TCP/IP network over this dark fiber." I wonder if the fiber being used is related to this story, from a month ago?

Comments

Re:hehe

[Twirlip of the Mists]

Are you trying to be funny, or are you just simple? Dark fiber is just what it sounds like: fiber-optic cable in the ground that doesn't have any light passing through it. It's not in use, in other words.

you guys all suck

[Twirlip of the Mists]

I know it's a Friday night (in some places) but man, the responses on this Ask Slashdot really suck so far.

Short answer: you can set up a TCP/IP network over a dark fiber link for as little as a few hundred bucks, if you can find equipment for a good price. Here's how.

I'm going to make a couple of assumptions here; correct me if I'm wrong. I'm going to assume, first of all, that each link you've got access to is actually a pair of links; that's the way dark fiber is almost always sold. Second, I'm going to assume that you've actually got a dark fiber link, as opposed to buying a lambda. (Buying a lambda means that the telco is letting you use one frequency of a dense wave division multiplexed [DWDM] link. Not the same as dark fiber in the literal sense, but the same in most practical senses.) Finally, I'm going to assume that the telco has provided you with the necessary repeaters on the line so that you can actually push light from one end to the other without any additional hardware. If your telco has sold you (or given you, whatever) "dark fiber," chances are that all three of these assumptions are true.

If all of those things are true, then you're in a really good position. You can run anything across these fiber links that you could run across a shorter length of optical cable: FDDI, Ethernet (any speed), Fibre Channel, FireWire, HIPPI, whatever you want.

You said "cheapest," and what's cheap depends on what's available. If you can get your hands on a couple of old Ethernet switches with 10BASE-F or 100BASE-F (which are simply 10 Mbit and 100 Mbit Ethernet over fiber optic cable instead of copper cable) you're in business. Just plug the dark fiber into a switch at each location and poof! A single TCP/IP network running across the fiber to both sites, at 10 or 100 Mbps depending on what you can find.

My last company had, among other things, some Bay Networks (now Nortel, I think) stackable Ethernet switches with 24 100BASE-T ports and two 100BASE-F ports. I think they sold for about $2,500 when new (in 1998 or so), but should now be available for a lot less used. If you can find some of those used you'll be in good shape. Asante also makes switches like these; I've never used them, so I won't vouch for them, but you can buy them.

Another option would be to bridge Ethernet to FDDI; switches that do this should be available for really cheap, if you can find them, because FDDI fell completely out of favor in the mid-1990's. FDDI runs at 100 Mbps, just like 100BASE-F, but it has to be bridged, and sometimes this can cause problems with packet splitting and MTU sizes, especially on Cabletron switch gear. Unless you're looking at an absolutely killer deal, avoid the FDDI option.

If you want to go with something more up-to-date, you can run Gigabit Ethernet over the fiber links. It'll cost more, but you'll get better bandwidth. A good idea might be to buy a couple of cheap 100BASE-T switches with 1000BASE-T gigabit uplink ports (about $150 each), then equip each switch with a 1000BASE-T to 1000BASE-SX media converter (as little as $200 each).

Any of those solutions-- 10BASE-F, 100BASE-F, 1000BASE-SX, bridged FDDI-- would require nothing more than a switch with the right media type at each end; you wouldn't have to mess with routers or anything, and you wouldn't have to do anything fancy with your IP network. In fact, you wouldn't be limited to running just IP. You could run anything that can be carried over Ethernet: AppleTalk, NetBIOS, whatever.

If you get the gear for a reasonable price, you can run any of those networks for really, really cheap. When the links aren't being used for video, plug 'em in to the switches and go to town. When you're ready, just unplug 'em and go back to video. The link will be down, but neither the switches nor the computers will care.

Re:you guys all suck

[toast0]

good information, one thing i think you may have missed is that there are apparently multiple (more than 2) access points. If you're going to run the fiber at gigabit w/ the converters, then if all the links share a location for one end (which would likely be in the main tv building, but who knows) and you would need a gigabit switch capable of handling all that fiber there. Of course, if its actually more of a ring layout, then each switch would need two gigabit fiber connectors, and you'ld really want more of a router than a switch, since otherwise it'd get ugly. It'd probably be better to have a router in each building anyhow, but it wouldn't be strictly needed, with creative network design.

Re:you guys all suck

[Twirlip of the Mists]

Nah. We're talking about doing this on the cheap. For the central office-- assuming all roads lead to Rome, if you know what I mean-- you might choose to have one switch with the same number of optical ports as you have fiber links, or you could just as easily have one switch with one optical port each for every fiber link. There's zero need to do routing here. If it turns out to be cheaper to buy 6 (or whatever) switches with 12 100BASE-T ports and 1 100BASE-F port each than to buy a switch with 6 100BASE-F ports-- which will likely be the case if this guy is buying used gear-- then you can just cascade or stack all the switches together. It will work fine, because the "three hop rule" only applies to dumb repeaters, not to switches.

I repeat: there's zero need to do routing, even if it's not a hub-and-spoke network. If one building connects to the next which connects to the next, just set it up as a bus.

Remember, the most important word in the question was "cheapest."

The Cisco way to do it.

[Mordant]

http://www.cisco.com/en/US/products/hw/modules/ps8 72/ps4025/index.html

plus a Catalyst switch at each end:

* Cisco Catalyst 2948G

* Cisco Catalyst 2980G-A

* Cisco Catalyst 2950 Series

* Cisco Catalyst 3550 Series

* Cisco Catalyst 4000/4500 Series

* Cisco Catalyst 6500 Series

buy some gigabit transceivers

[Peartree]

buy some gigabit transceivers, plug em into a gigabit switch. BAM! you've got a gigabit MAN at your disposal

Re:No they DON'T suck

[Twirlip of the Mists]

Without knowing the fiber type and the length of the runs it is not possible to answer the question

You're new to this whole "dark fiber" thing, aren't you? When a telco sells you "dark fiber" they're either literally selling you unused MMF, with repeaters in place, or they're selling you an unused lambda. In either case, the interface to customer equipment is multi-mode fiber, and you can run anything over it that you would run over a shorter piece of fiber.

But, what are the distances? Are the runs one kilometer or are they 30 kilometers? What type of fibre is it? How may connections are there in the fibre? Has the fibre been tested(characterization) to determine dispertion levels due to fibre quality, distance and connections.

If you were pulling your own glass, those would all be relevant questions. Since these connections come from a telco, the telco takes care of all of those things for you.

Look, buying dark fiber is like buying a dry pair from the telephone company. You have a pair of wires on one and and a pair of wires on the other, and you can use them as if they were opposite ends of the same piece of continuous copper. Are they really? No. The signals on your dry pair pass through switches and muxes from here to there. But the telco guarantees that the dry pair will act like a single piece of wire.

Dark fiber is the same way. No matter what is actually between you-- DWDM mux and demux, repeaters, microwave links, whatever-- the telco guarantees that the dark fiber link will act like one long piece of MMF.

Going further, what is cheap? It's rather subjective, don't you think?

Sure, it would be subjective if he'd said "cheap." If he'd said, "What's a cheap way to built a TCP/IP network over dark fiber?" there would have been hundreds of good answers. But he didn't say "cheap." He said "cheapest." And my answer, to my knowledge, comes down as the cheapest possible way to run TCP/IP over dark fiber.

Cheap to some people means a couple hundred dollars. It's highly unlikely that he could build the network for that.

It's highly likely that he could build the network for that, if he could get ahold of cheap used 10BASE-F or 100BASE-F gear. With the current business environment, the market is positively saturated with this kind of gear for pennies on the dollar.

The poster didn't come close to providing the required information with the question and got what he deserved.

Yup. As of the time that I wrote my post, the submitter had received about a dozen smart-assed, sarcastic responses and two helpful ones. Par for the course for an Ask Slashdot.

Re:Nothing to do with dark fiber

[Twirlip of the Mists]

He hasn't even explained what sort of fibre it is! Single mode? Multi mode? Frequency division?

It doesn't matter. When you buy dark fiber from a telco, you're given an MMF connection for each end. The link behaves just like it's a nice short run of multi-mode fiber. What actually happens in between-- DWDM, repeaters, microwave links, whatever-- is irrelevant. In fact, if everything is working properly, you'll never even be able to tell that there's anything going on in the middle at all.

I guess a lot of people are confused by the term "dark fiber." It's hardly ever literally true. When you buy "dark fiber" from a telco, what you're getting is an analog optical link to do with what you will. You can run anything over it that you can run over ordinary MMF. Is it ever actually, literally, "dark?" Hardly ever.

Can he afford a fibre ring?

Read the submission again. The telco is providing these links in a hub-and-spoke topology for free. "Can he afford a fiber ring" is a completely irrelevant question.

What sort of redundancy does he need?

None. They're going to use the links for IP traffic only when they're not being used for video. It'll be an ad-hoc network.

How long are the fibre runs?

It. Does. Not. Matter.

There are plenty of solutions here - ATM between sites with LANE, GigE into some 3550s, 10baseF into tranceivers

Fortunately, the submitter gave you a hint. He said "cheapest." Would ATM with LANE be "cheapest?" Of course not. Would Cisco gear be "cheapest?" Of course not. You're not even trying to be helpful, are you?

This guy obviously doesn't even know the extent to which he's in over his head.

You're just trying to make it sound more complicated than it is. Dark fiber is, far and away, the simplest form of long-range communication known to man. Well, maybe smoke signals or cups-and-string are simpler. Shine a blinky light down one end, and it'll come out the other. The question before the group is what's the cheapest way to turn Ethernet into blinky light and back again.

Re:Nothing to do with dark fiber

[narziss]

I'm not sure where you get your information from, but it's responses like this that really bother me. Under FCC guidelines, when a *LEC purchases "Dark-Fiber" they are purchasing an unused pair or unlit fiber between points A-Z; the only equipment between said locations are patch panels and splice huts. And it is SM fiber, 99.9% of the time (When's the last time you saw a bundle of MMF in a manwhole or fiber hut?

The cheapest way to put up a TCP/IP network via dark fiber would probably to be a cheap, used Cisco GE capable ethernet switch with a 5486 (SM/Long Range) GBIC.

If you're going to give advice about telecom, know what you're talking about.

Thanks

Re:Nothing to do with dark fiber

[nathanh]

I don't know how you got moderated up to Informative: 5 because you're spouting nonsense.

The run-length always matters.

The dark fiber is probably single-mode, not MMF, but there's no information to tell us either way.

I can't even imagine the confusion in your mind to lead you to think microwave might be involved! This is dark fiber, not a data service!

ATM with LANE might be cheaper if he can also share voice costs over the fiber. You can find some amazingly cheap second-hand ATM switches these days. Thank-you Dot-Com-Bust!

If he already has Cisco gear then he probably has a GBIC hole ready and waiting for a tranceiver. But without knowing how much bandwidth he needs how could you tell whether he even needs the capacity of GigE?

The question never even mentioned Ethernet, so I don't know where you got the idea that he wanted to know how to convert "blinky light into Ethernet" and back again. He said "TCP/IP network". TCP/IP is not dependent on Ethernet.

I stand by my first post and my defence of the person who got moderated to "flamebait" for saying the truth. This isn't a straightforward operation. If the person who submitted the Ask Slashdot doesn't think things through - or hire somebody to do the thinking for him - then he's going to waste money on hardware before finding out it doesn't do what he wants. Then he will have to waste money again, and again, and again, until he gets something that works to his satisfaction. This kind of irresponsible spending might have been par for the course during the Dot-Com-Boom but it's getting a little hard to bear these days.

Re:No they DON'T suck

[Meleschi]

Okay.

This is getting silly...

When's the last time you ever received dark fiber from a telco?

If the fiber run is going to be over 50 miles or so, it will usually go through repeaters of some type.

HOWEVER if the runs are shorter than that, it's really not the telco's responsibility to put in repeaters for you. As soon as repeaters come into play, you end up with a limitation on that fiber, regardless of what equipment you have at the ends.... OC3, OC12, OC48, whatever the repeaters are rated at is now your limitation.

For the short spans (<50 miles)that don't require repeaters, you're free to go as high as you can with whatever equipment you can afford. There are short range, medium range, long range, and extended long range cards (that I have experience with) that will do more than get the job done. And you're free to choose the speed of card you wish to purchase, as long as the fiber that is run is the correct type for the card you purchase.

Short range cards if I remember correctly, are usually Multi-Mode fiber. Medium to Extended Long Range cards are almost aloways single mode.

Stop running around and saying that having dark fiber means it's lit up by the Telco. It's not. The telco's responsibility is to provide fiber connectivity however is most appropriate considering the distances involved.

Re:Nothing to do with dark fiber

[freebase]

Ok.. Here's there deal. Working for a telco, I have some first hand knowledge here.

When a customer is willing to pay the HUGE fees associated with a "dark fiber" cross country, that's typically what they get - dark fiber.

When your telco is selling you "dark fiber" for local use, metro ethernet, whatever, as long as it stays in the same LATA, and is handled by the same carrier, it's actually just a DWDM wavelength on an already lit ring.

A lot of the time, but not always, the lambda (DWDM wavelength) will be delivered to the Customer Premise on MMF from a shelf in the basement, attic, electrical room, telco room, telco hut down the street, or somewhere near by. If the customer is large enough, they might even rate their own shelf.

Sometimes, though, customers request SMF, which can normally be handled as well. Usually, with SMF, though, a customer is buying a service such as an OC-48.

Now... what's the diff? With the "dark fiber" metro connection above, you've normally got $#@% for redundancy, unless you buy enough of those "dark fiber" pairs to implement it yourself. The telco probably won't give you much of an SLA on it. If they do... you're definitely getting a lambda.

On the other hand, though, the OC-48 service probably has a good (decent) SLA that can be negotiated to an acceptable level.

It's all about price, performance, reliability and control.