Slashdot Mirror
Building a TCP/ IP Network Over Dark Fiber?
1101z asks: "Well I work for a public access station in a city where a second cable/phone/internet company has moved just started operating. Part of there deal with the city was to let us have (for free) dark fiber links between several location in the city and our studio, so that we would be able to cablecast live from those locations. As the computer guy I would like to be able to interconnect computer networks that already exist at several of those locations, when we are not using the fiber for cablecast. The question is what is the cheapest way to build a TCP/IP network over this dark fiber." I wonder if the fiber being used is related to this story, from a month ago?
"dark" fiber... is that like anti-fiber? does it have transmissive capabilities, without you able to see it? :)
Goal: TCP/IP over dark fiber.
So far as I can see, you can't do it.
As soon as you try, the fiber won't be dark anymore, invalidating one of the conditions. You can have dark fiber, or fiber with TCP/IP (or just pretty lights for that matter) but not both at the same time.
-- MarkusQ
"Dark fiber" is a nice buzzword, but has nothing to do with your question:
"I don't know how to design a network, can someone do it for me?"
Even if someone was willing to do this for you, the answer's no, since you've given absolutely no details about sites, number of users, applications.. really, anything useful to go on other than that you want to use TCP/IP.
There is no 'network in a box'; everyone's requirements are different. If you would post some of your requirements, we might be able to give you some ideas.
a real source of information.
ITtoolbox
I can not for the life of me remember the name of the manufacturer, the one thing I do remember is that the little rackmount box was expensive. What's your budget like?
Your question, and the way you asked it implies that you have no clue.
You really need to be clueful to build a WAN/MAN, so I would suggest hiring someone who can do it.
Conformity is the jailer of freedom and enemy of growth. -JFK
I know it's a Friday night (in some places) but man, the responses on this Ask Slashdot really suck so far.
Short answer: you can set up a TCP/IP network over a dark fiber link for as little as a few hundred bucks, if you can find equipment for a good price. Here's how.
I'm going to make a couple of assumptions here; correct me if I'm wrong. I'm going to assume, first of all, that each link you've got access to is actually a pair of links; that's the way dark fiber is almost always sold. Second, I'm going to assume that you've actually got a dark fiber link, as opposed to buying a lambda. (Buying a lambda means that the telco is letting you use one frequency of a dense wave division multiplexed [DWDM] link. Not the same as dark fiber in the literal sense, but the same in most practical senses.) Finally, I'm going to assume that the telco has provided you with the necessary repeaters on the line so that you can actually push light from one end to the other without any additional hardware. If your telco has sold you (or given you, whatever) "dark fiber," chances are that all three of these assumptions are true.
If all of those things are true, then you're in a really good position. You can run anything across these fiber links that you could run across a shorter length of optical cable: FDDI, Ethernet (any speed), Fibre Channel, FireWire, HIPPI, whatever you want.
You said "cheapest," and what's cheap depends on what's available. If you can get your hands on a couple of old Ethernet switches with 10BASE-F or 100BASE-F (which are simply 10 Mbit and 100 Mbit Ethernet over fiber optic cable instead of copper cable) you're in business. Just plug the dark fiber into a switch at each location and poof! A single TCP/IP network running across the fiber to both sites, at 10 or 100 Mbps depending on what you can find.
My last company had, among other things, some Bay Networks (now Nortel, I think) stackable Ethernet switches with 24 100BASE-T ports and two 100BASE-F ports. I think they sold for about $2,500 when new (in 1998 or so), but should now be available for a lot less used. If you can find some of those used you'll be in good shape. Asante also makes switches like these; I've never used them, so I won't vouch for them, but you can buy them.
Another option would be to bridge Ethernet to FDDI; switches that do this should be available for really cheap, if you can find them, because FDDI fell completely out of favor in the mid-1990's. FDDI runs at 100 Mbps, just like 100BASE-F, but it has to be bridged, and sometimes this can cause problems with packet splitting and MTU sizes, especially on Cabletron switch gear. Unless you're looking at an absolutely killer deal, avoid the FDDI option.
If you want to go with something more up-to-date, you can run Gigabit Ethernet over the fiber links. It'll cost more, but you'll get better bandwidth. A good idea might be to buy a couple of cheap 100BASE-T switches with 1000BASE-T gigabit uplink ports (about $150 each), then equip each switch with a 1000BASE-T to 1000BASE-SX media converter (as little as $200 each).
Any of those solutions-- 10BASE-F, 100BASE-F, 1000BASE-SX, bridged FDDI-- would require nothing more than a switch with the right media type at each end; you wouldn't have to mess with routers or anything, and you wouldn't have to do anything fancy with your IP network. In fact, you wouldn't be limited to running just IP. You could run anything that can be carried over Ethernet: AppleTalk, NetBIOS, whatever.
If you get the gear for a reasonable price, you can run any of those networks for really, really cheap. When the links aren't being used for video, plug 'em in to the switches and go to town. When you're ready, just unplug 'em and go back to video. The link will be down, but neither the switches nor the computers will care.
I write in my journal
http://www.cisco.com/en/US/products/hw/modules/ps8 72/ps4025/index.html
plus a Catalyst switch at each end:
* Cisco Catalyst 2948G
* Cisco Catalyst 2980G-A
* Cisco Catalyst 2950 Series
* Cisco Catalyst 3550 Series
* Cisco Catalyst 4000/4500 Series
* Cisco Catalyst 6500 Series
I worked in a city government with the exact same arrangement. We used ethernet - yup -ethernet at 100Mbps over the single mode fiber with a copper to single mode converter at each end.
With some of the long haul gigabit stuff, it seems like this shouldn't be that hard as long as the distance limitations are reasonable and the fiber is point to point.
freeswan
freesbd ipsec
both found using google with a blod alcohol of at least 3 o/oo (it's about 8 in the morning in norway, and just typing this took at least 10 minutes).
Acts@core.mailboks.com Acrux@core.mailboks.com Adam@core.mailboks.com Adar@core.mailboks.com Ada@core.mailboks.com
that it's 1:43AM and i'm high as hell watching fear and loathing
Everyone seems to be focusing on puting fibre switches at each location, which I think is an unnecessary expense. The way I read the question you already have existing network at each location, and you just want to hook them together.
It seems to me that you could just get a few PCI fibre NICs and use them to set up existing machines at each location as bridges. I don't remember how much they cost, but it would definately be cheaper than switches. You'd have to make sure you had the right plugs/jacks, obviously.
It seems to me that it would be a pretty simple thing to do.
Under capitalism man exploits man. Under communism it's the other way around.
buy some gigabit transceivers, plug em into a gigabit switch. BAM! you've got a gigabit MAN at your disposal
I work in an office that has something called "CAT5e" cable. I hear that it can be used for computers as well as telephones! What equipment do I need to make a "network" that doesn't involve spending any money. Also if anyone could tell me what I can do with my "network" once I've built it as I dont know.
I did try googling but I just got funny looks.
Ahh, the dark fiber is strong in this one..... he will be a good network warrior.
As part of a deal with a new pet store, the city has been provided several free "dark" carrier pigeons.
Can someone please help me discover what additional hardware I need to do RFC 1149/CPIP?
I would also try "goo-gling" but I'm a grown man and it sounds like something more approbate for a baby to do.
The previous posts, though unforgiving and potentially offensive are right on the money! Your recommendation fails to take into account the two most important details of such a network. Details that the poster failed to provide. They are fibre type and distance.
Without knowing the fiber type and the length of the runs it is not possible to answer the question, and anyone who has a clue about it would know that type and distance are imperative.
Your recommendations would be great, assuming that the fiber runs are multimode fibre at under 500 meters. But, the post was talking about a MAN which suggests far greater distances than your proposed campus network. But, what are the distances? Are the runs one kilometer or are they 30 kilometers? What type of fibre is it? How may connections are there in the fibre? Has the fibre been tested(characterization) to determine dispertion levels due to fibre quality, distance and connections.
Going further, what is cheap? It's rather subjective, don't you think? Cheap to some people means a couple hundred dollars. It's highly unlikely that he could build the network for that. Cheap to some other non-profits that need such a network could be in the millions, as could this project. What's cheap to him? Budget information is imperative with an optical network. You have to have a power budget for the optics(not dollars) and you have to have a financial budget.
The poster didn't come close to providing the required information with the question and got what he deserved. Your recommendations, though well intended, are not applicable to the situation, in other words, just some wild ass guess(SWAG).
RFC1149 networks are not terribly complicated but, they can require a fair bit of support equipment to keep them running at their best. Below is a list of indispensible items.
Pidgeon house at each site.
Lots of bird seed.
Dietary supplements(not required but, recommended)
Cold climates need heaters for the pidgeon house.
Shovel (For, you know...)
Hose (To wash out pidgeon house)
Dogs (required in areas that have foxes, weasles and so forth)
Beer (Keep maintenance people happy)
Junk food (Maintenance people should never be hungry)
The long answer is that if you haven't done this before you better get some consulting help. Chances are you are talking about a ring topology and are going to be linking sites with different networks.
Perhaps you have telco supplied networks for each office hooked up over DSL or T1. If that's the case, have fun getting routing working without having the telco people disconnect you. Good luck reconfiguring the telco routers for that matter.
Routing complex networks is tough. Do you already have VPN's interconnecting these sites? Are you going to be introducing redundant routes? How are you going to manage these routes? IOS can suck if you've never seen how to configure routing processes. Routing software is also complex. Ripv2 is about as simple as it gets and it doesn't offer much control over which route you take -- the only metric is hop count. OSPF has design guides as big as phone books.
If you have a bunch of nats at your different locations, do the networks overlap? Are you going to have to renumber your networks?
If you are just playing in your spare time, you won't be able to do this for under a few grand. Fibre connections are generally not cheap. If you're lucky you could put a few fibre nics into a couple of linux boxes, but I don't foresee those nics being under $400 each.
If this is to be a business network, do it right from the start or you'll make yourself look stupid. People expect the stuff they don't understand to just work. There will be very little tolerance if services are going up and down and your fibre links are to blame.
-- DrZaius - Minister of Sciences and Protector of the Faith
Assuming that you would want to have your video equipment plugged in all the time and want to hijack its bandwidth only when it's transmitting blank stream, you can hook up (entirely in software) a weird setup which would dump data packets over digital video link (as if they were streams of pixels) and decode them on the other end, assuming that your video streams pass thru some kind of video caption cards inside more or less general-purpose computer... :)
;-) )
Yeah, it might be slow, but as a next step you might consider some steganography: why not use couple LSBs in each pixel for data and the rest for video?
(If anyone cares to moderate this, please mod it as "funny", not "informative"
Paul B.
The TOS stuff isn't relavent to dark fiber, because in effect you aren't going outside of your private network, so you control everything. If you also run and own your own PBXs in both locations, you should be able to connect them over the fiber as well. This probably would not be voice over IP, but it could be.
WRT FDDI, in my experience, FDDI was already well on the way out for the last of the dotcom period. I'm not surprised that a lot of it didn't work well, and if I was sitting on a cache of FDDI hardware, I'd probably junk it and take the right off. I'd be interested in a more informed report, on what are/were the more popular technologies. I thought ATM was big with the telcos and big pipe providers. The telcos also have a set of standards are protocols that are more "circuit switched" and related to ISDN where 64K channels that support a single voice channel are agregated to T1 (24 channels at 1.5M in the US, 32 channels elsewhere), and then into higher bandwidth channels.
Yes, it has been three years since the boom, so a lot of the used equipment dumped on the market is probably in use someplace else (hopefully, if you are CISCO). I supose it is likely that whatever is still available (if anything) is likely to be stuff you don't want (e.g. broken FDDI stuff), but again, this is where the consultants come in. Besides, the bust isn't finished, so some stuff will become available. DirectTV DSL is being taken off line (a pain for me), and I doubt they have an use for all of the equipment that will be taken out of service.
I worked with some underground fiber (owned by the city water department) a couple of years ago. It was running a low-speed serial link between a PLC and a computer. My task was to run ethernet over this link (at 10MB or 100MB speeds). The problem was the fiber was a little older and was a larger size. We BARELY got 10MB to run over it. 100MB was impossible without VERY expensive equipment.
If you'd post some specs on fiber size, termination, and distance, we could give you better answers than "just a couple of routers with fiber ports".
LOAD "SIG",8,1
LOADING...
READY.
RUN
I tell you what I'd use. I'd invest in simply invest in either 100FX or Gigabit Ethernet. How big is your city (demensions)? Is the fiber single mode (SMF) or multi mode (MMF)? Almost certainly SMF. How many micron (fiber diameter)? 100FX over SMF can easily travel 20-40Km. Long haul 100FX over SMF (10 micron) handles around 66Km. If it's 10 micron SMF, Gig-E LX will run up to about 32,808ft. Gig-E LH will run up to around 70Km. Each vendor does their own thing though too. Some vendors have out done themselves on distances. And technically the IEEE standards for distance are a minimum not the maximum that many vendors treat it as. With a good routed infrastructure and some QoS you could easily run your video over you IP network simultaneously with your generic network traffic. It's not hard. Also, I haven't even touched on wavelength division multiplexing. Do some research into the possibilities and you'll do fine.
Hmmm...seems everyone wants to either be a smart*** or overanalyze this. First off: It appears that he is working with the local cable company, so, actually, he has already defined what sort of issues and designs he will have with this. Dark fiber from a cable company means a point to point link. If he has multiple offices, this will generally mean that there will be a hub and spoke design on the fiber from a central location to the edge sites. In addition, the local cable co will most certainly provide the distances and loss budget for the fiber, making selection of equipment an act of trivia. The only thing not defined was the type of network he really wants, but in this case, it is also probably completely irrelevant. If you can't put all your services over TCP/IP, then you shouldn't be maintaining a network. First off, gear selection. I work for a Cisco-centric company, so most of my experience is with Cisco, but I have worked with Foundry and Nortel and a few others. If you want reliable, Cisco is certainly acceptable, and you can always put a contract on it to meet your needs. Second, protocol selection. Well, he wants TCP/IP. This will certainly be easy enough. Engineer the backend properly (IP addressing 101, and get a router for the links, your done. If you need to convert Analog Video to H.323, there are plenty of vendors who can do this. If you need COS, again, there are plenty of solutions. Layer 2 Selection. My recommendation would be to go with GigE. This is certainly cheaper than going with outdated FX technology if you buy new, and more reliable. When you have the distances and loss budget of the fiber links, you can select the appropriate GBIC's for the link, and if you get light at both ends, you have a network. In Cisco terms, I'd go with a 3550-12G at the core if you have more than 2 locations. This will be a reasonable fiber concentrator ($9,999 list) and also does QOS, policing, and routing (both packet and protocols, such as EIGRP). At the edge, depending on how much intelligence you need, you can put in Cisco 3550-24's, either EMI or SMI flavor. The EMI boxes are routers (same code as the 3550-12G. The SMI boxes have port routing in the newest code, but won't run routing protocols. These boxes will allow you to converge your network (data, VOIP, and video) and really don't have a horrible price tag. The SMI lists at $2,999 and the EMI at $4,999. This doesn't include GBIC's but the max cost on those is long-haul (Cisco ZX) which lists at $5,995. The intermediate reach LX is only $995. Don't mess around with MMF (SX GBIC's) if you don't have to. Even if you do short haul, this will be a more stable solution and will allow you to repurpose gear in the future. The big gotcha with this is support of the fiber. Most of the time Cable dark fiber contracts mean that the customer is responsible for paying for repair of the fiber. Find out who is responsible for the fiber, and make sure you budget appropriately if the cable company is going to charge you for repairs on the lines. If they won't do it, find someone who can (check the contractors who already work with the cable company, they already know the systems and people) and get them on retainer or whatever. Many schools and governments have ended up shocked because they had to find someone in the middle of the night with a fusion splicer to fix a damaged link. Or the cable company sent them an outrageous bill for repair after the fact. This sucks, because it is hard to budget for an 18-wheeler gone awry in an ice storm. If you do it this way, you can treat the fiber just like Ethernet, and be done with it. You don't have to relearn protocols, and you will have a decent growth path for the future. My networks (that look just like this!) are WAN's that I treat like LAN's, and they are easy to support. Gybrwe
Others have offered hardware suggestions for DIY, but if it was me I'd consider asking the local power co. if perhaps they would consider trading access to fiber in those buildings for hooking you up point-to-point.
I say this because where I am (Toronto, Canada), Toronto Hydro has a butt-load of fiber capacity in over 50% of downtown buildings (prime real estate). The company I used to work for sells internet access (among other things) in a good number of those buildings, and is presently talked to Hydro about getting a) a 10 Mbps internet connection directly from them and b) several "LAN extensions" connecting the building in question (basically VLANning the fiber circuits in each building so that they can see each other). Apparently they're offering this service incredibly cheaply - cheaper than going through Bell Canada or AT&T Canada for conventional T1 or Ethernet circuits.
Your power co. may have (or want) a similar infrastructure in place.
Dark fiber has been proposed by scientists to be fiber which does not transmit meaningful data, that cannot be directly detected by observing the passage of any form of electromagnetic radiation or packets thereof, but whose existence is suggested because of the seemingly overwhelming bandwidths of bad or useless information which exist on the Internet.
Says Dr. Unne of MIT, "The question of whether dark fiber really exists is vital to any theory concerning the origin and ultimate fate of the Internet."
The Internet as we understand it now started with the 'Dot Com' when it began a period of very rapid expansion. Astronomers and cosmologists alike have been trying to figure out whether the Internet will continue to expand forever or whether it is on its way to oblivion in the form of a 'Dot Bomb.' It all depends on the ratio of the actual density of relevant data to anti-relevant data (particles such as SPAMMYons and pron-i-trons).
Cosmologists denote this ratio with the Greek letter Moo. If Moo is less than one then gravity will cause the Internet to recollapse, if greater than one it will perpetually expand, and if equal to one it will expand at decelrating rate perpetually. The critical ratio of the Internet has been worked out to be about 1 objective fact per 3 misleading/incorrect assertions, which is very little considering that it is far greater than the highest intellectual vacuum that conservatives on Earth could ever hope to achieve.
Many cosmologists feel that there is much more fiber in the Internet than is visible through our browsers, and they have observational and theoretical proof for their beliefs:
"Websites such as 'Goatse' and 'Slashdot,' 'Spank the Monkey' banner ads, and 'Loan consolidation' spam mails are practically irrefutable proof that there is an enormous wellspring of misinformation that feeds off the naive. One might even go so far as to say that there is an as-yet poorly-described malevolent force causing the expansion of evil data," claims Rev. Poindexter. "What kind of bandwidth could support -- what kind of pipe is fat enough -- to support this? And are we helpless to keep track of it all?"
- a.c.
Look,
Without the additional details that are "required" by some of the posters, you can only speculate at an answer. Twirlip Of The Mists has done that, and had some very reasonable suggestions.
However, for a complete and assuredly valid answer, we simple need for information. As he said "let us have (for free) dark fiber links" that may mean that he really has nothing but fiber between buildings. Telcos aren't known for their sweeping generosity. He may have a lambda, which is absolutely reasonable in a metro area, which would give him something to work with.
If the fiber has sufficient bandwidth, he could split the fiber into data and video traffic (my high school used an OC3 in this manner for ITV classes and Internet access), but this would likely cost a good deal more.
The issue I have with the whole thing is "...when we are not using the fiber for cablecast." If you want cheap, as Twirlip Of The Mists suggests, that will mean (I could be wrong) physical disconnection of network cabling at both ends each time you broadcast, and then re-connection after broadcast. Would you have trained network people at each "public access" location, or would you end up driving around town before shows? Is it worth it to you?
Granted public access television isn't rolling in money, but to make your life easier, you probably want something you don't have to physically connect each time. Of course, you could get a fiber switch, and some X10 appliance switches and the Home Connect kit. Then you could just call up and switch off the data and switch on the feed.
I want it to be clear that I am not saying that would work.
That what was all this school was for... to teach us how to solve our own problems. -- janeowit
Having the link available only when not being used for video sounds a bit naff. The dark fibre should have a far higher bandwidth than a single (or even several) video feed requires.
Consider using FDMA to multiplex both signals over the fibre simultaneously.
http://www.cisco.com/warp/public/cc/pd/rt/12000/pr ofiles/mimos_cp.htm
OK so Cisco is expensive. But they've replaced an _obsolete_ piece of hardware for our customer for _free_ before. So they do (or at least used to) give back some of their gross margins to their customers.
Cisco's solution should work without repeaters for up to 50 miles/80km.
If the distances are a magnitude shorter than that (e.g. 3-5 km (15 km for single mode fibre) ), you could just try ethernet over fibre. Look for vendors like blackbox, dlink etc.
But once the network becomes useful, switching it off just for cablecasts may not be viable. So you may have to run what you cablecast over IP.
I've got runs! What a country!
You're right, I wouldn't say informative. But it is interesting, as the idea might be crazy enough to be doable (as some of the data-on-VHS posts suggest - actually, the video modem code could probably be adapted for this...?).
If you're feeling REALLY ambitious, you could put the decoding software for your steganography online and let anyone with access to the video stream decode the steganographied data from it, and thus run your own MP3 radio station or something...