Rolling Your Own Internet Connection?

budGibson asks: "Recently, I have become interested in rolling my own Internet connection using a T-1 or T-3. I realize this means maintaining my own DNS, routers, etc. A quick trip to the local phone company's web site suggests this is possible, as does a tour around ATT's data services division . I know that Slashdot at one point in time was going over a T-1 before co-location. Are there any non-vendor resources on what I need to set up my own on-site connection? Are there any tips on what to avoid? What to ask for?"

Comments

setting up a connection

[Clubber Lang]

I'm not sure of the costs in your area, but if I recall correctly a voice T1 will cost you close to $1000/month in southern ontario. Providers like to charge waaay more for data, even though it's basically the same line. A T1 is essentially 24 voice channels at (I think) 64k/sec, but you *must* lose at least one channel for data control per set of T1s, and it's better to have one channel per T1 (much safer). A T3 is like 30x the bandwidth of a T1, so it's kind of like saying "I'd like either plain rice, or filet mignon for dinner". Costs rise accordingly.

Re:setting up a connection

[tzanger]

This isn't really true. When you do a voice T1, this is true, but when doing data, you'll normally only lose 8kbit/sec to overhead (using ESF/B8ZS)... So rather than 1.544Mbit/s, you get 1.536Mbit/s.

Close, but not quite right.

A DS1 (T1 without the electrical specification) is 193 bits sent 8000 times per second. One of these bits is reserved for framing; i.e. you can't touch it. That leaves 192 bits sent 8000 times per second. Calculate that out and you get your "raw" speed of 1.536Mbps.

B8ZS came from AMI. AMI is used on voice circuits only -- it has no provision for excaping long strings of zeroes and in the voice-only days it didn't matter. It was statistically next to impossible to keep an 8-bit PCM coded line at 00000000 for any length of time. However when data circuits started popping up they needed a way to keep clock sync for long strings of zeroes.

B8ZS is a method to encode long strings of zeroes (8 to be exact) -- Like AMI, 1's are sent as an alternating stream of +1 and -1 to keep the net DC voltage of the link at 0V. Every '1' bit allows the clock recovery circuitry to sync up to keep the bit detection centered in the middle of every bit. '0's, however, are the absence of a pulse. Long strings of 0's cause the clock recovery circuitry to start falling out of sync. B8ZS sends eight zeros as a specially coded binary string: for example +1 +1 0 0 -1 -1 0 0 +1 +1. I made that up; I don't believe that is the actual pattern but what is important to note is that instead of the 1's being alternating in polarity, they are the same polarity. This is known as a BiPolar Violation (BPV). Normally the recovery circuitry sees this as an error but if it sees a particular pattern, it instead spits out eight 0's. Because it's a BPV, there is no need to use any kind of escape code so that you can tell the difference between eight 0's and whatever code they use to represent it.

That leaves framing. Remember that 1 frame bit at the end of each 192 bit sequence? In olden times that frame bit was always a '1' and was used to help keep the endpoints in sync. Since newer equipment is very good at keeping frame sync, that bit is used as an 8kHz channel between the two DS1 endpoints and can send out of band (OOB) singal information between them. Things like putting remote ends into loopback, status information, etc.

The telco grouped 12 frames together and called them a Super Frame (SF, and yes I do think of wrestling every time I go over this). Later on they decided to group 24 frames together and call them an Extended Super Frame (ESF). Why the grouping? Robbed Bits.

Robbed Bit Signalling is a way for the telco to indicate the line status. On hook, off hook, busy, ring(back). Your telephone conversation is sampled at 8bit/8kHz and then trunked into a DS0 on the DS1. There are 24 DS0s in a DS1. 8*24 is 192, plus the frame bit is 193. Now to indicate line status over the trunk, the telco created SF and uses that group of 12 frames to steal the LSB of every DS0 on the 0th and 6th frames of the SF to give them A and B signalling bits. Two bits gives you 4 line states. Later on, they wanted room for expansion, so ESF came along and the DS0 LSBs are stolen on the 0th, 6th, 12th and 18th frames of the ESF. Traditionally those C and D bits are just a copy of the A and B bits, but this isn't guaranteed.

ISDN doesn't use robbed bit signalling; they take 23 DS0s for data (as 23 8-bit channels) and then take the 24th DS0 for signal information. Since a DS0 is 64kbps, there's a lot of waste if you just use a D channel for a B channel (or even for 23 B channels) -- that's where NFAS (Non-Facility Associated Signalling IIRC) comes into play: You can gang up to 8 PRIs to share one D channel. So you lose 1 BRI on the DS1 with the D channel, but the other PRIs have 24 D channels. We use that to max out the number of dialup lines we can provide per NAS. Of course, the problem with doing NFAS is that if you lose your D channel, you lose up to 7*24+23 voice/data circuits as well. Most NFAS installations have a backup D channel as well.

So, after that long-winded explanation: You lose no data due to the overhead of ESF/B8ZS. You lose your frame bit which drops you to 1.536Mbps instead of 1.544Mbps, which is the raw speed of a DS1. You don't even lose out if you use a channelized T1 over a nonchannelized one. You *do* lose out on the full capacity of a DS1 if you have it provisioned for PRI instead of DEA (DEA is a Canadian term, I forget what the U.S. term is), since you lose a DS0 to signalling (and with NFAS that problem becomes smaller, but you still lose out on your "theoretical maximum"). And if you're a voice user (this includes analog modems), you lose a bit every 6 frames but since you don't know when the telco is going to steal that LSB, you essentialy have a 56k line. v.90/v.92 modems actually do try and determine when the telco robs the bits when they trainup but that still doesn't push you closer to 64k.

It gets a bit messier when you bring DS1s together into DS2s and DS3s, because there are "slop" bits which are used to overcome the problem with all these DS1s coming in out of sync from each other.

You can cut some corners...

[zsazsa]

We just installed a T1 at work through Qwest and we tried to do everything we could by ourselves that we could. We were pretty successful. We were just insistent that we didn't want to use their VPN, management, DNS servers, and domain registry. We would have wanted to go with a more major player like UUNET or Genuity, but Qwest offered us the best 1 year contract at about $800/mo for a site pretty far away from their POP downtown. Qwest did handle all the wrangling with the local telco, but this was welcome. Dealing with the telco is voodoo magic sometimes. From our research we found it's just as much to get a T1 from a major bandwidth provider than from a local ISP, so whatever floats your boat..

Something that we used to save a bundle was using a WAN card with an integrated CSU/DSU in our existing Linux router machine. We ended up spending just $250 on a Sangoma S508/FT1 on eBay! The card is just ISA but I doubt we'd be upgrading the current routing machine to something with just PCI anytime soon. Not only was it cheap as hell, it was a piece of cake to set up to boot. I wouldn't do a T1 any other way. Sangoma rocks and has been supporting Linux for years (they also support BSD, Windows 9x, NT, Solaris, etc etc.) Check out their website for their goodies. Of course, I have no affiliation with them other than dealing with their great stuff. There are other WAN cards out there of course.

Ian

Not that hard.

[NetJunkie]

People do this for companies every day.

First, you need a providor. I just priced T-1s for our office. They are falling in the range of $1K-$1200/month for an unmanaged connection. This means I handle the routers and everything, they give me the line. For that much you'll get an SLA and a ocnnection to a good providor such as AT&T or Sprint. If you want cheaper go down a tier in providor to someone that buys their bandwidth from the big guys.

After that you'll need a router. The Cisco 2600s are popular. You can get a T-1 card for them with internal CSU. Probably $3K new, $2K used on ebay.

After that it's up to you. Want your own domain? Host your own DNS. Set up a web site...whatever. Don't forget security. You can do pretty decent filtering on the router itself.

With this setup you'll also get a block of IPs and a full T-1 with 1.5Mb/sec up and down. So you can host what you want and do what you want. Is it worth this much over a $40 2Mb/sec cable modem? Not to me.