Slashdot Mirror


User: ender+1978

ender+1978's activity in the archive.

Stories
0
Comments
1
First seen
Last seen
Profile
(view on slashdot.org)

Comments · 1

  1. Bandwidth Costs Explained on Breakdown of Bandwidth Costs? · · Score: 1

    Apologies in advance, this post is lengthy.

    The monthly fee that a user pays is based on a fair amount of items. I will try to summarize what it takes (without particular pricing) what the ISP needs to do to provide you with service. It is easier to show what infrastructure a new player would need to create so I will follow that path. I will go through what and ISP should have to provide you with reliable service. The ISP in question will be a small regional ISP, serving 25000 DialUP customers, 2000 residential DSL Customers (at 1.5 Mbps Down or 128 kps Up) as well as 500 SDSL Business customers. The ISP also provides server collocation and web hosting. The point of this post is to show that the cost providing a given user with bandwidth is not just the raw cost of the traffic.

    Building Rent or Lease:

    The ISP must house its equipment (and or or your collocated server) in a secure location. They must ensure that they have strong physical security as well as enough room to house all of their staff and equipment. Equipment needs to be in a climate controlled room. Industrial strength AC is not cheap, you also need to worry about multiple power sources (possibly two sources of power as well as a large a large generator and UPS). Now that generator needs to be maintained (usually via a support contract) and the UPS batteries have a limited life span. Depending on the quality of power in a given area, you may need to be running off UPS at all times. Such load on a UPS means that the UPS batteries will likely need to be replaced in a few years (36 months or less).Monthly rent is a recurring cost. While equipment such as a UPS is a capital cost, it still takes quite a while to pay it off. Some equipment (like the generator) may not be paid of for 5 years or more.

    Staff:

    An ISP of size above should be able to sustain itself with 20 employees. For 24 or 7 coverage of one person (sysadmin) to be available at all times, one needs 5 employees. As I write this, I had to change the size of the ISPs user base as the amount of employees needed would have likely outstripped the cost. Larger ISPs may be able to justify a SYSADMIN 24 or 7 but smaller ISPs will likely need to have a person with less skills watch the network and then page the more expensive SysAdmin.

    You need someone to answer the phone (an administrative assistant) or a PBX. This PBX should be able to handle at least 2448 calls (when you service goes down, your customers will call and you'll need to offer them something better than a busy signal). You will also need to have staff to take care of the finances and sales. You will need to have a small tech support team (which can be staffed by High school or College kids at a lower cost than a full time experienced worker). At 2530K customers, 20 people may not be enough to run the entire company. You may need 30 tech support personnel alone.

    Collocation Facility:

    Since this ISP is to offer collocation services, they will need to build cages for customers to have access to. Being a small ISP, paying a security guard 24 or 7 is out of the question. A 24 hour NOC tech may be able to act as escort but this may not be feasible. What you will then need to do is offer 24 or 7 access (through some sort of card access system). Card Access systems are not inexpensive. Racks for customer equipment (locked such that customers cannot interfere with one another's service are more expensive than open racks). You will cable each rack with a few drops (data and voice) such that a customer can purchase both an Internet connection from you as well as possibly be able to order a T1 from the Telco). Your own customers may want to start providing dialup services of their own (or provide VOIP or calling card services). Your customers are becoming your competition.

    You will need to run this cable in proper cable trays and have this done by certified technicians (doing it yourself is likely out of the question unless you have staff that have done this before).

    You will give each customer a 100Mbps port on a switch (you will need to figure out how you will be throttling each user to their allotted or purchased bandwidth). Every 24 ports will cost you about $2000. You will need to aggregate traffic from each switch to a higher class switch upstream (this may not be an issue if the aggregate traffic from the entire switch is under 100Mbps)

    DialUP:

    With 25,000 customers, lets assume that you can oversubscribe your dial in lines 10:1. This means that you need 2500 phone lines to help ensure that your users do not get busy signals (the will cancel if they get them on a regular basis). This also means that your average user must only use the connection for about an hour a day. The more customers you have, the more you can play with oversubscription of the dialup ports. You will always have a user that is connected 24 or 7 as well as a user that does not use their connection at all (or for 5 minutes a month). The ratio's of your user base's usage will vary and cannot be easily determined ahead of time.

    These 2500 lines need to be purchased from the local Telco or a CLEC. In such bulk rates (four DS3's worth of voice). One may expect to pay lets say $10 per port or phone line). This equates to about $25,000 or month for just the phone lines themselves. Lets not forget about install costs for the DS3's as well as some kind of local loop charge. If you decide not to use DS3's but T1's instead, your costs are likely to be much higher per port. Now you have just bought some phone lines but now need remote access servers (like the Cisco AS5800). You will need to buy the chassis (which is bare, as well as power supplies. For the 5800 you will need to buy Modem Cards (20K for 144 modems, you will need 5 of these cards per DS3). DS3 Cards for the 5800 (two per box). Now you of course will be buying all of this new and with a support contract for 4 hour repair or replacement. Buying equipment on
    Ebay (especially without a support contract) is a disaster waiting to happen.

    At this point, you have a fairly empty room with a few DS3's and dial up servers. You have no content to offer your customers.

    Servers:

    At 25K customers, you will need a wide array of servers to suit customers needs (you thought that all you needed to provide them is internet access !!!). Lets assume that for this application, your unit cost per server is $2000 (all your servers will be rack mountable of course to maximize the use of space, you will need a KVM system to console into each server.

    You will also need to invest in some mass storage arrays (few hundred GB) such that servers do not have local storage and things like web content and mail spools are accessed by the servers. If you intend to offer Usenet access, you will need a high powered server as well as a few hundred GB of storage (today's daily news feed is a few hundred GB). This news feed will likely costs you $$$ as well as eating a few megabits of transit 24 or 7 or 365. You will have a few spare drives (these will of course be hot swappable SCSI drives as IDE drives are just not built to last in an ISP environment)

    3 Mail Servers (this will offer redundancy to both the residential and business users). Two mail servers are not likely to handle the load.

    2 Web servers for residential users (lets assume you give each user 10 MB of web space).

    5 commercial hosting servers (its amazing how much traffic a few thousand web hosting accounts can generate).

    You will keep a steady supply of spares. Hopefully you have purchased these servers from a reliable vendor and not tried to build them yourself. If you built them yourself, you should have a few hot spares ready of every part (or two spare servers).

    Lets not forget about another 5 misc servers (for DNS, RADIUS..)

    You will also need a number of servers for your own internal uses.

    DSL:

    The ISP is just reselling DSL from the local ILEC or CLEC. They do not have the capital or the user base to pay for DSLAMS. Users are expected to purchase their own modems. If the ISP sells the user the modem, they will make a $25 profit per modem. Of the $40 fee that the user pays for their DSL connection. Only $8 makes its way to the ISP, the rest is given to the CLEC for interconnection. The ISP is responsible for all services (MAIL or WEB) as well as Internet connectivity as well as support.

    Bandwidth:

    Up to this point you are an Internet provider without Internet connectivity :)

    To define pricing of upstream bandwidth we have to differentiate between transit and

    peering. The two terms are difficult to define and their use is often abused.

    Peering: is a free exchange of data (someone will need to subsidize the cost of the link between the two parties as well as the equipment at the two ends, this cost may be shared or assumed by a particular party). Some peering is paid for (at a cost that is significantly cheaper than transit.

    Transit: the privilege to cross one providers network to get to another. Transit may
    include a full Internet routing table (which means that the provider offering the feed is either a Tier 1 ISP with its own interconnection agreements to various networks (The Internet is nothing other a collection of many large Networks, accessible via mainly the Tier 1 providers).

    Transit purchased for a Tier1 provider is priced at approximately $200600 or Megabit. The quantity in which you purchase transit will determine the price. The price of this connectivity may not include facility costs (Ethernet or OC12 from the nearest point of Interconnection of the carrier to your location). It does not include the costs of routers that can handle this type of traffic (these costs will be mentioned later).

    Transit can also be purchased from an NSP (Network Service Provider) , a lower Tier
    provider. This provider may still have an extensive network, but a network smaller than the Tier1. This NSP will have extensive Peering and Interconnection agreements, this NSP will still have to purchase connectivity(Transit) from a Tier 1 provider. The NSP sells connectivity at lets say a 30% discount to the Tier1. Since this NSP has extensive peering, they can send a significant portion of their traffic via cheaper peering links. If an NSP purchases transit at $200 or Megabit and its cost of peer traffic is about $40 or Megabit, the more traffic that goes via private peers, the more money the NSP makes. The NSP may even get luckier in having customers access one another's content (where the NSP is getting paid by the sender and receiver for the same traffic).

    In its truest form, a peering agreement should be equally beneficial to both partiers. The two providers should be sending one another roughly the same amount of traffic. Failure to do so may mean that one party pays more than the other for sustaining the link. In order to peer with another provider one must be able to meet at a particular point and interconnect two routers. This may be done via a private facility (DS3 for example) or at an Interconnection point where providers meet. Exchange points are common in larger cities as an ISP only needs to pay for one expensive link into the exchange and will have access to many other peers. These exchange points may be free (where the cost is just that of getting there) or may charge a fee to enter the facility and or or a monthly bandwidth cost. Smaller
    regional ISPs will likely have users that visit local content (if they are lucky a dialup user may be accessing a web site hosted at the collocation facility and no traffic needs to transit the peering or transit links). Sheer scale in the size of the ISP is what helps these costs saving to be realized.

    An ISP of the size stated above may not be big enough to justify private peering but should look at collocating at an exchange. If no exchange is available in a given city, they may need to pay for facility to an exchange a considerable distance away (this may still work out cheaper than transit).

    Now we must determine how much bandwidth this provider needs.

    2500 Lines at 53Kbps (the fact that the upstream is slower is not really relevant as
    asymmetrical bandwidth contracts are less common). 2500 x 53Kbps = 132.5 Mbps (lets assume that not all dialed in users are using their connections at full speed, we can cut this amount to 50-60 Mbps at peak)

    2000 ADSL Lines x 1.5 Mbps (we will assume only 1 or 40th of these users will be using their connection to full capacity at any given time, we will discount all other usage at that time). 70 Mbps will be allocated to DSL users. Some may find it unusual that the dialup users use just a little more bandwidth as the dialup users. This calculation is based on the assumption that the DSL users will be on the network for short periods of time (one out of every 40 customers will be using their connection at its fullest, 1in20 at 50% , 1in10 at 25%, 1in5 at 10%). No servers are allowed (low bandwidth personal mail and web servers are tolerated).

    500 SDSL Users at 1.5 Mbps (we now have to assume that 1 in 20 users may be using their connection at full speed). Many of these users may also run servers. 35 Mbps will be needed for just these users.

    Web hosting (this is a very lucrative market for ISPs as many users websites have little or no traffic , very heavy oversubscription is common): 15 Mbps

    Collocation: 20 Mbps

    ISP Operations (corporate web space, mail servers, users web content) 10 Mbps

    Sub Totals:

    DialUp : 50 Mbps
    Rees DSL : 70 Mbps
    Bus DSL: 35 Mbps
    Misc : 10 Mbps
    Web Hosting or Collocation : 35 Mbps

    Total Bandwidth Needed : 200 Mbps (more may be needed at peak)

    One now needs to purchase 200Mbps of bandwidth.

    Connectivity :

    60 Mbps from Tier 1 Provider A (Ethernet Based , $450 or Megabit) = $27,000
    90 Mbps from Tier 1 Provider B (Ethernet Based , $350 or Megabit) = $31,500
    45 Mbps via Private Peering Links (DS3 , $90 or Megabit) = $4050
    5 Mbps Internal Traffic (Free)

    Provider A = $2000 Port Charge Per month
    Provider B = $3000 Port Charge Per month
    DS3 = $8K month (no exchange point in vicinity)

    Total Bandwidth Costs : $71,000 or Month

    Routers:

    A Router capable of routing this amount of traffic will cost approx $200k (well, you'll need two because of redundancy). Since you have two routers that are a little less powerful they will cost you $300K in total. Don't forget 510K per year for the service contract for you router and switches.

    Am I Making Money?:

    Monthly Costs:

    2500 Lines From Telco = 25K
    Phone lines for Business Use = 2k
    Bandwidth = 71K
    Utilities: 10K

    Total Costs:108K or month

    Revenues:

    25 DialUp Users x $15 = $375,000
    2000 Res DSL * 8 = $16,000
    500 Bus DSL * $400 = $200,000
    Web Hosting 1500 * $20 = $30,000
    Collocation: $500 per half rack + $400 per megabit = $10,000 Rack Space + $8,000 = $18,000

    Total Revenues = 639k or month

    What's Missing?:

    Initially this looks great the ISP is making a profit of $500k or month. What we have forgotten are the capital costs as well as salaries

    10 Tech Support Technicians x $35k per year = 350K
    10 System Admins x $50k per year = $500k
    1 Network Engineer x 60K per year = 60K
    5 Sales x $50K per year = $250K
    3 Administrative x $30K per year = $90k
    2 Executives x $80k per year = 160K
    1 President (Owner) x $120K per year = 120K

    Salaries = $1.5 Million or year

    Equipment or Capital Costs = $3 Million for all needed equipment and build of Data Center

    At this point it appears the ISP is doing well. To get here however, the ISP has spent a great deal of time and money. They had been loosing money for the first few years and their loans were accumulating interest.

    Equipment costs cannot be looked at as a long term investment any more. In the three years that I worked for an ISP in the past, we changed out terminal servers 3x (from reg 33.6 Sportsters that users dialed into, to remote access servers from US Robotics to offer X2 to a highly scalable V.90 platform).

    If I look at these costs, I think that my bandwidth costs may be too conservative. Users may want to use their connections a great deal more. I have also not had to price out DS3's for voice so $10 or port is just an example.