Slashdot Mirror
Breakdown of Bandwidth Costs?
WCityMike asks: "What is the origin of the cost of bandwidth? For instance, if I'm being charged for an apple, I know that, theoretically, the cost of that apple is going towards the purchase of apple seeds, the land on which the apple trees are grown, the fertilizer and water that helps the trees grow, and the salaries of those who pick the apples, clean them, box them, and send them to market. When an Internet provider charges someone hundreds of dollars in bandwidth costs because they were Slashdotted (or Farked) and their bandwidth use shot up, what costs have the Internet provider incurred, and why does it cost them what it does? Is there usually any sort of markup going on along the line, or are people just passing along their own expenses down the line to the end user?" It would be interesting to note the most important factor contributing to bandwidth costs. How much of the total costs are tied to infrastructure versus the human component (technicians, sysadmins, technical support and so forth)?
...the problem is burst. Period.
When you buy an apple, you can eat the whole apple at once (tossing the core into the compost pile when you're through), or you can cut the apple up into slices and eat those slices over a few hours' time. If you eat the entire apple in one sitting, it doesn't cost any more than if you'd cut the apple into 16 slices and eaten one per hour.
Bandwidth isn't like an apple.
Bandwidth is purchased in one of three ways (or sometimes several or all three ways at once):
1. A connection with a maximum of X amount of throughput at any given time, "unmetered" (ha ha)
2. A connection with a maximum of X amount of throughput at any given time, plus a restriction on how much throughput can take place in a given month
3. A connection with a maximum of X amount of throughput most of the time, which can increase on the fly to some multiple of X for short periods of time, with a restriction on how much throughput can take place in a given month
The third type of connection is usually known as "burstable." What it means is that the ISP sells you a line which is physically capable of, say, 3Mbps transfer; but they sell you this line with the understanding that 99% of the time you're only going to be using a maximum of, say, 1Mbps of that capacity. You're allowed to "burst" over your 1Mbps average up to 3Mbps.
Some ISPs restrict these "bursts" to certain lengths of time, some restrict the "bursts" to certain amounts of data (e.g. you can burst to 3Mbps for 1GB at a time). Either way, you're typically given a certain allotment of "bursts" per month. Any more "bursts" and you pay extra.
When your site is Slashdotted, you generate hella "burst," hella quick, and hella strong. As a result, your hosting company may wind up exceeding the allowable "burst" (either in time or in bytes) according to their contract.
Unlike the apple - where you can "burst" it all into your stomach at once with no penalty - bandwidth is sold on the contingency that it's going to be used over time. Use it all at once and you get fucked. That's pretty much it.
For my organization, about 45% of the customer's cost goes to pay for bandwidth. The rest is mostly people costs.
- We provide very limited end-user technical support.
- We provide a very specific service. We don't try to do "anything that could bring in a buck". I think a lot of ISPs get into the trap of "anything for money"... which results in a LOT of hidden costs. This should be MBA 101 stuff, but it's amazing how trapped people get doing things for zero benefit.
- Before we implement anything, we look at the potential costs and benefits.
- Everything has to have a complete process in place before we move it into production. Otherwise, on-going support costs skyrocket.
- We have a very clear contract with our customers. We trust them; they tryust us; they don't misuse us, we don't misuse them.
This is simply basic IT business stuff. But most small bandwidth resellers (and many large providers) fail at it.
Not all ISP's peer for free with other ISP's. In a lot of cases, they have to "settle up" for their peering costs at the end of the month, per megabyte, with their peers. UUNet is big on doing this with their peers and it's where they make 50% of their income...
Cogent and AOL just had a fallout about this - Cogent felt that they should have free peering with AOL but AOL felt Cogent should pay for usage because most of the traffic going to AOL was for Cogent customers and not vice versa.
(see 'Peering' Dispute With AOL Slows Cogent Customer Access and Paid Peering )
So yes, the short answer is that upstream ISP's do pay per Mb in a good amount of instances if they aren't a huge transit hub. Obviously some ISP's aren't subject to this and just charge by the MB figuring the more traffic you're getting, the more money you're making from their connection.
Sig (appended to the end of comments you post, 120 chars)
Anyway, my bandwidth was paid for. I had a full T1, Sprint didn't care how much traffic I sent across it, they were set to cover it.
He was very sorry about it. It cost me about 2 hours of downtime for the rest of my customers. It took me about an hour to respond once I found out there was a problem and pulled the plug on his box. I issued refunds to customers that were affected.
I charged him the cost of those refunds, plus $35 for an hour of my time.
Note that this probably didn't cover all of my costs. I wasted most of a day adjusting bills of those affected, which I didn't charge for. The agreement I had with this customer was special. He was allowed to use all extra bandwidth for his peering box, but was not allowed to infringe upon the rest of the customer's requests. This was before QOS packet stuff so it was kind of a manual 'keep on eye on the MRTG graphs and make sure were aren't maxed out' kind of arrangement.
> the cost of that apple is going towards the purchase of apple seeds
Apple cultivars are clones, you don't grow them from seeds but cuttings, scion wood. Seeds only matter in breeding.
I do not know how the big boys do it, they have to have teams of skilled techs, phone support people, support infrasturture, people to manage said infrastructure, oh yeah, and the actual network to provide the bandwidth.
I do not begrudge that fact that I have to pay a lot for bandwidth. I am actualy happy that I pay so little for what I get. However the barrier to entry into the market will make many people thing twice about doing it. If I knew then what I know now... well I probably would not have gone into this business.
As for the costs to the ISP when a site gets slashdoted, the cost is slow connectivity and frustrated clients. It is a delecate balance, making sure you have enough capacity, and staying in the black. There is a huge lead time to getting additional bandwith avaiable, unless you are a large organization and can afford standby bandwidth.
I have to keep a close eye on operations to make sure that bandwidth is not being over taxed by a single client, and that that client knows about thier usage. I have had several clients get hacked (open ftp Windows servers are a favorite). Suddenly they are pushing a full T1 of data out and they are scrathing thier head wondering why the Internet is so slow today...
First off, an increase in load does not mean you need to increase the number of people who work for you (sysadmins, techs, etc.)
Second, costs are obviously passed to end users with a mark up (however little).
Third, cost of bandwidth DECREASES AS BANDWIDTH ITSELF INCREASES. At least with ISPs. ISPs who buy bandwidth in bulk actually reduce their cost per Kbps (that is, we get charged a lower per Kbps rate if we buy 45Mbps outright than 64Kbps). For example, we used to pay $1600 for a 64Kbps link but now we pay only $6000 for a 2 Mbps link (we're outside the U.S. so this includes physical connections) instead of $51200 ([2048Kbps / 64Kbps] x 1600).
HOWEVER, buying in excess usually result in part of the bandwidth being unused but the ISP still pays for it. That's where the numbers game starts, where the ISP divide the cost of the bandwidth plus a mark up, to get what they charge customers.
Obviously they can't charge customers on a per Kbps rate to recoup costs otherwise the customers would be up in arms. Now here's a secret (ssshhh!), most Internet traffic is bursty, that is, customers don't always utilize the entire bandwidth (at least, not at the same time) that the ISP guaranteed(?) - you access a webpage, read it (while you read, you don't utilize bandwidth) and access another page. ISPs rely on that fact when coming up with prices. That is the reason most do not like it when you upload stuff in P2P nets. Most probably they just use the RIAA, DRM, whatever, as an excuse to block access to P2P. Why? Because the minute you offer upload access over P2P nets, chances are high that you'll be utilizing whatever bandwidth they promised you. They don't like that because if a significant number do that it'll throw off their numbers (remember, they didn't expect you to actually consume the entire bandwidth so they didn't charge you based on a per Kbps rate). Noticed this trend lately with so-called broadband providers who "guarantee" you so-and-so amount of bandwidth/speed?
You asked for the breakdown of bandwidth cost. Yes, most of it is caused by having to get additional bandwidth (larger pipe to handle larger traffic) but it is actually holding on to that large bandwidth (with very few users paying for it) which is the cause of you getting slapped with a huge bill if you go over the allotted amount. Their cost would be lower if they didn't have to keep excess bandwidth to give to you when the time comes that you actually need to burst over when you're slashdotted.
We solved this problem of having to offer competitive prices while still being able to pay for bandwidth by charging on a per-MB basis (with a minimum of course). We give enough free MB to cover a month's worth of surfing/browsing (done thru extensive study of usage patterns) but the minute customers go to P2P nets, we hit them with an excess usage fee. This limits them from keeping their connections 24 hours doing nothing but downloading mp3's and mpegs.
And we don't even have to hide behind RIAA or DRM.
ALL our customers btw, are satisfied with this arrangement. Only those who go to P2P nets and download recklessly are the ones complaining (until now, all those who exceeded our allotment by a large margin have ALL been caught using Kazaa, iMesh or Morpheus).
That is probably 15% of thier fiber capacity, not total capacity. The other cost of hooking up the fiber is the routers and integrating it into the "net". The reason is that the price of getting bandwidth is so low now that is not cost effective for them to hook the fiber up.
Counter-intuitive? The fiber is only one part of the cost and it is fixed since it is in the ground alredy. There is the capital required, and the human cost to support hooking up and maintaining the fiber. If they hooked up all the fiber they would drive down the costs of bandwidth even more, and thus lose more money. So they leave the fiber dark until it make economic sence for them to light it up.
That is not a simple question to answer since there are several factors that have to be taken into consideration in determining price. Different type services, e.g. dialin lines, colocation, dedicated bandwidth, etc, have different factors.
The primary thing to remember is that the ISP business model is based on setting prices for the "average user". They do not directly pass on the cost of each user to that user. For example, a dialin phone line costs $25. That is the cost of just the line. It doesn't include equipment, employess, or bandwidth. The reason ISPs charge $20 or less is that the average user doesn't tie up the line 24/7 so that they can 7 to 10 users per dialin line.
The same is true with bandwidth. An average customer who gets a dedicated T1 usually averages 50% or less of the lines capacity. So the ISP can sell T1 service to two or three customers for each outgoing T1.
Another factor is peak vs average load. The ISP has to provide facilities to handle the peak load. Yet, most prices are set according to average load. Check this site if you want a graphic example of the slashdot effect.
All this presents a problem for ISPs in pricing for users want fast speeds but have low averages, e.g. colocated web sites or DSL lines. They can't charge for the maximum possible load. If they did, DSL lines would run closer to $1000 per month instead of $50. If they stictly base it on average load, the low users are subsidizing the bandwidth of the high users.
To resolve it, they have come up with a price tier that gives an incentive to users to, accurately as possible, estimate their usage. They make it cheaper for the user to pay for the next tier above their average monthly usage rather then paying for the tier below their monthly usage.
I feel like picking a fight with everyone who thinks they are right. - Rainmakers
The ISP already owns (or leases) a bunch of fiber going wherever. They already own a bunch of routing hardware. Those were initial costs, which may not come again, unless business is good.
There are a bunch of costs that they continue to have.
Staff. Everyone from the salesman trying to sell you a line, to the support staff answering the phones at 3am when you can't remember what your netmask is suppose to be.
Building costs. Like, those lines run somewhere, right?
Fees. Everything else. :) It costs money to run your fiber into someone elses equipment (peerings), or to share fiber across an ocean, or a satellite link.
:) That's how they make money..
:)
So, they can't directly make money back for all those costs. They just turn around and say, "We'll charge you for the bandwidth usage", and voila, they do.
From what I understand, the actual cost of running a DS3 (45Mb) or a OC3 (155Mb) isn't too far apart.. But they'll sure charge you a lot more for it.
Like with dialup providers, they expect to need so many customers to cover expenses, then they turn a profit. If they can support 200 customers, and need 20 to make a profit, they don't start charging all the customers less as the number of customers increase.. The ISP owners just start driving nicer cars.
Serious? Seriousness is well above my pay grade.
Yes. Only the big boys have free peering between themselves. And, even they have costs associated with peering: cost of colocation at a peering point, bandwidth between the two service providers, etc.
Or do they have 'back and forth' arrangements where the ISPs only pay for the difference between in and out traffic?
Some do this. However, it has always been a hot button issue with ISPs. Some ISPs, e.g. UUNET, have a higher percentage of web servers. While others, e.g. Earthlink, have a higher percentage of users. Who exactly is providing more value to the other. Is Earthlink supplying the viewers to the web sites? Or, is UUNET supplying the web sites to the viewers?
I feel like picking a fight with everyone who thinks they are right. - Rainmakers
How much money does it cost an ISP to be a good ISP? A lot more than you expect! Take an OC-3 line. This can run you upwards of US$50,000 PER MONTH! This is just to connect your customers to the major communications backbones of the internet, so they get reasonable bandwidth.
But now you need to connect those communications lines to something so that your customers can actually get onto the internet. How 'bout a couple of routers. A couple of new Ciscos could run you upwards of US$100,000. Oh, wait a sec! You need servers so your customers can put up their websites, and to manage their accounts and stuff? Servers are the "creeping featurism" of providing service. The more customers you want, the more servers you have to add. For good, high-quality, industrial grade servers, let's take an average price of $2,000. I haven't even mentioned all the Cat-5 cable and switches you're going to need internally to connect all your equipment together.
For that matter, how on earth are your customers going to physically connect to your service? Dial-up? Well, you will probably need a CLEC to be installed at the local phone company. Be a cable provider? You'll have to string lines up around town. That's expensive! Either way, you are now a utility, and you will have to license yourself as such. THAT costs some bills. The more customers you have, the more stuff is filling up your OC-3 bandwidth. In order to maintain a higher quality of service, you're probably going to need to add another OC-3 -- or more economically add a couple of DC-3's to help spread out the load. $! $! $! It's really starting to add up! Whew!
Just to connect alone, you are looking at a recurring cost of better than $500,000 per year. Equipment costs will probably run you another $300,000. But you have to maintain all that. There's another cost for maintenance contracts. Believe me! You can't run all this by yourself! You will need to hire people just to help run this. And sell your services. And deal with irate customers. And accountants to watch your books so you don't put yourself too deep in the red. Parasites- er, uh, "lawyers," to keep you out of trouble with RIAA when some cheese-eating high school boy decides that paying someone for their music so the musician can make a living goes against the constitutional rights to free listening...
And here's something that few people stop to think about: all these gadgets feed on electricity. There's yet another whopping, monthly recurring cost! Oh, well...
These are the things that your money is going towards. Admittedly, I'm listing out all the top tier stuff, here. When you want to handle industrial and commercial uses, many customers at that level want guarranteed levels of service. Just to run this level of ISP can very easily run you over US$1,500,000 per year. You had better keep your customers VERY happy in order to make enough money just to cover your costs.
At a lower level -- say you just want to be a neighborhood server for people in your part of town. Or maybe, all you want to do is be a hosting service, where people can park their web sites and pay you per month. A Mom-an'-Pop type set up could probably be pieced together for under $50,000, but you still have to connect to the internet so your customers sites serve to the world at a reasonable bandwidth. A DC-3 line could handle this at under $15,000 per month. You'd need quite a few customers to cover that cost, at least 200 at $75 just to match that price. More to make a profit.
A long time ago, before Broadband. When 2400 baud modems were considered lightning fast. It was quite possible to set yourself up with a couple of computers, a couple dozen modems and a T1 and call yourself an ISP. In fact, that's how a few of today's ISPs got started. Take a look at Pair.com". They are a classic example of a world class hosting service that got started in a garage. Nowadays, just to keep up with the competition, you have to put out over 200 time more cash just to get started.
Gee, I bet you wish you had that $350 million dollar Powerball lottery ticket now, don't ya?
Whew! This water sure is cold!
I have done a fair amount of shopping on hosting and spent 70,000 USD on it last year. At the top, are the big network providers (UUNet, Qwest, AT&T, etc.) These companies have substantial costs associated with technology and connectivity. Their customers are typically end user ISPs, hosting companies, and colo (colocation)facilities, so support is a very minor cost. They have a commisioned sales force. Next in the chain, we have end user ISPs, hosting companies, and colo facilities. End user ISPs provide an obvious service. Hosting companies and colo facilities are somewhat intertwined. Large hosting companies have their own datacenters and connections. Colo facilities have separate racks that are leased out. The customer is responsible for the connectivity, but these places are typically prewired with every sort of network provider available. As an added complication, Many colo customers are small hosting companies. Let's look at the cost impacting factors at each of those levels: Network providers: Infrastructure: High Connectivity: High End user support: Nearly nonexistant End user ISPs: Infrastructure: Moderate Connectivity: Moderate End user support: Very High Hosting Infrastructure: Moderate Connectivity: Varies on account size End user support: Varies on account size Here is the problem with hosting: The largest companies are running large *nix servers with huge numbers of accounts on them. These are the 10-300 USD accounts. You pay for a minor amount of bandwidth and hardware, but the fixed costs of collecting payment and providing support make up a large portion of your bill. The high overage charges are there to discourage you from exceeding your planned bandwidth. These companies are paying for a fixed connection typically in 100Mbps and 1Gbps units. They need to keep a certain level of utilization and keep burst down to maximize the return from their connectivity costs. An additional cost of overage is in hardware. Those large servers are balanced with a high, but manageable number of accounts. When several accounts are at high burst, there is a possibility of slowing other accounts and getting complaints. The cheapest way is using dedicated servers and a fixed connection size. The problem with this is that it is not cost effective until you are using a fair amount of bandwidth. The reason it becomes so much cheaper here is that there is no uncertainty for the hosting company. You are also generally responsible for administration of these servers. Colo is pretty much the same except that you install your own servers in a facility instead of leasing a server from a hosting company. I hope this helps. I have explored many options and I have learned a great deal about hosting from being a large account.
Thought there have been several responses relating to burstable pricing, it's important to understand that burstable models are more of a marketing concept than a network costing one.
/can/ permit overbooking to occur.
Considering that IP service is in the class of packet sharing telecom - meaning you do not have a fully reserved, "100% yours and only yours" pipe from location A to location Z (end to end), your costing models have to be a bit more complex than adding the sum of the parts. But it should probably start there. Packet share != Overbooking, although packet share networks
To explain costing on a more common tier two or smaller level (tier one costing has much more to do with transport costs and a bunch more variables not interesting here), take your typical tier two or smaller ISP who we'll say is connected only to a single upstream, and that upstream provider is a tier one (multihomed at x+ bilateral and/or multilateral facilities; e.g. Sprint or AT&T Worldnet).
That ISP will have:
- an egress transport cost: e.g. a T1 to Sprint and perhaps a local loop cost to get to the tier one carrier; sometimes this is bundled by the tier one provider
- a facility cost: router, colocation, switching and all the associated things to figure out what to do with customer traffic and send it where
- an ingress transport cost: this is the transport cost from the customer to the ISP (e.g. T1 to a local frame relay cloud from the customer, then another portion from the cloud to the ISP, or a similar model in DSL, or a fixed point-to-point line)
Plus applicable switching/routing/data center/network core/backoffice/customer support/billing costs.
Then the fun comes in calculating an aggregate the ISP wishes to use in predicting how many seats it can sell on its T1 until the plane can get full - understanding that if it oversubscribes the T1, it may have the possibility of bumping passengers (such as slow performance; people getting less than what they paid for).
(Note: You'd be surprised how many smaller ISPs don't even understand the concept of aggregation - many rural ISPs in our parts think Internet comes in T1 and put 500 DSL home connections on the T1 without thinking twice. "Order another T1? Why? We already have Internet!" Plus there are other reasons they never go past a single T1 - their Cisco 2600 only has two ports (T1 in, T1 out), they don't know how to load balance more than one T1, and fundamentally it's too expensive for them)
Aggregation becomes the process of determining how to share that cost, since it's a shared network. It includes variables like time of day (business customers will usually demand more bandwidth during business hours; residential after hours, etc.), whether the customer has a bandwidth guarantee in their service level agreement, etc.
After all of this calculation and checking of assumptions, the ISP may calculate it into a cost per Mbps as we do.
so they have to overbook. It's all about getting the most out of what resources you have.
Not always true. Overbooking can be perceived as ineffective loading - an error, in otherwords, to be avoided. Your power utility doesn't run a constantly overbooked network - that'd have serious regulatory and technical consequences. But when you've got very little backoffice expertise in a smaller ISP, they won't know how to develop traffic models and purchase effectively.
Overbooking can be avoided both by developing good subscriber models and by enforcing them (ugly reality: if you're using 1 Mbps up/down sustained and are paying $35/month, you're not paying for what you're consuming and you'll cause a problem somewhere along the line). Additionally, you're going to see better aggregation and closer matches to "what you pay is what you get" as the last mile part of the business matures. Much of the success of an ISP is in how it buys and how closely it matches what was bought with what was sold - you can't run a $2 all-you-can-eat prime rib buffet, nor can you charge $10 for one that has nothing but dinner roles.
Really, overbooking and the opposite end of the spectrum (giving excessive bandwidth without charging for it) are mistakes made early in the development of the business. Successful providers will tighten up both ends, providing the service people ordered and paid for, and smart customers will understand that in order to get what they paid for, they shouldn't also expect a free lunch.
*scoove*
Let's look at cost first. Typically, service providers think of costs as some percentage of the dollars coming in for a given service. They break these down into several categories. Although each company is different, here's a simple example:
- Cost of goods sold: These are dollars that the companies spends to provide the specific service. More on this later.
- "SG&A": Sales, General and Administrative. Otherwise known as "overhead" -- cost associated with running the company that aren't tied specifically to a service (management salaries, office space, etc...). Sometimes sales salaries and marketing are broken out from this on a per-service basis, other times they are lumped into the general company overhead costs.
- Margin: This is the money left over after the company's costs are taken care of. If the company sells everything directly at retail price, this is equal to profit. However, if a company has resellers or other types of "sales channels", they will be paid a certain percentage of the margin for selling the product. Discounts also come out of this chunk.
Very rough rule of thumb: cost of goods sold should *never* exceed 50% of "retail" (undiscounted) revenue in the telecom / service provider space. Of course, if a company is desperate for business they may give away very large portions of the margin percentage in the form of discounts, etc...The "cost of goods sold" for a typical hosted dedicated server generally break down into several major categories, that may or may not be broken out as different "services":
- Hardware cost & software licensing: This is the fixed purchase cost of the dedicated server itself, and any associated software licenses. This is typically paid directly by the customer up-front, or broken down into lease-like payments and buried in a flat monthly fee.
- Data center: These are costs of having the server sitting in the data center, including lease cost of the space, supporting rack and LAN infrastructure, electricity, cooling, security, etc... Typically these costs are figured based on averages per square ft. of server room space or by rack space.
- Operations / management: These are the costs of management services provided for the server: NOC staffing, management / monitoring system cost, tape backup costs, etc... In the case of colocation, this may be minimal or non-existant, while it's very significant for dedicated / managed services, since you're paying for system administrators to upgrade your OS, apply patches, etc...
- Customer service / support: Costs for the right to call someone on the phone and get help with your service. This primarily includes call center infrastructure & support staff salaries. Again, may vary widely depending on the level of service selected.
- Bandwidth: At a minimum, this is the cost of the routing infrastructure, the cost of the WAN part of the NOC staff and systems, and the monthly costs paid to the upstream providers for big dedicated pipes. Your average hosting company is not running their own fiber or even buying dedicated circuits: they are buying IP transport from a large ISP, and pay for a certain amount of bandwidth. Usually, they'll lay in a very large fiber pipe to the ISP's local POP, and then activate additional bandwidth as needed. I can't break down the long-haul ISP's costs for providing that bandwidth, but presumably it breaks down similarly.
Now, a few words about pricing, and specifically bandwidth pricing, since the poster was interested in that. Aside from bandwidth, the costs above are reasonably predictable from customer to customer and month to month. The more predictable a cost is, the more likely a service provider is to lump it in with a bunch of other costs and charge a flat fee. The less predictable a cost is, the more likely it's going to be metered and broken out.In general, customers demand / want flat, fixed, predictable prices. This is why ISPs charge $19.95/month for dialup rather than $.05 / hr. Of course, at some level the ISPs price for dialup is proportional to actual use, but they've figured out that, on average, they can make money at a $19.95 flat fee, and customers are willing to pay it. Of course, customers who use less are subsidizing the service of customers who use more. But as long as the *average* cost remains low enough that the low-usage customers feel they are paying a fair price, it works out.
Server bandwidth is a case where it doesn't work out. The amount of bandwidth that a single static web server is capable of consuming is quite stunning, and goes up along with Moore's law. Many hosting customers choose a dedicated server not because they need gallons of bandwidth, but because they have some sort of custom app or want full administrative control of their system. These folks aren't willing to pay a price that would cover the "average" cost of bandwidth across all an ISPs customers, which may include large streaming media systems or pr0n hosters. So the ISP meters / measures the bandwidth and charges each customer appropriately.
However, it's a bit more complicated than that. You'll notice that, if a customer buys a lot of bandwidth, they pay a lot less per GB than a small customer. The smallest customers may pay ridiculous prices for bandwidth if they exceed their plan (case in point the Farked "boobies" page that racked up $400 in bandwidth charges in two days). It's obvious that SPs are charging much more than their "costs" for the bandwidth in these cases, and you'll notice that the same thing is true for cell phone plans, etc... What's going on?
The answer is that it's all about predictability. A service provider must maintain adequate capacity for providing service. If they don't have enough upstream bandwidth, service quality for their entire user population goes down the toilet. It takes a long time to add additional capacity- new fiber needs to be run, new equipment purchased, etc... As a result, service providers are always buying new capacity in advance of demand, and to do this accurately they must be able to forecast demand. This boils down to getting accurate forecasts from their customers, which is impossible to do directly. Instead, as capitalists, they put economic incentives in place to motivate customers to predict their maximum demand accurately by pricing in tiers, or packages. Buy buying a particular bandwidth package (say, 20GB/month) you are effectively telling the provider that's how much you are planning to use, and they plan their capacity accordingly. Deviating from their master capacity plan is very, very costly for the service provider, and accordingly, the "right" to deviate from your plan as a customer is proportionally more expensive.
Another way to look at it. Customers who buy the smallest bandwidth package are the least "valuable" to the company because they are making no revenue commitments. They are also the most easily able to double, triple, or quadruple their demand from month to month because their servers are capable of consuming so much more bandwidth than they are buying. On the other hand, a large, multi-server customer is a more valuable customer, and is likely to be using a larger portion of the available capacity of their servers, and thus less likely to have wild changes in the amount of bandwidth they consume. The small customer thus pays much more per GB of bandwidth than the large customer, especially if they exceed their plan. In a lot of ways it's like the airline industry or any other industry where buying additional capacity is expensive and/or time consuming: you pay a premium for being able to use that capacity on short-notice.
Well, that's more than you probably ever wanted to know. Hope it's been educational.
-R
ISPs are probably what can be termed "price searchers."
Ugh... and this merited an insightful? Must be lots of leftover mod points:-) Don't mean to harp on an aspiring young mind, but...
but largely I imagine this is negligible--the bandwidth is there whether or not it is used.
Your professor has fallen prey to the fiber glut myth. Yes, even in our parts of flyover country, it is "fiber, fiber everywhere, but nowhere to connect." Countless miles of 100-lane highways with no onramps.
But when you realize that most of the cost is in the onramp and the interchange (last mile and switching, respectfully), there isn't this mythical nearly free bandwidth.
As a few other posters have noted, ISPs are out to make money.
As witnessed from the events of Worldcom, Global Crossing, Level3 and countless other carriers, probably not enough were out to make money. Please explain though, why would this comment be relevant (not to mention "insightful")? If your employer is not out to make money, RUN. Got stock in a company that is not out to make money? SELL. Buying service from a company not out to make money? LEAVE.
Making money permits us to be around tomorrow to provide you service, give people jobs, and pay our vendors and shareholders. If you feel otherwise, a good cure would be some reading to discharge the mush your professor presented.
A price searcher has a set cost that is basically unrelated to the amount of service they sell.
IF sum(ISP) = bandwidthcost, AND bandwidthcost is fixed, then this would be true. Neither is correct. I'm buying from a multihomed, dual OC3 IP carrier and have a 1 gigabit pipe to them myself, and pay around $200/month per Mbps of bandwidth. Doesn't seem fixed to me. It isn't to my upstream either, and last time I looked, Sprint and AT&T didn't give free extra T1s after the first one was bought.
They simply match up the price/quantity demanded at that price that leads to the highest revenue and thus profit.
This model might apply more to someone like Microsoft, Nokia or Motorola who have a set R&D cost to engineer a product and very little reproduction and distribution cost (e.g. reducing cellphone circuitry to ASIC). But it doesn't fit the carrier business.
*scoove*
But when you are buying and selling intangible goods like electricity and bandwidth on the open market,
I haven't looked into the issue of bandwidth cost in detail, but I know for sure that it is possible to get a good hard number on the cost of producing a kilowatt-hour of electricity, and it is done all the time. Companies know what their capital investments, fuel costs and operating expenses are or are likely to be, or can estimate the effects of changes in those costs. Before ANY power plant is built you can bet all of this is calculated to a fare-thee-well by the utility company. And there is no bullshit involved in this calculation. I know, I used to do that sort of calculation as part of my job (in a different, but related industry).
The situation with internet bandwidth may be a bit more complicated by fact of the matter is that it's an industry where things are unsettled and changing rapidly, both from an economic and regulatory point of view so making predictions is likely to be a lot riskier. But accounting practices are not the issue - cost engineering is a well established and definitely non-bullshit field. It's the unsettled nature of the business that makes predictions difficult.
Do you think it's really possible for UUNet, Sprint, and AT&T to precisely calculate the cost of transmitting a MB from point A to point B through their vast communication networks?
I think that they know very well how much per day it costs them to run their network, what their capital investment is and how much traffic in MB it carries each day. From that it is actually very easy to calculate what the cost is to move a MB from a to b. I am also sure they have a very good idea what their marginal costs are.
There's no law of supply and demand being followed by these bandwidth cartels.
I disagree with that strongly. Supply and demand is ALWAYS in force. If you raise the price too high, the demand will go down. Period. Look at the current situation with consumers today - we have many people using dialup because they don't want to pay the higher cost of the increased bandwidth of broadband.
And they certainly don't have to worry about setting the price to high. What are you going to do, start laying wire for your own personal Internet?
If the price goes to high you can BET somebody will start launching satellites or running more cable, or launching stratospheric planes, or installing wide area wireless, or deploying whatever other technology seems best at the time.
Good analogy and good points. Power companies know the cost of building a power plant of any particular size. The trick is to estimate the capacity required. Build one too large and you have to pass the cost of the unused capacity onto your users. Build one too small, and you have brownouts and blackouts. ISPs can also calculate the exact costs associated with supplying bandwidth.
There are several differences between the two industries. One is that most power companies are government regulated industry. If they guess too high, they can pass the cost of the unused capacity directly along to the users. The only choice the user has is to pay the higher costs or reduce their use of electricity. They can't switch providers. If they guess too low, the consumer has to put up with the brownouts until more generating capacity can be built.
ISPs don't have that luxury. If they guess to high, they have to eat the cost of the unused capacity since their users can easily switch to cheaper providers. Guess too low, and they lose customers to providers that aren't maxed out.
It's the unsettled nature of the business that makes predictions difficult.
This is true in both industries also. I am sure that the power companies calculations were shot to hell when air conditioning, TVs, etc were introduced. The advantage the power companies have is that they are stictly usage based. They will earn more money as people increase their usage.
ISPs pricing is a mix of flat rate and usage based pricing. The flat rate portion is based on the "average user". When something occurs to change what the average user consumes, the ISP has to eat the cost of the additional usage until they can adjust their prcing plans. Due to the compeition, it is not as easy for them to increase their costs.
DSL is an example of this. The original pricing models were based on the assumption that residential users would download the same amount of traffic as they did when using dialup. They assumed that the only difference would be in the "burstiness" of the traffic. However, P2P made that assumption invalid.
I feel like picking a fight with everyone who thinks they are right. - Rainmakers
Actually the fees usually are to recoup costs. Most ISPs that serve end users pay upstream for bandwidth. There are bulk deals cut, but unless you're someone with leverage (like a big carrier, or a backbone provider) you're looking at paying at least $150/mo per megabit measured at 95% peak (this means the top 5% of 5 minute usage averages are thrown out each month). This price can be as high as $500/mo per megabit that I've seen. If you pay for a T1 with bandwidth included, this price is attached to the cost of your circuit.
Depending on how your ISP is setup, this could be the only bandwidth cost you get hit with - if your ISPs colo racks happen to be in the same building as their upstream provider for example. This is rarely the case. In most cases you'll find that there's an extra $80-$500 per megabit involved. This can be the other $500-700 for the T1 if it's just a T1 involved, or it can be a partial cost of a T3 or OC line. If it's OC you will probably be adding some value for the relatively new (in accounting terms at least) and extremely expensive equipment on either side of the line.
If you're talking about a home DSL then you may have 2 or 3 more similar layers involved. A payhaul to/from your DSL provider, and then the DSL provider's internal network (DSL provided through the LEC usually doesn't count since they basically have free excess capacity they use to bottom out their prices to make the market not worth expanding for others).
On top of this, you have the customer complaints and the image to deal with if you just slammed the top of the ISPs line. If the ISPs network can handle ~42 Mbit and they normally run 15 Mbit, and you just added 7 Mbit for a few hours.. you will probably only get sacked with bandwidth costs. If you just peaked out the line that means that other, legitimate customers probably couldn't get some function done, which means they're now upset with the ISP and probably thinking their network is poor, underpowered, or oversubscribed - they now have their eye out to switch to another provider. This hits the ISP pretty hard if they already have 50% excess capacity - meaning they are looking to grow to fill a lot of that. You're going to get sacked with a hefty "you just caused us a lot of grief" fee on top of the other costs in this case.
So basically, the costs are going to be:
$80-$500/Mbit per month for every hop in a traceroute up until the last hop that isn't in your ISP.
$150-$500/Mbit per month for bandwidth from upstream provider.
I run a small wireless ISP that covers a five county area, three of them in a 700k population urban area, two rural counties that border the city.
:-)
A single T1 from a reputeable provider (Sprint, UUNet, AT&T on good days) still costs around $1,000 in this part of the country. You can get slightly cheaper T1s from [IC]LECs ($800 or so) but if you later multihome with a tier 1 provider you'll end up lopsided. If you grow enough to need a significant fraction of a DS3 your cost per T1 equivalent can drop from the $1,000 mark to about $500.
I've posted this little equation a number of times before and I don't think its worn out yet.
one 256k full time music/movie trader == 4 channels on a T1 == (4 channels/24 total channels) * $1,000 ~= $160/mo cost to an ISP, dropping to $80/mo if they're a big ISP.
In addition to circuit costs I pay antenna site leases for building roofs, towers, and the like. I've got a small office and the debt on the equipment used in the network.
Unlike various venture capital created wireless beasts flopping around in my market, my company is 'organic' - it has financing (money used to buy equipment) but no funding (money used for operations). My partners and I all have other sources of income (consulting, equipment sales, very tolerant spouses, etc) and we won't draw paychecks until we've got the recurring revenue to support that.
We face one other wireless player in our space with a first generation bridged only network and a not so good reputation (I love you guys! Don't change a thing!). We also face a large, agressive cable company turned CLEC, DSL is widely available from the ILEC and another CLEC, and the ILEC has a fairly useable, cheap ISDN offering.
At the entry level for business we get between $75 and $130 for a connection up to 768k. We charge $250 per megabit with the assumption that anyone using that sort of speed is going to have VPN apps on the connection and they'll require additional hand holding.
In addition to the business customers we have one of the rural counties that is actively producing residential customers @ $29.95 monthly. I wouldn't miss them if they all dropped dead tomorrow, since they're mostly those that wore out their welcome at their local DSL provider and now they're Kazaaing up my network.
T1s are full duplex. Did you know this? DSL/cable users are amazed when they learn this fact. I like to explain it to them on a phone call rather than in person, so I can interrupt them while they're telling me a T1 can't be 1.5 mbit symetric. Think about it
So, I've got N T1s x 1.5 mbits of inflow and the same amount of outflow. The business customers use bandwidth from about 0700 - 1700, the residential customers from about 1500 when kids get home from school till about 2400 - basically two completely separate markets I can serve with the same inflow bandwidth. The ratio of inflow to outflow is about 5:1 with 'normal' customers.
The Kazaa thieves upset this 5:1 rule of thumb greatly and should really be counted as a wholesale symetric customer and charged accordingly. The CEO won't approve this charge plan, so I'm approaching it BOFH style, and anyone running Kazaa or similar services has just volunteered to be a victim^h^h^h^h^h^h subject in my various QoS experiments.
How did we decide how much to charge?
You need to understand that while I have an in depth understanding of what people are doing with the IP network I manage I *don't care* what they're doing, so long as they're not annoying the others trying to use it, or doing stuff that will get various three letter government agencies serving subpoenas to my office.
I happen to like playing with IP networks which makes being CTO a pretty fun job, but we could just as well have a warehouse full of various widgets that we buy for $2 and sell for $3 - its just an investment.
All networks built with more equipment than you can carry in one large RubberMaid(tm) storage container are investments. They depreciate a certain amount monthly, they have various expenses involved in operation including the next tier ISP cost, and hopefully, if you're doing things right, you're ending up with more $$$ on the 31st of December than you had January 1 of the same year.
So, there you have it. Its a *business*. Take a business class if this is all fuzzy to you, or better yet just start an ISP on your own and we'll see how long you think sharing MP3s and movies via the internet is a 'right'.
I am very easy to get along with, but I don't have time to waste being nice to people who are being stupid. -Theo
Colocation centers and Dense Wavelength Division Multiplexing are to blame for most of this, along with Moore's Law making computers (and therefore routers) cheaper. A decade ago, it cost a lot of money to put fiber in the ground, and the fiber had a fixed capacity (typically 1.7gbps), and you could get some economies of scale by putting a bundle of fibers in the same trench, since the cost of the fiber was much less than the right-of-way or installation cost. DWDM lets you run many wavelengths on the same fiber. In ~1996, this meant 8 wavelengths of 2.4Gbps; today it's 80-160 wavelengths of 10 Gbps. Adding more capacity still costs you money, because you need to add repeaters in the middle and expensive electronics at the edges, but the costs of those keep dropping and repeater distances keep growing, so it makes sense to buy the edge equipment you need now and upgrade later when you need it, since the prices of the hardware are diving fast. The marginal costs on a backbone fiber aren't zero, but they're pretty close.
Colocation centers also change the economics radically, because they can buy a small number of fat pipes, which have much lower price per bit than skinny pipes, avoid paying the telcos much for access lines, because they can be located near big ISPs, can support thousands of machines in the same location, and to the extent that their customers are sending bits to each other instead of the outside world, can provide lots of bandwidth on very cheap LANs inside their buildings instead of to an ISP who charges them money. Before Exodus fell apart, there was a huge ecosystem of providers who sold services to each other inside their colo spaces, and some of the other hosting providers did that also.
A related business is carrier-neutral colo centers like Equinix or the Seattle Westin Building or LA's 1 Wilshire, which convince a bunch of telecom and ISP carriers to build connections into their buildings, rather than selling to hosting users, which lets the carriers connect to each other using cheap interconnects. So a big ISP would buy an OC48 pipe into the building from an access provider, buy a rack to put their routers on, and then use Gigabit Ether to connect to the other ISPs. Also, they can put a telco fiber into the center, so if a small customer wants a T1 to one of the carriers, they can just add the circuit onto the fiber. But how do you price bandwidth in this environment? For the colo company, it's easy; they're selling real estate and charging a flat fee for fiber patch panel connections (how they make money doing that is a different question, but at least they know their costs, so they can do prices that aren't artificial.) But for the carriers, is there any reason to run a peering connection at 100 Mbps vs. 1Gbps? Not really; their router hardware costs a bit more, but it's still a drop in the bucket.
Then there's Dark Fiber access and Dim Fiber access. It's not as common as George Gilder predicted, but there are places where it's available. The basic model is that the access provider isn't messing with routers or switches; they're acting more like a construction company, installing fiber and renting right-of-way (or conduit space) and letting the customer light it up themselves at whatever bandwidth they feel like. Sometimes this is a hybrid business - a provider might have a big lit ring around San Francisco Bay and sell wavelengths on the ring connecting to dark fiber on the last mile, or alternatively they'd have a big bundle of hundreds of fibers around the Bay and do patch panels. Some of that is technology dependent, because of distance limitations for different speeds, and therefore it's also influenced by geography - anything inside Manhattan doesn't need a a repeater, or from Manhattan to nearby parts of New Jersey where the real estate's a bit cheaper and there's less backhoe risk, but some distances in the San Francisco Bay Area are a bit too far for repeaterless operation at some speeds, and Los Angeles is really huge. Also, while fiber technology used to be just for long-haul telecom or lower-cost LANs, the Storage Area Network people have recently developed technologies like Fibre Channel for connecting disk drives to CPUs at 20km distances - there are starting to be a lot of banks and brokerages in Manhattan that have disk farms in New Jersey colo centers, and instead of their mainframes being all in the same building as the traders, they may be spread out geographically, often with primary capacity in one location and backup in another.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Because the market doesn't have the volume like the US what is happening is that everything is charged as a markup. So if the City Council charges the Telco rental on the cable pits (It happens trust me) then this is passed on in the monthly costs. Because Australia used to only have one large bandwidth wholesaler (Telstra), everyone was charged 18c per Mb to download data from the Internet. Once the other Telco's built up their own infrastructure they followed similar models and thus is the issue of most ISP's charging each other for data downloaded from their networks. In the end though whereever the data ends up that person ultimately pays.