Slashdot Mirror
Managing Bandwidth and Bandwidth Costs?
"I'd like to illustrate the second concept. When you have your (for example) T1 and you're not really using it, you are still paying for all that bandwidth. It's like the car that sits in your garage, you're still paying insurance and car payments on it even though you're not using it. But then you put up a new game, serve new media or suddenly become the 'Site of the Day' and your bandwidth is flooded and maxed out. For that case, it's like you've bought a car that only goes 40 miles an hour but while the demand exists and only while that demand exists, you need a car that goes 150 miles an hour. You don't want to pay the money for a car that goes 150 because you only need it occasionally. Later, you know you'll need that car to go 220 but you're not there yet.
So if this makes sense with regards to bandwidth, it is like you'd want burst-bandwidth depending on need. Do any of you face this problem? If you do and have solved it, I'd love to hear about your strategy. Once this is solved, we get back to the first question, how do you manage that cost, put a number on it and either fit it in to your business model or pass it on to your customers?"
We can do it the slackware way - tar it up and let someone else do it.
Or we can do it the debian way - let someone else tar it up and then do it.
But we like the Slashdot way most. Post it up and watch the traffic floooooooooooooooow....
Too many to spell out here.. Search google, LOTS of people sell bandwidth at X mbits/s with a BURST to x mbit/s.
You figure our how much you will estimate you will use as a total per month, and you pay for that. You cruise at 2mbit/s mostly, then you explode to 145mbit/s and at the end of the month you average out to 6mbit/s.
The idea is you BUY your peak speed, but pay for a low average..
EVERYONE does this now, call around... It was kind of silly for you to ask Slashdot how to do this. UGH.
- Voxel
Modesty is one of life's greatest attributes
Where possible BitTorrent is your friend. If you can use it, let your customers help out with the bandwidth usage. They will probably get the file(s) quicker and you won't have servers getting torched.
how can we get our company's bandwidth setup so we can survive a slashdotting?
maybe if he got some comments from the peeps at osnews or cnet...
vodka, straight up, thank you!
If you want to pay for data by the gig, rather than by the pipe size, just sign up with an ISP which allows that.
At the ISP where I work we offer fiber connections that allow for increased bandwidth for certain periods of time. For example, our burstable connections are usually around 1-3 meg for normal times, then burstable up to 10 megs. I'm sure you can find something suitable to you.
:-)
*blatant sales pitch*
If your buisness is near Southern Ontario, check out our website at www.sentex.net. We rock
Here's a free tip --- don't post your site to slashdot.
Congratulations - you passed this one.
Anyway the answer to all your prayers is obviously BitTorrent.
Whenever Savvis has end of quarter sales you can negotiate the price a ton.
Right now they're about 700/month for a managed T1 with local loop included.
1) Agree with your ISP on a standard data rate, burstable to X as needed.
2) Use RTG to monitor traffic in and out, making sure that you know what switch/ports/etc. that client is using.
3) Charge the client (this is usally done based on 95th percentile).
4) Profit!!!
libertarianswag.com
This is what Akamai and the like are sold for.
It's almost certainly going to be cheaper than just buying bandwidth.
Or you could go for the approach of colocating your own box somewhere central for the heavily hit stuff.
Even this will be a whole lot cheaper and won't impact on your normal traffic to your organisation.
ALTQ
This guy is way out there
I don't know what solution you'll eventually decide on, but to test it you can post a link to the file on AskSlashdot.
:)
My blog
If you need proof that Kast is better than BT in this situation look at http://konspire.sourceforge.net/BitTorrent.shtml .
Hope it helps.
As soon as we all have 50mpbs 802X connections and, anonymous P2P software. This kind of question will never be asked.
With 100 gig hds, and reliable high speed connectivity 24/7. It's pretty easy to see what could be built from that.
Someone wants to download the new 150mb CS or BF1942 upgrade? Just enter the name of the file, select it...and the P2P software does the rest. Initiating multiple downloads from about 20 or so nodes in parallel. Then you just glue the program together once you have downloaded all the pieces.
Mirroring is such a hack, and dynamic bandwidth is the last gasp of the client-server paradigm. Let's move on.
The government has a defect: it's potentially democratic. Corporations have no defect: they're pure tyrannies. -Chomsky
that offers "pay by the GB". The hosting company I work for has GiGE links but only pays for the exact amount of traffic we push through the link. The monthly cost on the line is minimal, or nonexistent depending on the provider.
Since we have more then one link as well, it gives us redundancy and the upper hand to negotiate the best price per GB, so we can send 90% of our traffic out that link. If the next month a different provider comes back with a cheaper price, we switch it around and send the 90% out their link. Within days of cutting the traffic off for a link, we can usually expect a phone call from the sales rep with lower price offer.
Any idle links we have don't cost us anything extra, since we _will not_ deal with any provider that doesn't offer pay by the GB. Paying for the raw link speed, regardless of how much traffic you push through, or paying 95th percentile prices are all mostly a rip off.
Open Source Time and Attendance, Job Costing a
Couple of different options. First, you could talk to the Content Delivery Networks (CDN's) like Akamai or Digital Island. They can probably help you (for a price).
Another option is colocation. In particular if you have short traffic spikes. Many colocation places charge your for at a '95 Percentile'. This will cut out about 3 days worth of 'peek traffic' and you only pay for the maximum bandwidth you use after removing the top 5%. Just make sure the colocation place has enough bandwidth to handle the spikes.
Some ISPs (e.g. Yipes) offer flexible contracts that allow fast (daily?) bandwidth changes. So if you announce a new version of your product, you can increase your bandwidth until the rush is over.
One hint: Try to move the large file/content away from your 'importants' networks, so other things like e-mail keep flowing even if the content site is running into issues due to load.
---- join dshield.org Distributed Intrusion Detec
It seems to me that this is exactly the type of problem that places like akamai and cable and wireless (was digital island) are trying to solve. Pay only for the bandwidth you use, leverage their existing distributed architecture, profit. You can try to get bustable bandwidth etc, but in the past I've found it to be more expensive. Things may have changed since them (a year ago) but you should still look into a content delivery network.
Based on recent research at the University of Waterloo, you may well be able to treat the bandwidth usage as a risk factor and treat the option to buy more bandwidth as exactly that: an option on a real commodity. You would likely be able, then, to price the value of waiting to invest versus the value of investing now with a given expected return. Basically the cost of holding off on investing would then be quantifiable and you could choose the best time for investment.
There has been some good research done on this lately which you can read up on at the U. Waterloo Scientific Computation Group which did the work in co-operation with telecoms and the Finance department. The math is perhaps a little heavy going, but the results may put you on a firmer footing than doing the same computation with NPV or similar methods.
Disclaimer: I'm currently doing research with this group, though not exactly on this topic.
Let's face it. Most of the suggestions above are useless. Since when is a company going to officially distribute stuff via Kazaa or BitTorrent? Sorry, but when Microsoft says 'To download our latest Service Pack, use Kazaa' then pigs will be flying. It's so unprofessional.
The easiest solution is not to host it yourself, but to use specialized file hosting ISPs. There are lots of these around, and it's a trivial task on Google to find one at the price you want. These are ISPs that entirely focus on hosting large files for download, with servers optimized for that job.
There's no point in lagging out your regular servers which are probably optimized for something else.. and a dedicated file host can scale as you go.. which would usually cost you a packet.
Start with a best guess for amount of bandwidth based on number of users and type of usage. Then monitor and adjust. If you pay for uneeded bandwidth you may be losing money. If underestimate bandwith needs and lose business, you may lose money...
Some things to consider checking into:
Use SNMP and RMON to manage and monitor your wide area connections. You will be able to do trending on your traffic to see what percentage of used is used during any predetermined interval. Free tools like MRTG are a great place to start.
Good network hardware, maybe most specifically your edge router(s) and firewall(s), that allows you to configure some sort of priority queuing or traffic shaping. Cisco's QOS features in its IOS based routers are known to be particularly strong.
A service or dedicated person to make sense of the traffic reports. It is an ongoing process.
You will also likely want to consider measuring "latency" as well as bandwidth consumption.
Also note that during a one minute average, 50% use can be 30 seconds of 100% and 30 seconds of 0%. Be aware that many of the bandwidth utilization numbers that are reported can be averages. You will want to look for peaks as well.
Good Luck.
~8^]
How to handle this situation primarily depends on where your site is hosted.
If you have your servers hosted with co-location provider like AT&T, Globix, Cable and Wireless (ha ha), Verio ot the like then you'll almost always have the option to "burst" above your monthly allotment of transfer per month - problem solved. Just do some simple math and figure out what is the max of the LAN connection of the provider, how much your server(s) can handle with respect to transfer and you'll be able to figure out if you need to add hardware or NIC's to handle the load. Generally speaking the "bursting" is usually calculated fairly. Also, in months that you use less bandwidth you won't have to pay for a higher class of service above the already agreed upon monthly transfer rates.
However, It doesn't sound like this is the case. A t-1 has a physical capacity to transfer 1.544 Mb/sec. You will never be able to "burst" above this rate. As long as your servers are at the end of this pipe they will never be able to transfer more than ~ 180K/sec to the internet at large. Your options at this point are:
* Add more capacity to your hosting site (expensive).
* "Partner" with a company like Akamai for content delivery. It'll be a few thousand dollars plus the cost of bandwith to set up the content redistribution. Not really a bad deal if you are serious about delivering your content to your users. They also have great reporting about who and where your users are coming from.
* Your last option is to set up a shared/dedicated hosting account with a provider that charges you by the GB. That way you only pay for what you use + the monthly cost of the server. Try interland they have some good deals.
The bottom line is that if your site is hosted at the end of a physical connection you own - it's not going to be enough. You'll need a datacenter environment that has a PhatPipe to the internet and a machine or machine(s) that can handle the throughput. If you have the cash go with Akamai - they are good at what they do.
to clarify, it was a FreeBSD x86 server. The
largest ftp site in the world.
For every annoying gentoo user, are three even more annoying anti-gentoo crybabies. Take Yosh from #Gimp for example.
P2P realizes the two facts that you obviously don't:
1) Not everyone uses their Internet connection 24 hours per da
2) Most people don't need hard drive space these days (i.e. storage is cheap)
Combine these two developments, and you have a lot of upload bandwidth sitting idle. I would argue that P2P becomes MORE effective, not less, as you move to legitimate files, because people are more likely to leave it running when they aren't afraid of the RIAA/MPAA tracing their connection down. Since ISPs have been reluctant heretofore to ban P2P traffic (after all, it is a driving factor for adoption of high-speed Internet access), all those hours people are sleeping with a P2P program open is free downloads for the rest of us.
There are valid complaints against P2P (untrusted incoming executable data, very high latency, no centralized validation of data, etc.) Use them instead of the bandwidth non-issue if you don't like P2P.
First of all, you need to know what's normal for your network, and how your bandwidth is actually being used:
h no logies_white_paper09186a008017f93b.shtml
d ex .pl?i=Technologies&f=1387
http://www.linuxgeek.org/netflow-howto.php
http://wwwstats.net.wisc.edu/
http://www.arbornetworks.com/products_sp.php
Then you need to make use of technologies which allow you to provision accordingly:
http://www.cisco.com/en/US/tech/tk543/tk757/tec
http://www.cisco.com/pcgi-bin/Support/browse/in
Juniper and other vendors have equivalent capabilities, I'm just not very familiar with them. But the concepts are the same.
Be sure to tell your colo or file hosting provider what your projected usage is, and how many megabits you may want access to, to assure that they can handle it. You may also want to make a courtesy call a day or so prior to each launch to let them know what to expect.
Remember when Eddy Van Halen got tounge cancer a couple years ago? THAT was a busy weekend for their website, which we host. Of course, they didn't have any warning, but boy-o, that was bigger than any slashdot effect that I've ever seen. We also host O'Reilly (the computer book folks), so we certainly see plenty of slashdotting.
We're at: http://www.sonic.net/sales/colo/
Shop around - but keep in mind that buying from someone near your intended downloader may help you with both latency and costs. The SF Bay Area has the best pricing for bandwidth, and the lowest latency connections to the highest number of users - that said, if your target market is on the east coast, you should be in Hearndon, VA or NY or Boston.
-Dane Jasper (Sonic.net)
-- Dane Jasper Sonic.net, Inc.
Have the timeout on a file reasonable, though. FTP files generally don't change that frequently, so hold those for 48 hours. HTTP pages can change a lot more often. Make sure the caching server is configured to respect the HTTP flag to not cache, and don't cache unmarked web pages for more than a few minutes.
This should be sufficient to spread out the load some. I prefer to have a few "extras" in there, though. If I know there's going to be a popular file released, it makes sense to have a CRON job pre-load that file into the cache during off-peak hours. That way, it's in the cache and ready for users by the time anyone gets round to requesting it.
If there are a few web sites that are VERY popular but largely static, set up a "neighbor" cache, which ONLY caches those web sites, with a very long time-out. Use a program like Harvest to grab the entire site, via the cache, and you'll then have everything ready for the users. (It'll also be searchable, via Harvest, which'll be a bonus.)
The second option is at the network layer, and should be used only if the above is not sufficient. Enable "diff-services" and "Quality of Service" in the kernel. How to do this depends on the OS you use. Linux and all the *BSDs support these options, but how you set it up varies with each.
Once you've done that, enable "HQF" or "CBQ" queueing discipline, and attach those to the FTP and HTTP services. Configure them such that each user is guaranteed a certain level of bandwidth for that service, if they request that or more. They get more only of nobody else needs it. (This is usually described as a "soft ceiling".)
You also want to enable the "RED" networking option.
This isn't as hard to do as it sounds, and it can massively ease network congestion.
(CBQ = Class Based Queueing. HQF = Heirarchical Queueing Function. RED = Random Early Detection.)
Once you've applied both the "high level" and "low level" solutions, your network congestion should be massively eased. Again, though, use pre-loading for the cache as extensively as you can to ease those peak-time burdens.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
He'd be happy to make money customizing it for people...
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
What I would do is colocate all my servers in a datacenter.
Take a burstable bandwidth, let's say that can burst to 100mbs, but to control your bandwidth in most time to ensure you do not go over the cost, you configure your router to not allow more than let's say 1mb of bandwitdh or whatever you want as a maximum and willing to pay for in normal time.
You should then monitor your bandwidth usage in real time, as well as the logs on the machines, and adjust the traffic shaping to the amount of traffic you want to allow.
For example, you know what on that day, you will do a marketing operation, and you are willing to spend $xxx more for the bandwidth, you then change your setting right before your marketing plan to the maximum of bandwidth you are willing to pay.
my 2c...
Who in their right mind modded the parent to +5???
"Bandwidth trading" escapades have bancrupted Williams and contributed to Enron having to go ever more aggressive with their "creative accounting." It reeks of the same foul smell as the "real options" nonsense. (Note: the article referenced even has "real options" in the keywords).
Basically you've got a whole bunch of not-so-smart people who can fake expertise in nearly anything as long as they don't get their hands dirty (aka academics) colluding with some not-so-smart MBA's in executive positions - and then you've got a problem for everyone around them.
This is a problem that needs to be priced based on actuarial type methods (a'la insurance). Just because you can write a (usually bad) price process for "bandwidth" doesn't mean that you can logically apply all the consequent financial mathematics.
And, here's a party-pooper - real options are not options at all - no matter what tenured clowns like Pindyck and Dixit want you to believe.