Slashdot Mirror
Managing Bandwidth and Bandwidth Costs?
"I'd like to illustrate the second concept. When you have your (for example) T1 and you're not really using it, you are still paying for all that bandwidth. It's like the car that sits in your garage, you're still paying insurance and car payments on it even though you're not using it. But then you put up a new game, serve new media or suddenly become the 'Site of the Day' and your bandwidth is flooded and maxed out. For that case, it's like you've bought a car that only goes 40 miles an hour but while the demand exists and only while that demand exists, you need a car that goes 150 miles an hour. You don't want to pay the money for a car that goes 150 because you only need it occasionally. Later, you know you'll need that car to go 220 but you're not there yet.
So if this makes sense with regards to bandwidth, it is like you'd want burst-bandwidth depending on need. Do any of you face this problem? If you do and have solved it, I'd love to hear about your strategy. Once this is solved, we get back to the first question, how do you manage that cost, put a number on it and either fit it in to your business model or pass it on to your customers?"
We can do it the slackware way - tar it up and let someone else do it.
Or we can do it the debian way - let someone else tar it up and then do it.
But we like the Slashdot way most. Post it up and watch the traffic floooooooooooooooow....
Too many to spell out here.. Search google, LOTS of people sell bandwidth at X mbits/s with a BURST to x mbit/s.
You figure our how much you will estimate you will use as a total per month, and you pay for that. You cruise at 2mbit/s mostly, then you explode to 145mbit/s and at the end of the month you average out to 6mbit/s.
The idea is you BUY your peak speed, but pay for a low average..
EVERYONE does this now, call around... It was kind of silly for you to ask Slashdot how to do this. UGH.
- Voxel
Modesty is one of life's greatest attributes
Where possible BitTorrent is your friend. If you can use it, let your customers help out with the bandwidth usage. They will probably get the file(s) quicker and you won't have servers getting torched.
If you want to pay for data by the gig, rather than by the pipe size, just sign up with an ISP which allows that.
At the ISP where I work we offer fiber connections that allow for increased bandwidth for certain periods of time. For example, our burstable connections are usually around 1-3 meg for normal times, then burstable up to 10 megs. I'm sure you can find something suitable to you.
:-)
*blatant sales pitch*
If your buisness is near Southern Ontario, check out our website at www.sentex.net. We rock
Here's a free tip --- don't post your site to slashdot.
Congratulations - you passed this one.
Anyway the answer to all your prayers is obviously BitTorrent.
1) Agree with your ISP on a standard data rate, burstable to X as needed.
2) Use RTG to monitor traffic in and out, making sure that you know what switch/ports/etc. that client is using.
3) Charge the client (this is usally done based on 95th percentile).
4) Profit!!!
libertarianswag.com
This is what Akamai and the like are sold for.
It's almost certainly going to be cheaper than just buying bandwidth.
Or you could go for the approach of colocating your own box somewhere central for the heavily hit stuff.
Even this will be a whole lot cheaper and won't impact on your normal traffic to your organisation.
If you need proof that Kast is better than BT in this situation look at http://konspire.sourceforge.net/BitTorrent.shtml .
Hope it helps.
As soon as we all have 50mpbs 802X connections and, anonymous P2P software. This kind of question will never be asked.
With 100 gig hds, and reliable high speed connectivity 24/7. It's pretty easy to see what could be built from that.
Someone wants to download the new 150mb CS or BF1942 upgrade? Just enter the name of the file, select it...and the P2P software does the rest. Initiating multiple downloads from about 20 or so nodes in parallel. Then you just glue the program together once you have downloaded all the pieces.
Mirroring is such a hack, and dynamic bandwidth is the last gasp of the client-server paradigm. Let's move on.
The government has a defect: it's potentially democratic. Corporations have no defect: they're pure tyrannies. -Chomsky
that offers "pay by the GB". The hosting company I work for has GiGE links but only pays for the exact amount of traffic we push through the link. The monthly cost on the line is minimal, or nonexistent depending on the provider.
Since we have more then one link as well, it gives us redundancy and the upper hand to negotiate the best price per GB, so we can send 90% of our traffic out that link. If the next month a different provider comes back with a cheaper price, we switch it around and send the 90% out their link. Within days of cutting the traffic off for a link, we can usually expect a phone call from the sales rep with lower price offer.
Any idle links we have don't cost us anything extra, since we _will not_ deal with any provider that doesn't offer pay by the GB. Paying for the raw link speed, regardless of how much traffic you push through, or paying 95th percentile prices are all mostly a rip off.
Open Source Time and Attendance, Job Costing a
Couple of different options. First, you could talk to the Content Delivery Networks (CDN's) like Akamai or Digital Island. They can probably help you (for a price).
Another option is colocation. In particular if you have short traffic spikes. Many colocation places charge your for at a '95 Percentile'. This will cut out about 3 days worth of 'peek traffic' and you only pay for the maximum bandwidth you use after removing the top 5%. Just make sure the colocation place has enough bandwidth to handle the spikes.
Some ISPs (e.g. Yipes) offer flexible contracts that allow fast (daily?) bandwidth changes. So if you announce a new version of your product, you can increase your bandwidth until the rush is over.
One hint: Try to move the large file/content away from your 'importants' networks, so other things like e-mail keep flowing even if the content site is running into issues due to load.
---- join dshield.org Distributed Intrusion Detec
It seems to me that this is exactly the type of problem that places like akamai and cable and wireless (was digital island) are trying to solve. Pay only for the bandwidth you use, leverage their existing distributed architecture, profit. You can try to get bustable bandwidth etc, but in the past I've found it to be more expensive. Things may have changed since them (a year ago) but you should still look into a content delivery network.
Based on recent research at the University of Waterloo, you may well be able to treat the bandwidth usage as a risk factor and treat the option to buy more bandwidth as exactly that: an option on a real commodity. You would likely be able, then, to price the value of waiting to invest versus the value of investing now with a given expected return. Basically the cost of holding off on investing would then be quantifiable and you could choose the best time for investment.
There has been some good research done on this lately which you can read up on at the U. Waterloo Scientific Computation Group which did the work in co-operation with telecoms and the Finance department. The math is perhaps a little heavy going, but the results may put you on a firmer footing than doing the same computation with NPV or similar methods.
Disclaimer: I'm currently doing research with this group, though not exactly on this topic.
Let's face it. Most of the suggestions above are useless. Since when is a company going to officially distribute stuff via Kazaa or BitTorrent? Sorry, but when Microsoft says 'To download our latest Service Pack, use Kazaa' then pigs will be flying. It's so unprofessional.
The easiest solution is not to host it yourself, but to use specialized file hosting ISPs. There are lots of these around, and it's a trivial task on Google to find one at the price you want. These are ISPs that entirely focus on hosting large files for download, with servers optimized for that job.
There's no point in lagging out your regular servers which are probably optimized for something else.. and a dedicated file host can scale as you go.. which would usually cost you a packet.
P2P realizes the two facts that you obviously don't:
1) Not everyone uses their Internet connection 24 hours per da
2) Most people don't need hard drive space these days (i.e. storage is cheap)
Combine these two developments, and you have a lot of upload bandwidth sitting idle. I would argue that P2P becomes MORE effective, not less, as you move to legitimate files, because people are more likely to leave it running when they aren't afraid of the RIAA/MPAA tracing their connection down. Since ISPs have been reluctant heretofore to ban P2P traffic (after all, it is a driving factor for adoption of high-speed Internet access), all those hours people are sleeping with a P2P program open is free downloads for the rest of us.
There are valid complaints against P2P (untrusted incoming executable data, very high latency, no centralized validation of data, etc.) Use them instead of the bandwidth non-issue if you don't like P2P.
Be sure to tell your colo or file hosting provider what your projected usage is, and how many megabits you may want access to, to assure that they can handle it. You may also want to make a courtesy call a day or so prior to each launch to let them know what to expect.
Remember when Eddy Van Halen got tounge cancer a couple years ago? THAT was a busy weekend for their website, which we host. Of course, they didn't have any warning, but boy-o, that was bigger than any slashdot effect that I've ever seen. We also host O'Reilly (the computer book folks), so we certainly see plenty of slashdotting.
We're at: http://www.sonic.net/sales/colo/
Shop around - but keep in mind that buying from someone near your intended downloader may help you with both latency and costs. The SF Bay Area has the best pricing for bandwidth, and the lowest latency connections to the highest number of users - that said, if your target market is on the east coast, you should be in Hearndon, VA or NY or Boston.
-Dane Jasper (Sonic.net)
-- Dane Jasper Sonic.net, Inc.
He'd be happy to make money customizing it for people...
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
What I would do is colocate all my servers in a datacenter.
Take a burstable bandwidth, let's say that can burst to 100mbs, but to control your bandwidth in most time to ensure you do not go over the cost, you configure your router to not allow more than let's say 1mb of bandwitdh or whatever you want as a maximum and willing to pay for in normal time.
You should then monitor your bandwidth usage in real time, as well as the logs on the machines, and adjust the traffic shaping to the amount of traffic you want to allow.
For example, you know what on that day, you will do a marketing operation, and you are willing to spend $xxx more for the bandwidth, you then change your setting right before your marketing plan to the maximum of bandwidth you are willing to pay.
my 2c...