Slashdot Mirror
Managing Bandwidth and Bandwidth Costs?
"I'd like to illustrate the second concept. When you have your (for example) T1 and you're not really using it, you are still paying for all that bandwidth. It's like the car that sits in your garage, you're still paying insurance and car payments on it even though you're not using it. But then you put up a new game, serve new media or suddenly become the 'Site of the Day' and your bandwidth is flooded and maxed out. For that case, it's like you've bought a car that only goes 40 miles an hour but while the demand exists and only while that demand exists, you need a car that goes 150 miles an hour. You don't want to pay the money for a car that goes 150 because you only need it occasionally. Later, you know you'll need that car to go 220 but you're not there yet.
So if this makes sense with regards to bandwidth, it is like you'd want burst-bandwidth depending on need. Do any of you face this problem? If you do and have solved it, I'd love to hear about your strategy. Once this is solved, we get back to the first question, how do you manage that cost, put a number on it and either fit it in to your business model or pass it on to your customers?"
how can we get our company's bandwidth setup so we can survive a slashdotting?
maybe if he got some comments from the peeps at osnews or cnet...
vodka, straight up, thank you!
Whenever Savvis has end of quarter sales you can negotiate the price a ton.
Right now they're about 700/month for a managed T1 with local loop included.
1) Agree with your ISP on a standard data rate, burstable to X as needed.
2) Use RTG to monitor traffic in and out, making sure that you know what switch/ports/etc. that client is using.
3) Charge the client (this is usally done based on 95th percentile).
4) Profit!!!
libertarianswag.com
Yeah, but you probably want to put out a branded version of BitTorrent with a nicer user interface.
;)
The current UI looks like puked out in 2 days
Also it might make sense to provide a BitTorrent version that installs through ActiveX. The download version should be offered for Linux, Mozilla and Opera users.
Christian
--- Eat my sig.
If you need proof that Kast is better than BT in this situation look at http://konspire.sourceforge.net/BitTorrent.shtml .
Hope it helps.
As soon as we all have 50mpbs 802X connections and, anonymous P2P software. This kind of question will never be asked.
With 100 gig hds, and reliable high speed connectivity 24/7. It's pretty easy to see what could be built from that.
Someone wants to download the new 150mb CS or BF1942 upgrade? Just enter the name of the file, select it...and the P2P software does the rest. Initiating multiple downloads from about 20 or so nodes in parallel. Then you just glue the program together once you have downloaded all the pieces.
Mirroring is such a hack, and dynamic bandwidth is the last gasp of the client-server paradigm. Let's move on.
The government has a defect: it's potentially democratic. Corporations have no defect: they're pure tyrannies. -Chomsky
May end up costing us on an "as-used" basis. Wouldn't it be nice, actually, for the occasional e-mail reader, web-browsing, non-bandwidth-hogging person, to only pay $5/month for the bandwidth they consume? Of course for most of us, it'd end up costing us $80/monthly for our DSL/Cable connections, as we download gigabytes a month, at blindingly fast speeds. Sort of like cellphones are used now-a-days; on an as-used basis, with a chunk of pre-paid megabytes, or whatever. Of course, if you're an ISP, there's the issue of counting those megabytes, which could be solved by bandwidth-accounting software (which I'm sure there are plenty of, but here's one I know of for Windows-based systems.
Based on recent research at the University of Waterloo, you may well be able to treat the bandwidth usage as a risk factor and treat the option to buy more bandwidth as exactly that: an option on a real commodity. You would likely be able, then, to price the value of waiting to invest versus the value of investing now with a given expected return. Basically the cost of holding off on investing would then be quantifiable and you could choose the best time for investment.
There has been some good research done on this lately which you can read up on at the U. Waterloo Scientific Computation Group which did the work in co-operation with telecoms and the Finance department. The math is perhaps a little heavy going, but the results may put you on a firmer footing than doing the same computation with NPV or similar methods.
Disclaimer: I'm currently doing research with this group, though not exactly on this topic.
Let's face it. Most of the suggestions above are useless. Since when is a company going to officially distribute stuff via Kazaa or BitTorrent? Sorry, but when Microsoft says 'To download our latest Service Pack, use Kazaa' then pigs will be flying. It's so unprofessional.
The easiest solution is not to host it yourself, but to use specialized file hosting ISPs. There are lots of these around, and it's a trivial task on Google to find one at the price you want. These are ISPs that entirely focus on hosting large files for download, with servers optimized for that job.
There's no point in lagging out your regular servers which are probably optimized for something else.. and a dedicated file host can scale as you go.. which would usually cost you a packet.
Why is it that BitTorrent is the fix-all solution for the /. crowd? I for one HATE BT, because, seeing as I am on DSL, I usually get no more than 20K/Sec even when sharing to my maximum potential. Why you ask? Because due to the assymetrical nature of ADSL, once my upstream gets clogged, it cannot facilitate the neccisary ACK's that need to be sent out to the other end in order to ensure my download stays up.
In other words, if I'm uploading at my maximum speed, I can't reply quickly enough to tell the nodes I am downloading from that the last packet got through, so they resend it instead of the next packet until I can squeak out an ACK or two to let them know to move on.
BT is a nice concept, but in practice it needs work.
"The saddest words of mice and men, are not those which were, but should have been."
I'm running into the problem of hosting several different sites on shared servers, and need to be able to tell each customer what their useage is - what are you all using to do this?
"Going to war without France is like going deer hunting without your accordion." - Jed Babbin
A local dialup ISP just sets the router to stop forwarding for 45 seconds at a random interval, somewhere between 5 and 45 minutes after the last stall.
You wouldn't believe how many dialup customers use KazaaStein or other music stealers. Really ruins their downloads, but the text people don't seem to mind.
The music stealers go elsewhere the first time a competitor offers a deal for new customers. We've found 5% of our customers use 95% of bancwidth, give them to the competition.
Totally. Akamai really isn't that expensive when compared to buying your own bandwidth. The only thing about Akamai (from what I've heard, I've never used it) is that you have to plan updates a few days ahead of time to give them time to propagate amongst the various Akamai balancing servers. But have you ever noticed that downloads from Akamai are usually the fastest around? And that pretty much every corporate high-traffic site uses them (CNN, Apple, IBM, ETrade, FedEx, Yahoo, etc.) so I think they'd be fine for your small usage.
> One problem might be corporate customers who have to pay for their bandwidth
Actually I was working on a P2P software that solves this problem,
by introducing "neighbourhood". That means, when several machines
in the same LAN receive the update, they will prefer to connect to
each other rather than to a machine outside on internet. This
SAVES money (on ingres), provided that exgres bandwidth isn't
excessively donated back to the P2P community.
Have the timeout on a file reasonable, though. FTP files generally don't change that frequently, so hold those for 48 hours. HTTP pages can change a lot more often. Make sure the caching server is configured to respect the HTTP flag to not cache, and don't cache unmarked web pages for more than a few minutes.
This should be sufficient to spread out the load some. I prefer to have a few "extras" in there, though. If I know there's going to be a popular file released, it makes sense to have a CRON job pre-load that file into the cache during off-peak hours. That way, it's in the cache and ready for users by the time anyone gets round to requesting it.
If there are a few web sites that are VERY popular but largely static, set up a "neighbor" cache, which ONLY caches those web sites, with a very long time-out. Use a program like Harvest to grab the entire site, via the cache, and you'll then have everything ready for the users. (It'll also be searchable, via Harvest, which'll be a bonus.)
The second option is at the network layer, and should be used only if the above is not sufficient. Enable "diff-services" and "Quality of Service" in the kernel. How to do this depends on the OS you use. Linux and all the *BSDs support these options, but how you set it up varies with each.
Once you've done that, enable "HQF" or "CBQ" queueing discipline, and attach those to the FTP and HTTP services. Configure them such that each user is guaranteed a certain level of bandwidth for that service, if they request that or more. They get more only of nobody else needs it. (This is usually described as a "soft ceiling".)
You also want to enable the "RED" networking option.
This isn't as hard to do as it sounds, and it can massively ease network congestion.
(CBQ = Class Based Queueing. HQF = Heirarchical Queueing Function. RED = Random Early Detection.)
Once you've applied both the "high level" and "low level" solutions, your network congestion should be massively eased. Again, though, use pre-loading for the cache as extensively as you can to ease those peak-time burdens.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Look at wondershaper.. I have it set to prioritize my upload stream (don't bother having it reduce queueing on my downstream, hasn't been necessary) by putting ACKs, DNS, irc, ssh, ping, and http (port 80 target; haven't addressed https) in the high priority queue, and everything else in bulk. Works pretty well, although I'm considering adding more ram on the NAT box and running a local DNS server again; Comcast's are flakey.
"'Tis great confidence in a friend to tell him your faults, greater to tell him his." --Poor Richard's Almanac