Slashdot Mirror
A Crash Course on Network Bandwidth Metrics?
Kind of Blue asks: "I work for a small software development company in India, providing development services for a company in UK. We connect to the UK client through VPN; and our correct staff strength is around 13. We are going in for a major upgrade in our internet connection — and owing to the size of the firm, cannot afford the services of a networking expert/consultant. Hence I, a layman, have been asked to look into the matter and decide on the ISP and the bandwidth. I have a vague idea about the required bandwidth — it must be around 512 kbps(remember, it's India I am talking about!) and must be a persistent connection, since we use source control softwares connecting to servers in the UK.
There doesn't seem to be a 'networking for dummies' kind of resource on the web. No one seems to talk of network metrics anywhere. So, can Slashdot give me a crash course in what I need to know?"
"Our present ISP gives us a DSL connection of 512 kbps on 1:4 sharing. There are frequent disconnections; and hence loss of work while code check-ins. As we are increasing in strength, I am also looking at more bandwidth. But what bamboozles me is how are these things measured? Will I get a better bandwidth if I take a 512 leased line on 1:2 sharing? When the staff doubles, should I upgrade to a 512 connection on 1:1 sharing or must I take 1 Mbps on 1:4 sharing?(There's a huge price gap between the 2 here in India) In any case, how does one decide the optimum bandwidth required for a bunch of 15 developers on VPN and Source control?"
it must be around 512 kbps(remember, it's India I am talking about!) and must be a persistent connection, since we use source control softwares connecting to servers in the UK.
Have you actually measured how much bandwidth your source control software application consumes? To me that'd be the very first step, before you look into upgrading from one voodoo number to another. Real data is very often the key to good decisions.
More data, damnit!
Dear slashdot:
I'm trying to outsource your jobs. However, I don't know all that I need to know. Would you mind providing free training in your spare time for your replacement?
OTOH: you probably don't need as much speed as you do reliability. I would guess that an unsharing situation will work better for you.
You might, over the short term, insert an extra hop in your network in front of the ISP hop, and measure avg bandwidth there over the course of a business day. As long as you have enough total bandwidth / second to match your typical usage (allowing for some margin, and expected growth) in a business day, all you'll cope with is some latency.
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
Unless there are also developers in the UK constantly accessing the code repository there's no good reason to use source control outside of your office. It'll be considerably cheaper to buy a small server with a RAID array, stick Subversion on it, and put it on your network, than it would be to be transfering files all the time. If the company in the UK that you work for insist on having control of the code then write something that syncs diffs between your local repository and the UK nightly/a few times a day/hourly/whatever. 13 coders aren't likely to be checking in more than a couple of megabytes of changes every hour.
Note: SVN/CVS/SourceSafe/whatever can probably do this already, IANAEIVS (I am not an expert in versioning systems).
http://twitter.com/onion2k
One thing you could look at is to use subversion as your source control system, as it only transfers diffs accrss the network so is very good with low bandwidth connections.
and our correct staff strength is around 13
So after adding in your incorrect staff, you get a total of?
How much data are they pushing onto the source control server? Why not just move the source control server to India?
Contention ratios only help to ensure that you get the full 512Kb/s, it won't increase your reliability I wouldn't think. You will have to increase your total bandwidth size, not just change your contention ratio when you get more staff.
...so I will just say what half the other people here are thinking.
Maybe you can outsource this project to the west since there are plenty of very well experienced people who know this stuff who could use the work.
I don't hate India or Indians, but it will be a cold day in Hades before I give assistance to a shop whose only purpose in this world is to displace jobs formerly occupied by my neighbors.
Yes and No. Right now you are setup on a 512K/s line, shared with up to 4 other customers. This means that if Customer A is using 400k/s, and Customer C is using 100k/s, then customers b and c get to share 12k/s. (without getting into the discussion of prioritizing and QOS, I'm way oversimplifying) Or, depending on the setup, each of you might be setup to only get 128k/s when all of you are using your connection. Now, if its the first case, and you have one other customer that is hogging all your traffic, then you will benifit from moving to a 1:2, or even just getting rid of them on the same circuit. However, the only way to be sure that you have 512k/s if you need it is to make sure you have your own connection, and a garuntee from the ISP. (in the US, these are called "Business class" DSL circuits in many areas) It will be MUCH more expensive.
What are we going to do tonight Brain?
... to england.... its loverly here...
But then you might find your job outsourced to india....
reliability of an internet service is not always dependent on bandwidth used.
Most times it depends on your ISP and their upstream ISPs' network, quality of service, etc.
Note: it is possible you are reaching your real bandwidth limit (what the ISP makes available to you) at different times which causes delays resulting in timeouts. Might be volume of traffic over the VPN link or it might be non-VPN traffic to the rest of the internet.
If you want more bandwidth (hedged bet for the future) and better reliability, then what you want is redundant ISPs and VPN concentrators that support redundant ISPs and more importantly support load sharing.
I have no relationship with the following company except I evaluated one of their products earlier this year for non-VPN redundant ISP problem..
http://www.astrocorp.com/
Given what you say about large price delta between DSL@512 w/1:2 vs 1Mbps w/1:4, you would be better off with two DSLs each @512 w/1:2 from two different ISPs.
Note: The key to redundancy is making sure each of the ISPs you choose don't share the same upstream ISPs.
Not sure of the market in India but this is hard in many locations where the local telco is the only upstream provider.
This sig space tolet, reasonable rate.
Dear Esteemed Readers of Slashdot,
I have been assigned the task of sending email for my employer located in Kontagora, Nigeria. Can one of you explane to me how this internet works. Experience with bank transfers is helpful.
Thanking you kindly,
Mister Mbata Odiaka
http://news.bbc.co.uk/1/hi/world/south_asia/527267 2.stm
Deleted
Rather than "Ask Slashdot" I suspect that he might have been better off going here:- clinic.htm
http://www.mtholyoke.edu/~ebarnes/python/argument
(just predicting the next 20 comments...)
When I started my current job the company had minimal bandwidth requirements, a dedicated T1 and only 7 people using the connection. And they all complained about the "slow" connection. Problem was latency. 250ms to the ISP's gateway was typical and often worse. Given that displaying a typical web page involves several DNS lookups and many connections those quarter-seconds added up quick.
We switched to a new provider (actually had to, the old one was in bankruptcy) and people were stunned by the improved speed. Of course the "speed" was the same - they were both T1s. Only the latency had changed.
Using the appropriate transfer method, bulk data transfers may not be significantly impacted by bad latency. But anything involving lots of small transfers and/or short connections can be killed by bad latency.
~~~~~~~
"You are not remembered for doing what is expected of you." - Atul Chitnis
You say you can't afford to figure this out with an expert. My response is that you're asking your competition to help you cut your costs so that you can better compete with them.
Enjoy the answers.
-Tim Louden
Have you guys ever seen the questions that people from India post to USENET? Its hilarious: they post a short synopsis of their question, and usually don't include enough information. Then at the end they have the gall to add something like: "I NEED TO KNOW IMMEDIATELY SO LET ME KNOW THE ANSWER QUICKLY, RAJA" as if they are talking to paid tech support! I am assuming its just a communication/translation problem.
More to the point, there are calculators to solve this kind of questions on the internet. Try google.
You could have figured out how much bandwidth you don't have by posting a link to your local server. Being Slashdotted would've shown your ISP you need more bandwith!
Then; Take the number of connections at your end before your connection was severed, and add that to 'half' the number of connections when your ISP's server/connection maxed out, multiply by the number of toilets in your office, and divide by Pi.
Why don't you just outsource your development work to the U.S.? Then you won't have to worry about bandwidth.
you've come to the Slashdot community which is mostly Americans to ask us to help you out. Someone who is taking our jobs away from us. I don't think so.
Cyberbite Networks - Web Hosting, Dedicated Servers & Colocati
Welcome to the world of business. Are you starting to understand why we charge as much as we do outside of India? It's because we have to support our business infrastructure and still make a profit for all our effort! Don't like it, go back to India. Oh wait...
your developers to stop downloading porn and illegal music at work.
Cyberbite Networks - Web Hosting, Dedicated Servers & Colocati
You're (ostensibly) a professional services shop, seeking to provide (ahem) professional services to another company, over the internet. You plan on using the internet as your delivery method for the services rendered. Now, why on earth would you ask Slashdot for professional services advice? What if your customers instead decided to go it on their own for internet services? While a 13 person shop may be a small company, I refuse to believe that you cannot spend the money necessary to have a PROFESSIONAL consultant come in and advise you, at the very least, what your bandwidth needs are. I would surmise that your clients expect you to act like a professional shop - I'd recommend hiring a professional, instead of trying to skimp and asking Slashdot.
This article could probably be filed in the 'I need legal advice' portion of the archives.
(Also, if you're a programmer, at least try and check out Google. If you're technical enough to program, you're technical enough to grep at least some small portion of networking.)
One of the major things to consider is that TCP's performance is also bound by the recieve window size[2]. Since TCP can only send more data as it's ACK'd due to this window, RTT plays a factor.
So also consider the "bandwidth delay product"[1] to where you're connecting to and compare this to your recieve window size. Consider checkking to make sure increasing your recieve window size won't get you better performance. This needs to be done at the end of the connection that's recieving the data, which for a checkin is likely to be in the UK...
Another thing is that TCP slows down whenever it sees loss as it considers loss to be due to congestion. The amount of time it takes to recover from this is based on round trips (ie latency again). This has a far more complicated formula to work out, see my webpage http://www.wand.net.nz/~perry/max_download.php for a simple calculator to work it out for you.
Since you're in India I suspect that the delay to your UK collegues is probably going to end up going the "wrong" way around the world and will therefore due to the speed of light in fibre your latency is going to be high.
[1]: Easily calculated by the obvious formula bandwidth*delay=recieve_window_size. Perhaps a more useful version of this is rwin/delay = maximum_tcp_bandwidth.
[2]: Not to be confused with the congestion window which is a different concept.
Almost all of the Japanese imports sold in the US are manufactured in America (USA). You are still helping the US economy when you buy one.
I'll throw in a comment along with everyone else... Why would U.S. Slashdot users (the majority, I believe) help you with your technical problems so you can undercut our development costs and take away our potential work? Part of the cost of doing business in India is (apparently) the high cost and crappy bandwidth of the internet access. (Most of the people on freelancing websites say they have a 256k or 512k cable connection. That is a joke compared to the 3 to 8 Mbps connections we have here, which are in turn a joke compared to the 10 to 100 Mbps connections offered elsewhere.) I would try hiring someone who knows about networking, or perhaps buying a book or three, instead of trying to get free help from Slashdot. We're not stupid, and do have something of a self-preservation instinct.
At least, thank you for using proper grammar and not phrasing things in the usual "Our internet connection is being of 256 kbps, and we are needing to upgrade it" manner. It's the most irritating mannerism, and it's one specific thing. I'm usually understanding of language barriers, but if I consistently mangled or abused a certain verb form in another language, I would expect the listener to be annoyed (my Spanish teacher usually is).
It would be nice if, given that you appear to understand the difference between Internet access in the US and India (ours is much faster), you would explain the different notation for bandwidth. Essentially, it's a clever way of hiding the fact that a 512kbps line doesn't give you 512kbps download *or* upload, it gives you 512kbps *total*, split between download and upload. It makes the line sound faster than it is.
Since apparently they express bandwidth in India differently, you'll have to do a little math. I believe (from some hasty Googling) that x:y sharing means x parts upload and y parts download, out of x + y parts total.
So that means that if you have an n kbps line with x:y sharing, you get n*y/(x+y) kbps down and n*x/(x+y) kbps up. An example would be your current 512k line with 1:4 sharing. You get 102.4 kbps up and 409.6 kbps down. Just to drive the point home again about why outsourcing isn't perfect, I get 768 kbps up and 3072 kbps down. :-b
By the way, you can't have a "rough idea" that you need 512 kbps bandwidth total. Bandwidth is directional. You can know that, for example, you use about 256 kbps down, or that you use about 128 kbps up. But knowing the total won't help.
And, finally, I would recommend that you get the 1 Mbps leased line, since, first, it will solve your bandwidth problems much easier than reallocating the 512 kbps line, and, second, it will cost you more money, which will increase your rates closer to what U.S. programmers charge.
Didn't he have to compile numbers for WEekly Network USage and Annual Network USage?
Downmodding is the refuge of the weak. Don't downmod, make a better argument!
Have a look at a Riverbed Steelhead http://www.riverbed.com/. They do some good things with regards to TCP acceleration as well as application level acceleration. I've got some, and on a long haul connection (64Kb ISDN over 3000km) it effectively doubled my bandwidth and halved the latencies (paying for the units in about 18 months).
While profiling your activity is a good idea, and you can make some educated guesses when you expand the bandwidth, it's worth giving a bit of thought to a more creative use of the bandwidth you have.
These things (by the way) also run Linux - so you can trust them to work.
""There are frequent disconnections; and hence loss of work while code check-ins.""
Is this the key reason why you need a VPN?
[BTW, I hope that you are not using MS SourceSafe...]
Have you considered alternate revision control systems?
On the (not too expensive) commercial side, I am a fan of Code Co-op, from http://www.relisoft.com/. I've used it with a development team in China a few years ago, and they were definitely worse-off than you. Free trial available for download.
First, what you're asking isn't really about bandwidth metrics. There's really only one way to measure bandwidth. There are other factors that contribute to connection usability... notably packet loss (probability distribution) and latency. Those are not network bandwidth metrics, though.
Second, you're asking the wrong question. You seem to be presenting data to slashdot and asking them to make a decision for you. You need to figure out what you don't know, and ask those specific questions.
If you were asking for some tools with which to measure connection properties, I'd suggest:
http://dast.nlanr.net/NPMT/
It doesn't make a whole lot of sense for a development team to primarily use a source control system located thousands of miles away, particularly when they have a lousy internet connection.
I fault the American citizens for their purchasing and voting decisions.
don't blame "us" - "we're" just the sheep. Blame the criminals who legislated themselves a banking monopoly - that'd be the quasi-public "federal reserve" system.
Federal Reserve "prints" money to pay for the budget deficit. Printing money (monetary inflation) causes price inflation, and price inflation causes producers of goods to cut corners to keep their costs down, hopefully just a little while longer than their competitor.
In recent decades, the most popular "corner" to cut has been the corporation's american workforce. First it was low skill workers - textiles, shoes, etc. Little by little, outsourcing has worked its way up the supply chain, and today most consumer merchandise is produced in Asia.
The "federal reserve note" dollar will crash, as all fiat currencies eventually do. It shouldn't be too long - no more than a year, I'm sure. There are consequences for running record deficits, you know...
The good news is that the dollar crash will take out the families with "old money" too, and we can return to a more egalitarian economic system. It'll be rough, for a bit, but definitely worth it.
Learn the rules so you know how to break them properly.
www.teslabox.com
...There's a big "but", usually, to try and curb hyper-inflation interest rates are raised. So now your 5% PA mortgate is a 50% PA one. However, if you were smart enough to get into a fixed rate mortgage and there's still time before it converts into a variable rate one you should be a happy camper, if you can feed yourself that is. If you have savings, that 50% interest should come in handy for that too!
I am NaN
Poster cammoblammo has already called you on it ("Ah, there's your point: 'He's Indian, therefore he's ripping me off'"), but your attitude towards Indian people sucks. I am very sorry if you or friends are unable to work because senior people in their employers found other people in the world to do the work cheaper, but that does not justify the attitude that has you say "why help them out when they are stealing our jobs". Your bosses are selling your jobs to the lowest bidders -- Indian technology companies aren't stealing anything. Please reconsider your attitude.
There are 3 main things you should care about to figure out what is the right circuit:
h tml
Port Speed:
How fast is the connection regardless of sharing. This determines how fast information gets out of your office and on to the internet. Think of your file as a big string of ones and zeros and you need to take and put each one of those onto the wire at the rate of your port speed (ie. 512K, 1MB, etc). Therefore it will take twice as long to serialize something at 512K vs 1M. Since the Internet pipes between your office and the other end are going to be much larger than you office connection you only care about this for your office and the office on the far end. Note that its not going to help you even having 512K at this end if your other end is slower since everything will need to deserialize there at the slower speed. The lower port speed between the town offices will cap the theoretically fastest time you could transfer a file in.
Bandwidth:
Bandwidth is a measure of throughput over time. So using your 1MB 1:4 shared example, my port speed is 1MB but my bandwith is 256K because I will average to this amount of throughput over time. Therefore it matter how this sharing work (I'm not familiar with how they do it in India). If your traffic is measured and shaped to conform to this over time, think of this as the sustained amount of data you can transfer (like FrameRelay). If its just truely shared and if the people you are sharing it with are not using it you can (like cable), think of this as the theoreticaly minimum amount of sustained traffic you can transfer (note overhead from protocols will actually bring this down some).
Round Trip Time:
Not all paths through the internet are the same and not all protocols are the same. This is how long it take for you packet to get from one place on the internet to another and will very from carrier to carrier. If you don't have a choice of carrier, then there isn't much you can do about this. If you do, try and understand what their RTT for the two closest points to your offices that they will give you and do they provide any guarentees around maximum RTT. The lower the better, because many protocols were designed to work over high speed local area networks (LANs) and significantly degrade over wide area networks (WANs). Fileshares like NFS and CIFS are great examples of protocols that were never designed to run over a WAN and even with protocols designed for WANs advanced TCP and caching techniques can significantly improve performance of transfers. Because of these protocol issues, a bunch of companies have come out with products that optimize these protocols for travel over the WAN. Depending on exactly what your doing, you may get better performance by deploying one of these products instead of upgrading your circuit. Contact a local reseller and check out the Cisco Wide Area Application Services Software as an example http://www.cisco.com/en/US/products/ps6870/index.
Net-Net:
Depending on the protocol, look into using a product to optimize the transfers instead of just throwing a bigger circuit at it.
If you are sending bursts of traffic look for the higher port speed on a shared line.
If you are sending sustained traffic look for the higher guarenteed bandwidth.
Hope this helps.
Here's one firm you won't have to worry about stealing your jobs.
Well, let's give them a fighting chance at least. Ok, here's what I do to estimate bandwidth needs: Minimum bandwidth x number of users @ peak time. In your case- I would say you need a 1.5 MBit T-1 line or better. If you can't afford to do that in your current contract, then you need to renegotiate your contract for more money.
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
" we didn't ask for them, management imposed them on us for reasons we're still not quite aware of --"
Managment was looking at replacing you with them.
When training someone who will be taking your job, train them badly, foist all problems on them, communicate with them at a time when it seems reasonable, but in reality it isn't so they are always the delay.
You owe nothing to a company that will drop you like a hot potato, and you own nothing to whome ever they will replace you with.
Fight for your job, because the governmant and managment will not.
For tjose of you think this is a horrible attitude, rest assured people in other countries would do this to you.
The Kruger Dunning explains most post on
first, use the MS network tool fdisk.
Follow the procedure, you want to do a network latence repair know has 'partitioning'. It splits the bandwidth up among applications accross your hard drive. This will dramatically reduce bandwidth.
You versioning system is going to be a big drain, I suggest starting with that one.
The Kruger Dunning explains most post on
Warning: I'm a network designer working for a telco so I have clue!
For low latency and reliability, get a fractionnal T1 instead of DSL. Technically you can go from 64kbps up to 1.536mbps in increments of 64kbps. Interleaved DSL always has at least 10ms latency. A point-to-point T1 can be as low as 1 ms. Some provider can dual home you in their core. This way you don't loose your connection when they perform software/hardware upgrades on their switches.
DSL is cheaper but when it breaks, most telcos will serve you last. DSL breaks more often that T1 anyway. T1 is less susceptible to noise/interference than DSL and T1 are delivery on better quality copper loops than DSL. You can end up with good DSL in practice but you never know.
If possible get the service from who ever owns the last mile. If you take it from a reseller, the reseller has to contact the local telco for tests & repairs. I've seen cases where there are 3 intermediary companies.
In the UK uou can usually only get a fixed rate for a period of up to 5 years, after which you convert back to a variable rate. Of course, at that point you switch to another mortgage and/or renogotiate a new fixed rate (IIRC).
I am NaN
At the level of issue he's talking about, it might well be hiring a consultant for 2 hours. Heck, we'd do it for two hours, but a local consultant probably costs less for a day.
My other piece of advice is similar to what I've already seen: Ignore bandwidth, look for reliability. As recently as 2003 we had 4 users in the US on a single 56k line - and really, it was fine if you weren't trying to download big files. But it was a rock solid 56k that didn't lose any packets.
If you have actual DSL disconnections, you do not have a bandwidth problem... (Unless you're getting disconnected from the VPN only because something took too long)
If bandwidth was really expensive and you wanted to be really hardcore you could setup the Linux Advanced Routing stuff (or possibly just get a gaming router, depending on the model, but I haven't tried) and set it up to make sure that it responds to interactive connections rapidly at the expense of any other downloads. But from the sound of it you might need some help getting that setup.
Looking for freelance Actionscript (Flash/Flex) or ColdFusion work and/or freelance developers. Email me, put Slashdot
The government DID step in. And instead of implementing some sort of system that would tend to balance the standards of living, they have created INCENTIVES for offshoring.
a ks_for_.htmlR J8OVF&b=32409
http://techpolicy.typepad.com/tpp/2004/03/tax_bre
http://www.americanprogress.org/site/pp.asp?c=biJ
These were high up in Google, that's my only endorsement of these sources. It was a hot issue in 2004...
Looking for freelance Actionscript (Flash/Flex) or ColdFusion work and/or freelance developers. Email me, put Slashdot