Slashdot Mirror
Comcast Cheating On Bandwidth Testing?
dynamo52 writes "I'm a freelance network admin serving mainly small business clients. Over the last few months, I have noticed that any time I run any type of bandwidth testing for clients with Comcast accounts, the results have been amazingly fast — with some connections, Speakeasy will report up to 15 Mbps down and 4 Mbps up. Of course, clients get nowhere near this performance in everyday usage. (This can be quite annoying when trying to determine whether a client needs to switch over to a T1 or if their current ISP will suffice.) Upon further investigation, it appears that Comcast is delivering this bandwidth only for a few seconds after any new request and it is immediately throttled down. Doing a download and upload test using a significantly large file (100+ MB) yields results more in line with everyday usage experience, usually about 1.2 Mbps down and about 250 Kbps up (but it varies). Is there any valid reason why Comcast would front-load transfers in this way, or is it merely an effort to prevent end-users from being able to assess their bandwidth accurately? Does anybody know of other ISPs using similar practices?"
Doesn't Comcast advertise this "SpeedBoost" as a feature - the language in their ads is something like "get massive super speed for the first 10MB of a download, then it will revert to your provisioned line speed"... So, it actually *is* a good thing rather than something to pad bandwidth tests, and it does generally help your general user, right?
This is because of powerboost. As I understand it, powerboost makes the first 20MB download at a higher rate than your advertised bandwidth. Since bandwidth tests are done on such small files, you get a worthless result. The idea is that people who download lotsa of relatively small files get better performance, where as people downloading a lot of huge files like ISO images, full length movies, et cetera willg et initially good speed but after 20MB will feel like they are getting gipped.
Derek Greene
Comcast? Dishonest? Say it ain't so!
All kidding aside, this wouldn't surprise me too much. Comcast (and probably all other providers) are advertising this super-mega-intarweb speed as "up to x mbps." So, theoretically, as long as *one* site can provide data at that rate, their marketing garbage still stands. Even if 99.9% of the other websites top out at 4kbps, if Speakeasy's speed test says it can transfer a file at 15mpbs, technically Comcast is correct. They are giving you "up to 15mbps."
Proudly supporting the Libertarian Party.
Most internet browsing is with relatively small amounts of data, so wouldn't front-loading of this nature noticeably increase browsing performance? Since this kind of performance is noticed by the majority of users it would seem to be something that increases their perception of their connections' speed.
I'm not saying that Comcast might not be cheating on purpose for speed tests, I just think that there might be another reason behind it other than just to make their test scores artifically high.
Sounds like they have simply optimised their network to favour "bursty" usage, for example web browsing. This would seem a sensible thing for a consumer ISP to do.
Some consumers may not notice the speed increase when downloading smaller files, such as text-based e-mails and simple Web sites with few graphics. However, customers who frequently download large files, such as software, games, music, photos, and videos will now download at speeds that are faster than ever before. For example, PowerBoost significantly reduces the time it takes to download a one hour television program. Comcast subscribers at the 6 Mbps tier would reduce their wait time in half - from 4 minutes and 29 seconds to 2 minutes and 15 seconds. And MP3 fans will be able to download music files as fast as 2.2 seconds! See more here
Full Tilt
Um, this isn't a new concept, nor is it particularly sneaky:
http://en.wikipedia.org/wiki/Token_bucket
You just might want to check that their connections are properly tweaked as far as RWIN, MTU, etc go. 14/5 compared to 1.2/290 is a vast, vast difference that should never happen if they are paying for a certain tier of service, even if it is advertised as "up to" that higher rate. I'd also do a smoke ping and line quality tests, etc over at BroadbandReports, because there is something definitely not right with those connections if that is the average drop in performance. There may also be mis-configured firewalls, routers, mis-provisioned lines, water leaks, etc causing such issues.
My advertised and provisioned rate via Atlantic Broadband cable is 5/512. I am actually getting closer to 6 or 7 down and 468 up at all times due to some tweaking I did. Even the AtlanticBB tech seemed a bit shocked that I was getting more than 5 down, and said it was unusual, but they wouldn't re-provision the line or anything because of it. I count myself lucky, because Verizon's service here is absolute rubbish - $25.00/month for 1.5/768 DSL that, shall I say in the politest way possible, isn't actually working for more than two weeks per month because they are too cheap to replace lines that were put up in this town sometime in the 1950's at the latest (Not to mention they never actually bother to show up for scheduled appointments to rewire buildings that were constructed pre-1900, such as mine - big old Victorian type home turned into apartments).
Powerboost does mess with speed testing, however those "tests" are very rarely accurate anyhow, as I can rate higher on a test to Seattle or Los Angeles than I do to say Pittsburgh, Toronto or NYC, which are MUCH closer to where I live (by several thousand wire miles). It's more accurate to calculate your average rates by downloading/uploading large files from/to a university/public FTP or something, at least in my experience.
@Mindless Drivel: 100% of Twitter posts ever Tweeted.
I know that this is slashdot but I'll try to answer some of the OP's question anyway. Of course I won't do any original research myself, but rather rely on information from the previous posters or make things up as I go.
Q1. Is there any valid reason why Comcast would front-load transfers in this way?
Yes. Most requests from browsers are for short files. By upping the speed for short requests, pages will render faster. This is a plus for the user, as he spends less time idling. Long downloads on the other time are expected to take a while to complete; the user expects to be able to walk away from the computer for a while. Thus Comcast can argue that they have greatly enhanced the experience of the web browser by stealing a few cycles from the downloader. I would welcome such a plan as long as the ISO downloading speed is reasonable.
Q2. Is it merely an effort to prevent end-users from being able to assess their bandwidth accurately?
It would have that effect on a poorly designed bandwidth test. Bandwidth testers try to make the download size long enough to counteract tcp connection costs and to average over variations in download speed. Comcast has just given them another variable to take in to account. Interestingly, there are some test suites that are designed to detect what Comcast is doing and give them extra credit for it. They bill their tests as real world throughput tests. They want to indicate what the effective bandwidth is while browsing web pages that reference many images or javascript files.
Q3. Is Comcast cheating?
If Comcast is just doing this when accessing known test sites then they are cheating. If this is their policy for all connections then the worst that can be said is that they are optimizing their service to a particular class of users (surfers as opposed to downloaders). If you are in this category, then you should be happy.
PowerBoost only accelerates the connection if the average speed you've been getting over the past 30 seconds* is less than the speed you are rated at/paid for. So if you have a 6 Mbps connection, that's 768 KB/s max. PowerBoost will raise that to up to 2 MB/s for a little less than 15 seconds, making your average for the past 30 seconds equal to 768 KB/s. After that, no matter how many new connections you open, your connection stays at 768 KB/s. But if your connection gets interrupted/throttled for a few seconds, you may get another boost after it resumes, until you are back to 768 KB/s 30 second average again.
*it may be slightly more/less than a 30 second average. Boosts seem to last about 10-15 seconds, which would make sense with that number.
Iperf, or something like it is what you should be using for speed tests. Set up the daemon on a machine that you know you need to access and tell it to send a ton of data a few times. See what the results are. Those speed tests test how quickly you can communicate with some random server that you'll never need to send any presentations or video files to in day to day business.
I can assure that they do absolutely do this, and it is really annoying.
It's really bad on uploads -- I just ran a test and I got 300 KB/s for the first 5 megs, then it degrades 100 KB/second over the next few megs, so that by the time you have uploaded 14 megs you are getting close to 40 KB/S in upload speed, and the connection is so bad that the shared digital phone line does not have enough bandwidth to have a phone conversation. Stop the upload and start it up again, and you get 330 kb/second, with the same degradation curve.
For downloads they do the same thing, but not so severely -- I downloaded a 67 meg file and it ran at about 750 KB initially, but then dropped to around 350-400 KB/S (according to the FTP app) about halfway through.
So for anyone using the connection for smaller file sizes (like the speed tests) you seem to get "blazing" speeds -- I ran the test at a couple of the internet speed test sites and they both think that I have 12000-14000 kb/s download speed and 2700 kb/s upload speed.
So if I didn't have any other way to measure it, I would think that I was getting way more than I paid for, rather than something that in reality is very pitiful.
That can't be right. From your description, it sounds like a genuinely good and beneficial to the user idea. Where's the catch ?
Forget magic. Any technology distinguishable from divine power is insufficiently advanced.
If you want a commercial-grade link you expect to saturate, pay for it! Otherwise, you are stealing from other users and the ISP should throttle you to be fair to them.
In operating system theory, it is well known that a scheduling algorithm called "Shortest Job First" yields the least total waiting time. The SJF algorithm is usually implemented by giving a "new" job high priority, and then reducing the priority gradually as the job accumulates resource usage. The algorithm was developed in the 1960's to allow time-sharing operating systems to provide rapid keystroke response, while continuing to process large batch jobs in the background.
For communication systems, the same principle applies. The only difference is that the network is sharing a different resource (circuit bandwidth), instead of cpu time. The "new" connection gets high priority, and then that priority is reduced as the number of bytes/packets transferred over that connection increases. This allows rapid response for interactive applications, like browsing or editing, while also allowing the network to process large data transfers in the background. To apply it to datagram traffic, the switch just keeps a priority for each source/destination address-pair in cache, and any pair that is not in the cache is regarded as "new".
This has been pretty much standard practice in packet communication switching for a very long time. There is no surprise here, at least not to those of us who have not been doing communications network programming for a few decades.
http://www.dslreports.com/shownews/Slashdot-Keeps-Rediscovering-Comcast-Powerboost-91976
Yeah, I did some testing on my own a while back and my theory is that it's a token-bucket sort of thing, implemented in the modem. Whenever your aggregate bandwidth is less than X for more than a certain amount of time, it allocates a "token" and resets your cap to 2*rated. The longer your connection is non-busy, the more tokens you get, up to a certain point (when the bucket is full). Then, when you start moving some data, and you go over your rated limit (which, after all, is half of what the modem is giving you), it starts taking tokens out of the bucket. When the bucket is empty, it re-caps you at your rated speed, and no more boost until you start collecting tokens again, which means a period of inactivity.
;)
And yes, as the other commenter pointed out, this is actually an entirely sensible way to deal with "bursty" internet use and improve user experience without actually buying any more bandwidth. It would be really sweet if Comcast didn't do other stupid shit
This is not at all uncommon. Having worked as a network engineer for an ISP for 7 years just prior to my current job, I can tell you that this is common practice, especially if they are using any ATM. One of the reasons is that most session based transactions (web pages, email downloads, etc) are over in a few seconds. It actually provides better throughput and congestion control for the entire network to allow the initial transaction to burst at a higher speed, since a huge portion of those transactions are over very quickly.
:)
Think of the connection as a large pipe (your cable connection) with a small outflow valve (your modem), connected to a larger, higher pressure pipe (your ISP). Until your local pipe is full, you can put water into it as fast as you desire. But once it is full, the volume slows down because you can only put in as much as you are taking out (your cable modem connection/outflow valve). So what speakeasy and various other speed testing sites see is the effect of filling up your local pipe (your connection to your ISP).
What a large file download shows you is the actual throughput.
BTW, this is also a quick, very simplified explanation of bandwidth (how much data you can pack into the pipe) vs. throughput (how fast you can actually pull data through the pipe).
He could always try using OpenDNS as a test to see if his ISP's DNS service is the issue:
208.67.222.222
208.67.220.220
512 MB RAM, 20 GB disk, 200 GB transfer, five datacenters. $19.95/month.