Testing 3G Networks Across the US

PCWorld recently tested the 3G networks of AT&T, Verizon, and Sprint in 13 different cities across the US. They've now posted the results, which show that Sprint and Verizon are neck-and-neck for reliability, while AT&T has consistently higher upload rates. From the article: "Across more than 20 testing locations in each of the 13 cities we tested, Verizon had an average download speed of 951 kbps. Verizon demonstrated good reliability, too; the network was available at a reasonable and uninterrupted speed in 89.8 percent of our tests. Sprint's 3G network delivered a solid connection in 90.5 percent of our 13-city tests. Sprint's average download speed of 808 kbps across 13 cities wasn't flashy (at that speed, a 1MB file downloads in 10 seconds), but dependability is an important asset. The Sprint network performed especially well, both in speed and in reliability, in our test cities in the western part of the United States. The AT&T network's 13-city average download speed in our tests was 812 kbps. Its average upload speed was 660 kbps. Reliability was an issue in our experience of the AT&T system: Our testers were able to make a connection at a reasonable, uninterrupted speed in only 68 percent of their tests." What have you noticed about the various carriers in your city?

Comments

And then the commercials

[IndustrialComplex]

So based on the results of this test, I think we can expect commercials from all three carriers claiming that they are the fastest* 3g network around with the best reliability**.

Even as an owner of an iphone who knew what he was getting, Apple/ATT's commercials really get under my skin with their claims regarding the speed/capabilities of the phone. Of course, they get away with it with a 0.3 mS flash of text that informs us that the performance was artificially shortened. Comcast is another company whose commercials strike me as pure lies and misinformation based on a grain of truth***.

*For some definitions of fast
**The network will reliably not cause your phone to collapse into a singularity.
***Results from Brooklyn Bridge Sales LLC.

Re:And then the commercials

[MrCrassic]

When I had AT&T, reliability was a problem. First, the 3G network wasn't as mature as it is now, so upload/download speeds were only slightly greater than EDGE (usually 400 kbps or so). Second, and most concerning, was that I frequently had issues getting HSDPA service; I was often on the EDGE (2G) network, which was disappointing considering how much I would have been paying for it.

Needless to say, I only had AT&T for less than the 14-day trial period. I tested all of this on a Treo 750.

Re:And then the commercials

[Luscious868]

Comcast is another company whose commercials strike me as pure lies and misinformation based on a grain of truth

As a general rule of thumb never trust claims made in an advertisement. One of the great things about the Internet is that there are a plethora of sites out there that you can turn to get a better idea of how products and services really work. I rarely buy an expensive product or service these days without checking it out first.

Re:And then the commercials

[girlintraining]

Comcast is another company whose commercials strike me as pure lies and misinformation based on a grain of truth

Well, the problem is they advertise "speed" and "availability", neither of which are really decent metrics. Speed is a crap metric because it's a scalar (math) quantity, and most often is measured by peak, rather than average, or worst-case. And "Availability" depends entirely on the service level agreement. If the power goes out, does that count toward availability? No, because "they" can't control it. Routine maintenance on their network? Nope. Lightning strike? Nah. So they can say almost whatever they want and get away with it because of some clever word-play. You'll be careful to note that in these endless commercials about high speed internet from any company they're careful to never put any number in it except the phone number to call. So it's not so much that they're lying -- it's really more that they're speaking sweet nothings, which is perfectly legal (and disingenuous).

Us geeks know that network performance isn't a scalar (math) quantity. Bigger numbers don't mean shit. It's the matrix of bandwidth (in bytes), latency (in milliseconds), packet loss (a percentage), all averaged over a long enough time-frame (hour, day, week, month, or billing cycle) to account for all systemic variables (bandwidth caps, network load averages, etc) is what matters.

I suppose you could derive from this information a weighted index, but it would still be largely useless to the average consumer. The problem is when you get down to brass tacks, different users have different needs. A heavy game player's internet needs will likely be low bandwidth, but low latency. A few milliseconds of extra time, or a few lost packets, will make that user's experience very poor. Someone who has an internet-TV has a large need for bandwidth, but latency is not an issue (even if the transfer is delayed by hours it might not matter). And then there's the little old grandma who doesn't do anything but check her e-mail and read CNN. If it wasn't for latency problems, she could be using a modem and never know the difference between either. Especially if she installed Vista -- god, network latency is nothing when it takes 8 seconds to render the downloaded page.

Comcast delivers an acceptable experience to a certain class of internet users and has crafted their service accordingly. The problem is that this service isn't tiered or can be adapted to serve several different markets. There is only one service, one market, and if you don't like it--you may not have any other options. Comcast is constrained by a need to maximize profitability, minimize costs, and is using an infrastructure which they are unwilling (or unable) to modify to deliver an acceptable experience to a larger user-base. There's no competition in most of its markets, and hence no reason to invest in doing so. The lack of competition ensures that Comcast's prices will continue to inflate while the number of customers who receive an acceptable experience will fall.

The bandwidth caps being imposed now are not the (direct) result of TV-over-internet competing with its internet offering, and instead the logical result of a lack of competition with its internet service. Any business in the same position and market(s) as Comcast would be doing the exact same thing, because Comcast doesn't exist to bring internet to the masses, so we can all celebrate the information age and live in peace, tranquility, and gigabytes of free porn. They exist to make money for their shareholders.

And the reason why service is shit in so many parts of this country isn't because of Evil BigCorp and their profiteering ways, but rather;

a) Infrastructure costs are a very high barrier to entry into the market. The United States is a big place with a low population density (taken as a whole) compared to other geographical regions like, say, Japan. The cost per customer is higher because there's a lot more wire and

Re:And then the commercials

[sycodon]

Ha!

The only thing you can do when in a two year contract. Bitch and moan.

BTW,
I can be in the middle of a call, stopped...not moving, 5 bars, great connection. Then nothing. Signal gone, can't connect again.

Seems like a brown out type of thing. I bet they don't have enough capacity.

Verizon wins

[Lulfas]

Looking at the charts, Verizon wins pretty handily unless you're in specific cities looking for a phone. They are the fastest downloader in 7 of the 13 spots, and most reliable in 7 of the 13 spots. Kind of funny that the home of the iPhone doesn't manage a decent reliability in any city besides Boston. Only thing they really lose on is upload speeds. Although, it is weird they didn't test in Los Angeles.

It's not the cities, it's the spaces in between

[BadAnalogyGuy]

In a mobile culture like America's, we live a significant portion of our lives on the road. On holidays like today, we aren't, like 19th century Europeans, stuck in our hovels waiting for Ebenezer Scrooge to hand deliver a Christmas duck. Rather, we get out and drive, drive, drive all over this great, goddamned country.

So there's only so far 3G networks can take us if the coverage is only within city limits. When our cars are hooked up to cellular networks for data services, what good is it to have exceptional coverage in town when you're 100 miles from the next town? Empty spaces and big skies just prove how big this place really is, and it's all about living and moving and getting out there and getting to the next place that is what it's all about, man.

Get me some coverage in Yosemite. Death Valley. Appalachia. Crater Lake. Yellowstone. Shasta. Mt. McKinley. Grand Canyon. From Blaine, WA to Miami, FL. San Diego, CA to Eastport, ME. Cover it all and let us get on with really living in this great big country of ours.

Re:It's not the cities, it's the spaces in between

[satsuke]

While to believe the commercials from the larger players, there will never be absolutely seamless coverage across the nation because ..

1. There are places nobody lives (or it's economically unfeasible to cover)
2. Transmit powers are 1/12 of what they were in the analog era
3. They can't just throw a tower up anywhere

Back when analog bag phones were the norm, I never found anyplace without coverage .. why? Because on analog they had a nominal 3 watt transmit power on the phone, which let you have towers dozens of miles apart and still get a reliable signal. Today's mobiles operate at .25 of a watt or less, and since the 3G spec devices currently at or becoming the norm are based on CDMA technology (CDMA or WCDMA/HS?PA), the transmit power will only go down based on the load of the tower. (Under CDMA, the transmit power decreases when the load rises, lowering the noise floor and allowing more devices on the tower, with the net effect of creating islands of service if the network has hot spots and they don't plan accordingly).

As far as towers and stuff are concerned, I remember reading an article from upstate new york about a stretch of state highway that had pristine views, and a very high mortality rate in the winter because nobody had cell service up that way. The local government bodies sued and cajoled the cell carriers to build coverage, but wouldn't let them put the tall towers up to allow service in an economically feasible way. Net result, no coverage and more death, but the view was still great.

This information is useless.

[areusche]

This information is useless unless we as consumers can get to use our service to its fullest. I don't care if I have 5 solid meg wireless connection up and down if I fill up my monthly quota of 5gb data transfer. If the carriers were more transparent about the softcap I think everyone would appreciate it. Say something like "We have a 5gb limit on our service. This means that if you exceed 5gb of data consumption in a billing cycle your internet speed will be slowed down to 200kbps." To seriously believe this is still going on today confuses me to whits end.

My results on ATT

[Savior_on_a_Stick]

I use an HTC - ATT Tilt branded smartphone.
I'd like to point out that the testing methodology is not remotely suited to use in selecting a carrier.

Average is useless.

Verizon's has coverage that is far and away the fastest in areas not within major metropolitan areas, whereas ATT does not.

Sprint has traditionally been known as Highway Wireless, meaning that they tend to have excellent coverage along interstate highways, but when veering more than a mile or two from the highway in search of a late night fuel up, you'll lose signal much more frequently than with Verizon.

In the Portland Oregon metro area. Verizon does have the most granular coverage, and ATT has the fastest HSPDA speeds.
It should be noted that hspda speeds are significantly higher than vanilla 3g, and if speed is your primary criterion, 3g only phones are out of the running.

Granular coverage notwithstanding, ATT has the best voice and data coverage in my employer's physical locations in Portland.

However, my experiences do agree with the report with respect to ATT data dropouts.
The reason for the dropsouts seems to be prioritization of voice traffic over data at peak times.

ALL of the carriers have issues with capacity during peak times - like 5pm rush hour.
Because of the tight convergence of cell using driver along major arterials, and the towers that serve them, it's not unusual to drop a call when moving from cell to cell.
Data is no different in this regard, but added is the fact that consumers are more sensitive to inability to place a call than they are to data not flowing, hence the prioritization of voice.

On my commute route,

Re:Combination of AT&T + NYC = screwed

[BadAnalogyGuy]

Not only that, you also have to live in New York City.

The test seems to be bogus

[rwwyatt]

Disclaimer: I work for a Data Card Manufacturer

Without the actual procedure for the tests, it is difficult to say if PCWorld'ss are any good. I am not familiar with the software used as no major industry provider uses it. The standard tool in the industry is Windcatcher

It really depends on the way the test is run. The problem actually relates to the TcpWindowSize as it should be increased to at least 128Kb for HSPA based networks and for CDMA as well.

Another major issue is that Data Cards don't inherently support streaming. Streaming is often used as a secondary PDP context and this will have a major negative effect. Were they in a handover region or not? On HSPA, every other cell is an interferer so throughput should be measured with a Single Carrier in the active set. It is still possible to be in a handover zone while in a parked car.

Did they use the carrier supplied good coverage locations? Randomly may not cross the panaroma of RSSI.

As well, the latest modem from Novatel Wireless is the USB760. I also believe the latest Sierra Wireless card is the compass something or other. Did they use a Y Cable? Did they use an external antenna? What model of PC did they use as TRP/TIS makes a huge difference in low coverage areas?

Without more data, I would still say there isn't sufficient evidence to form any conclusions from their article.

AT&T

[paimin]

Well, I'm posting this from an AT&T 3G connection, and I can say it's absolutely relia[[&2$188:..NO CARRIER

Reliability more important than speed

[PhrackCreak]

I have AT&T. I live in San Francisco. AT&T regularly drops calls. I cannot make calls from home without dropping them a minute or two into the conversation. I could not make calls from work until they installed an expensive repeater. Notice that AT&T lost EVERY SINGLE reliability comparison.

For my needs, that makes them the worst provider.

90%

[bugi]

Since when is 90% reliability even remotely acceptable?