Testing 3G Networks Across the US

PCWorld recently tested the 3G networks of AT&T, Verizon, and Sprint in 13 different cities across the US. They've now posted the results, which show that Sprint and Verizon are neck-and-neck for reliability, while AT&T has consistently higher upload rates. From the article: "Across more than 20 testing locations in each of the 13 cities we tested, Verizon had an average download speed of 951 kbps. Verizon demonstrated good reliability, too; the network was available at a reasonable and uninterrupted speed in 89.8 percent of our tests. Sprint's 3G network delivered a solid connection in 90.5 percent of our 13-city tests. Sprint's average download speed of 808 kbps across 13 cities wasn't flashy (at that speed, a 1MB file downloads in 10 seconds), but dependability is an important asset. The Sprint network performed especially well, both in speed and in reliability, in our test cities in the western part of the United States. The AT&T network's 13-city average download speed in our tests was 812 kbps. Its average upload speed was 660 kbps. Reliability was an issue in our experience of the AT&T system: Our testers were able to make a connection at a reasonable, uninterrupted speed in only 68 percent of their tests." What have you noticed about the various carriers in your city?

Comments

And then the commercials

[IndustrialComplex]

So based on the results of this test, I think we can expect commercials from all three carriers claiming that they are the fastest* 3g network around with the best reliability**.

Even as an owner of an iphone who knew what he was getting, Apple/ATT's commercials really get under my skin with their claims regarding the speed/capabilities of the phone. Of course, they get away with it with a 0.3 mS flash of text that informs us that the performance was artificially shortened. Comcast is another company whose commercials strike me as pure lies and misinformation based on a grain of truth***.

*For some definitions of fast
**The network will reliably not cause your phone to collapse into a singularity.
***Results from Brooklyn Bridge Sales LLC.

Re:And then the commercials

[MrCrassic]

When I had AT&T, reliability was a problem. First, the 3G network wasn't as mature as it is now, so upload/download speeds were only slightly greater than EDGE (usually 400 kbps or so). Second, and most concerning, was that I frequently had issues getting HSDPA service; I was often on the EDGE (2G) network, which was disappointing considering how much I would have been paying for it.

Needless to say, I only had AT&T for less than the 14-day trial period. I tested all of this on a Treo 750.

Re:And then the commercials

[Luscious868]

Comcast is another company whose commercials strike me as pure lies and misinformation based on a grain of truth

As a general rule of thumb never trust claims made in an advertisement. One of the great things about the Internet is that there are a plethora of sites out there that you can turn to get a better idea of how products and services really work. I rarely buy an expensive product or service these days without checking it out first.

Re:And then the commercials

[sycodon]

I have AT&T for my cell phone. Even though I live in a major urban area, it drops calls all the time. Their service and coverage sucks.

Re:And then the commercials

[girlintraining]

Comcast is another company whose commercials strike me as pure lies and misinformation based on a grain of truth

Well, the problem is they advertise "speed" and "availability", neither of which are really decent metrics. Speed is a crap metric because it's a scalar (math) quantity, and most often is measured by peak, rather than average, or worst-case. And "Availability" depends entirely on the service level agreement. If the power goes out, does that count toward availability? No, because "they" can't control it. Routine maintenance on their network? Nope. Lightning strike? Nah. So they can say almost whatever they want and get away with it because of some clever word-play. You'll be careful to note that in these endless commercials about high speed internet from any company they're careful to never put any number in it except the phone number to call. So it's not so much that they're lying -- it's really more that they're speaking sweet nothings, which is perfectly legal (and disingenuous).

Us geeks know that network performance isn't a scalar (math) quantity. Bigger numbers don't mean shit. It's the matrix of bandwidth (in bytes), latency (in milliseconds), packet loss (a percentage), all averaged over a long enough time-frame (hour, day, week, month, or billing cycle) to account for all systemic variables (bandwidth caps, network load averages, etc) is what matters.

I suppose you could derive from this information a weighted index, but it would still be largely useless to the average consumer. The problem is when you get down to brass tacks, different users have different needs. A heavy game player's internet needs will likely be low bandwidth, but low latency. A few milliseconds of extra time, or a few lost packets, will make that user's experience very poor. Someone who has an internet-TV has a large need for bandwidth, but latency is not an issue (even if the transfer is delayed by hours it might not matter). And then there's the little old grandma who doesn't do anything but check her e-mail and read CNN. If it wasn't for latency problems, she could be using a modem and never know the difference between either. Especially if she installed Vista -- god, network latency is nothing when it takes 8 seconds to render the downloaded page.

Comcast delivers an acceptable experience to a certain class of internet users and has crafted their service accordingly. The problem is that this service isn't tiered or can be adapted to serve several different markets. There is only one service, one market, and if you don't like it--you may not have any other options. Comcast is constrained by a need to maximize profitability, minimize costs, and is using an infrastructure which they are unwilling (or unable) to modify to deliver an acceptable experience to a larger user-base. There's no competition in most of its markets, and hence no reason to invest in doing so. The lack of competition ensures that Comcast's prices will continue to inflate while the number of customers who receive an acceptable experience will fall.

The bandwidth caps being imposed now are not the (direct) result of TV-over-internet competing with its internet offering, and instead the logical result of a lack of competition with its internet service. Any business in the same position and market(s) as Comcast would be doing the exact same thing, because Comcast doesn't exist to bring internet to the masses, so we can all celebrate the information age and live in peace, tranquility, and gigabytes of free porn. They exist to make money for their shareholders.

And the reason why service is shit in so many parts of this country isn't because of Evil BigCorp and their profiteering ways, but rather;

a) Infrastructure costs are a very high barrier to entry into the market. The United States is a big place with a low population density (taken as a whole) compared to other geographical regions like, say, Japan. The cost per customer is higher because there's a lot more wire and

Re:And then the commercials

[sycodon]

Ha!

The only thing you can do when in a two year contract. Bitch and moan.

BTW,
I can be in the middle of a call, stopped...not moving, 5 bars, great connection. Then nothing. Signal gone, can't connect again.

Seems like a brown out type of thing. I bet they don't have enough capacity.

Verizon wins

[Lulfas]

Looking at the charts, Verizon wins pretty handily unless you're in specific cities looking for a phone. They are the fastest downloader in 7 of the 13 spots, and most reliable in 7 of the 13 spots. Kind of funny that the home of the iPhone doesn't manage a decent reliability in any city besides Boston. Only thing they really lose on is upload speeds. Although, it is weird they didn't test in Los Angeles.

Re:Verizon wins

[Anonymous Coward]

I'm not sure any of them really "wins". They all show trouble maintaining even "one 9" in reliability - in fact only one of them made that - and just barely at that.

It doesn't surprise me in the least though, as I live in a city of 120,000 - in a housing development, not out in the boonies, and right up the street from my house is "no signal" even for voice calls. Coverage and reliability on these carriers in the US is always a problem and with scores between 68% and 90.5% all you can really infer from that is, "these folks really shouldn't even be selling this service until they have it working.". (Perhaps they should have it in beta still.)

Re:Verizon wins

[Dun Malg]

The problem with NYC skewing the averages is that on average most people in the US aren't in NYC.

It's not the cities, it's the spaces in between

[BadAnalogyGuy]

In a mobile culture like America's, we live a significant portion of our lives on the road. On holidays like today, we aren't, like 19th century Europeans, stuck in our hovels waiting for Ebenezer Scrooge to hand deliver a Christmas duck. Rather, we get out and drive, drive, drive all over this great, goddamned country.

So there's only so far 3G networks can take us if the coverage is only within city limits. When our cars are hooked up to cellular networks for data services, what good is it to have exceptional coverage in town when you're 100 miles from the next town? Empty spaces and big skies just prove how big this place really is, and it's all about living and moving and getting out there and getting to the next place that is what it's all about, man.

Get me some coverage in Yosemite. Death Valley. Appalachia. Crater Lake. Yellowstone. Shasta. Mt. McKinley. Grand Canyon. From Blaine, WA to Miami, FL. San Diego, CA to Eastport, ME. Cover it all and let us get on with really living in this great big country of ours.

Re:It's not the cities, it's the spaces in between

[satsuke]

While to believe the commercials from the larger players, there will never be absolutely seamless coverage across the nation because ..

1. There are places nobody lives (or it's economically unfeasible to cover)
2. Transmit powers are 1/12 of what they were in the analog era
3. They can't just throw a tower up anywhere

Back when analog bag phones were the norm, I never found anyplace without coverage .. why? Because on analog they had a nominal 3 watt transmit power on the phone, which let you have towers dozens of miles apart and still get a reliable signal. Today's mobiles operate at .25 of a watt or less, and since the 3G spec devices currently at or becoming the norm are based on CDMA technology (CDMA or WCDMA/HS?PA), the transmit power will only go down based on the load of the tower. (Under CDMA, the transmit power decreases when the load rises, lowering the noise floor and allowing more devices on the tower, with the net effect of creating islands of service if the network has hot spots and they don't plan accordingly).

As far as towers and stuff are concerned, I remember reading an article from upstate new york about a stretch of state highway that had pristine views, and a very high mortality rate in the winter because nobody had cell service up that way. The local government bodies sued and cajoled the cell carriers to build coverage, but wouldn't let them put the tall towers up to allow service in an economically feasible way. Net result, no coverage and more death, but the view was still great.

Re:It's not the cities, it's the spaces in between

[ChrisStrickler]

I was in Yosemite for the last week, and my blackberry had no connection (T-Mobile). My friends had the rest of the big carriers. AT&T (iPhone) had antenna coverage but dropped almost every call within the first minute after connection. Verizon (Razr2) would show no connectivity but we could still make calls out and have them stay connected for 10-15 minutes (we ended up calling our wives/families in shifts on the verizon phone). The Sprint (Samsung flip) phone had data and sms/mms but couldn't make a call - even though it was showing full connectivity.

Hardest thing for me is the payment rates between them. The Verizon phone with 900 minutes and no SMS/MMS was $115 a month. The Sprint phone was $79.99 a month, 400 minutes including an unlimited data plan, AT&T was $100+ and my phone (T-Mobile) is $80 a month for two phones with 700 minutes and unlimited data/mms/sms/email.

Anecdotal evidence, but from my brother at the Grand Canyon, on the surface, everyone had connection (same situation, bunch of carriers) except for the MetroPCS/Cricket users. In the Canyon AT&T showed full connectivity but would not connect, Verizon still connected as did Sprint and T-Mobile, but all 3 dropped consistently.

My biggest concern is El Paso. At any random moment I lose data connectivity as I am randomly connected to an antenna in Mexico that charges roaming data on my plan (I have my phone setup to disconnect data if it detects it will be roaming for data). Also, there are periods of no signal for all 4 carriers across parts of Texas from El Paso into San Antonio, but getting outside of Houston towards Louisiana and it is actually worse.

Re:It's not the cities, it's the spaces in between

[ScrewMaster]

The local government bodies sued and cajoled the cell carriers to build coverage, but wouldn't let them put the tall towers up to allow service in an economically feasible way. Net result, no coverage and more death, but the view was still great.

It's interesting to compare that attitude (on both the part of government and the providers) to how AT&T was originally created and managed by the Federal Government. Universal coverage was mandated, as were QOS standards, and the old AT&T put in service to all kinds of places that weren't remotely "economically feasible". That was the price of their monopoly, and by and large they lived up to that end of the bargain.

Of course, there were a lot of things our government handled more intelligently back then.

So crap speeds?

[Anonymous Coward]

So Australia actually has significantly faster 3G networks than America... Wow!

This information is useless.

[areusche]

This information is useless unless we as consumers can get to use our service to its fullest. I don't care if I have 5 solid meg wireless connection up and down if I fill up my monthly quota of 5gb data transfer. If the carriers were more transparent about the softcap I think everyone would appreciate it. Say something like "We have a 5gb limit on our service. This means that if you exceed 5gb of data consumption in a billing cycle your internet speed will be slowed down to 200kbps." To seriously believe this is still going on today confuses me to whits end.

My results on ATT

[Savior_on_a_Stick]

I use an HTC - ATT Tilt branded smartphone.
I'd like to point out that the testing methodology is not remotely suited to use in selecting a carrier.

Average is useless.

Verizon's has coverage that is far and away the fastest in areas not within major metropolitan areas, whereas ATT does not.

Sprint has traditionally been known as Highway Wireless, meaning that they tend to have excellent coverage along interstate highways, but when veering more than a mile or two from the highway in search of a late night fuel up, you'll lose signal much more frequently than with Verizon.

In the Portland Oregon metro area. Verizon does have the most granular coverage, and ATT has the fastest HSPDA speeds.
It should be noted that hspda speeds are significantly higher than vanilla 3g, and if speed is your primary criterion, 3g only phones are out of the running.

Granular coverage notwithstanding, ATT has the best voice and data coverage in my employer's physical locations in Portland.

However, my experiences do agree with the report with respect to ATT data dropouts.
The reason for the dropsouts seems to be prioritization of voice traffic over data at peak times.

ALL of the carriers have issues with capacity during peak times - like 5pm rush hour.
Because of the tight convergence of cell using driver along major arterials, and the towers that serve them, it's not unusual to drop a call when moving from cell to cell.
Data is no different in this regard, but added is the fact that consumers are more sensitive to inability to place a call than they are to data not flowing, hence the prioritization of voice.

On my commute route,

Combination of AT&T + NYC = screwed

[nigel999]

So the numbers verify: If you live in New York City and have AT&T, you have the worst 3G service of any carrier in that city.
Not only that, you have the worst service of any city that AT&T covers.
Not only that, you have the worst service that ANY carrier provides in any city.

Screwed.

Re:Combination of AT&T + NYC = screwed

[BadAnalogyGuy]

Not only that, you also have to live in New York City.

Re:Combination of AT&T + NYC = screwed

[Anonymous Coward]

And the light at the end of the tunnel is new jersey.

"it's the spaces in between"

[zogger]

That's exactly what this new network is for, where you have good normal cell or wifi coverage, it will use that, outside that coverage, it will be using the satellite.

I live in NYC

[hamburgler007]

And I wish I got half the speed on my iphone they claim for AT&T. Reliability is a big issue too.

The test seems to be bogus

[rwwyatt]

Disclaimer: I work for a Data Card Manufacturer

Without the actual procedure for the tests, it is difficult to say if PCWorld'ss are any good. I am not familiar with the software used as no major industry provider uses it. The standard tool in the industry is Windcatcher

It really depends on the way the test is run. The problem actually relates to the TcpWindowSize as it should be increased to at least 128Kb for HSPA based networks and for CDMA as well.

Another major issue is that Data Cards don't inherently support streaming. Streaming is often used as a secondary PDP context and this will have a major negative effect. Were they in a handover region or not? On HSPA, every other cell is an interferer so throughput should be measured with a Single Carrier in the active set. It is still possible to be in a handover zone while in a parked car.

Did they use the carrier supplied good coverage locations? Randomly may not cross the panaroma of RSSI.

As well, the latest modem from Novatel Wireless is the USB760. I also believe the latest Sierra Wireless card is the compass something or other. Did they use a Y Cable? Did they use an external antenna? What model of PC did they use as TRP/TIS makes a huge difference in low coverage areas?

Without more data, I would still say there isn't sufficient evidence to form any conclusions from their article.

Road warrior perspective

[cybrchld]

I'm constantly on the road traveling and have had all three services, i can tell you Verizon has had the best coverage around the county, specially rural areas, whats the point of having the fastest network if when you need it you cant connect. this is where ATT and sprint fall short on.

AT&T

[paimin]

Well, I'm posting this from an AT&T 3G connection, and I can say it's absolutely relia[[&2$188:..NO CARRIER

AT&T vs Sprint or Verizon

[Dun Malg]

I've noticed other important differences between AT&T and the others: when I go outside the US, my phone isn't a fucking useless brick. I'm also not stuck driving to a tech support office if my phone craps out, I can just put the SIM card in a different phone. I can also order phones with interesting features from foreign countries and they work.

I wonder why they left T-Mobile out. I'm with AT&T currently would love to see where the other major GSM carrier stands.

It's a good thing they didn't test T-Mobile's 3G

[DragonTHC]

It would have been a laughingstock.

T-Mobile's 3G is like Swiss cheese here in Miami.

And when it does work, it's usually less than a megabit.

Bad SIM can cause dropped calls

[DJRumpy]

Not sure if this is your issue, but I had the same problem and I live in the Dallas / Fort Worth area. They tend to have excellent coverage. The problem was with my sim card. The contacts erode or become corroded or the card itself becomes warped. Replacing it resolved my drop calls issue. The AT&T folks may squawk but you can usually get a replacement for free if you suspect it's faulty.

Re:It's a good thing they didn't test T-Mobile's 3

[WiiVault]

Thats because while tmobile has almost zero 3g coverage nationally, they also have few users with 3G phones.

Reliability more important than speed

[PhrackCreak]

I have AT&T. I live in San Francisco. AT&T regularly drops calls. I cannot make calls from home without dropping them a minute or two into the conversation. I could not make calls from work until they installed an expensive repeater. Notice that AT&T lost EVERY SINGLE reliability comparison.

For my needs, that makes them the worst provider.

90%

[bugi]

Since when is 90% reliability even remotely acceptable?