Bad Connections Dog Google's Mountain View Wi-Fi Network

itwbennett writes "Google launched its Mountain View, CA public public Wi-Fi network in August 2006. It was one of the first public wireless Internet services in the U.S. and was intended to provide free service across the city. But in 2012, one year after Google signed a 5-year agreement to continue the service, it started a slow decline to the point of being unusable. 'We started noticing it in very large files, things like operating system updates, but now it's on files as small as 500 kilobytes,' said Rajiv Bhushan, chief scientist of pharmaceutical startup Livionex and a long-time user of the network. A recent test by IDG News Service resulted in a total failure to get a working Internet connection at a dozen sites around Mountain View, including in the city's main downtown area and directly in front of Google's headquarters." I've had disappointing results trying to connect to several other public wireless nets around the U.S., both privately sponsored and municipal. Do you know of any that work especially well?

Comments

The solution

[TechyImmigrant]

Just unplug it and plug it back in again.

Re:The solution

[Richard_at_work]

No no no, they are holding it wrong!

Re:The solution

[slashmydots]

You forgot, they also have to strap it to a blimp.

Re:The solution

[Man On Pink Corner]

"In other news, something made by Google turns out to be a half-assed implementation of a good idea, unfavored by management and consequently determined to be a career-limiting move for Googlers unfortunate enough to be assigned to it. Consequently it is allowed to fall into disrepair, and will be scheduled for decommissioning at a time carefully calculated to maximize user inconvenience. Ric Romero has film at 11, so stay tuned for that."

Re:The solution

[AmiMoJo]

That's not actually a completely daft suggestion. A lot of wifi gear tries to automatically pick the clearest channel, but usually sucks at doing it while in operation. Rebooting disconnects all clients and gives it a change to do a full scan of all channels before selecting one, possibly switching to a less congested frequency.

Re:The solution

[AlphaWolf_HK]

Or perhaps it's just oversaturated. Wifi doesn't have unlimited bandwidth you know. After enough people find out that they can stop paying for their regular ISP and just hop on a free wifi you'll start to run into problems.

Re:The solution

[Que_Ball]

Would it shock anyone to know I actually did this reboot to a malfunctioning public Wifi base station recently and it worked?

I had a client moving into a new commercial location where the local cable company (Shaw) has one of their public Wi-Fi terminals installed.

They did not have their own network connection yet (booked for a few days later) so we just joined their computer to the public network but it was horrible. The connection showed moderate to high packet loss which was strange because the base station was in the roof a few feet away. Even doing a ping test to first hop (the base station) was showing the packet loss problem. Increasing the packet size on the ping tests showed the problem was got worse as you increased the packets so anything that wanted a sustained download and not small little transactions was suffering worse effects.

So I went into the back, found the power injector for the base station and cut the power. Plugged it back in, and after the reboot it was working well. No more packet loss, and a usable connection.

Maybe Shaw needs to update the firmware on these Cisco base stations they are using.

Re:The solution

[lsatenstein]

Or perhaps it's just oversaturated. Wifi doesn't have unlimited bandwidth you know. After enough people find out that they can stop paying for their regular ISP and just hop on a free wifi you'll start to run into problems.

===
This network problem is one which I call "teething". At low internet speeds (700-1.2kbits) everything is tuned for arrivals of packages and the queuing of packages for forwarding.

At higher speeds, the buffering of packets must be much much larger, as the number of queued packets can vary from instant to instant. Ergo, forwarding and resending of packets takes more cushioning. Peeks and valleys in buffering need to be placed at every interchange point. Timeouts and the like are also important.

In effect, to go up in speed from dsl/cable to fibre, takes some serious tuning and learning a new system management paradigm.
     

Re:The solution

[zizzo]

I can tell you as a bonafide resident of Mountain View the net work is not over saturated. It is simply unusable. There was a brief time where the secure variant worked passably well but that doesn't even work now. I honestly suspect the problem is just the access points are not receiving any physical maintenance and are falling into disrepair. There's enough alive to maintain the visible SSID but that's about it.

isn't wifi like the old layer 1 hubs?

[alen]

you transmit into the air and everyone receives the signal and the receiver has to filter out any traffic that is not meant for itself

too much data being transmitted by people in the area and the connection is useless. even my home wifi is almost useless during peak times at night since i have two dozen or so other people with wifi around me

Re:isn't wifi like the old layer 1 hubs?

[deanklear]

(These are general ideas and may not be technically accurate... feel free to correct me)

There are several problems with WiFi technology itself. First, there is no contention management for wireless. When you're wired in, collisions are detected quickly, so you can saturate the connection near its theoretical limits without too many errors. (There's a promotional video about this from Meru Networks, but it is fairly educational.) By contrast, WiFi will roll through a larger bit of data and then ask for confirmation of receipt, which can lead to a lot of problems as radios talk all over each other. This is not a problem in regular office environments, where walls, floors, and furniture can provide separation so the radios can "hear" things that are closer. However, get into an open air environment and add a bunch of devices at once, and everything flatlines as the access points attempt to orchestrate several hundred devices in range, including interference from other radios within "hearing" distance on the same channel.

The second issue is one of limited channels. Originally WiFi was designed to move a tiny amount, and I think you could actually split off 802.11b into 11 discrete channels. As data needs grew, they consolidated 11 channels into 3 discrete channels for 802.11g (4 in the EU, I believe) and that's where it stands: a 3 lane road for 2.4GHz. 5GHz has more channels, depending on where you are in the world, but right now they are unreliable as the requirement for many of them is to be compatible with DFS, which means that if there is a certain signal being broadcast, your access points are expected to abandon that channel immediately. I think there are changes in the works from the FCC and although it only introduces 30% or so of new spectrum, it happens to cross multiple channels, so it may be like going from 9-12 channels to 20 or so. Combined with the more limited range of the higher frequency, having 20 discrete channels opens up a lot of options for basic broadband in public spaces. (Well, it did until the new ac standard came out, and I haven't even bothered to read it because these massive spectrum widths are going to be a nightmare, and I'm in a different line of work these days.)

However, none of this solves the "microcell" design of WiFi, where the client makes the decision on what radio to connect to instead of the access point. Your cell service, for instance, works well because the tower instructs the client so it can perform handoffs, reduce the data rates, and make other adjustments to keep things from choking up. I have sat and watched an iPhone cross over multiple access points and hundreds of feet to connect across a stadium for no explicable reason. (That's true for every wireless device, but I'm picking on iOS because they are notoriously noisy, always flooding the air with useless beacons, trying desperately to connect to stored wireless networks even when they aren't around.)

I have deployed Xirrus, Aruba, Extricom, Unifi, and some other products in dense situations, but as far as I know, the only pseudo non-microcell options available are from Extricom and Meru. Although I haven't used Meru, I can say that Extricom has been the most reliable in very dense environments, since they use some tricks to keep the air quiet, and they do not introduce beacon traffic with the addition of more radios. (Disclaimer: I have worked with the guys from Extricom quite a few times, and I think they are very capable, so take that opinion with a grain of salt.) Xirrus works pretty well in corporate environments, and their reporting interfaces are great, but I was disappointed that their sales staff continued to deny problems in 2.4GHz long after it was obvious that they didn't have a workable solution for super dense deployments. But maybe they just didn't know.

Anyway, ignoring all of that technical garbage, the

Re:isn't wifi like the old layer 1 hubs?

[egamma]

Very informative, nice post. A couple of comments.

First, there are three nonoverlapping channels for 802.11b/g. So there's 11 channels, but transmitting on channel 1 means that you are putting noise on channels 1-5; transmitting on channel 6 means that you are putting noise on 1-10; and transmitting on channel 11 puts noise on 7-11. If you transmit on channel 3, that means you are disrupting 1-8, so it's best to simply use 1, 6, and 11.

5Ghz doesn't have as much of a saturation problem because of the smaller distance that a usable signal travels. Sure, 24Ghz would be even better, but then I wouldn't be able to use the signal on my driveway or probably even my bedroom. I can see 24Ghz being good for apartment use, or college dorms, although dorms should have their own wifi.

Re:isn't wifi like the old layer 1 hubs?

[Immerman]

Each channel is a slightly different frequency - and somewhat like an FM radio a decent wireless router/card will "tune in" on the frequency it's connecting on,and tune out the others, preventing neighboring signals from "talking over each other". IIRC it's recommended to leave one or two "dead" channels between overlapping hotspots to minimize crosstalk, but even adjoining channels have much better signal rejection that trying to establish two independent datalinks over the same frequency, which is what happens if your neighbors router is operating at the same frequency as yours. The situation is actually even worse than a hub - imagine you have multiple *completely unrelated* networks all trying to simultaneously talk over the same wire.

Of course with a few dozen overlapping hotspots you're going to have some unavoidable crosstalk, there's less than a dozen different wi-fi channels. However most hotspots are factory configured to run on the same channel, so moving a few channels away from the congestion can allow for a much better connection, especially if you're at a decent range from anyone sharing the same channel. There's even free software available (I forget the name) that will show you the number and strength of the signals on the various channels so that you can make an informed decision about which channel will likely have the least interference.

Re:isn't wifi like the old layer 1 hubs?

[jandrese]

One thing you didn't touch on: A lot of Wifi chips are really really bad. Like they'll crash randomly and repeatedly when connected to certain kinds of access points. Sometimes it is the access point that crashes. For the most part the chips reset themselves and continue on, so it's just a momentary interruption, but when it happens over and over you'll really start to notice.

Re:isn't wifi like the old layer 1 hubs?

[bruce_the_loon]

No, once you change the channel, you change the RF and IF tuning so the radio's analog signal stages effectively ignore the other frequencies. Only the demodulated data wave from the frequency on that channel gets through to the digital processing part of the WiFi device.

To be physics correct before some expert goes postal, the antenna does receive the other frequencies, but the RF stage (2.4Ghz level) eliminates them by speed-of-light physics, not by digital processing of anything.

Re:isn't wifi like the old layer 1 hubs?

[AmiMoJo]

One major problem I see is that many APs default to channel 11, and their auto-channel-selection code only seems to pick channels 1, 6 or 11. Those kinda made sense for 802.11b, but now they just waste available spectrum.

802.11g is 20MHz wide, so should use channels 1, 5, 9 and 13. 802.11n can go up to 40MHz, in which case we pretty much just have channels 3 and 11 left...

There is also channel 14. It is only supposed to be used in Japan and only for 802.11b (10MHz channel width), but I find you can usually activate it for 802.11g in any part of the world with a few simple software tweaks. Probably a bad idea but because channel 14 is spaced 12MHz above channel 13 instead of the usual 5MHz it is usually uncongested, even if channel 11 is flooded.

Re:isn't wifi like the old layer 1 hubs?

[AmiMoJo]

I wish there was better software available. The most often recommended one is InSSIDer, and here it recommends channel 1 has having only one other distant AP. If I use channel 1 I get constant drop-outs. Channel 4 is fine for me, although my neighbour's wireless subwoofer used to drop out when I was using it. He stopped using it a while back, so it's a double win for me.

WiFi with anal probe

[Animats]

The vast majority of attempts didn't even get as far as the log-in screen, which requires signing into a Google account to connect.

That's Google. "Public" WiFi with data mining.

Re:WiFi with anal probe

[Anonymous Coward]

Maybe we need a "free beer" type quip to help people distinguish public-access and public-ownership. and probably to understand the difference between them,

Re:WiFi with anal probe

[tibit]

Everybody who wants a good service may^W can use a satellite dish looking directly to AP in order to overpower them all.

That's actually very bad since WiFi by default suffers from hidden node problems. It'll break down the performance for everyone. In WiFi, the collision detection only works when nodes can all hear each other. Someone who knows better, feel free to correct me.

Seems to be a systemic problem.

[doubledown00]

This is just one user's opinion, but slow gradual declines seem to be the hallmark of Google projects. They work well when they're shiny and new, but over time the projects are neglected and deteriorate. Similar things have happened with Google Voice and Google Docs.

Re:Seems to be a systemic problem.

[Anonymous Coward]

As both a Docs and Voice user, I can't related to your comment of deterioration. However, they don't get much in the way of new development, which I suppose could be considered neglect.

Though compared to Maps where there is significantly development and "improvement," they also don't makes sure a new version has the same feature set as the old. Navigation in particular has really taken a nosedive on Android.

Re:Seems to be a systemic problem.

[eclectro]

I wouldn't necessarily agree with you, look at google maps which have improved in quality, But this happened through a dedicated commitment to the project.

But in the other hand I have noticed that there are google "quality control" issues which I have experienced. A few of the patents in Google Patents have bad and unreadable pages (especially with the drawings) or are scanned crooked as an example.

I suspect that the wireless network is oversubscribed where you have people jamming each other (and the router) to get into a local router. This can be a problem with any radio technology when you have too many users on a channel. And google probably does not make money on this project and hence allocate resources for other things.

But this is a larger issue with WiFi networks in general. People think it's an infinite resource and can replace traditional wires everywhere. I assure you, it is not and can not.

Re:Seems to be a systemic problem.

[pspahn]

Though compared to Maps where there is significantly development and "improvement,"...

Funny thing is, I still use Voice every single day, but the only times I use Maps is when it runs as the map platform for some other service. If I want to look something up on a map, I use Bing Maps now. Google maps have become too slow and clunky in the last year. Tiles simply do not update fast enough.

If that's what you call "development and improvement", I'll stick with "neglected" just about every single time.

Abuse of tool?

[Tablizer]

"I cannot download a new OS or Gone with the Wind in HD on free wi-fi."

Shocking!

Re:Abuse of tool?

[spire3661]

Did you know that HP printer drivers are now up to 160 MB? its not just movies that are large files anymore.

consumer products

[fermion]

Pretty much Google's consumer products seem never to be improved or maintained. At first the consumer products were critical because this is why people like me allowed Google to set cookies, while 2o7 and the like were blocked. Of course we are now in world where tracking on the web goes beyond cookies, so maybe Google does not think it needs to provide a service beyond search to entice users.

In any case the decline of the WiFI is not surprising. It is like Google docs, now Drive, that started off as a really competitive product, but the office applications has never been updated so the features continue to lag. OpenOffice makes it look like vintage 1990.

Seriously. If MS were competent they could destroy Google with Bing and MS Windows Phone. But that is how the game works. Google does not have outrun the bear, it only has to outrun MS, which isn't that hard.

It's free. C'mon.

[skidisk]

I live and work in Mountain View (not for Google). Look, the thing is free. What do you expect? I can log in and use it reasonably well. I certainly wouldn't depend on it for my only connectivity, but it works well enough when I need a quick piece of data or need to send something and don't have cell service or am using a wifi device. Just chill.

What I have seen

[Groo Wanderer]

While I haven't used the Google service yet, I see similar problems in a lot of public areas like airports where I happen to find myself a lot. It seems to be more of an issue with the non-direct data traffic like the auth services, ads/gateway tasks, and DNS. More often than not it is one of these 'services' that are unrelated to the traffic that are acting up.

One example is the wi-fi networks in the Minneapolis or San Fran airports. You can log on, and then getting an IP, getting on the "I agree" screens, the videos you have to watch etc etc are all dog slow to one degree or other. The Delta lounge in the Minneapolis and San Fran airports are very extreme examples of this problem especially when they were T-Mobile (damn their black souls). You would 'get on' and then nothing or something trivial really slow.

Once on you would have decent ping times and some speed tests would be OK but anything that needed 'extra services' was pain. Changing your DNS to something you have or a know fast provider helped a lot which tells me the NAS/Radius/whatever server they use was overwhelmed. Now that I am thinking about it I should do a traceroute next time I am on to see what is happening in more detail, I am curious.

My first bet is that the majority of these services go through a single auth/security box that is under-CPUd and forces everything out a single overloaded link. If anyone has the time. I also wouldn't be surprised if DPI had a hand in it too, especially from Google.

41 kbps download, 2 kbps upload

[theodp]

Google Groups: Complaint to the City of Mountain View about Google WiFi Service. And, from January: Amid complaints, Google promises WiFi upgrades.

"Bad Connections Dog"

[terminalhype]

What is a "Bad Connections Dog" and why is it Googling Mountain View's wi-fi Network? Possibly it is looking for a Good Connections Cat?

Re: "Bad Connections Dog"

[TheGratefulNet]

not to worry, its bark is worse than its byte.

Re: "Bad Connections Dog"

[slashmydots]

Yo dawg. I heard you like bad connections so we found you a dog that has a bad connection, dawg.

Project Loon promises 200+ kbps download

[theodp]

So, would Mountain View be better off with the balloon-powered Internet of Project Loon, which offers 3G speed or better?

Not Surprising

[sigipickl]

There are dozens of reasons why Wi-Fi doesn't scale to the masses. Especially outdoors or in large spaces. Here are a few:
- Wi-Fi is half-duplex. Only one transmitter can broadcast on a channel at any given time. If the transmitting radio is slow (weak connection, older technology, bad-driver, etc...), then all other devices must wait for the transmission to end before they can get their airtime to transmit.
- A Wi-Fi radio that conforms to the Wi-Fi spec must co-operate when on the same channel as other wireless networks near it. This means that the google APs should be honoring the management traffic and broadcasts from other Wi-Fi radios near them. In a place like Mountain View, there is a *LOT* of Wi-Fi.
- 802.11n performance is dependent on multi-pathing. An AP on a pole in the middle of a park doesn't give much in the way of surfaces to reflect a signal off of. You end up at my first point- slow transmission, lower cell capacity.
- While two clients on an AP each can "hear" the APs transmissions, they may not "hear" each others'. Collisions galore.
- The ISM bands that Wi-Fi operates in are full of non-Wi-Fi interference. Wireless baby monitors are notorious for killing Wi-Fi, as are cheap wireless video cameras. Cordless phones,motion detectors, microwave ovens, remote control toys all play a part in the general noise within these RF bands.

Internet for free?

[iocane4me]

They are using 5 GHz Alvarion BreezeAccess VL's at 54mbs for their short backhauls. (The diamond looking things on the telephone poles) Alvarion is dead and the BreezeAccess is dead. My experience using that hardware is that is can be a little flaky and it takes someone in the know to get them running adequately. I always got a lot of dropouts and could never stream YouTube reliably through an Alvarion pipe. There are a couple hundred of them in Mountain View. Also, the original agreement was back when Google needed Mountain View for all that space on Shoreline and Charleston, the new park, fire station, and ways to deal with traffic, Google needed to keep Mountain View happy. Mountain View needed a big company to replace all those dead ones in 2008 So the free Internet looked good for all to get what they wanted. It was the first Google municipal installment and there was much to learn. The hardware they are using is from a dead company and Google is going fiber to the home now, but everything is paid for. Why not renew for another 5 years, it doesn't have to work it just needs to garner press so that Kansas City (and others) does not get the drift that Google will pull out after five years. The same is true of Milpitas. Google bankrolled that in 2008 through a non-profit when Earthlink bailed on that project.. How is Milpitas doing, they are still upgrading. Everyone knows if you want free Internet in Mountain View you go to a cafe on Castro that doesn't change their passwords.

A counter-example

[Brett Buck]

Just down the road from Mountain View, the old (now defunct) Sunnyvale minicipal Wi-Fi worked pretty well when it was running. I was a long way away from the nearest transceiver but I never had any significant problems with connecting or with throughput.

    It went defunct because the funding went away, I think, not because it didn't work.

        Brett

Re:1-6-11 vs. 1-4-8-11, B vs. G vs. N.

[AK Marc]

1-4-7-11 or 1-4-8-11 (or 1-5-8-11) are "better" than 1-6-11. 1-6-11 guarantee no frequency overlap, at a cost of greater in-band interference (As fixed numbers of devices fight for a smaller number of channels). And, as the frequency falls off, so does power, so that, for a reasonably spaced network, 1-4-7-11 are non-overlapping.

One of the other oft ignored issue is that other 802.1b clients are not "interference" but "contention". Bandwidth goes down only slightly worse than linearly. But with "real" interference (something that doesn't practice CSMA/CA), you'll get worse performance because the interference won't ever back off.