Slashdot Mirror
NTT DoCoMo Asks Google To Limit Android Data Use
An anonymous reader writes "NTT DoCoMo has had enough of Android's effects on its mobile network in Japan. Following a service disruption due to Google's Android VoIP app, the company is now asking Google to look at reducing Android's data use. In particular, the amount of time allowed between control signals being sent either by official apps or 3rd party ones. Typically these occur as often as every 3 minutes, but scale that up to thousands of apps on millions of handsets and you can see the issue DoCoMo has. So, does DoCoMo need to invest more in its infrastructure, or is Android a data hog that needs reining in?"
Slopes never have enough room for American habits.
Americans are never efficient enough for those angry slants.
I say we kill 'em all and let God (or Morman God if you are Glenn Beck or Mitt Romney) figure it out.
You could be a sister wife for a pedophile in your next life!
If all they are asking is for Google to optimize its network usage, as the article seems to imply, go all out.
If its telling Google to try and control the amount of bandwidth the users decide to use, well, I think they are going to have a little trouble getting that done.
"reign" a kingdom. "rein" a horse. Please.
Having been to Japan, and several locations around the world, I can say with fair certainty that NTT DoCoMo has the best network service I have *ever* seen. It allowed me to measure what is due to the iPhone's failings, and what is due to the network operator's failings. By contrast, in New York, AT&T makes getting signal a game of hide and seek. France stands somewhere in between the depths of AT&T and glory of NTT DoCoMo.
All this to say that if NTT DoCoMo feels Android is unoptimized... than I pretty much take their word for it.
Both Apple and Google need to be aware of their bandwidth usage, but it is not just those two, but the app developers as well. Better to spend a few more CPU cycles and compact the data a little more than to bring down the network. XML is fine, but hardly the most efficient way to transmit data, especially not without compression.
On the other hand, the providers must realize that the trend are for increasing data usage, as we take our daily communications with us, rather than sitting at home with our fixed line broadband connections.
It's some "control-traffic" issue that you, the customer, really shouldn't have to think about. It's nothing to do with an application that routes voice traffic over a data connection, thereby disrupting the carrier's finely-crafted billing practices.
The Japanese used to be very pragmatic people. What happened to them? Some have said the South Koreans have taken over. I am inclined to agree when I look around my home.
Android is 'open'. The Japanese build and maintain their networks. The problem they describe is can be stemmed by a [simple] software 'update'. Why don't they do just that?
Again, what happened to the Japanese?
I live in NYC and rarely have any problems getting a signal on AT&T. Actually getting any data or calls to my phone over that signal, however, is a distinct challenge. ;-)
I often experience dropped calls and slow data rates while my phone happily shows "5 bars" of signal.
"Andloid's not done until VOIP won't run."
the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
Uh... Control signals have nothing to do with overall data consumption quantities. When you send a text message, you send the 160 or so bytes of data through control signals. The issue here is that Android doesn't control the way its apps try to contact the towers, basically hammering them if they don't respond properly. This issue is one of the reasons Android has massive standby battery drain problems, as detailed in this 300 page xda thread: http://forum.xda-developers.com/showthread.php?t=1179809
The galaxy nexus has its own 100+ page thread dedicated to battery drain on standby.
So, does DoCoMo need to invest more in its infrastructure, or is Android a data hog that needs reigning in?"
Isn't the Slashdot mantra always "It's open source! Therefore, it's someone else's problem when it comes time to pay the bills!"?
Everybody gets what the majority deserves.
DoCoMo
MoDaCo
Comodo
Where is the originality?
So I've got one of them gen-u-ine android phones. Which apps are supposedly hammering the network?
My guess is the GOOG latitude GPS plotting thingy must be updating fairly often. So in the big list of blacklisted apps, I've got one "i donno maybe guess".
This is important to me because my batteries life is very short... Enabling latitude position tracking thingy meant my battery died in about 10 hours (which is an issue for a guy who works 10 hour shift bracketed by a modest commute), so I shut it off, gaining me at least 6 more hours, making it very easy not go thru a working day without charging. I wonder if I disable "something else" if I'll magically gain yet another 6 hours... or more...
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
really... does anyone use CDs anymore? Especiallly CDs filled up with a bunch of crap?
TFA:
Typically these occur as often as every 3 minutes, but scale that up to thousands of apps on millions of handsets and you can see the issue DoCoMo has
I still don't see the issue they have... what is the issue? What are these "control signals"... is it TCP traffic or cellular network control signals? What resources are they consuming? Have other carriers had the same kind of problem or is it specific to DoCoMo? What update frequency do they recommend if 3 minutes is too often? 4 minutes? 6 minutes? 60 minutes?
The problem is that pesky Shannonâ"Hartley theorem. Basically it says that your data throughput is equal to your channel bandwidth and SNR. With a wired connection, there's a lot you can do there. With wireless, there's serious limits. SNR is fixed by environmental conditions, unless you are willing to up the S part a lot which requires lots of power and thus isn't suitable for mobile.
Bandwidth is limited since you have to share the airwaves with lots of stuff and certain frequencies are good for certain things. So going with 60GHz might sound great because a 1GHz channel would be no problem, but you find that it has trouble with attenuation due to water in the air, never mind walls. 700MHz cuts through walls pretty nice, but your channel will be much smaller.
Then of course there's the real sticking point: You share with everyone else in the same area. If your throughput is, say, 100mbps total you are sharing that 100mbps between anyone on the same segment as you. So have 100 users, all trying to go full blast and you'll each get 1mbps or so.
The only solution is to build out the segments smaller. However there's limits to how much of that you can do. You can't realistically segment the network down to an arbitrarily small level.
The "JUST UPGRADE THE NETWORK!" thing that geeks like to scream is very unrealistic in the case of wireless. Yes, with fibre we can have shit tons of bandwidth, when you are talking carriers in the hundreds of Terahertz, you can have some bigass channels, you can have lots of channels, SNR is quite good in fibre and you can always just lay more of it. However wireless has some real physical limits you butt up against.
We have to accept that when you are using wireless, especially longer range wireless, that there are limits to deal with.
So what? The world is heading to MORE digital data usage, not less, that's a fact. And the prices for use are going to drop, that's also a fact. Maybe NTT DoCoMo needs to up their capacity, not worry about throttling it so much. Words all providers need to take to heart.
Python: 'And then suddenly you have a language which says "we're all stuck with whatever the whiniest coder wants".'
Considering that Google has a SERIOUS interest in making control signal intervals as long as possible for battery life purposes - if they are too often, the carriers only have themselves to blame. Too many carriers have aggressive NAT firewalls with short TCP connection timeouts, and it's much better for the handset AND the carrier to send a keepalive within that timeout period than to have to detect a dead connection and set up a new one.
Google "netpiculet" or look at my last post earlier on this article for an eye-opener of how network providers shoot themselves in the foot.
If Google is sending control signals too often - DoCoMo should take it up with the carriers that deployed broke-ass NAT boxes that forced Google to do this.
retrorocket.o not found, launch anyway?
Given that this is a shortage, and that a shortage is when demand exceeds supply, DoCoMo can either add supply by investing in its network, or they can reduce demand by raising the cost of bandwidth, at least during peak periods. This gives DoCoMo two options to eliminate congestion on its network.
Raising the cost of bandwidth could be done pretty easily by setting caps on peak hour data usage.
Any sufficiently unpopular but cohesive argument is indistinguishable from trolling.
I pretty much agree. Plus, upping their capacity will make them reduce power emissions from their antennas which will calm down the GSM-is-microwave freaks.
Write boring code, not shiny code!
Was this a third-party app or an official Google app? If it's third-party, Google has no control. One packet every 3 minutes is, honestly, pretty damn good for some apps. Skype can cause a device to wake up once every few seconds (which is the main reason it's an epic battery hog).
Google's own apps are about as efficient as they can be in order to minimize periodic data, because keepalives and checkins wake the device, draining battery. The problem is that some major carriers have broken NAT boxes ( http://conferences.sigcomm.org/sigcomm/2011/slides/s374.pdf ), forcing Google to reduce keepalive intervals so that these carriers don't kill TCP connections - which forces an expensive (in terms of time, battery, and network resources) connection setup sequence.
retrorocket.o not found, launch anyway?
640k ought to be enough for anybody.
XKCD:Xeric Knowledge Comically Dispen
*YOU* started it by superlow tcp timeouts. Give the users a public IP without double- and tripple-nat that's mainly intends to screw with our sessions for profit-maximisation (by breaking IM and voip-apps), *then* you can demand that users don't use the data their ****** PAYING for!
I know it makes articles sound more dramatic and controversial, but sometimes the answer is obvious:
"So, does DoCoMo need to invest more in its infrastructure, or is Android a data hog that needs reigning in?"
The answer is clearly "both". Apps and android should optimize their data usage, doing so increases battery life and gives a better user experience all around. If DoCoMo is identifying particularly troublesome apps, then that is helpful to decide where to start hacking. _Also_, DoCoMo should be upgrading its infrastructure. It is clear that data use will only rise, and they certainly would like to have more customers, apps reducing their usage is a good thing, and will create a better experience, but not a solution to the problem that people actually have uses for all that data, and said usage will grow.
http://notanumber.net/
So after Google tells them to go screw themselves, what are they going to do? Ban the operating system?
The solutions to these problems are simple. If you have 100 customers, and you have 100MB/s of bandwidth on your network, don't sell them all 5MB/s plans. ITS SIMPLE MATH. To blame this on Google is like an all-you-can-eat buffet complaining to a local taxi service that they're dropping off too many fat people. Who do you expect to pay for all-you-can-eat? Twiggy?
Seems like they really shouldn't oversell their network.
You can't offer high end services and phones and expect people not to use them. This sounds awfully familiar to North American telecos offering Unlimited* service and complaining when users actually use their bandwidth.
That's because AT&T is suffering from the same problem NTT DoCoMo is suffering from - control channel congestion. You're getting 5 bars alright, but the big problem is stuff like dialing and establishing data connections consume control channel bandwidth. Dropped calls happen because your phone's trying to switch towers and can't because it can't get a word in edgewise on the control channel.
Slow data ditto - the phone on 3G needs to establish multiple PDP data sessions to get 3G speeds, and if it can't talk to the tower because the control channel is busy, well, it suffers.
Control channel congestion (caused by all this plus texting) is why AT&T service can be horrible, despite having plenty of free channels available for data and voice. It's what took T-mobile down once (a bad IM app overloaded the control channel).
Think of it as the old-timey POTS phone days where you lifted the handset and told the operator who you wanted to talk to. And now have lots of people do the same and the operator's now overloaded trying to establish and tear down connections, leading to phone calls not going through, the operator not responding to you, etc.
Should Android be more efficient, and every app it runs. OF COARSE. There is not such thing as efficient enough, but Google has to decide if it is worth
their money right now.
Should the network provide stop wining because their network is running up against the physics of
allowing too users on it and just fix the problem. Probably.
Sorry, but don't take on more subscribers then you can service. And if their are devices that you don't like their affect feel free to ban them.
If that doesn't make you money , that's called business. It's balance. Either provide what people want , or go out of business.
âoeTolerance applies only to persons, but never to truth. Intolerance applies only to truth, but never to persons.
Sprint's Dan Hesse said the iPhone is so data-efficient, it will help Sprint keep its mobile data plans unlimited. He claims they do not ping the network near as much, and are better at handing off data to wireless networks. http://www.forbes.com/sites/elizabethwoyke/2011/10/26/sprint-ceo-iphone-will-help-us-keep-unlimited-data-plans/ Assuming that was a comparison to Android?
I'd guess they've already upped their capacity, and they probably think you should up yours.
...the future crusty old bastards are already drinking the Kool-Aid.
*Then* they'll have bandwidth problems.
IANANetworkEngineer, but it sounds to me like you are saying we need a separate network for the handheld devices, to deal with the different TCP settings. Is that even possible with a cell carrier? Can we do something like a VLAN?
"Crude and slow, clansman. Your attack was no better than that of a clumsy child."
DoCoMo has admitted that their core packet switcher capacity is insufficient. The recent downtime they had, they hit 16 something packets/sec when the system only had capacity for 14. And they are predicting hitting 20 soon. Apparently they use some sort of traffic prediction capability to manage their network, but how that is applied (edge/tower throttling?). So this is basically DoCoMo saying they were woefully unprepared for the smartphone revolution in terms of network capability (but note this is not necessarily an issue of bandwidth at the tower level, since japan has an enormous concentration of towers). They were halfassing smartphones only a year or two ago, and doing network gear rollouts are multi-year endeavors, so their earlier plans are now inadequate. DoCoMo and the other carriers are already throttling inside the core Tokyo area at the tower level it seems as a temporary measure to cover their core inadequacy. DoCoMo got caught with their pants down on smartphones since imode seemed to be king here until the iPhone showed just how bad the dumbphones here were in terms of usability.
They try to blame the recent disruptions on Google and a domestic android VoIP app service. DoCoMo's only solution is to spend the big money on their core network that they haven't been doing enough of. But they've been blowing their wad on their 3.9G LTE rollout (Xi).
A secondary issue is that DoCoMo is EoL their MOVA(2G) service since they have to vacate a frequency band, so you have a lot of people switching up to new data intensive dumbphones and smartphones running on FOMA(3G-UMTS). The deadline is March to jump up before service is cut, so perhaps the recent performance problems in the last 6 months may be attributable to the changeover users upgrading and blowing out the UMTS infrastructure that was installed quite a while ago and will not be upgraded (likely replaced with Xi(3.9G-LTE)).
I wonder why nobody mentioned it, but moving to IPv6 will largely mitigate the issue. Every device will be able to have a public IPv6 address and keepalives don't have to be frequent to cater to evil NAT boxes. As a fact, keepalives will not be needed at all at the network level, but only in the application if it is to provide presence notification. Think UDP over IPv6.
Exactly. And increasing control channel capacity is not free, as it reduces resources available for users data. Plus the control channel need robust encoding so are comparatively more expensive than data on average. Plus the standards have some flexibility on dimensioning control channels, but it's not fully flexible either. There's some assumption built in.
So operators may increase the amount of resources allocated to signaling, but this is only possible up to a point and would reduce data capacity (so increase cost per user level data).
So the way to get out of this with everyone wining is to make sure mobile phone applications use the network efficiently, and don't waste signaling capacity just out of ignorance. It seems both Google and Apple provide centralized polling / notification services already, so blaming them may be unfair. But I'm pretty sure there's a lot of developer education still to do. And maybe adding tools in the platform to see what are the wasteful applications would help. Knowledgeable users could spot the offenders, and make some noise so they improve their behavior.
There is a nice study on this topic from last year, showing why Android push protocols (the control signals being referred to in the article) can be data hogs. Although the same can be said for push-based protocols on iPhone I guess.
A paper and a presentation available here:
https://sites.google.com/site/sachinkagarwal/home/publications-talks/gis-2011-infocom-2011
In particular,
https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxzYWNoaW5rYWdhcndhbHxneDoyMDVmZTBjYzU3MGQ5MmI2&pli=1
Slides 7,8,9 where the author says that multiple HTTPS connections are being continuously maintained by the Android device.
Abstract:
"Push message delivery, where a client maintains an “always-on” connection with a server in order to be notified of a (asynchronous) message arrival in real-time, is increasingly being used in Internet services. The key message in this paper is that push message delivery on the World Wide Web is not scalable for servers, intermediate network elements, and battery-operated mobile device clients. We present a measurement analysis of a commercially deployed WWW push email service to highlight some of these issues. Next, we suggest content-based optimization to reduce the always-on connection requirement of push messaging. Our idea is based on exploiting the periodic nature of human-to-human messaging. We show how machine learning can accurately model the times of a day or week when messages are least likely to arrive; and turn off always-on connections these times. We apply our approach to a real email data set and our experiments demonstrate that the number of hours of active always-on connections can be cut by half while still achieving real-time message delivery for up to 90% of all messages."
or they could just deploy ipv6 and avoid the need for broke-ass NAT boxes mucking around with the content of user's TCP sessions and packets.
It's funny that you say that, because based on (admittedly half year old) data that an app developer collected about reconnect rates, Japan was by an order of magnitude the worst country with regards to number of reconnects that this app had to perform (DoCoMo was the second-to-worst carrier around the world).
Reconnects happen because the cell carrier closes a connection or times out--a good cell carrier won't change your IP address or RST your connections when you switch towers, but a bad one might decide to assign a new IP address each time. On some apps, reconnection may consume up to 1MB of bandwidth each time as they attempts to resync data (Yes, good apps shouldn't do this, but I have seen it happen.)
The problem is not Android -- the problem is the shitty QoS that most mobile carriers put on their networks, combined with the fact that they often kill connections at the NAT layer without notification, time out connections over unwanted ports and block protocols that they don't like.
The end result is that everything on a cell network has to happen over port 80 or port 443, with the SSL negotiation overhead that involves, combined with sending keepalives every 4 minutes. Yes, Android is unoptimized. DoCoMo might be doing everything right, but they bear the price of all of the terrible cell carriers that go out of their way to block data (AT&T, T-Mobile, I'm looking at you). Android 4.0 has a Data usage monitor that helps a ton in debugging misbehaving apps, but data is a fact of life.
That said, Apple may have made a good decision by forcing app developers to use push notifications when the app is in the background. Android messed up push notifications by tying them to Google Talk and Android Market -- this means apps that require push will not run on a large fraction of android devices around the world (including the Kindle Fire). The result is that apps don't use push and implement their own (often buggy/wasteful) push system.
Finally, if DoCoMo doesn't want users to send/receive data, then limit their bandwidth for crying out loud. Don't whine when you provide fast service and people use it. What is complaining to the OS manufacturer going to do? They provide a platform, not the apps or the service they run on.
Yes, if I had mod points, I'd mod you up.
I'm an app developer, and I've had to deal with countless network problems (usually NAT's dropping connections without RST) that ended up being resolved by stupid strategies like "f it, lower the keepalive interval to 5 minutes", and killing a connection if it was not ack'ed in X seconds (you can be more agressive with killing TCP connections by adding protocol-level acks on client&server).
But despite this, I've managed to reduce bandwidth greatly by making my protocol independent of TCP connection -- in other words, I connect, tell the server who I am, and keep going with my connection, slowly making forward progress even if my layer 3 connection is killed every few seconds. At this point, TCP port 443 becomes basically a heavyweight datagram protocol (with a SSL handshake) because you can't rely on anything.
I'd rather use push notifications, but they have two glaring holes: 1) You can't rely on messages arriving on time. This means it's useless for a VOIP app where you expect it to ring within a few seconds. 2) Google C2DM requires that you have android market installed. This means your app won't work on half the phones around the world.
DoCoMo would be stupid to just sit there and yell, but it's never wrong to ask Google to take some burden, especially as they make money from each other.
The article claims Google's notification system is particularly egregious.
Also, Apple's system should be differentiated at least to some degree. Android gives developers more opportunity to utilize the network in the background. Apple's APIs give developers much less leeway in that regard.
At that point you're probably better off writing your own reliability protocol layered over UDP...
retrorocket.o not found, launch anyway?
That's one thing I wish wireless providers would "get" - rather than marketing insanely high transfer rates with ridiculously low caps, they should sell tiered rates...
The T-Mobile and AT&T "unlimited" policy of full speed up to a ridiculously restrictive throttle wall is ridiculous - it makes FAR more sense to reduce the initial speed, and make the throttle a little gentler. If throttling of high users weren't so severe (a "soft cap" that degraded as your usage got higher, instead of a "hard threshold" after which your device is nearly unusable) people wouldn't be so damn pissed off.
I would have no problem with trading peak and average speeds in order for not worrying about "brick wall" overage thresholds. Plus, this would be better for the network - instead of congestion when people's billing cycles reset the limits, data would be spread more evenly over the month.
retrorocket.o not found, launch anyway?
If you can't handle the data use, stop selling the phones in such high quantities. I have no sympathy for cellular providers. They were blind with greed for years seeing how profitable smartphones can be, and now when they have finally sold millions and millions they try to blame anyone but themselves for their networks shortcomings.