Slashdot Mirror


Wrong Number: Why Phone Companies Overcharge For Data

MrSeb writes "A recent study (PDF) conducted by UCLA professor Chunyi Peng shows that carriers generally count data usage correctly, but those customers who commonly use their device in areas with weak signal strength or to stream audio or video are often overcharged. Peng and three other researchers used data gleaned from an app installed on Android smartphones on two different carriers. The issue appears to be in how the system is set up to count data usage. Under the current scenario, data is charged as it is sent from the carrier's network to the end user. What does not exist is a system to confirm whether the packets are received, and thus preventing charges for unreceived data. Peng demonstrated this in two extreme circumstances. In one case, 450 megabytes of data was charged to an account where not a single bit of it had been received. On the flipside, Peng's group was able to construct an app which disguised data transfers as DNS requests, which are not counted by the carriers as data usage. Here they were able to transfer 200 megabytes of data without being charged. Overall, the average overcharge is about 5-7% for most users. While that does not seem like much, with unlimited plans gone and data caps in style that could pose potential problems for some heavy data users. Could you be going over your data allotment based on data you never received? It's quite possible."

36 of 105 comments (clear)

  1. DNS not counted? by ccguy · · Score: 2

    DNS requests, which are not counted by the carriers as data usage.

    I'd love to see a source for this.
    Well, a source to anything actually, but it seems to much to ask around here these days...

    1. Re:DNS not counted? by joostje · · Score: 5, Funny
    2. Re:DNS not counted? by BumboChinelo · · Score: 3, Insightful

      It is true, I work for a supplier of GSN nodes and most operators configure GGSN/PGW to classify DNS and TCP SYNs as free. However, this freebie is overset but charging retransmission in the telco IP network. Personal experience in comparing sent bytes vs received bytes on remote and charged bytes in CDRs. Wiresharking on Gn interfaces pointed out the origin of extra bytes in CDR. So there never is a free lunch

    3. Re:DNS not counted? by Zocalo · · Score: 2

      Who needs Iodine? Anyone with access to an an SSH server could bind it to port 53, then just establish SSH tunnels to it to do all sorts of stuff, not least of which are avoiding all sorts of data charges and local content filtering. Hypothetically speaking, of course... ;)

      --
      UNIX? They're not even circumcised! Savages!
    4. Re:DNS not counted? by GameboyRMH · · Score: 2

      That's assuming port 53 traffic is free rather than DNS traffic being free - not the same thing.

      --
      "When information is power, privacy is freedom" - Jah-Wren Ryel
    5. Re:DNS not counted? by Anonymous Coward · · Score: 2, Interesting

      I also work for a supplier of GSN nodes (with charging in particular, hence posting as anonymous coward). I know we can't work at the same company, because where I work, a major bug like that would halt any delivery until it's fixed. There's no way you'll have more bytes in a PGW/GGSN CDR than what you see on Gn. A bug report would be written immediately. Any internal packet drop in our system will not be counted in CDR:s. Packet drops on Gn? Not going to happen. So you're either making that up, or your Wireshark setup is miss configured, or you work for.. Nokia? :)

    6. Re:DNS not counted? by isj · · Score: 4, Informative

      The operators that my employer delivers charging solutions to rarely use the byte counters from the GGSN because they charge he traffic types differently, e.g. "free facebook" while other traffic is charged based on byte counters from the DPI box. It is true that some of the newer GGSNs/ASN-GWs have adequate DPI capabilities so they can classify the traffic with enough granularity and the byte from them can be used.

      There is one slightly fishy thing in the study (yes, I read the fine paper). Their test with logging on, go idle, move to radio-inaccessible room, then have server start steaming UDP to the phone (which will be dropped due to inaccessibility). In my experience the SGSN/GGSN quickly signals that the user has gone offline (PDP session termination), and the stream of UDP from the server is blocked at the DPI or the GGSN. Sounds like the operator that the study used has a major bug in its charging setup where PDP session termination doesn't also stop the IPuser association.

    7. Re:DNS not counted? by Scarred+Intellect · · Score: 2

      I'd love to see a source for this. Well, a source to anything actually, but it seems to much to ask around here these days...

      Seems to me the study is the source. Called a "primary source" where I come from.

    8. Re:DNS not counted? by profplump · · Score: 2

      Actually many hotspots *do* require you to use the local DNS server. It's pretty trivial, since that's part of the DHCP config, so it's not like end-users have to do anything to make that work. In those cases your data needs to be close enough to the DNS protocol to get propagated upstream, which is basically what iodine does.

      Also note the SSH more or less requires TCP, so even if you allowed port 53 outbound you could limit access to the UDP protocol and effectively block SSH and many other tunneling tools, even if they are set to port 53. No DPI required. Sure, you'll prevent TCP DNS queries but that's rarely an issue of the kind of devices connecting to semi-open hotspots, particularly in the pre-payment phase of the connection.

    9. Re:DNS not counted? by Anonymous Coward · · Score: 2, Insightful

      A strong bias and a poor conclusion. Rather, a poor methodology. Since you also appear to work in the industry - I'm wondering if you notice what I notice? The two methodologies they used for detecting traffic at the mobile... They used an android application that does not require root access (e.g. isn't taking pure IP level statistics) and the Traffic Status API.

      Two pretty big problems. 1) They don't measure TCP Retransmissions... This is uhh a big "duhh" - but more disturbing they don't measure IP packet overhead. E.g. the application they used uses the same API their homebrew application uses - both of which look at actual TCP and UDP payloads (reassembled in the case of TCP). IP Packet Overhead, SYN's, SYNACK, FIN, RESET. Are those measured? Nope.

      Basically, the methodology is flawed and the title is sensationalist. If this is the quality of research coming out of American Universities then we're fucked. Anyone who has a minor clue can read the paper and come to some astonishing conclusions:

      1) They don't measure IP Packet Overhead
      2) They don't measure TCP Overhead
      3) They don't measure TCP Retransmissions

      I'd wager most of the "over charged data" they're claiming is coming from #1 and #2, just setting aside #3. And with regards to #3 - it's clear that it's to the consumer's benefit to measure traffic at based on the number of IP Packets, as opposed to measuring traffic on the RAN. If the consumer had to pay for RAN - it would create a billing nightmare. Those who require a lot of RAN retrans would be screwed.

      Finally, the idea of measuring actual payload received on the device, and then reporting that back to the network is simply laughable. It's stupid at best, retarded at worst. The overhead in this reporting would drive the cost of data usage up and would only hurt to consumers. Considering the possible signalling load, let alone the security implications...

      Overly educated morons talking about shit they don't understand. It's no wonder CS grads in this country can't find jobs. They should demand their education $$$ back - because they clearly didn't learn enough to actually be effective in the workplace....

  2. If only it worked both ways by Anonymous Coward · · Score: 5, Funny

    Well I sent a check for my monthly bill... not my fault you didn't receive it.

    1. Re:If only it worked both ways by Anonymous Coward · · Score: 5, Funny

      Reminds me of a joke. Telephone lineman is recruited into the army. Gets to the rifle range for practice. Everyone is shooting, targets are being hit by all but the lineman. Sarge comes up and asks why his target got no hits on it. Lineman ties a rag over the end of his rifle and fires off a round, shows the sarge the rag with a hole in it and says, "No problem at this end."

  3. Link to study by ccguy · · Score: 4, Informative
  4. they could charge more... by joostje · · Score: 4, Funny

    I guess that means the operators will shortly release an update for the phone OS's to also charge for the data the phone sent but wasn't received by the operator.

  5. Step 1 by Spy+Handler · · Score: 3, Funny

    1. write an app that disguises streaming porn as a DNS request
    2. ???
    3. Profit!

    1. Re:Step 1 by Pieroxy · · Score: 2

      http://code.kryo.se/iodine/

      It turns out you can already disguise all of your traffic as a DNS request.

  6. Re:Wow, your mobile networks SUCK. by game+kid · · Score: 4, Funny

    Thank you for your suggestion. at&t(R) is committed(tm) to rebuilding the nation's largest 4G network this year with your input. To pay for the buildout* you will notice a 10% 2012 Nation's Largest 4G Network Improvements Fee along with a 200% SMS price increase. Thank you again for choosing at&t(R), with the nation's largest 4G network.

    *Buildout subject to cancellation without notice. Just kidding about that last part, we cancelled it while you were reading that sentence. The fee stands to pay for the costs of typing the prior two and commissioning a forthcoming Gartner study to validate their market impact.

    --
    You can hold down the "B" button for continuous firing.
  7. Re:Why do phone companies overcharge for data? by Merls+the+Sneaky · · Score: 2

    To get rid of the taste of the food.

  8. The trouble is. . . by dtmos · · Score: 4, Insightful

    Resending a packet due to a missed ACK takes up air time, just like it did sending it the first time, and the carriers have no control on where the user will be. If they make their systems robust enough to move their present average packet reception rate from an already-good 93-95% to, say, 99%, this will only enable their users to move down another floor in their sub-basements, or another few city blocks, or another cubicle row deeper into the building, before the average goes back down again -- after all, wireless systems have limited range. The cost of the new infrastructure would be roughly twice that of the previous one ("increasing coverage is increasingly expensive"), and you're going to pay for the cost of the infrastructure either way in your air-time charges.

    Look at it this way: Even if the company only charged for packets successfully received, it would just increase their rates by (1/0.95) - 1 = 5.3% to (1/0.93) - 1 = 7.5% to maintain the same cash flow. Plus it would have to start keeping track of the success or failure of each packet transmitted, and put that into its billing scheme. That's a database PITA I don't want, thank you very much.

    1. Re:The trouble is. . . by samjam · · Score: 3, Informative

      Keep track isn't hard. Telco's already have optimisation for tcp re-delivery from the mobile gateway so that the distant sender doesn't have to re-send the missing packet, the telco can do that.

      This service improves tcp performance over a mobile network and is important for customer retention.

      Maybe not all telco's do it but I doubt it.

      http://www.iith.ac.in/~tbr/teaching/docs/transport_protocols.pdf
      http://www.cs.sunysb.edu/~jgao/CSE370-spring06/lecture17.pdf

    2. Re:The trouble is. . . by broknstrngz · · Score: 4, Informative

      Indeed, all sorts of TCP splitting techniques exist. However, there is only so much data such a device can temporarily queue to keep retransmission on the terrestrial side. If you run a network with 10 million subscribers, it becomes very interesting.

      The mismatch comes from the fact that operators collect CDRs on the terrestrial side of their GGSNs. So even if the mobile subscriber does not need to resend a segment, the terrestrial retransmission is still accounted, as are the duplicate ACKs sent by the Internet host.

      You simply can't expect both having the cake and eating it. High latency links come with trade-offs.

      I work for a provider with much higher RTTs (~1200ms). The 5% reported by the study is exactly what we are seeing.

    3. Re:The trouble is. . . by broknstrngz · · Score: 5, Interesting

      Also, most people have no idea of the optimization techniques operators use.

      Navigate to any content heavy website. If your mobile browser allows you to, try to see the source of the page.
      Chances are you will see all whitespace trimmed, all CSS and JS inlined. All pictures are compressed in a lossy
      fashion to reduce their size.

      There is also HTTP request coalescing. If you request "/", the whole page will be retrieved, then processed as
      above and delivered to the mobile browser in a single reply. How many GET requests do you save? A lot.

      If there were no such techniques, one's mobile bill would be almost twice as high and the browsing experience
      would be 4 times as slow.

    4. Re:The trouble is. . . by Anonymous Coward · · Score: 3, Interesting

      HTTP request coalescing is a mixed bag, since the all the stuff you inline will be resent when you request the next page. If you have carefully reduced the size of your html pages and set your CSS and JS to be basically cacheable forever, you may want to add Cache-Control: No-Transform to your headers to request that no such thing is done. The same header should theoretically prevent your images from being re-compressed with an absurdly bad JPEG quality setting.

    5. Re:The trouble is. . . by Archangel+Michael · · Score: 2

      Why is it then, that bad wifi on my phone is immensely faster than good Cell (3G)? One or two bars on wifi loads so much faster than full bars on my 3G? Is their network that over saturated that even with a good signal that they just can't transmit? It is even more obvious on mobile optimized sites. Just sayin.

      --
      Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
  9. Vodafone Netherlands by hankwang · · Score: 5, Interesting
    It happened that I wrote down the status of my data usage over the month July, so here is my anecdotal experience for Vodafone Netherlands:

    * Android Droidstats usage logger: 369 MB (2012-07-31 22:16h)
    * Android "My Data Manager": 337 MB (2012-07-31 22:16h)
    * Vodafone online usage monitor: 307 MB (up to 2012-07-30 17:46h)
    * Phone bill for July: 343 MB (since a couple of months they actually mention the total; before I needed to use a perl script to parse the PDF invoice and add the data usage of some 200 separate data sessions)

    When I asked about the differences a few months ago, the Vodafone customer service told me: "The information on your Vodafone account online is the real usage. Numbers from data usage apps are not reliable." But I highly doubt that I used 36 MB over the last day of the month, so it seems that within Vodafone they have different systems.

    My train commute (where I use most of my data) passes through an area with bad coverage, so I would have expected a bigger difference based on the theory that packet loss accounts for most of the difference.

    1. Re:Vodafone Netherlands by mkraft · · Score: 2

      With AT&T, I discovered that data usage takes about a day to be entered into the system, so the extra data could have come from the prior billing cycle.

      In my case, I had about 800 MB of data left on the last day of my billing cycle so I decided to download a 300 MB file. The data usage didn't show up in AT&T's system until the next day, so despite showing the correct usage date on my bill, the data was applied to next months data cap.

      Basically, don't trust usage monitors, apps or even your bill to be correct since data isn't applied the way you would think it should be.

  10. Word Play by Dunbal · · Score: 2, Insightful

    "Overcharged". What a sweet word to use instead of other words like fraudulently charged or stolen. No, you were simply "overcharged". Don't worry about it. And your bill is due by the 1st.

    --
    Seven puppies were harmed during the making of this post.
  11. Re:It should be charged anyway by gl4ss · · Score: 2

    Data takes up network capacity whether the device receives it or not.

    if you're an operator..
    step 1. hire some flooders.
    step 2. profit.

    essentially this is why they should sell by speed, not by usage.

    --
    world was created 5 seconds before this post as it is.
  12. Re:ET Phone home? by Hatta · · Score: 2

    The nature of TCP includes positive acknowldgement and retransmission. If you don't get that acknowledgement, don't charge the user for that packet.

    --
    Give me Classic Slashdot or give me death!
  13. Carriers are businesses by Pete+Venkman · · Score: 3, Insightful

    Do you really think what carriers charge is how much it actually costs to send a text message?

    1. Re:Carriers are businesses by Pete+Venkman · · Score: 2

      If a business only charges actual cost for a product, they cannot make a profit. I want some companies to earn a profit, especially if I'm a stakeholder. If we're lucky, we have jobs (and paychecks) that are a result of some company making a profit. No, I'm not talking about hookers-and-blow-for-everybody kind of profit, either.

    2. Re:Carriers are businesses by Mr.+Slippery · · Score: 2

      Yes, carriers charge pretty much what it costs to send a text message, and keep the infrastructure that the message traveled over running.

      The problem is that carriers also charge for a voice call what it takes to connect that voice call, and keep the call that the message traveled over running -- even though it's the same infrastructure used for text messages. Nothing like double billing.

      People who work at carriers aren't all driving around in Ferraris, you know.

      Of course not. Employee salaries are an expense to be minimized. The profits go to shareholders, not employees.

      --
      Tom Swiss | the infamous tms | my blog
      You cannot wash away blood with blood
  14. Re:It should be charged anyway by Guppy · · Score: 2

    step 1. hire some flooders.

    While I don't expect legit carriers to crap-flood for profit, it seems this presents a perverse economic incentive not to invest in improving the quality of their networks.

  15. I still have an unlimited data plan by sandytaru · · Score: 2

    It's only gone from the big carriers. I use Metro PCS, and have 4G in the city where I live, along the major route I commute to Atlanta, and of course have it in Atlanta itself. (The trade off is having text-only outside of the network.) If you live in a rural area and need one of the big guys to provide coverage, you're screwed, but if you live in a metro area, you'll probably get a better deal with a smaller regional carrier.

    --
    Occasionally living proof of the Ballmer peak.
  16. So wait... by Sprouticus · · Score: 2

    All I need to do is run a website or service on UDP/53 and mobile users wont ever get charged? It can't be that easy.

  17. Re:Another Exploit by mysidia · · Score: 2

    Another thing is, you could attempt to download far more data than your connection is capable of. For example, making a portable device participant in a BitTorrent network, where the client intentionally attempts to download as much data as possible -- at a high loss rate, the data your phone isn't capable of receiving still gets sent by the tower, and therefore still congests the wireless portion of their network, but your device never receives the data, because it's not capable of it.

    Packet loss is an end-to-end connectivity thing; it is no specific party's responsibility. Ultimately, the end user has to take responsibility for requesting packets that they are unable to receive.

    It would probably be better if the phones warned the user about high packet loss or implemented a temporary self-shutoff or backoff. And wireless providers offered SLA credits, for data discarded before it's even attempted sent by the provider's radio towards the user's phone (e.g. Output discards).

    Otherwise, packet loss by the phone itself, or loss of radio transmission, is a result of network congestion. There actually is very good sense in the user having to pay as if data was received when they are requesting packets that are being lost due to congestion --- the user is participating in the congestion that should be prompting the network provider to upgrade the network in that area, so they should pay more for causing that congestion.

    The providers should just clarify that dropped packets do count, and choose appropriate pricing.