Slashdot Mirror


Beware of Using Google Or OpenDNS For iTunes

Relayman writes "Joe Mailer wanted to download an iTunes movie recently and his Apple TV told him it would take two hours. When he switched his DNS resolver settings, the download time dropped to less than 20 seconds. Apparently, iTunes content is served by Akamai which uses geolocation based on the IP address of the DNS request to determine which server should provide his content. When you use Google or OpenDNS to resolve the Apple domain name, all the requests to Akamai appear to be coming from the same location and they're all directed to the same server pool, overloading that pool and causing the slow downloads. The solution: be wary of using Google or OpenDNS when downloading iTunes files or similar large files. Use your own ISP's DNS servers instead or run your own resolving DNS server."

13 of 348 comments (clear)

  1. Opposite Experience with Adobe Download by eldavojohn · · Score: 5, Informative
    First, I was able to verify this with the iTunes download. My Cox DNS was 20 seconds while my Google DNS was 2 minutes 10 seconds.

    But I just tested this on my own by using a different source that uses Akamai: Adobe.

    So I picked a file at this URL: http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.4.0/enu/AdbeRdr9.4-1_i486linux_enu.bin

    Sure enough, the initial server directed me to 72.215.224.16 with this partial tracert:

    4 12 ms 10 ms 10 ms mrfddsrj02gex070002.rd.dc.cox.net [68.100.0.145]
    5 17 ms 14 ms 12 ms ashbbprj01-ae0.0.rd.as.cox.net [68.1.0.220]
    6 12 ms 15 ms 12 ms 72.215.224.16

    Firefox told me this would take 3 Minutes and 35 Seconds.

    Then, I set my DNS to the 8.8.8.8 and 8.8.4.4 addresses and tried it again. This time I was sent to 72.246.30.19 with this partial tracert:

    4 11 ms 12 ms 14 ms mrfddsrj02gex070002.rd.dc.cox.net [68.100.0.145]
    5 13 ms 11 ms 13 ms ashbbprj01-ae0.0.rd.as.cox.net [68.1.0.220]
    6 17 ms 17 ms 13 ms ge13-1.br01.ash01.pccwbtn.net [63.218.44.125]
    7 21 ms 18 ms 12 ms akamai.ge13-4.br02.ash01.pccwbtn.net [63.218.94.142]
    8 17 ms 18 ms 13 ms a72-246-30-19.deploy.akamaitechnologies.com [72.246.30.19]

    Surprisingly, this second server that I was directed to using Google DNS only took 10 seconds to download the same file. I did it a second time and it took 30 seconds.

    Now after restoring my default DNS resolution that URL continually directs me to 72.215.224.40 and the download is as speedy as the Google DNS. If I switch back to Google DNS it now continually directs me to 72.246.30.32 so you can see that there's some load balancing going here that apparently can be divvied up by geographic location for some of their customers. Apparently Apple needs to investigate the same solution that Adobe is using from Akamai. Which doesn't consider everything from Google DNS being fulfilled from a west coast replication server?

    --
    My work here is dung.
    1. Re:Opposite Experience with Adobe Download by TooMuchToDo · · Score: 4, Insightful

      The problem is Akamai is using DNS to determine location, when it should be determining the geolocation of the client IP and throwing an HTTP redirect to the proper server. You can't rely on a client using the "proper" geographically-located DNS server.

    2. Re:Opposite Experience with Adobe Download by icebike · · Score: 4, Insightful

      Well the point of the GP's Anycast comment was that simply using 8.8.8.8 as your dns server is not sufficient to pinpoint WHERE your dns comes from.

      8.8.8.8 will resolve to different physical machines depending on the load balancing that Google is doing.

      Your dns request might be served out of California on one hit, and out of Ireland on the next hit.

      Akamai, by paying attention to where the DNS request came from is doing it fundamentally WRONG, because they could actually deny service (for national licensing reasons) based on location of the DNS server when the actual user was in a totally different (and legal) location.

      --
      Sig Battery depleted. Reverting to safe mode.
    3. Re:Opposite Experience with Adobe Download by TheRaven64 · · Score: 5, Insightful

      Not really. An HTTP redirect means that you make an initial connection to one server, and are then told that what you really want is on another. This adds a small amount of latency, but typically well under half a second. The original server is then not used for the remainder of the download. Akamai is typically used to serve large files - at least a few megabytes - so this extra hop doesn't add much overhead, and does make the geographic distribution much more efficient.

      Using the DNS server's IP to determine the address of the client is fundamentally broken. There are other cases where it can fail spectacularly, such as when you have a computer sitting on two networks - it always sends DNS requests on one and then picks the less-loaded network for other connections, so the DNS tells it to go to the right server for the DNS cache's network and the client uses the other network. You can also have serious problems with resolvers caching addresses in a laptop - if you move between two networks (e.g. 3G and WiFi) and the server's address is cached, you'll find that you're going to the wrong server.

      --
      I am TheRaven on Soylent News
    4. Re:Opposite Experience with Adobe Download by vrmlguy · · Score: 4, Insightful

      A HTTP redirect system requires extra http request to be made without even knowing where the client is. You could be making that request from South Africa to North America, adding big delay.

      An HTTP redirect is typically resolved in less than a second. If your download is going to take over a minute, that's less that 1% of your total download time.

      When 99.9% of internet users use their local ISP's DNS it just makes sense to build it like that. Sure it's not perfect for the odd geeks who have changed their DNS settings, but that is so small minority it doesn't make any sense to slow down their whole system.

      When your power users don't use their local ISPs, it makes sense to at least filter out those requests for special handling. Right now, if I were asked to chose a CDN, I'd have to recommend someone other that Akamai, if only because I use OpenDNS at home. If I were Akamai, I'd want to route Google and OpenDNS requests to servers that provide HTTP redirects based on the client's IP.

      Akamai has been in this business for a very long time and has their infrastructure on datacenters all over the planet. They know what they're doing.

      If they knew what they were doing, they wouldn't have this problem. Level 3 was able to grab Netflix, which indicates at least some customer dissatisfaction, and Limewire is growing at twice Akamai's rate. Do you work for Akamai, or are you a free-lance apologist?

      --
      Nothing for 6-digit uids?
  2. Namebench DNS tool by maggotbrain_777 · · Score: 5, Interesting

    This afternoon, I found a tool from Google Code called namebench which tests response times against multiple DNS servers and give recommendations based upon a number of query types. The results returned when checking the 'censorship tests' were interesting. Seems a number of sites (wikileaks, isohunt, stormfront) returned 'incorrect' results across DNS servers. I'm going to try this over the next couple of days and see if any of my browsing speeds improves.

  3. Re:Good advice - Always use your ISP for DNS by shentino · · Score: 4, Insightful

    Only if I trust them not to fuck with it.

  4. Re:You would think. by Timothy+Brownawell · · Score: 4, Informative

    They only find out your IP address after it's too late.

    1. Your computer asks a DNS resolver where the server is.
    2. The DNS resolver asks Apple's (well, Akami's) DNS server where the server is.
    3. The DNS server guesses the closest server, but all it has available to work with is the address of the resolver.
    4. Your computer uses that answer to contact the server and download whatever. If it was given the wrong server, it's too late now.
  5. Re:Good advice - Always use your ISP for DNS by MachDelta · · Score: 5, Funny

    MIL - I realized after a few seconds that probably stands for "Mother-In-Law", but the mechanic in me instantly interpreted it as "Malfunction Indicator Lamp."

    Shortly after that I had a chuckle upon realizing that they're both things no one likes to see.

  6. Re:Good advice - Always use your ISP for DNS by CheerfulMacFanboy · · Score: 5, Funny

    yeah, since you can't set secondary dns servers in any modern os...

    Sure you can, Primary and secondary are setup to openDNS on my router...

    Do they resolve wooosh.com?

    --
    Fandroids hate facts.
  7. Re:And how is this news? by xnpu · · Score: 4, Interesting

    BTW - Remember when Google proposed to modify the DNS protocol to pass on the end-users IP? This is exactly why.

  8. Re:Hrmmm by Desert+Raven · · Score: 5, Insightful

    No, it's not particularly elegant. But on the other hand, split-horizon DNS is nothing new or magical either. Nor would I classify it as "abuse". The capability has been there since the early days of BIND.

    In the DNS trade, we refer to it under the category of "stupid DNS tricks"

    That said, it does have some significant advantages over other techniques.

    #1, It's protocol-independent. Sure you can do intelligent redirects with HTTP, but not everything in the world is HTTP
    #2, Even with HTTP, in order for it to work, you have to now change the name of the server, and often the links to internal content. Your initial request to www.domain.com will now have to be redirected to hostx.domain.com or www.location.domain.com etc., and links on the pages to content servers will also have to be altered. This can be confusing to end-users, and may require additional SSL certs. It's also a code maintenance issue.
    #2a, While the renaming seems trivial on first glance, it has HUGE implications for search engines, etc, since those "local" servers will get indexed instead of a generic name
    #2b, It also means that a calculation will have to be made by the web server deciding where to redirect you to, then the actual redirect, increasing load and latency. DNS solutions are "pre-computed" and thus do not have similar issues.
    #2c, If you solve 2a by checking every request at every location, you make 2b much worse
    #3, It's simple.

    Downsides:

    #1, Third-party DNS recursive services throw it off. (There is a proposed RFC that would allow for such recursives to pass the originating network in the request)
    #2, It makes DNSSEC a right royal PITA (Much more than it already is)

  9. Re:Good advice - Always use your ISP for DNS by hazem · · Score: 4, Interesting

    Use your own ISP for DNS.

    Do you have any tips for keeping your ISP from directing a "server not found" to one of their crappy ad-ridden search pages? I think that's a major reason people choose DNS servers that aren't at their ISP.