Beware of Using Google Or OpenDNS For iTunes

Relayman writes "Joe Mailer wanted to download an iTunes movie recently and his Apple TV told him it would take two hours. When he switched his DNS resolver settings, the download time dropped to less than 20 seconds. Apparently, iTunes content is served by Akamai which uses geolocation based on the IP address of the DNS request to determine which server should provide his content. When you use Google or OpenDNS to resolve the Apple domain name, all the requests to Akamai appear to be coming from the same location and they're all directed to the same server pool, overloading that pool and causing the slow downloads. The solution: be wary of using Google or OpenDNS when downloading iTunes files or similar large files. Use your own ISP's DNS servers instead or run your own resolving DNS server."

Comments

Opposite Experience with Adobe Download

[eldavojohn]

First, I was able to verify this with the iTunes download. My Cox DNS was 20 seconds while my Google DNS was 2 minutes 10 seconds.

But I just tested this on my own by using a different source that uses Akamai: Adobe.

So I picked a file at this URL: http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.4.0/enu/AdbeRdr9.4-1_i486linux_enu.bin

Sure enough, the initial server directed me to 72.215.224.16 with this partial tracert:

4 12 ms 10 ms 10 ms mrfddsrj02gex070002.rd.dc.cox.net [68.100.0.145]
5 17 ms 14 ms 12 ms ashbbprj01-ae0.0.rd.as.cox.net [68.1.0.220]
6 12 ms 15 ms 12 ms 72.215.224.16

Firefox told me this would take 3 Minutes and 35 Seconds.

Then, I set my DNS to the 8.8.8.8 and 8.8.4.4 addresses and tried it again. This time I was sent to 72.246.30.19 with this partial tracert:

4 11 ms 12 ms 14 ms mrfddsrj02gex070002.rd.dc.cox.net [68.100.0.145]
5 13 ms 11 ms 13 ms ashbbprj01-ae0.0.rd.as.cox.net [68.1.0.220]
6 17 ms 17 ms 13 ms ge13-1.br01.ash01.pccwbtn.net [63.218.44.125]
7 21 ms 18 ms 12 ms akamai.ge13-4.br02.ash01.pccwbtn.net [63.218.94.142]
8 17 ms 18 ms 13 ms a72-246-30-19.deploy.akamaitechnologies.com [72.246.30.19]

Surprisingly, this second server that I was directed to using Google DNS only took 10 seconds to download the same file. I did it a second time and it took 30 seconds.

Now after restoring my default DNS resolution that URL continually directs me to 72.215.224.40 and the download is as speedy as the Google DNS. If I switch back to Google DNS it now continually directs me to 72.246.30.32 so you can see that there's some load balancing going here that apparently can be divvied up by geographic location for some of their customers. Apparently Apple needs to investigate the same solution that Adobe is using from Akamai. Which doesn't consider everything from Google DNS being fulfilled from a west coast replication server?

Re:Opposite Experience with Adobe Download

[sumdumass]

I though Google used Anycast just like the rest of the large providers. Perhaps it's a routing issue where Google's servers are separated a bit geographically from certain people and the servers they are wanting to connect to?

Re:Opposite Experience with Adobe Download

[sleeper0]

Two hours vs. instant streaming isn't a localization issue, you can easily stream 1-2mbps (or much more) from half way around the world. ~100ms in latency is nothing with a fat, non time sensitive stream like recorded video.\

It sounds like the specific POP the google DNS server is being fed is overloaded with traffic. It should be fairly easy for Apple to resolve the problem on their end, by simply not resolving to overloaded pops (they shouldn't ever anyway).

Other video cdn backed services (like netflix) don't suffer POP overloading on public DNS servers like GTE or open.

Re:Opposite Experience with Adobe Download

[JS_RIDDLER]

from this discussion http://tech.slashdot.org/story/10/05/29/1355252/How-CDNs-and-Alternative-DNS-Services-Combine-For-Higher-Latency there are some really good comments from OpenDNS http://slashdot.org/comments.pl?sid=1669064&cid=32389146

Re:Opposite Experience with Adobe Download

[wagnerrp]

It sounds like the specific POP the google DNS server is being fed is overloaded with traffic.

That sounds exactly what was surmised in the summary.

When you use Google or Open DNS to resolve the Apple domain name, all the requests to Akamai appear to be coming from the same location and they're all directed to the same server pool, overloading that pool and causing the slow downloads.

Re:Opposite Experience with Adobe Download

[fuzzyfuzzyfungus]

I'm not familiar with exactly what the options and price sheet are from Akamai, they aren't nearly as 'consumery' about it as Amazon EC2 is, more of a 'our rep will call you' sort of thing; but one wonders if Apple(who serves huge amounts of audio and video at relatively low prices, and presumably fairly low margins, is cheaping out in a way that Adobe isn't...

Presumably, Akamai uses their geolocation trickery because local deliveries are faster and cheaper. No need to traverse numerous hops, possibly controlled by others prickly about peering, and so forth. However, unless Akamai is secretly denser than a sack of hammers(which seems out of character) they should know that using the massively overloaded system, no matter how local, is going to be worse than using a more distant; but more lightly loaded, one. However, if there is a cost difference between local and distant, that isn't necessarily something that they would offer to their customers for free...

Re:Opposite Experience with Adobe Download

[bennomatic]

So here's an interesting element to this: corporate clients of hosted SAAS products which use CDNs for delivery. If I'm in the Munich office of company X which uses hosted software from company Y, and if my local DNS server is completely subordinate to the central nameservers in X's NYC datacenter (don't laugh; I've seen it happen), then chances are when I'm accessing Y's software via Akamai, I'm going across the pond to get to an Akamai POP in NYC, then going over the Akamai network to access Ysoft. Needless to say, the advantages of a CDN disappear at that point.

Re:Opposite Experience with Adobe Download

[rekoil]

The geographic information isn't the issue. It's the fact that there are a very large number of clients using the same pools of DNS resolvers. Akamai uses those resolvers' IP address to map the client to a cache pool; if there are too many requests from the same netblock, they'll all get sent to the same cache pool, overloading it.

At some point, Akamai's load feedback system will notice this and direct users to a different pool, but it's a reactive measurement.

Re:Opposite Experience with Adobe Download

[TooMuchToDo]

The problem is Akamai is using DNS to determine location, when it should be determining the geolocation of the client IP and throwing an HTTP redirect to the proper server. You can't rely on a client using the "proper" geographically-located DNS server.

Re:Opposite Experience with Adobe Download

[icebike]

Well the point of the GP's Anycast comment was that simply using 8.8.8.8 as your dns server is not sufficient to pinpoint WHERE your dns comes from.

8.8.8.8 will resolve to different physical machines depending on the load balancing that Google is doing.

Your dns request might be served out of California on one hit, and out of Ireland on the next hit.

Akamai, by paying attention to where the DNS request came from is doing it fundamentally WRONG, because they could actually deny service (for national licensing reasons) based on location of the DNS server when the actual user was in a totally different (and legal) location.

Re:Opposite Experience with Adobe Download

[eggnet]

Anycast is not a magic pill.

If you set up one DNS server on the east coast, another on the west coast, with anycast, it will still suck.

You'd need an anycast DNS server in every ISP in every metro area to match end users simply using their ISP provided local DNS servers.

Re:Opposite Experience with Adobe Download

[devxo]

Then Akamai would only be a load balancing service. It's purpose is also to serve you content fast and geographically close - doing HTTP redirect would add unnecessary hops, perhaps to the other side of the world.

Re:Opposite Experience with Adobe Download

[devxo]

But that is mostly an uncommon situation. 99.9% of internet users will use their ISP's or otherwise close DNS.

Re:Opposite Experience with Adobe Download

[icebike]

No, you don't need one within every ISP.

A few at major backbone hubs routing to your fastest least busy server is all you need.

Nobody would use anycast if it required a DNS server in EVERY ISP.

The point here is that routing a request to a server near the DNS server is fundamentally the wrong thing to do. You route to the server closest to the end user.

Re:Opposite Experience with Adobe Download

[the-it-guru]

But what if it's the ISP co-location network that's overloaded, not the Akamai server itself at that facility? (eg a Denial of Service attack against some other site hosted at the same ISP)...

Re:Opposite Experience with Adobe Download

[A1rmanCha1rman]

This is an answer to your sig rather than a response to your valid analysis of the DNS resolution / load balancing conundrum:

"Why do we want smartphones when there are so many stupid users?"

To balance the smart / stupid ratio.

Re:Opposite Experience with Adobe Download

[@madeus]

I assert the OP is correct and Akamai's approach is what's broken IMO.

Using geolocation based on the IP of a DNS server is bad approach IMO. Sure there are a lot less DNS servers out there than clients, but there are obvious problems with it and the over head of IP geo location look up and an HTTP direct per request is very small

A redirect system would use a tiny amount of bandwith, and easy to build a system to cope with, and you wouldn't neccerrily need to have it hosted all over the globe like they do with the cache servers, not if it's only doing HTTP 302/303 redirects. It would be simple to impliment and very robust too - much less likely to cause problems than something as crude as DNS server IP geolocation.

You could also have the network layer handle routing traffic to the 'least cost' route, which would be a really good way to do it - apart from it being more complicated to manage, not least given all the different providers Akamai have to do deal with, Though I'm guessing is that's what Google are doing in any case (and I remember managing after DNS servers hosted in different countries but with the same IPV4 addresses 10 years ago, so it's not as if it's a new concept).

Re:Opposite Experience with Adobe Download

[devxo]

A HTTP redirect system requires extra http request to be made without even knowing where the client is. You could be making that request from South Africa to North America, adding big delay.

When 99.9% of internet users use their local ISP's DNS it just makes sense to build it like that. Sure it's not perfect for the odd geeks who have changed their DNS settings, but that is so small minority it doesn't make any sense to slow down their whole system.

Akamai has been in this business for a very long time and has their infrastructure on datacenters all over the planet. They know what they're doing.

Re:Opposite Experience with Adobe Download

[TheRaven64]

Not really. An HTTP redirect means that you make an initial connection to one server, and are then told that what you really want is on another. This adds a small amount of latency, but typically well under half a second. The original server is then not used for the remainder of the download. Akamai is typically used to serve large files - at least a few megabytes - so this extra hop doesn't add much overhead, and does make the geographic distribution much more efficient.

Using the DNS server's IP to determine the address of the client is fundamentally broken. There are other cases where it can fail spectacularly, such as when you have a computer sitting on two networks - it always sends DNS requests on one and then picks the less-loaded network for other connections, so the DNS tells it to go to the right server for the DNS cache's network and the client uses the other network. You can also have serious problems with resolvers caching addresses in a laptop - if you move between two networks (e.g. 3G and WiFi) and the server's address is cached, you'll find that you're going to the wrong server.

Re:Opposite Experience with Adobe Download

[rjstanford]

The point here is that routing a request to a server near the DNS server is fundamentally the wrong thing to do. You route to the server closest to the end user.

And how do you do that? After all, they've already made their request and you have to have something answer it. Sure, you could tell them to talk to someone else, but that's a potentially very slow bounce (at internet speeds). Aha! When they connect and ask for the file, that's actually the 2nd time they communicate with you. The first time they provide a string, an "address" so to speak, and ask machines that you control what IP address they should talk to. Perfect! That even has to be a separate request so we're not adding latency. You can figure out where they are through that lookup request, get their IP (or the IP of their immediate upstream, same thing for this purpose), and give them the best IP address for their 'net location.

Oh, that's what they're doing already? Yeah... maybe, just maybe, they're smarter than random /. armchair quarterbacks are at this. If you start routing your internet requests through geo-random proxies, services that try to help you based on your geolocation are going to be confused. There's no way around that. The fact that you're just proxying some of your traffic only means that just some services are going to be confused. Still your (collective) fault.

Re:Opposite Experience with Adobe Download

[AmberBlackCat]

Akamai, by paying attention to where the DNS request came from is doing it fundamentally WRONG, because they could actually deny service (for national licensing reasons) based on location of the DNS server when the actual user was in a totally different (and legal) location.

So if we find a DNS server in the right country then we might be able to download videos that are blocked in our own country?

Re:Opposite Experience with Adobe Download

[Anonymous Coward]

Akamai, by paying attention to where the DNS request came from is doing it fundamentally WRONG, because they could actually deny service (for national licensing reasons) based on location of the DNS server when the actual user was in a totally different (and legal) location.

Not necessarily. After the DNS address is resolved to an IP, the client sends its HTTP request to that IP, and location-based policy can be applied at that point based on the actual client IP.

There are still complications, like what to do if the client sends an (easily forged) X-Forwarded-For header. Do you apply your location-based rules based on the connecting IP or the claimed original IP? But these are questions for the content owner rather than technical problems.

Re:Opposite Experience with Adobe Download

[Midnight Thunder]

We ran into this sort of issue with a high availability setup. The nice thing with the DNS approach is that it will always be www.myco.com, no matter the IP it resolves to. On the other hand if you delegate this to application server then you have to redirect to a server with a different name, such as www1, www2, etc

If we had been dealing with non transactional stuff where state was in the cookie the first approach would have worked, but we weren't and we had to ensure the browser was always talking to the same server, so we ended up using the second approach.

One other thing is that there is a certain assumption that there is only one DNS server between you and a root server and that none of them cache.

Maybe an alternative would to have DNS servers support geographical zones for the IP addresses, so it is the client that can decide which server to speak to. There would have to be a notion of default zone for compatibility and fallback.

Re:Opposite Experience with Adobe Download

[Anonymous Coward]

Well the point of the GP's Anycast comment was that simply using 8.8.8.8 as your dns server is not sufficient to pinpoint WHERE your dns comes from.

8.8.8.8 will resolve to different physical machines depending on the load balancing that Google is doing.

Your dns request might be served out of California on one hit, and out of Ireland on the next hit.

Akamai, by paying attention to where the DNS request came from is doing it fundamentally WRONG, because they could actually deny service (for national licensing reasons) based on location of the DNS server when the actual user was in a totally different (and legal) location.

No, Akamai is not "doing it wrong". They keep at least one server cluster co-located with your ISP, usually they'll have one in each market area. When you lookup off your ISP's DNS it will route your request through their local Akamai server, which is only handling load from your ISP. If you use a 3rd party service like OpenDNS it has no ability to know about the private Akamai server and will either send you to a public one or just direct to the source. It's your ISP which is deciding which server you hit, not Akamai.

Akamai uses pre-emptive site caching based on worldwide usage patterns, so it's not so much just a matter of the size of the file, but the popularity of the site and the content. They will actively push content out to local Akamai cache servers when pattens shift. Their business model is based on helping your ISP keep traffic from hitting their peering and transit links, thus reducing bandwidth costs.

Re:Opposite Experience with Adobe Download

[Anonymous Coward]

No, you don't need one within every ISP.

A few at major backbone hubs routing to your fastest least busy server is all you need.

Nobody would use anycast if it required a DNS server in EVERY ISP.

The point here is that routing a request to a server near the DNS server is fundamentally the wrong thing to do. You route to the server closest to the end user.

No, you're not understanding how Akamai caching works. They aren't the ones making any decisions about which cache server you hit for starters- it's the DNS provider which decides which Akamai server IP to return when you send them a query. Now, I don't know if OpenDNS, or Google, use Akamai servers at all. The way Akamai does business is they setup a co-location deal with your ISP (yes, every ISP they can find), and drop a caching cluster into various locations. The server caching prevents a lot of duplicate traffic and can massively reduce load on transit, backhaul, and peering circuits within your ISP's network; they don't handle external traffic just your ISP's local load.

So what usually happens is that your ISP's DNS sends you to the Akamai server which is closest to you- a private cluster on your ISP's network. When you do a lookup off a 3rd-party DNS server, it really doesn't know much about where you are coming from based on your IP address, so it has to go with what it does know- it's own location.

Re:Opposite Experience with Adobe Download

[calzones]

They are doing it wrong in the sense that this is only a historically decent way of determining location and distributing load accordingly.

As we move forward, we will become far less likely to rely on our ISPs for such things. Using google DNS or OpenDNS has become a much bigger deal than just a handful of geek elites and is becoming widespread.

Count on it gaining even more traction with the general public the more the ISPs try to shape our packets.

And finally, there's no reason to expect that we will be still using the same method to resolve addresses forever. More and more of the internet is moving toward distributed/decentralized/p2p service and the adoption of mobile devices will probably accellerate that process.

Sure, Akamai had a neat idea at first, but they need to work it out with services like OpenDNS for a more reliable, forward-looking solution.

Frankly, any process that makes assumptions is flawed. Sometimes it's necessary, but when the pitfalls become revealed, the correct response is NEVER "oh, but the assumptions are good, the people are just doing it wrong and violating the assumptions"... the correct response is "look like we have to change our assumptions. we have new information. let's find a better way to do this."

Re:Opposite Experience with Adobe Download

[vrmlguy]

A HTTP redirect system requires extra http request to be made without even knowing where the client is. You could be making that request from South Africa to North America, adding big delay.

An HTTP redirect is typically resolved in less than a second. If your download is going to take over a minute, that's less that 1% of your total download time.

When 99.9% of internet users use their local ISP's DNS it just makes sense to build it like that. Sure it's not perfect for the odd geeks who have changed their DNS settings, but that is so small minority it doesn't make any sense to slow down their whole system.

When your power users don't use their local ISPs, it makes sense to at least filter out those requests for special handling. Right now, if I were asked to chose a CDN, I'd have to recommend someone other that Akamai, if only because I use OpenDNS at home. If I were Akamai, I'd want to route Google and OpenDNS requests to servers that provide HTTP redirects based on the client's IP.

Akamai has been in this business for a very long time and has their infrastructure on datacenters all over the planet. They know what they're doing.

If they knew what they were doing, they wouldn't have this problem. Level 3 was able to grab Netflix, which indicates at least some customer dissatisfaction, and Limewire is growing at twice Akamai's rate. Do you work for Akamai, or are you a free-lance apologist?

Re:Opposite Experience with Adobe Download

[calzones]

Why not just assign each Apple TV mac address to a download pool?

Re:Opposite Experience with Adobe Download

[CastrTroy]

Because MAC addresses are only used at layer 2, and don't travel the whole length of the packet. The MAC address changes between each router the packet passes between. The MAC address seen by the Apple servers are just the MAC Address that apple has on their own routers in the datacenter. Apple has no way of seeing the initial MAC address from the Apple TV unit initiating the request.

Re:Opposite Experience with Adobe Download

[Shin-LaC]

An HTTP redirect is typically resolved in less than a second. If your download is going to take over a minute, that's less that 1% of your total download time.

But if you're using Akamai to serve web page content, one second is a huge delay. Adding one second to the download of a CSS stylesheet is likely to have a bigger impact on browsing speed than all of the JavaScript improvements browser makers have been boring us with this year.

Re:Opposite Experience with Adobe Download

[calzones]

It was mostly a rhetorical question. Knowing Apple, they would go by serial number instead. Or they could go by iTunes account.

The point being: presumed location is not a good way to load balance. Balance your load based on either random distribution, or better yet, planned distribution.

The easiest and cheapest way (not the best) to do this is simply give each new AppleTV owner an ID and assign them to a pool and make sure all your pools are ideally populated.

Re:Opposite Experience with Adobe Download

[icebike]

Look, if Google can use anycast, why do you find it such a stretch that Akamai couldn't as well?

Re:Opposite Experience with Adobe Download

[CastrTroy]

The problem here being that users might be more or less active. It might just so happen that a lot of really active users get put on the same node. And then a bunch of inactive users get placed on another node. So then you end up with an over-utilized node and and under-utilized node. So what you would really have to do is have real time assigning of clients to different nodes. This is what Akamai is supposed to do. The problem is that they assign clients to nodes based on the location of their DNS Server. They should really have a better approach, and I assume they will come up with something better in the near future. I'm not sure why they would assume that somebody's DNS Server is geographically close to the client. It isn't the case in many situations. Although now the problem has gotten out of control, because not only is the DNS server not geographically close to the clients, but a single DNS server is being used for a very large number of clients. The same thing could have happened if a large ISP decided to use a single DNS server for all their clients, much the same way google DNS is acting as a single large DNS server.

Re:Opposite Experience with Adobe Download

[icebike]

That is a distinct possibility with the way Akamai is doing business, yes.

Can't say for sure that this happens, there may be subsequent checks in place.

But you are generally free to use any DNS servers you want, and most ISPs will serve DNS both ON and OFF their network. Having been saddled with an ISP with dodgey DNS servers I have often simply substituted IPs for a fast ISP in another state into my resolv.conf.

Re:Opposite Experience with Adobe Download

[calzones]

We are 100% in agreement.

Location is highly problematic not just because DNS is an unreliable indicator of location, and not just because location is really practically meaningless on the internet... but because even if it were reliable and meaningful, locations have a bad way of suffering from demand overload of resources (internet or otherwise) due to regional events (sporting events, holiday celebrations, natural disasters, weather patterns, etc).

I expect the future will bring even more distrubted/decentralized access by way of p2p connectivity and discovery. By then, using an ip or dns server to determine resource allocation will be totally silly.

The system needs to be self-adapting and enable a near-real-time load re-distribution based on optimizing demand vs measured latency every so often. New requests should habitually be routed to servers with the lowest activity.

FWIW, there are often times when I'd rather simply be refused access until a decent node is available as opposed to put up with a node that can't keep up. It's worse with Apple TV because once you start streaming, the clock is ticking.

Re:Opposite Experience with Adobe Download

[windex]

I have not read a more ignorant comment on how DNS works on slashdot in my entire history of reading slashdot.

Akamai is doing DNS geolocation. The solution is to use a combination of DNS geolocation combined with http redirects (also based on client IP geolocation) to attempt to find a good close server. If an end-user is using a remote DNS server. This can even be mostly invisible with a CDN like Akamai. DNS servers do not decide which 'Akamai' IP to give anyone. Akamai's forward resolving DNS servers return a response they have crafted as 'close' to the requesting DNS caching server (e.g. Google DNS, OpenDNS, your ISP, etc). The caching server caches the result and sends it to the DNS client on your local system. End of story. DNS does not forward or proxy the DNS client's address through the caching server to the forward resolving server. Ever.

For the record, using Akamai DNS *without* their CDN service (e.g. load balance/geolocate only) when redundant sites, AnyCast, and BGP should be standard operating procedure in enterprise network deployments is fucking stupid.

Re:Opposite Experience with Adobe Download

[MintyGreenMedia]

Akamai has been in this business for a very long time and has their infrastructure on datacenters all over the planet. They know what they're doing.

Based on the "IPv6 is hard!" whining I've heard from their camp, I'm not so convinced they still know what they're doing.

Re:Opposite Experience with Adobe Download

[Anonymous Coward]

i'm sorry, but you are an idiot.

Dns is not proxying anything, it's requesting an ip address for a name. period.

If I want to use a caching dns server thats 1 hop from the backbone instead of the cacheless nightmare my isp tries to run, I should not be punished for it.

Akamai is doing it wrong, period.

Their service is primarilay used for LARGE FILES, where an extra 100ms of transaction delay at the start isn't going to hurt you.
Akamai was initially used to lower the latency on high volume web pages, and they are still using the exact same technology for high bandwidth needs. This is a mistake.
Additionally, they should damned well have the ability to monitor how much traffic they are sending to each server.
It doesn't matter if I have a 1ms ping time if i'm pulling a 5GB file, the throughput is more important.

Re:Opposite Experience with Adobe Download

[ottothecow]

+1

There was a (longish) comcast DNS outage in the chicago area not that long ago (maybe it was even more widespread).
The only reason I know this happened is that I signed on to facebook and saw dozens of people who would never touch a network control panel spreading messages with concise instructions on how to "get the internet back" by changing to the google DNS servers.

It is highly unlikely that these people went back in and changed back to the automatic comcast DNS servers once service was restored...now you have a bunch of people who could run into this akamai problem but have no idea what it really means to change your DNS settings (and who will be loads of fun for the iTunes support staff when they call to ask why netflix plays instantly but itunes wants 2 hours to start a streaming movie)

Re:Opposite Experience with Adobe Download

[Professor_UNIX]

Cox's Akamai node they host on their internal network is massively overloaded by all the Cox users that hit it. When I first changed to Cox last year I started noticing massive buffering when trying to access Akamai sites like Apple's movie trailer's site or even just updates for my iPhone. I was barely getting 512 Kbps downloading stuff. The minute I switched over to using Level 3's DNS servers my download times cut to a fraction of what it used to take and I was getting 20 Mbps.

I reported all this information along with detailed traceroutes and packet dumps and what was their response? "Reboot your computer. If that doesn't resolve the situation you may need to reset your cable modem." That was it. They totally ignored all the analysis and just told me the problem was on my end. I haven't used their DNS servers since.

Re:Opposite Experience with Adobe Download

[rekoil]

By all accounts, Level 3 got the contract primarily on price, not features - Akamai is typically the most expensive CDN out there.

IMO Netflix has enough traffic volume that they could build their own CDN and probably pay even less than what they're paying Level 3. Even if it costs more, the internal control over the thing may be worth the delta. Several other large sites have done exactly that already.

Re:Opposite Experience with Adobe Download

[TooMuchToDo]

The trick would be to use HTTP redirects for large files where 1-2 seconds of initial latency isn't going to cause a problem, but still use the DNS/UDP/Anycast method for small files where you don't care if they're load balanced or not.

Re:Opposite Experience with Adobe Download

[TooMuchToDo]

From what I've heard from internal Netflix IT folks, it's extremely dysfunctional in there, which may explain why it's easier for them to just cut a check and have someone else do it.

Now, if someone with IT competency were to purchase Netflix (Google? I mean, their job is to organize the world's information, and movie content *is* information), I'm sure you'd see a swift internal CDN roll out occur.

Re:Opposite Experience with Adobe Download

[Anonymous Coward]

An HTTP redirect is typically resolved in less than a second. If your download is going to take over a minute, that's less that 1% of your total download time.

I'm intentionally missing the point, but you fail at math here.

How is "less than a second" automatically less than 1% of "over a minute" when a second is 1.6% of a minute?

Re:Opposite Experience with Adobe Download

[rekoil]

From what I've heard from internal Netflix IT folks, it's extremely dysfunctional in there, which may explain why it's easier for them to just cut a check and have someone else do it.

Now, if someone with IT competency were to purchase Netflix (Google? I mean, their job is to organize the world's information, and movie content *is* information), I'm sure you'd see a swift internal CDN roll out occur.

I've heard similar, along with some highly visible departures in their systems/network engineering groups rumored to be due to exactly those decisions.

Speaking of cutting checks, has anyone noticed that the movies.netflix.com site is actually hosted by Amazon AWS?

macbook:~$ host movies.netflix.com
movies.netflix.com is an alias for merchweb-frontend-1502974957.us-east-1.elb.amazonaws.com.
merchweb-frontend-1502974957.us-east-1.elb.amazonaws.com has address 204.236.218.69

Re:Opposite Experience with Adobe Download

[Vrtigo1]

An HTTP redirect is typically resolved in less than a second. If your download is going to take over a minute, that's less that 1% of your total download time.

Over a minute means > 60 seconds, so 1 / 61 = .0164 = 1.6%, so while I appreciate the point you're trying to convey and I realize I'm just being nit-picky, your math isn't exactly 100%, no pun intended.

Re:Opposite Experience with Adobe Download

[Vrtigo1]

Mod parent up, he's got it exactly right. There are many problems with doing your geolocation at the DNS level rather than the client level. One prevalent example that springs to mind is a corporation with remote users. If the users are connected to a VPN that does split-tunneling, they're likely to be doing their name resolution via internal DNS servers, which are almost certainly on a different ISP's network and geographically not as close to the client as you could get by doing a geolookup on the client's IP. There is a performance hit by doing an HTTP redirect, but as others have said, it's very small when you're talking about a file of any significant size.

Good advice - Always use your ISP for DNS

[crusty_architect]

This is a very widespread practice now. Use your own ISP for DNS.

Re:Good advice - Always use your ISP for DNS

[jaymz666]

yeah, cause comcast and at&t never have DNS outages... Last month Comcast had a huge DNS outage, I didn't even notice it since I have been using openDNS for years. My MIL called me up saying her internet was down, I had her ping some IPs and they worked, but DNS didn't. Changed her over to opendns and it worked fine after that.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

Unless you have Road Runner, who can't seem to keep a resolving DNS infrastructure online reliably.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

Use your own ISP for DNS.

No can do. My ISP's DNS redirects unresolvable queries to a bunch of ads. Google's DNS works much better, and is easier (8.8.8.8) to remember.
It may work fine for Apple TV, whatever that is, but otherwise not so much.

Re:Good advice - Always use your ISP for DNS

[shentino]

Only if I trust them not to fuck with it.

Re:Good advice - Always use your ISP for DNS

[0100010001010011]

Too bad modern OSes only have a spot for a single DNS server. Otherwise you could add multiple.

Add Multiple.
Drop Timeout Time.
Enjoy.

If Comcast goes down, I'll fail over to Verizon/Google. If Comcast is up it knows my location.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

yeah, since you can't set secondary dns servers in any modern os...

Re:Good advice - Always use your ISP for DNS

[omglolbah]

Um, I know it is early (6am here for me...) but which modern OS only supports one server?...

Win7 supports a long list if you so desire, as does linux...

Or did you typo? :p

Re:Good advice - Always use your ISP for DNS

[jaymz666]

yeah, so using the DNS servers from comcast that are provided by DHCP never change for you, huh. Troubleshooting that mess is a pita.

Re:Good advice - Always use your ISP for DNS

[jaymz666]

Sure you can, Primary and secondary are setup to openDNS on my router...

Re:Good advice - Always use your ISP for DNS

[MachDelta]

MIL - I realized after a few seconds that probably stands for "Mother-In-Law", but the mechanic in me instantly interpreted it as "Malfunction Indicator Lamp."

Shortly after that I had a chuckle upon realizing that they're both things no one likes to see.

Re:Good advice - Always use your ISP for DNS

[devilspgd]

Which is fine, except that you'll hit the "wrong" Akamai servers for your network.

My question is why Akamai is using DNS for their geo-load-balancing rather than anycasting the content servers themselves.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

I'm guessing they don't have sarcasm where you're from.

Re:Good advice - Always use your ISP for DNS

[TheRealGrogan]

Umm no, I think I'll just pass on those services if they are that daft, thanks.

Fuck akamai... if any software delivery system or service is slow for me because of content distribution tomfoolery, I simply won't use it. I would never have anything to do with iAnything in the first place, though.

Most ISP's DNS servers suck... and the whole reason I started using OpenDNS is because the ISP's were slow to respond, and the primary was often out and there were delays until the resolvers queried the secondary.

Hell, even my ISP's DNS servers that I would otherwise get assigned aren't exactly local.

A big, fat, monopolistic communications company that didn't get broken up on our side of the fence (in Canada) that doesn't care about their customers. Unfortunately it's the best Internet connection (DSL) I can get where I live. I could throw a stone and hit houses on nearby streets that have fiber, but they aren't bringing it to me because there's nothing but dead people that live on my street. (and the rest are just summer cottages)

DNS is only meant to be used for resolving hostnames and IP addresses. Any other inference people choose to make from any part of it for any purpose is wrong.

Re:Good advice - Always use your ISP for DNS

[camperdave]

Um, I know it is early (6am here for me...) but which modern OS only supports one server?...

Grab your morning coffee and fire up the sarcasm detector.

Re:Good advice - Always use your ISP for DNS

[CheerfulMacFanboy]

yeah, since you can't set secondary dns servers in any modern os...

Sure you can, Primary and secondary are setup to openDNS on my router...

Do they resolve wooosh.com?

Re:Good advice - Always use your ISP for DNS

[a_nonamiss]

I guess everyone's mileage varies. I've been using RoadRunner in Central Ohio since 1998. (I was a residential beta tester back when you had to install a RoadRunner client and "sign in" to a proxy server using a Kerberos token.) I've had both residential and business class service, and I can only recall one DNS outage which lasted about an hour. Now, I won't say there weren't other outages. We had one winter where the physical circuit went down 5 times in 2 months due to weather-related problems. But DNS has always been rock solid for me.

Re:Good advice - Always use your ISP for DNS

[fast turtle]

and that's one of the reasons I added the Google DNS server as a secondary to my router. Solves the damn problem when TW/RR suffers another DNS failure.

Re:Good advice - Always use your ISP for DNS

[HeronBlademaster]

Use your own ISP for DNS.

When you first get a Comcast account, before you've registered your modem's MAC address with them, they give you an IP address but the DNS server they give you always points you at their registration server. Trouble is, the database that the DNS server reads out of can sometimes get out of sync with what modems are actually registered, and there's nothing Comcast's first- or second-level techs can do about it other than to tell you how to set your DNS servers manually to something else (they'll give you the IPs of the regional Comcast DNS servers). (This happened to my dad when he signed up.)

So... you'll forgive me if I'm wary of using the DNS servers my ISP gives me.

Re:Good advice - Always use your ISP for DNS

[Demonoid-Penguin]

This is a very widespread practice now. Use your own ISP for DNS.

Who's your ISP? Do they operate in Australia?

(currently) 3 Mobile, and Vodaphone, have very slow, and unreliable, DNS.

google.com and google.com.au averaging about 350 ms, at present neither ISP is able to resolve their own address (sigh) ping three.com.au PING three.com.au (203.37.69.133) 56(84) bytes of data. ^C --- three.com.au ping statistics --- 25 packets transmitted, 0 received, 100% packet loss, time 2015ms and that's while using a three connection! (how does that work?)

BigPuddle doesn't resolve address URL's that google DNS does (wikileaks, piratebay, and others)

I use a DNS cache on the firewall, pointed at googledns. I'll have to try opendns and compare.

Dunno about iTunes...

Re:Good advice - Always use your ISP for DNS

[d6]

my (former) ISP took a looooong time to patch that poisoning exploit last year. I quit using them for DNS then.

Re:Good advice - Always use your ISP for DNS

[wagnerrp]

We have a business class RoadRunner line in south west Ohio, and their DNS servers have all sorts of problems.

Re:Good advice - Always use your ISP for DNS

[aiht]

ping three.com.au
PING three.com.au (203.37.69.133) 56(84) bytes of data.


^C
--- three.com.au ping statistics ---
25 packets transmitted, 0 received, 100% packet loss, time 2015ms

But that did resolve.
I'm confused... are we talking about DNS lookups or ping?

Anyway, three.com.au doesn't respond to ICMP Echo Requests, no matter which ISP you're connected to. Many sites don't.*
If you want to test connectivity to a website, telnet to port 80.

* iinet.net.au, westnet.com.au, internode.on.net, bigpond.com.au - all allow pings.
three.com.au, vodaphone.com.au, virginmobile.com.au, telstra.com.au - all block pings.

Re:Good advice - Always use your ISP for DNS

[Z00L00K]

I already do, and since my ISP censors the internet through their DNS there is no alternative to go back to them.

And a cleaned up version of my config. It doesn't involve the ISP at all but queries the root servers on the net instead.

And as long as the ISP:s doesn't filter the DNS requests to the root servers this is the way to go right now.

options {
                allow-query {
                                127.0.0.1;
                                192.168.0.0/16;
                };
                directory "/var/named";
                pid-file "/var/run/named/named.pid";
                recursion yes;
                dnssec-validation no;
};

key mykey. {
                algorithm HMAC-MD5;
                secret "** Secretas... ***";
};

zone "." {
                type hint;
                file "root.hints";
};

zone "int.anon.org" {
                type master;
                allow-update { key mykey.;};
                file "int.anon.org.db";
                notify yes;
};

zone "1.168.192.in-addr.arpa" {
                type master;
                allow-update { key mykey.;};
                file "1.168.192.db";
                notify yes;
};

zone "localdomain" {
                type master;
                file "localhost.db";
                notify no;
};

zone "0.0.127.in-addr.arpa" {
                type master;
                file "0.0.127.db";
};

zone "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa" IN {
                type master;
                file "ip6.local.db";
                allow-update { none; };
};

Re:Good advice - Always use your ISP for DNS

[Z00L00K]

Sorry - It should have read use your own DNS...

Re:Good advice - Always use your ISP for DNS

[Z00L00K]

Which they do for me, so I run my own DNS.

Re:Good advice - Always use your ISP for DNS

[crusty_architect]

Anycast == Hard, DNS == Easy...

Re:Good advice - Always use your ISP for DNS

[Demonoid-Penguin]

yeah, since you can't set secondary dns servers in any modern os...

:-D

You're aiming too high for this crowd.... ;-p

Re:Good advice - Always use your ISP for DNS

[crusty_architect]

BigPond actually has the most robust DNS infrastructure in Australia..

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

Fuck akamai... if any software delivery system or service is slow for me because of content distribution tomfoolery, I simply won't use it.

You funny. You use Akamai in 90% of the places you think you do, and 80% of places you think you don't. Nice idea though.

Re:Good advice - Always use your ISP for DNS

[mysidia]

This is a very widespread practice now. Use your own ISP for DNS.

Your ISP uses the same DNS servers nationwide.. so how exactly do you think this is better than using Google or OpenDNS DNS servers?

Re:Good advice - Always use your ISP for DNS

[mysidia]

No can do. My ISP's DNS redirects unresolvable queries to a bunch of ads.

How long do you think it will be before your ISP uses dynamic packet inspection to transparently intercept DNS response packets you receive from 8.8.8.8 and replace NXDOMAIN responses with a 'fake' A record response for their ad servers?

Re:Good advice - Always use your ISP for DNS

[jaymz666]

They resolve it to some parked site.

Re:Good advice - Always use your ISP for DNS

[jaymz666]

Excellent question, a method that uses your actual location instead of where your DNS server is.

Re:Good advice - Always use your ISP for DNS

[KibibyteBrain]

Why use your ISP for DNS? Chances are their servers suck, and they will insert spam links for failed resolutions to add insult to injury for their horrible service. Find a server that is 1) geographically close and 2) measurably performs well. I personally use this tool for locating a DNS server that measurably works well with my connection: http://www.grc.com/dns/benchmark.htm .

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

Because anycast doesn't work well with stateful protocols (like TCP and HTTP).

Consider what would happen if the route changed in the middle of a download: you'd start talking to a server you had no session with.

Re:Good advice - Always use your ISP for DNS

[jaymz666]

if you're on a home network chances are pretty high your router is acting as your DNS server or setting the DNS servers for your machine via DHCP.

Maintaining DNS primaries, secondaries, tertiaries etc. on more than one or two machines can be a bit of a PITA too.

Re:Good advice - Always use your ISP for DNS

[Demonoid-Penguin]

But that did resolve.

You are (obviously) correct, and yes "we" are talking about DNS. You are not the source of the confusion....I pasted the wrong bit of bash history there, (that IP came from the IPCop cache).

The DNS address/es supplied by 3 Mobile show are currently 202.124.76.66 & 202.124.68.130 - I don't know if they are the ones I've always gotten. Using them I cannot resolve my.three.com.au to check my account.

Thanks for reminding me about ICMP, both 3Mobile and Vodadphone having been driving me nuts the last few weeks - dropouts, P2P filtering, account balance weirdness (somehow credited 1.4G last week!). Vodaphone currently resolves as does three (intermittantly), my.three.com.au doesn't (though the IP I have for it works). (I'm being told I should look at vodafail.com(?), and that both ISP are now one, or similar, as I try and write this).

telnet three.com.au telnet: could not resolve three.com.au/telnet: Name or service not known with clean dns cache

$ telnet 203.37.69.133 Trying 203.37.69.133... ^Cusing correct address works

Googledns works for everything except my.three.com.au

I shall now go and read up this site that supposedly will explain the problems, and take my confusion with me. Cheers

Re:Good advice - Always use your ISP for DNS

[FoolishOwl]

You can override the assigned nameservers at your router or in your OS.

Re:Good advice - Always use your ISP for DNS

[jaymz666]

Sure you can, and managing that for more than a couple of PCs is a PITA.

Re:Good advice - Always use your ISP for DNS

[Khyber]

"Consider what would happen if the route changed in the middle of a download: you'd start talking to a server you had no session with."

Isn't that what the header packets are for? So each packet knows where it came from and where it's going?

Re:Good advice - Always use your ISP for DNS

[hazem]

Use your own ISP for DNS.

Do you have any tips for keeping your ISP from directing a "server not found" to one of their crappy ad-ridden search pages? I think that's a major reason people choose DNS servers that aren't at their ISP.

Re:Good advice - Always use your ISP for DNS

[Khyber]

RR top-tier residential service in Redlands, CA. (15/2)

I have to reset the modem multiple times daily, or the router, because DNS is consistently fucked.

Re:Good advice - Always use your ISP for DNS

[Khyber]

How long do you think it will be before the first lawsuit happens over that, considering redirects without notification are against the rules as it's considered a function of spyware?

Re:Good advice - Always use your ISP for DNS

[Khyber]

It's blacklisted in my router at the root domain level.

Slashdot runs so much faster, now.

WHY MUST YOU LOAD SOMETHING WHEN I'M CLOSING YOUR TAB, SLASHDOT?

Seriously, that's a bunch of bullshit.

Re:Good advice - Always use your ISP for DNS

[Demonoid-Penguin]

BigPond actually has the most robust DNS infrastructure in Australia..

I have no means of knowing whether that is correct or not, and no opinions on it either. I have worked for Telstra (as a complex data tester - so I know a little of the infrastructure), but not TeleTech (BigPond).

Dunno how robustness came into this - I simply pointed out that BigPuddle dns "appears" "selective" about which addresses it resolves.

Mighty tempting to go way OT on this - but I need to have a quick read up on what's happening to my ISP's before dinner...

Re:Good advice - Always use your ISP for DNS

[nabsltd]

Sure you can, Primary and secondary are setup to openDNS on my router...

I really don't understand why such a high percentage of /. readers use anything other than their own DNS server (i.e., a DNS server in or behind their router).

It's insanely trivial to install a caching DNS resolver on just about any OS and there is also custom router firmware that does this.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

Some of us prefer NXDOMAIN responses instead of getting hijacked to a splash page full of advertising.

Re:Good advice - Always use your ISP for DNS

[crusty_architect]

So standard question. Why are you throwing unresolvable queries at your ISP's DNS??

Re:Good advice - Always use your ISP for DNS

[mibus]

Isn't that what the header packets are for? So each packet knows where it came from and where it's going?

Anycast breaks that though, because all servers in the anycast pool share the same address, so it confuses the server & clients when you add and remove servers from a pool.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

I just use my own. Query root servers, then query whatever. Faster than ISP. And no one controls it directly except myself.

IF you use OpenDNS or Google DNS or whatever, you are as prone to tampering as my own dns is.

Re:Good advice - Always use your ISP for DNS

[mibus]

Why use your ISP for DNS? Chances are their servers suck

I'd really consider seeing if you can find a better ISP (if it's possible in your area).

IMVHO, It's not too much to ask of an ISP that they have reliable and uninfested DNS.

(Disclaimer: I am in the sysops team that maintains resolvers at an ISP in .au)

Re:Good advice - Always use your ISP for DNS

[Vectormatic]

yup, and while some routers support manually setting a DNS, the integrated router/dsl modem i got with my new sub doesnt.. and i kind of hate the idea of putting a router behind my router for that kind of stuff (makes me feel like xzibit if you know what i mean)

and besides, its apple TV, who gives a frack? any card carying compu-nerd is gonna have his own multi-TB NAS setup with video-streamers under each TV

Re:Good advice - Always use your ISP for DNS

[RobertM1968]

This is a very widespread practice now. Use your own ISP for DNS.

I prefer using a DNS provider who doesn't serve me a Yahoo powered by Bing search page if I try going to a bad URL - unlike my "own ISP".

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

OpenDNS is fucking evil. Any DNS service that resolves non-existent domain names breaks all sorts of valid software.

Re:Good advice - Always use your ISP for DNS

[RobertM1968]

Anycast == Hard, DNS == Easy...

DNS also equals "stupid method of doing it". While most/many large ISPs have DNS servers geographically located, a number still do not. Heck, back in the 90's, UUNet ran three "public" (for their customers) DNS servers... if memory serves, they were Virginia, Chicago and Cali. Customers would use the closest two. Now... this was an ISP that was AlterDial, MSN's backbone and dial-in, AOL's backbone and dial-in, and, at the time, battling for the largest Internet backbone against IBM.

Don't know why they'd use a method known to be broken for various implementations/setups/ISPs and hope it works with larger ISPs' customers. Makes no sense at all.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

What kind of two bit "modern" OS are you running? Windows lets me specify as many DNS servers as I want.

Re:Good advice - Always use your ISP for DNS

[RobertM1968]

So standard question. Why are you throwing unresolvable queries at your ISP's DNS??

Hmmm... here's a few guesses:
(1) typo
(2) bad link from another site
(3) dead link on a search engine
(4) checking a domain that's been registered to see if it's active, parked or pointing nowhere

I'm sure there are other reasons...

Re:Good advice - Always use your ISP for DNS

[Krneki]

Because I trust Goggle/OpenDNS much more then I trust our politicians.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

Use your own ISP for DNS.

Unless you live in a country where ISPs have to censor and filter the web, like China or soon all of Europe.

Re:Good advice - Always use your ISP for DNS

[Evets]

The geo location on my ip has always been pretty accurate - within 25 miles. Running my own dns server alleviates the problem and keeps me away from relying on my ISP. (which accounts for at least half the downtimes I experienced prior to spending the 15 minutes it took to set it up in the first place.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

> Do you have any tips for keeping your ISP from directing a "server not found" to
> one of their crappy ad-ridden search pages?

Move your custom to a better ISP.

Re:Good advice - Always use your ISP for DNS

[FooBarWidget]

Yeah, "insanely trivial" to everybody who know all about DNS and its terminology ("zones", "authoritative server", "slave", etc) as well as moderate Unix skills. I develop Unix server software and I get questions all the time from people who don't even know what $PATH is and how it works. If you call installing a DNS server "insanely trivial" then you have no idea what that word means in the context of 99.9% of the world population.

Re:Good advice - Always use your ISP for DNS

[beelsebob]

But why would you? Even if it's easy, it's more effort than it's worth.

Re:Good advice - Always use your ISP for DNS

[JonJ]

He's not referring to regular users, but /. readers.

Re:Good advice - Always use your ISP for DNS

[@madeus]

Most big ISP's have really awful DNS servers, virtually all in the UK. Customers moan about this all the time on broadband forums.

I work for a very large ISP (several million broadband customers) and our suck, but no-one in the networks team will take ownership and fix it, or it seems admit to it. So I use Google DNS, as do my co-workers - we all know full well there is a problem but we are not responsible for the systems.

I worked an similarly sized ISP, same problem. Lousy management not enforcing any meaningful quality control (i.e. just box ticking ... of the wrong boxes) combined with lazy/incompetent/overworked people responsible for designing / build the systems (and a mix of overworked/under-qualified Ops guys).

Smaller ISP's seem to be a lot better at this (my guess is in smaller companies there is less dead weight in the employee pool - especially in the bozo management layer, where plausible deniability and responses of "I'll need more data on that" are king).

The solution to this is middle managers with a clue. Alas, having a clue is no longer as up near the top of the selection criteria as it once was in a more industrial era.

Re:Good advice - Always use your ISP for DNS

[NetServices]

Never trust your ISP

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

Looks like it...

Re:Good advice - Always use your ISP for DNS

[McTickles]

You blacklisted akamai? how?

Re:Good advice - Always use your ISP for DNS

[f3rret]

Ordinarily I'd completely agree with this, if it wasn't for the fact that my ISP - and most other Danish commercial ISPs - have decided to take a note from China and filter the hell out of the interwebs here.
Luckily they're nowhere as competent as China when it comes to internet filtering - incidentally, this is to their credit as the filtering was the gov't's idea and not the ISPs- so changing DNS providers lets you scoot comfortably around the filters.

While we're at it, is there anyway to manually define which Akamai node you're using? Like setting it in the HOSTS file or something?

Re:Good advice - Always use your ISP for DNS

[Morth]

But why bother? I've done that before, but no longer. It much more simple to do nothing... And probably faster too, since you'll have a much larger cache and will end up doing less DNS queries past the recursing server.

Re:Good advice - Always use your ISP for DNS

[imunfair]

I remember reading that Comcast has an opt-out for the non-standard 404 pages, I'm sure other ISPs have a similar option if you look hard enough. Probably easier just to use external DNS instead though

Re:Good advice - Always use your ISP for DNS

[Krneki]

Forget my stupid response, by custom firmware you mean like Tomato firmware for routers?

Re:Good advice - Always use your ISP for DNS

[TheRaven64]

It's been a while since I set up a DNS cache, but I seem to recall the steps were:

• Install.
• Start.
• Point /etc/resolv.conf at localhost.

Of these, the first two steps are hidden behind a point-and-click UI in a modern *NIX system and the third is often possible via some network settings UI, so you can do the whole thing without going to the command line. You don't need to know anything about DNS, because a DNS cache is not an authoritative server, doesn't control any zones, isn't a slave, and is usually configured to work out of the box in this state.

Setting up an authoritative DNS server is harder, but that's not the issue here.

Re:Good advice - Always use your ISP for DNS

[TheRaven64]

Until you've thrown them at some DNS, how do you know that they're unresolvable?

Re:Good advice - Always use your ISP for DNS

[rjstanford]

So was FooBarWidget, these days.

Re:Good advice - Always use your ISP for DNS

[rjstanford]

I dunno. You can unscrew a malfunction indicator lamp without affecting your family too much.

Re:Good advice - Always use your ISP for DNS

[rjstanford]

And how exactly would that fix their DNS problems?

If your router stops passing DNS requests through to the RR servers, that's a defective router and you should get a new one. If they have central server issues, resetting your router won't fix them.

Your experience is not normal. If it was a central issue then all RR customers would have to be resetting their routers many times a day. I guarantee to you that this is not happening. Take in your modem/router and swap it for another one. You'll be much happier.

Re:Good advice - Always use your ISP for DNS

[rjstanford]

It's blacklisted in my router at the root domain level.

This advice from the guy who also claimed that his DNS is provided by Road Runner, but it goes down several times a day until resetting his personal router (or modem) fixes it.

Sure.

I'm with Anonymous Coward on this one.

Re:Good advice - Always use your ISP for DNS

[Jah-Wren Ryel]

I remember reading that Comcast has an opt-out for the non-standard 404 pages, I'm sure other ISPs have a similar option if you look hard enough. Probably easier just to use external DNS instead though

Verizon just uses a different server address, I think it was the lame one's address +1.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

My ISP has terrible DNS service. Their DNS servers are often down, several hours a couple of times a week. To get around this I've been using Google's DNS. Better to have a slow connection than none at all.

Re:Good advice - Always use your ISP for DNS

[Midnight Thunder]

One reason people switched to Google and OpenDNS was because of their ISP's policy of hijacking non-existent DNS entries and redirecting to their search engines.

My alternative solution was simply to block the IP of the search host. Sure it did not resolve the broken DNS server issue, but does help work around one of the symptoms. What I would really want to do is put in an intermediate DNS server that can filter out any IP address entries matching their bogus one.

Re:Good advice - Always use your ISP for DNS

[Khyber]

Third router, fourth modem, second line replacement.

It's their shoddy local DNS. It's updating quite often. It's happening to every RR customer in my complex. I'm called quite often to do simple 30/30/30 to get them back on.

It's DNS, as I can ping any external IP just fine. So can the apartment people, including our landlords.

Re:Good advice - Always use your ISP for DNS

[Khyber]

By domain name, of course. You do know how to use a wildcard in your config script, yes?
*.akamai.net

You can also implement a blacklist by IP address. It's quite simple to find out which IP addresses are currently allocated to Akamai.

Adios Akamai with specialized whitelist exceptions for certain components to get some sites to work properly.

And I have to repeat the settings in NoScript as well but of course that's pretty trivial.

Re:Good advice - Always use your ISP for DNS

[Midnight Thunder]

What software DNS solution did you uses? Please don't suggest BIND, since that is real bitch to set up.

Re:Good advice - Always use your ISP for DNS

[Khyber]

How about you come over and watch it happen for yourself? That's a fully open invitation with full-paid plane ticket and couch space. And when you see me prove to you hands-down that it's DNS all the way (and I'm not switching to Google because their DNS propagation is slow as balls,) you get to swallow ten of my fresh bhut jolokia peppers as an "I told you so" and I get to record it and post it on youtube.

Deal? I just spent last week helping the Redlands Police Department track down the source of some random pair of video signals coming into my and other people's apartment (over-powered baby video monitor.)

I'm down for the next challenge. Please, come on by. Developing the tech for growing plants without light at all was much more complicated.

Re:Good advice - Always use your ISP for DNS

[TheRaven64]

Fedora came with a package for BIND preconfigured to run as a DNS cache - I presume it still does. Windows NT used to come with one preinstalled that you could just start by clicking in the services system. I've also configured BIND as an authoritative DNS server, and I agree that it was not the easiest program to configure, but as a DNS cache it was trivial.

Re:Good advice - Always use your ISP for DNS

[faedle]

Believe it or not, sometimes you have no choice.

Where I presently live, there is no DSL-based service available, only cable modem. If you are fortunate to live on the right side of the building, you can get Clear's WiMAX service, but that has issues.

Re:Good advice - Always use your ISP for DNS

[faedle]

I've "opted out" on Comcast, and yet if I use their DNS servers I still get Comcast's crappy page about 10% of the time. Nobody at Comcast can even figure out what the problem is, let alone fix it.

I gave up and just started resolving against Google.

Re:Good advice - Always use your ISP for DNS

[McTickles]

hosts file doesn't allow for wildcard... thats the problem

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

Use your own ISP for DNS.

Do you have any tips for keeping your ISP from directing a "server not found" to one of their crappy ad-ridden search pages? I think that's a major reason people choose DNS servers that aren't at their ISP.

The easiest thing to do might be to note the IP address of the ISP's squatting/bounce-page webserver and add a static route so traffic to that address is routed to localhost...

Re:Good advice - Always use your ISP for DNS

[Ksevio]

Typically ISPs have one set of DNS servers to give you the ads, and another set that you can manually change to to give you the normal message.

Re:Good advice - Always use your ISP for DNS

[Qzukk]

The MAC address is only visible on the local network. The first router you hit strips it and adds its own (assuming that the router's uplink is ethernet or some other link protocol that has a MAC).

The idea behind anycast is that I can install hundreds of DNS servers around the world, and give them all the same address but make them return different results, with the idea that the DNS server that is "closest" to the user would return a (non-anycast) address for "download.example.com" that is also "close".

Re:Good advice - Always use your ISP for DNS

[petermgreen]

isn't that what the MAC address of each machine is for
No Mac addresses are used to identify computers on a local network. They are not part of the packet that is routed over the internet. The internet identifies computers by IP address.

Anycast is basically a hack, you have multiple servers with the same IP address and route traffic to those servers based on where you receive it.

For a protocol like DNS this works great. Since dns is stateless and any server will give the same answer.

However with TCP based protocols the client and server establish a connection. If routing changes and sends the traffic to a different server the new server will have no knowlage of the connection and will not be able to continue it.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

Yep - at work we run our own recursive DNS server, because I trust myself.

We saw savings vastly more dramatic than this - 100 ms RTT dropping to near 1 ms RTT. (This is what happens in New Zealand - onshore, we have a lot of bandwidth and few places so it's all multimegabit fibre. Offshore, you're looking at 100 ms RTT minimum!

Re:Good advice - Always use your ISP for DNS

[userw014]

The MAC address isn't part of the IP packet.

Changing the anycast "endpoint" server should only matter for state-full protocols that last a "long" time. Most HTTP requests are relatively short, and most web pages HTML is sufficiently broken that the client (i.e.: the user clicking on the mouse) will just reload the page and try again.

Re:Good advice - Always use your ISP for DNS

[petermgreen]

Anycast certainly has the advantage that the server should be directed to the closest server in network terms and avoids the problem of users using remote dns servers.

but it also has a few major dowsides.

* if routing changes in the middle of a connection and sends the traffic to a different server any TCP sessions in progress will be broken. In the worst case with a router flapping or doing some form of load balancing between two routes that end up at different servers this could make it impossible to complete the download.

* anycast works with IP blocks that are big enough to be respected in global internet routing (which have to be requested from a rir with justification). This makes it much harder to be flexible about what content each server hosts than with dns (which works with hostnames which are essentially free so you can quite happilly give each block of content it's own hostname).

* with anycast the descision on which server traffic ends up at is made by other peoples routers. This makes it difficult to deal with an overloaded server.

For file downloads I think the best option would be to combine dns with http redirects. The dns will send most users to the most local server straightaway and those it doesn't can be dealt with through http redirects. For web pages the cost of the http redirect probablly isn't worth it.

Re:Good advice - Always use your ISP for DNS

[Pinback]

Use your own ISP for DNS.

Do you have any tips for keeping your ISP from directing a "server not found" to one of their crappy ad-ridden search pages? I think that's a major reason people choose DNS servers that aren't at their ISP.

Direct all household surfing through a squid proxy, and put the ad-page url in a deny list?

Re:Good advice - Always use your ISP for DNS

[shentino]

Unfortunately sometimes I'm behind a firewall that blocks outbound DNS, so using my "ISP" for DNS is mandatory.

Coincidentally enough there are shitloads of domain names on its blacklist that refuse to resolve.

Just wait until ISPs start doing that wholesale "to protect the children" *cough*

Re:Good advice - Always use your ISP for DNS

[userw014]

I fear I agree with you - but I run /boot/kernel, not /boot/vmlinuz

Besides, I want to do things on my LAN/WAN interface that conventional solutions don't allow.

Re:Good advice - Always use your ISP for DNS

[nabsltd]

Forget my stupid response, by custom firmware you mean like Tomato firmware for routers?

I don't use a "home" router, so I can't say exactly which firmware has a caching DNS server, but, yes, Tomato is an example of the kind of custom firmware I meant.

Re:Good advice - Always use your ISP for DNS

[nabsltd]

But why bother?

With all the hijacking of responses that every public DNS server seems to have, I don't trust any of them to return the correct answer.

And probably faster too, since you'll have a much larger cache and will end up doing less DNS queries past the recursing server.

As another comment mentioned, namebench is a good tool for showing just which DNS server is fastest.

For me, my caching server answered 30% of requests in less time than the fastest response from any external server, with about 50% of requests answered in less time than the average for the fastest external server.

Also, the whole point of TFA is that some content will be much slower if you use a public DNS server, even if the DNS response is faster.

Re:Good advice - Always use your ISP for DNS

[nabsltd]

This makes no sense.

If you run your own caching nameserver, then the only thing you have to trust is the authoritative server for the domain you look up, which you always have to trust.

Adding any other layer (Google, OpenDNS, your ISP, etc.) adds another entity that you need to trust.

Note that if your ISP blocks DNS requests to any server but their own, then there is nothing you can do without trickery like tunneling DNS through a VPN.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

He says he did it in routing. That is fairly easy. Just checked my little four port and you can use keywords or explicit addresses even there. Your local networking stack should be able to do the same and much more.

Re:Good advice - Always use your ISP for DNS

[WorBlux]

I wonder if an ipv6 implementation/protocol could/does solve this. Use the /64 prefix and have the suffix just be ::any in the DNS AAAA records. When the client gets a response it will know which specific server to talk to from that point on.

Re:Good advice - Always use your ISP for DNS

[JourneymanMereel]

You (or somebody who manages your network) has to actively choose to use OpenDNS and they advertise those redirects as a feature. That'd be like signing up for a service that automatically withdraws $10/mo from your bank account and then suing them for accessing your bank account.

Re:Good advice - Always use your ISP for DNS

[omglolbah]

Determining if something is stupidity or sarcasm on slashdot can be surprisingly hard sometimes :p

Being sleep deprived doesnt help *cough*

Re:Good advice - Always use your ISP for DNS

[omglolbah]

Mine also handles the job of being my home router behind the isp router :p

Got to love iptables and such

Re:Good advice - Always use your ISP for DNS

[bill_mcgonigle]

Only if I trust them not to fuck with it.

"with it" or "it up".

Re:Good advice - Always use your ISP for DNS

[rjstanford]

Then how does resetting your router "fix" the problem, if the problem's on their servers?

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

Last I tried, Comcast's "opt-out" uses a browser cookie which then causes it to serve a DIFFERENT non-standard 404 page, which just has less ads then their default page. Truly abysmal. And of course I have no other broadband choice in my area besides Comcast, so not like I can switch to one of the other 1 broadband providers in my state who also do the exact same thing.

Re:Good advice - Always use your ISP for DNS

[b4dc0d3r]

You're using this new technology called javascript, I can feel it. Turn that shit off for slashdot. You'll get used to it, and it won't try to reload stuff.

You may be using CTRL+W to close a tab, and I think slashdot has some sort of ASDW navigation thing going on. That's what I figured last time I investigated. So you could write a greasemonky script to intercept or disable the key presses, or just turn it off.

Re:Good advice - Always use your ISP for DNS

[Tacvek]

if you're on a home network chances are pretty high your router is acting as your DNS server or setting the DNS servers for your machine via DHCP.

Quite true, but to clarify for those not familiar, very few home routers actually run a recursive resolver. They generally act merely as a proxy, forwarding the requests onto the ISP's recusive resolvers, unless otherwise configured. This design allows for more localized caching, although I'm not sure if the common routers are actually configured to cache or merely always forward requests.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

The classic discussion system has been broken for months. I used to be able to read slashdot on a Kindle. To get all the comments you need to have javascript enabled and click more comments, at least for anonymous cowards.

Re:Good advice - Always use your ISP for DNS

[Khyber]

NoScript stops all /. Javascript and Flash.

It's not RSS, it's not CSS, gee, what would Slashdot need to load every time we changed pages or moved our view from one page to the next?

OF COURSE! AKAMAI ADVERTISING.

Thus demonstrating a possibility that slashdot, like the rest of America, is starting to specialize in the pre-packaging and distribution of advertising - AKA BULLSHIT.

AKA Slashdot is becoming Fox News.

Re:Good advice - Always use your ISP for DNS

[Khyber]

There are no HOSTS files in a typical router. What nonsense are you on about?

Re:Good advice - Always use your ISP for DNS

[m85476585]

Where I live, there's no DSL or cable, only Clearwire (an early wimax technology inferior to Clear's service. I can't remember if Clear and Clearwire are the same company or not), satellite, or cellular 3G. None of these are particularly good options, but I have Clearwire. It's been OK until recently during peak load times I start getting 1000-2000 ms ping times (around 100 is average) and extremely slow download speeds.

Re:Good advice - Always use your ISP for DNS

[Khyber]

Because it forces a refresh of the DNS with every restart, so I can get a connection for an hour or two before it dies again.

Hi, I used to work for Stream doing RR tech support. This is a COMMONLY KNOWN RR issue and has been since the 90s.

And I do have a packet sniffer looking for RST commands on the coax before the modem, none are being received so they're not PURPOSELY interrupting my service, but tons of DNS updates are coming through.

Which would correlate quite nicely with USA, Europe, and Asia's network status being just under 75% with some rather heavy packet loss - BAD DNS is quite often such a cause for bad network performance ratings like what I'm seeing globally right now.

Yay internettrafficreport for being such a useful tool to observe WTF is happening so you can determine if you're having a local issue or if it's more widespread.

Re:Good advice - Always use your ISP for DNS

[Paralizer]

Why don't you just run custom firmware on your router so you can have a local DNS server you idiot!

What firmware allows me to do this?

Oh I don't know that..

Re:Good advice - Always use your ISP for DNS

[Khyber]

And to add, looks like quite a few backbones are down.

http://internettrafficreport.com/namerica.htm

Texas, Nevada, Wisconsin, New Hampshire, Ontario, CA and Vancouver, CA are completely down, no wonder there are so many DNS propagation requests at this time.

This problem is not as bad when the network health is better globally and in my local continental region.

Re:Good advice - Always use your ISP for DNS

[Khyber]

If the MAC address isn't part of the IP packet then how does a router know which data packet was intended for which machine behind which router?

Because this seems to state otherwise you would NOT be able to identify the machine: http://en.wikipedia.org/wiki/MAC_address

Camfrog (an internet program) bans by MAC address. So does IRC servers. MAC is still part of the TCP/IP protocol. I can ban forum members on my forums by their MAC address, so I am not sure you're on your game regarding what you're talking about. MAC is THE BASIS for most of OSI Layer 2.

Re:Good advice - Always use your ISP for DNS

[devilspgd]

It occurs to me that DNS could be used as a primary load distribution solution with HTTP redirects being used as a fallback.

This would require CDNs to work with large public DNS providers to ensure that the DNS providers return CDN IPs that have a redirect.

For website hosting redirects probably aren't worth it especially since with many sites you'd end up redirecting every static resource. I'm not sure that the latency of hitting the "wrong" CDN pool is that big a deal anyway for web pages and their resources.

For anything over 1MB or so, the additional few hundred ms to redirect upfront will quickly be overshadowed by the significantly faster KB/s download speeds.

However, at this point redirects might not be simple to drop-in, does iTunes follow redirects? Phasing it in wouldn't be that difficult, clients could add a &allowredirect=true to tell the CDN that it's free to redirect users when appropriate.

Either way, DNS is a bit of a clumsy way to do it.

Even so, anycasting IP traffic isn't impossible but it does require some clever programming to allow sites to "transfer" a client back and forth. A CARP-like solution would be possible.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

That'd likely work for load balancing in one physical location.

The solution for UDP has simply been the fact that DNS clients just drop the "slower" packets after accepting the result from the first server.

The proposed solution for TCP was to have the client "know" that the IP it is trying to use is an anycast IP, when the server responds, it responds using it's "normal" IP and the client starts using the normal IP from that point on. Which is totally not going to lead to session hijacking or anything like that.

Re:Good advice - Always use your ISP for DNS

[jjk3]

Wait .... how is anycast harder, this seems way simpler to me then some DNS trickery. With anycast you have multiple servers answer to the same IP address and then announce that IP (or really block). Then routing figures out the least cost path. This is what routing is suppose to do. With DNS you need the DNS server to figure out where the client is and provide the best response. Since DNS doesn't have any path information (like routing does) you have to add all kinds of crap to figure out which answer to give. It sounds like in this situation the crap Akamai isn't the right crap.

Re:Good advice - Always use your ISP for DNS

[matthewd.net]

Unbound is a good choice. But even BIND isn't that scary for a purely non-authoritative recursive resolver.

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

If you're running any kind of server that does lookups against the clients, SMTP DNSBLs for example, you can't take the chance unresolvable hosts will be given a valid DNS response. Put another way, the ISPs that bastardize DNS to server ads or whatever are breaking the protocol, and they should know better.

Re:Good advice - Always use your ISP for DNS

[RobertM1968]

If you're running any kind of server that does lookups against the clients, SMTP DNSBLs for example, you can't take the chance unresolvable hosts will be given a valid DNS response. Put another way, the ISPs that bastardize DNS to server ads or whatever are breaking the protocol, and they should know better.

Very good point and very true.

Re:Good advice - Always use your ISP for DNS

[TheRealGrogan]

Actually I realize that (I often like to see what I'm connected to and use netstat). I think perhaps I wasn't completely clear. Note what I said though... "if any software delivery system or service is slow because of content distribution" I would stop using it.

This could be akamai, fileplanet's mirrors, whatever. If I get pissed off, I go elsewhere. For example, I have better downloads through Steam's content delivery system than Direct2Drive (FilePlanet) so I started buying from Steam instead. I can also go back to meatspace any time to buy my games, though I do so hate optical media.

If I can't at least get something I pay for at the full capacity of my meager DSL link (I can download about 620 kilobytes/sec at best) then fuck it.

I'm quite happy with the speed of my DNS lookups, as well as DNS changes refreshing admirably quickly when sites move, using OpenDNS and I'm not going back to my ISP's shitty ones because of Akamai. (therefore, I say "fuck akamai")

Re:Good advice - Always use your ISP for DNS

[Anonymous Coward]

Because my ISP has a dns failure every now and then, and I got tired of calling the hotline, configured google dns.
Life is beautiful since

Re:Good advice - Always use your ISP for DNS

[userw014]

The IP packet doesn't contain MAC information. However, an IP packet on some media (ethernet, optical fiber, ATM, DSL) will likely contain some kind of MAC header - but only so that the two devices communicating can recognize each other.

When the IP packet is forwarded from one router to another, the original MAC information is not copied (or is removed), but the new MAC information appropriate for communication between those two routers is added.

The Internet is a network of (physical) networks. MAC addresses on different media could have different addressing (although the ethernet MAC header has been very convenient for most.)

Unless, of course, you're tunneling ethernet frames over IP (such as you might do if you were bridging ethernets over commodity IP service.)

Re:Good advice - Always use your ISP for DNS

[McTickles]

well my rooter is provided by my ISP so I dont know what actually goes on in there. Also it is unlikely they know what goes thru it, on a side note...

thirst most!

[Anonymous Coward]

yes

Re:thirst most!

[blai]

whose DNS were you using?

Re:thirst most!

[JustOK]

1.2.3.45 same as my luggage uses.

good technical discussion of this at HN

[harlows_monkeys]

There's some good technical discussion in the Hacker's News discussion of this issue.

Re:Alternate solution

[Anonymous Coward]

iTunes is great on the Mac.

Namebench DNS tool

[maggotbrain_777]

This afternoon, I found a tool from Google Code called namebench which tests response times against multiple DNS servers and give recommendations based upon a number of query types. The results returned when checking the 'censorship tests' were interesting. Seems a number of sites (wikileaks, isohunt, stormfront) returned 'incorrect' results across DNS servers. I'm going to try this over the next couple of days and see if any of my browsing speeds improves.

Re:Namebench DNS tool

[countertrolling]

...namebench runs a fair and thorough benchmark using your web browser history...

No small gold mine there...

Re:Namebench DNS tool

[nabsltd]

The results returned when checking the 'censorship tests' were interesting. Seems a number of sites (wikileaks, isohunt, stormfront) returned 'incorrect' results across DNS servers.

I suspect there is something wrong with their database, as my local DNS server is returning the exact same IPs as all the public DNS servers.

One thing running that benchmark showed is that although most public DNS servers are better on average than my local server at handling hundreds of simultaneous requests, 20% of the requests to my local server were answered in 1ms, and nearly 30% were answered before the fastest public DNS server returned an answer (in 9ms).

Considering how easy it is to set up your own DNS server, and how many advantages it has as far as getting correct answers, and the fact that most queries will return much faster, I don't know why anyone would rely on a public DNS server.

You would think.

[SchizoStatic]

Why do they use the dns for the geo location and not the ip address itself? You would think that would make way more sense.

Re:You would think.

[nedlohs]

Their DNS server never knows your IP since if the name result isn't cached by the DNS server you are using then that server makes the request to them not your computer and hence they either see nothing or the IP of the DNS server you are using.

Re:You would think.

[Timothy Brownawell]

They only find out your IP address after it's too late.

1. Your computer asks a DNS resolver where the server is.
2. The DNS resolver asks Apple's (well, Akami's) DNS server where the server is.
3. The DNS server guesses the closest server, but all it has available to work with is the address of the resolver.
4. Your computer uses that answer to contact the server and download whatever. If it was given the wrong server, it's too late now.

Re:You would think.

[crankyspice]

Your computer uses that answer to contact the server and download whatever. If it was given the wrong server, it's too late now.

Why is that too late? See the IP address, issue a redirect to the appropriate server. In HTTP that's as simple as issuing a 302 response and a Location: header. (I haven't Wiresharked iTMS to see how it's connecting, but it would be a dead simple change to the protocol to insert a handshake step that occurs before the substantive transfer begins. Overhead would be negligible.)

Re:You would think.

[crf00]

I already knew this, but now it makes me think why not we instead use HTTP redirection strategy? That is, say the client hits the server directly at http://example.com/very-large-file.zip, the server detects the client's IP and permanently redirects it to http://[location].static.example.com/very-large-file.zip, where static.example.com is a subdomain managed by Akamai and [location].static.example.com always resolves to CDN node nearest to the specified location regardless of the client's IP address.

Re:You would think.

[tsj5j]

It's not too late at all.
The HTTP server can redirect you based on your location.

Re:You would think.

[Anonymous Coward]

Your making a couple of incorrect assumptions.
1. All content is delivered via HTTP. While most static content is, video streaming often using other protocols.
2. The client accessing the cdn can fallow redirects.

Re:You would think.

[xnpu]

Indeed. HTTP might be a bit slower and not benefit from the ISP's DNS caching, but in conjunction with the DNS method it would provide an acceptable correction method. Rather wait 1 more second for the download to start then to download at painfully low speeds.

Re:You would think.

[Anonymous Coward]

Your attempted answer is grammatically retarded and ambiguous in the extreme. Don't talk to people; it doesn't help.

Why do they use the dns for the geo location and not the ip address itself? You would think that would make way more sense.

What makes way more sense to the CDN is throughput, latency and low cost. You expect the CDN's content server to analyse the clients IP address and redirect the client to another, more optimal, server. That is a bunch of additional latency and cost that can be avoided by effectively performing the 'redirection' in DNS.

Google anticipated all this when they created their service. Google has published an IETF draft proposal that allows a client to embed part of its IP address (/24) in a DNS query. With that a client can use any compliant resolver and the CDN's authoritative server will produce the optimal response for the actual client.

Problem is the Google solution requires compliance from everyone involved; the client, or at least the client's resolver must add the partial client IP address to the query and the CDN must use this information in lieu of the resolver's address. Also, the firewalls, proxies, etc. in between must not strip this extra query information, and caches must be elaborated to include the new client info in the cache index tuple.

There are costs associated with Internet traffic. These costs are minimised by distributing content and reducing round-trips. The content distributors are going to do whatever it is they must do to achieve minimal costs. This is the cold, hard truth of the matter and what Paul Vixie or whomever else thinks doesn't automatically trump the bean-counters. If the IETF can't see their way clear to a solution that addresses the problem then the CDNs will make their own. If the Google solution is unacceptable then bring something else to the table; yapping at CDNs from the IETF peanut gallery won't help.

Re:You would think.

[deniable]

I may have to sniff some traffic going to Apple. For some reason their software updates kill our filters. I wouldn't be surprised to find they've made a 'custom' HTTP for themselves.

Re:You would think.

[TooMuchToDo]

Most of Akamai's caching platform is based around DNS for location awareness. This is because DNS (in most cases) can be easily anycasted (since it's stateless when all done over UDP) vs trying to anycast tcp connections done with HTTP.

Re:You would think.

[TooMuchToDo]

More info on my above point. If Akamai were to use HTTP instead of DNS for load balancing, complexity would increase in having to manage redirect clusters, as you couldn't anycast them over UDP like you can with DNS.

RFC 1546 - Host Anycasting Service

How UDP and TCP Use Anycasting

      It is important to remember that anycasting is a stateless service.
      An internetwork has no obligation to deliver two successive packets
      sent to the same anycast address to the same host.

      Because UDP is stateless and anycasting is a stateless service, UDP
      can treat anycast addresses like regular IP addresses. A UDP
      datagram sent to an anycast address is just like a unicast UDP
      datagram from the perspective of UDP and its application. A UDP
      datagram from an anycast address is like a datagram from a unicast
      address. Furthermore, a datagram from an anycast address to an
      anycast address can be treated by UDP as just like a unicast datagram
      (although the application semantics of such a datagram are a bit
      unclear).

      TCP's use of anycasting is less straightforward because TCP is
      stateful. It is hard to envision how one would maintain TCP state
      with an anycast peer when two successive TCP segments sent to the
      anycast peer might be delivered to completely different hosts.

      The solution to this problem is to only permit anycast addresses as
      the remote address of a TCP SYN segment (without the ACK bit set). A
      TCP can then initiate a connection to an anycast address. When the
      SYN-ACK is sent back by the host that received the anycast segment,
      the initiating TCP should replace the anycast address of its peer,
      with the address of the host returning the SYN-ACK. (The initiating
      TCP can recognize the connection for which the SYN-ACK is destined by
      treating the anycast address as a wildcard address, which matches any
      incoming SYN-ACK segment with the correct destination port and
      address and source port, provided the SYN-ACK's full address,
      including source address, does not match another connection and the
      sequence numbers in the SYN-ACK are correct.) This approach ensures
      that a TCP, after receiving the SYN-ACK is always communicating with
      only one host.

Emphasis mine.

Re:You would think.

[TooMuchToDo]

If HTTP was used instead of DNS, Akamai would have to have a redundant HTTP redirect cluster to manage the redirections. But you can't have one cluster, you need several for redundancy and speed, spread across the world. Also, you can't use anycast with TCP like you can with DNS, because it's TCP based, which means you lose load balancing and redundancy at the IP protocol layer.

DNS was and still somewhat is easier. No one really envisioned Google doing DNS and it taking off I think.

Re:You would think.

[wiredlogic]

It's too late for Akamai's form of transparent mirroring which doesn't hit their customer's servers if they already have the data they need. The insertion of Akamai's servers into the mix is done by substituting IPs and doesn't take place at a higher level of protocol.

Of course they ultimately have to serve up data to the requesting IP and they could verify that their original geo-location was accurate. Maybe someone should patent that idea quick since Akamai hasn't seen fit to implement it yet.

Re:You would think.

[Ash-Fox]

They only find out your IP address after it's too late.

But anycasting doesn't get the IP address too late?

Re:You would think.

[Ash-Fox]

More info on my above point. If Akamai were to use HTTP instead of DNS for load balancing, complexity would increase in having to manage redirect clusters, as you couldn't anycast them over UDP like you can with DNS.

As someone who has setup TCP anycasting. Just ensure that your anycasting routing is stable, that prevents pretty much any 'hop' issues.

See also, http://www.nanog.org/meetings/nanog37/presentations/matt.levine.pdf

Re:You would think.

[F.Ultra]

+1

The talk about UDP is not relevant since they still could use the DNS geolocation for the UDP connections. And I guess that most of their traffic is http anyways.

Re:You would think.

[nedlohs]

Why not express it more clearly then? And then go on the four paragraph rant.

Something like:

When you make a DNS query your IP address is not exposed to the server making the geolocation decisions. All it gets is the IP address of the DNS server you are using, and if the result is cached already it doesn't see the request at all.

Re:You would think.

[TooMuchToDo]

I sat in on that presentation (and am on the NANOG mailing list). I agree that it *can* work, but you can't rely on simple geographic diversity for route stability with a platform the size of Akamai's. Not enough knobs to twist when things break (although it seems to be working great for CacheFly).

Excellent presentation, btw.

Going to be at NANOG51?

Re:You would think.

[ista]

In fact, there are quite a few people out there using Anycast for TCP-sessions. It's really a matter on what timescale you're looking at. The networking guys see TCP as something to use for long-living connections - e.g. a BGP session running for days, weeks or even months. A flapping route in this setup will result in a broken session. But: what does this really mean to you? If your CDN distributes downloads which are "done" within a few minutes, such a rarely flapping route will result in a few broken sessions once a day out of millions of downloads successfully served. Compared to issues like non-working DNS, overloaded servers and filled lines, that's nothing and can actually enhance the overall CDN service.

A nice paper to read is this one from Matt Levine. He's working for a CDN provider using TCP-Anycast for years now and sums up the most important issues on TCP-Anycast.

Basically the most important one is that your anycasted servers really have to be spread far enough so that flapping routes at some peering point won't matter. As a rule of thumb, put one CDN loadbalancer on the US east coast, one to the US west coast, another one to western europe, one to Australia and one in Hong Kong. If you'd like to put multiple CDN loadbalancers to one continent, leave space between them, e.g. one box for each country/state.

Re:Alternate solution

[FictionPimp]

Very criminal when you consider that you do NOT need to install iTunes just to install quicktime.

Why

[xaoslaad]

I have to ask why they are playing games with dns rather than using some kind of LB solution to direct users to the closest server(s) based on the client ip address. Is this not feasible or is it cost prohibitive; the method theyre using seems crazy to me though i fully admit to not being up to speed on high level networking design.

Re:Why

[omglolbah]

The beauty of the DNS "trick" is that a user requesting say "yadiyadi.com/media/cheez.mp4" in Norway would get one IP and a client in say Australia would get a completely different IP. This makes the whole CDN implementation a whole lot easier as you avoid the whole negotiation issue by having the domain resolve to different IPs based on the source of the request.

This is overly simplified of course.

It works for the vast majority of users too.

Re:Why

[xnpu]

While it's arguably "prettier" I don't see anything wrong with old school redirects though. Either using 302's or "sourceforge" style.

Re:Why

[mysidia]

While it's arguably "prettier" I don't see anything wrong with old school redirects though. Either using 302's or "sourceforge" style.

Additional HTTP request round trips = Increased Page Request Latency (time that elapses between when a page is requested and when the response data starts to be received = Slower page loads

Re:Why

[TooMuchToDo]

You can anycast DNS with UDP and the client never knows the difference since it's handled with BGP and IP. HTTP redirects would be much more difficult to do from a reliability and scalability standpoint. Also, when it fails, it would be much less graceful.

Re:Why

[chgros]

I don't see anything wrong with old school redirects though.
Well, they only work for http, for one thing.

Re:Why

[xnpu]

[HTTP] I think that's what Apple is using here.

Regardless, not being able to do Redirects is a design choice of whomever came up with the protocol used. It can be added.

Re:Why

[Anonymous Coward]

SEO and branding for one.... Anycast as others have mentioned would be a better answer than using DNS.

Re:Why

[rjstanford]

Especially when the end-user is in Australia, using an DNS server that gives them an IP address for the initial communication that's in California. Then you have DNS:AU->US-CA->AU, HTTP:AU->US->AU(302), HTTP:AU->AU instead of DNS:AU->AU, HTTP:AU->AU. Now scale that up into the billions of requests, and there's some serious differences.

Re:Just wondering....

[Anonymous Coward]

It wouldn't...?

They aren't doing network shaping through packet analysis, they are using DNS to determine which server is geographically closest to you, which (usually) causes better network speeds.

Source location is a bandaid

[Anonymous Coward]

If some of the server pools are being overloaded while others are sitting relatively load free, source location is obviously not the best choice for load balancing. Sure, it may work most of the time but I'm sure ISP's dns server locations are not equally spaced around either. I am in VA and the Comcast DNS address I have are in NJ. I guess that is not too bad but how many people from Comcast are using those same DNS addresses?

Re:Alternate solution

[Excelsior]

Really? You mean on the Mac it isn't required to set up an IPhone or IPad that have no business relying on a desktop machine? You mean it isn't required I sync with it just to get Podcasts onto a device that already has internet connectivity? You mean on Mac it doesn't have a proprietary, signed procedure for syncing music to IPhone/IPod Touch/IPad, that makes it completely impossible to develop competing software without breaking the DMCA?

Sure the "ITunes experience" doesn't suck as hard on the Mac as it does on other platforms, but it still sucks. As GP says it's malware, only I would elaborate and say its malware that malicious to an entire industry.

Hrmmm

[StripedCow]

Such a basic operation, and still not working as intended? Something is terribly wrong here if you ask me...

It must be Apple's "magic" that's causing the trouble.

Re:Hrmmm

[mini me]

It is actually a pretty complicated problem, finding the IP address of the nearest host based on a domain name, solved in a fairly elegant way.

The only problem here is that you are being passed the address near Google instead of your ISP, which means that you miss out on the benefits of finding a host nearby that is only a few hops away.

Re:Hrmmm

[socsoc]

Elegant? Misusing DNS to make a CDN faster isn't elegant.

Re:Hrmmm

[Gadget_Guy]

It must be Apple's "magic" that's causing the trouble.

No, it is not Apple's fault. Anyone using Akamai would have the same problem. I think Microsoft use them for Windows Updates too.

Re:Hrmmm

[Desert Raven]

No, it's not particularly elegant. But on the other hand, split-horizon DNS is nothing new or magical either. Nor would I classify it as "abuse". The capability has been there since the early days of BIND.

In the DNS trade, we refer to it under the category of "stupid DNS tricks"

That said, it does have some significant advantages over other techniques.

#1, It's protocol-independent. Sure you can do intelligent redirects with HTTP, but not everything in the world is HTTP
#2, Even with HTTP, in order for it to work, you have to now change the name of the server, and often the links to internal content. Your initial request to www.domain.com will now have to be redirected to hostx.domain.com or www.location.domain.com etc., and links on the pages to content servers will also have to be altered. This can be confusing to end-users, and may require additional SSL certs. It's also a code maintenance issue.
#2a, While the renaming seems trivial on first glance, it has HUGE implications for search engines, etc, since those "local" servers will get indexed instead of a generic name
#2b, It also means that a calculation will have to be made by the web server deciding where to redirect you to, then the actual redirect, increasing load and latency. DNS solutions are "pre-computed" and thus do not have similar issues.
#2c, If you solve 2a by checking every request at every location, you make 2b much worse
#3, It's simple.

Downsides:

#1, Third-party DNS recursive services throw it off. (There is a proposed RFC that would allow for such recursives to pass the originating network in the request)
#2, It makes DNSSEC a right royal PITA (Much more than it already is)

Re:Hrmmm

[crusty_architect]

Elegant is over-rated. DNS is a great place to implement some good value trickery for 99.99% of customers.

Re:Hrmmm

[Anonymous Coward]

allow for such recursives to pass the originating network in the request

Which originating network? You aren't suggesting that caching be disabled, are you? Load balancing without application support should be done with anycast routing.

Re:Hrmmm

[Desert Raven]

The originating network of the client.

Caching would not have to be turned off, but it would get a lot more complicated.

Note that the draft expired, but I expect someone will eventually resurrect it.

EDNS Client IP

I don't use either

[93 Escort Wagon]

I've used our university's DNS servers as primary for over a decade, with whatever my current ISP is as secondary. I haven't had any complaints.

sounds like an apple problem

[Dan667]

doesn't apple still watermark all their content anyway? Seems like you should be buying it from somewhere else for both reasons.

Re:sounds like an apple problem

[greerga]

"As of the January 2009 Macworld Expo, Apple has announced that all music in iTunes will be available without DRM, and encoded at the higher-quality rate of 256 kbit/s."

http://en.wikipedia.org/wiki/ITunes_Music_Store

Re:sounds like an apple problem

[devilspgd]

Right... Without DRM, but with a watermark (in other words, if you download a Miley Cyrus song and share it, anyone else who gets access to it can track it back to you)

That being said, I have a lot of trouble getting upset over the fact that purchased content is watermarked. As long as I'm not distributing the content, who cares?

Re:sounds like an apple problem

[gnasher719]

Right... Without DRM, but with a watermark (in other words, if you download a Miley Cyrus song and share it, anyone else who gets access to it can track it back to you) That being said, I have a lot of trouble getting upset over the fact that purchased content is watermarked. As long as I'm not distributing the content, who cares?

There are of course all kind of conspiracy theories around, but the fact is that two things are different in your downloaded files than in my downloaded files: The Quicktime atom for "creation date" which is - guess what - set to the time when we each download the file, and there is information so that _your_ downloaded file will turn up in the "Purchased" section in iTunes on your computer. This is not a watermark at all. It's just iTunes not carefully removing forensic evidence that could be used against you.

If anyone cared there would be an anonymiser app for this.

Re:Alternate solution

[Jeff DeMaagd]

If only ignorance is criminal too.

Maybe at one time, iTunes was the only way to get Quicktime, but if that's true, that was years ago.

http://www.apple.com/quicktime/download/

I think you'd find some people saying QuickTime is criminal too, but I think that's a different discussion.

Re:Alternate solution

[hedwards]

That's true. However they do require you to install quicktime in order to get the codecs, unless I'm missing something. And for whatever reason Apple insists on not using any native widgets. Which means that not only are you installing crapware, but it also looks ugly on top of that.

Re:Just wondering....

[hedwards]

It wouldn't and really shouldn't. CDNs are there to ensure that the least amount of infrastructure is used for each request. Meaning that they try to put the server as close to your physical location as possible. If anything, net neutrality would encourage this as it would be easier to have a CDN covering both Qwest and Comcast in a given region or whatever the options are in your area.

Or use your own DNS

[La Gris]

I use to setup my own DNS at home and casually use forward zones when needed. I started this when ther was that issue with redirecting non existant names.

Sure, not every one should do this as it stress load root servers and some ISP may redirect UDP/TCP 53 to their own servers. BTW, that's still my way of using DNS.

Re:Alternate solution

[Anonymous Coward]

"As GP says it's malware, only I would elaborate and say its malware that malicious to an entire industry." - it might not be the most optimized piece of software on earth but to call it malware just exposes you as a someone who has hopped on the anti-Apple bandwagon because it's trendy.

Um .. duh

[Anonymous Coward]

This isn't Apple's fault. It's also not Akamai's fault. They're trying to provide the best user experience by directing a client to the "closest" server. This is accomplished by the global load balancer answering DNS queries with the IP address of a server that's close to the source. But, because of how DNS works, the only information they have to work with is the IP address of the client's DNS server ... not the actual client's IP address. So, if you use a DNS server that's clear across the country from you (or worse yet, one on a different continent!), you're likely to get directed to a server you don't really want to use.

Re:Alternate solution

[Anonymous Coward]

What the fuck are you smoking? The discussion is about being forced to install itunes in order to install quicktime, not the other way around.

M$ does it too...

[alanshot]

Microsoft does this too. After scratching my head over the past several weeks trying to figure out why I cant download M$ files worth crap half the time, this appears to be why.

Re:M$ does it too...

[Anonymous Coward]

What is M$? Is it some sort of FUCKING RETARDED way of typing MS? I think it is.

Re:M$ does it too...

[deniable]

It's a string in MS BASIC. I believe the relevant line is: 10 LET M$="MICROSOFT"

Re:Alternate solution

[Anonymous Coward]

yEA But Apple still sucks I mean come on, it's trendy for a reason. I like real Apples and the company Apple doesn't taste like a nice, fine, red delicious on a sunny afternoon, its sweet nectar dripping from my chin hairs and sliding its way down my chest to my erect nipples where it pools for a moment and then gravity finally exerts the full strength of its power, drawing the sweet juices farther down towards my waiting

Re:talk's cheap fucker

[DAldredge]

Amazon's MP3 store.

Re:Alternate solution

[devilspgd]

You don't need to install iTunes to install QuickTime. Sadly, you do need QuickTime to install iTunes. Which is the lessor evil depends on your needs, but I'd be thrilled to have iTunes alone without QuickTime, Bonjour or the host of kernel mode crap it installs.

So the moral of the story is...

[CAIMLAS]

So the moral of the story here is not that Google and OpenDNS services are bad, but that Apple's iTunes QoS methods are of "questionable quality" - at best.

How did this make Slashdot's frontpage, again? Maybe this should be filed as a bug report to Apple (do they read those?) instead.

Re:So the moral of the story is...

[Anonymous Coward]

No, the moral of the story is if you're using Google or OpenDNS and want fast downloads then don't use a CDN; pirate them.

Re:So the moral of the story is...

[bill_mcgonigle]

Maybe this should be filed as a bug report to Apple (do they read those?) instead.

Yes, but they're just marked 'duplicate' and you're not allowed to see the original.

Multiple DNS feature?

[Anonymous Coward]

Seems like it would be useful to use multiple DNS servers and then choose whichever one has the fastest download and abandon the other connections.

Do any browsers/OSs/whatever have this feature? As I understand it, the secondary DNS feature only uses the secondary server when the primary server is down.

Re:Multiple DNS feature?

[Jon Stone]

That's sounds very much like the default behaviour of ISC's bind, up until version 9.6 https://www.isc.org/software/bind/new-features/9.6

Randomize server selection on queries As a security improvement to make forgery a little more difficult, BIND 9.6 now attempts to make the order of the server selection for queries less predictable. Previously, BIND would prefer to query the server with the lowest round trip time (RTT). Now servers that haven't been tried yet have their RTT set to a random value between 0 ms and 7 ms. And the RTT values of servers which have been tried are now randomly changed up to 128 ms.

This algorithm also applies to DNS servers specified with the "forwarders" clause. A local bind installation with the ISP's and Google's DNS servers configured as forwarders would do what you want. The OS and applications would then be configured to use the local DNS server.

And how is this news?

[xnpu]

This applies to tons of GEO-optimized services and has been this way since day one. Really, how is this news?

Re:And how is this news?

[xnpu]

BTW - Remember when Google proposed to modify the DNS protocol to pass on the end-users IP? This is exactly why.

Re:And how is this news?

[khchung]

Really, how is this news?

Maybe because this involved both Apple and Google? Guaranteed tons of comments (and tons of hits) from fanboys on both sides.

Re:And how is this news?

[mysidia]

This applies to tons of GEO-optimized services and has been this way since day one. Really, how is this news?

In this case, I would call it GEO-degraded services.

Presumably some people who have ISPs actually near Google's DNS servers have the same issues even when using their ISP's DNS servers.

This is brokenness.. This is a fuckup that Apple needs to fix, not Google.

Re:And how is this news?

[A beautiful mind]

That would have been stupid though, unless you want to throw away the caching layer of the dns infrastructure.

Re:And how is this news?

[Anonymous Coward]

Still a bad idea, just like using the source of a DNS request for making a load balancing decision is a bad idea. The domain name system is a distributed, caching database. Introducing that kind of dependency is like setting the TTL to zero, i.e. disabling caching. See, Google and others even ignore the TCP specs just to shave off one round-trip time from their page load times, and then they propose to make DNS queries uncacheable? It doesn't make sense.

Re:Alternate solution

[dangitman]

Which is the lessor evil depends on your needs,

Well, since neither Quicktime or iTunes is leased to you, I guess that means neither is a lessor evil.

Re:talk's cheap fucker

[Anonymous Coward]

Amazon's MP3 store.

Strange world you live in, to recommend an Amazon "We delete your property" solution as "superior"

Re:Be wary of using iTunes.

[Anonymous Coward]

Amen! It's buying content from iTunes like highly immoral, well maybe not so immoral as buying from Amazon or a CD store, but immoral none the less.

Re:Alternate solution

[devilspgd]

Fair enough.

Re:Alternate solution

[Anonymous Coward]

fwiw, you do not need to sync to get podcasts. you can get them directly on the iThing without using itunes. at least with the most recent ios.

Old news, you've just figured this out?

[Anonymous Coward]

Let me get this right, you've just figured this out? People have been using DNS and IP based location load balancing for years google, yahoo, facebook, limelight networks, akamai you name it is doing it ie. content delivery networks and ip aka location based load balacing\site selection. e. Getting the content closer to the end users, improving experience, tayloring experience based on location. I remember for quite awhile several years google's appliances though apnic netblock was taiwan based, eventually databases were updated with the correct country code being attached to the allocation directing content to the correct country based experience. Old news.

Re:Alternate solution

[HeronBlademaster]

Quicktime has been around a lot longer than iTunes (but it was never less of a resource hog as far as I remember).

http://en.wikipedia.org/wiki/Quicktime#QuickTime_1.x

Re:talk's cheap fucker

[Anonymous Coward]

Amazon's MP3 store.

Strange world you live in, to recommend an Amazon "We delete your property" solution as "superior"

Ding Ding! We have a winner!

HTTP Live Streaming

[SuperKendall]

Pretty sure Apple is using all HTTP Live Streaming at this point, which in fact is all based on HTTP...

Also I have worked with a lot of applications that stream or play media now, and generally it's been done over HTTP - I'd say that's more the rule than the exception.

And if an HTTP client can't follow redirects it's not really an HTTP client - that's pretty basic stuff, I can't fathom there is anything that wouldn't obey a re-direct (unless it was doing so on purpose).

To who again?

[SuperKendall]

The moral the the story would appear to be that more people on Slashot need to read up on what CDN's are and who runs them.

Re:Alternate solution

[dakohli]

I really appreciated iTunes when it deleted all of my music from my iPhone when I synced up with a new Laptop. I thought I had authorized the computer, but then during the first sync, instead of transfering my purchased songs to the computer it just removed them from the phone. I know if was my fault somehow, I missed something along the the way. In the end, I figured that it would be just easier to get a new phone that I could control easier.

Six months ago, I was the owner of mini-Mac, iPad and a iPhone. Now, I no longer have the Mac, (back to windows and mostly Linux), have a Galaxy S Captivate, and the newest device is a eLocity A7 Android tablet. I will not say that there haven't been some bumps in the road, but I will say I'm happier overall.

Now, I would still recommend a Mac to some of my Family/Friends who don't like configuring their computers. In fact as one of the major techies in my family I encourage them all to adopt Macs, because then my life is a lot easier!

-My mistakes are just those, please accept my apologies. Tks

Or set up your own nameserver

[FoolishOwl]

With a little effort, you can set up BIND on your own system.

Re:Or set up your own nameserver

[karolbe]

I have set up pdnsd on my local Ubuntu box. This is a simple proxy DNS server, very easy to set up and gives good results (depending on network response time for cached domain is about 1ms).

Re:Or set up your own nameserver

[FoolishOwl]

I've heard good things about PowerDNS, and I think Wikimedia uses it.

I went with BIND just because it's standard, so the instructions for anything that involves nameservice includes instructions for BIND: configuring IPv6, configuring DNSSEC, configuring TLS/SSL, etc. There's never much load on my Linux box, so I don't worry about optimizing performance.

Non-native widgets

[tepples]

Apple insists on not using any native widgets.

How are the widgets in QuickTime and iTunes not native? Are they written in Java bytecode or something? Are they PowerPC, and run under the equivalent of Rosetta?

Re:Non-native widgets

[@madeus]

The poster is referring to the GUI I'm sure - in the most recent release of Safari for Windows they have now dropped that (finally!).

The Java analogy is not so far off, even if no Java style VM is involved, it's a throwback to OpenStep on Windows which does some target platform abstraction. Apple keep their own cross platform versions of the development tools running on other platforms - Interface Builder (now known as "Xcode") ran on Windows and on both Windows and Mac could cross compile (for Win/Mac/Solaris).

The approach Apple use for bundling QuickTime and iTunes on Windows is technically very sensible (of course they should re-use components as dependancies).

The problems are (1) the UI for iTunes is not fully native (2) the Apple Software Update app for Windows (which is a clone of the Mac version) is obtrusive to an unwarranted degree and (3) QuickTime has some obnoxious options selected for default install, and while being great on Mac OS X 10.6, sucks big time on Windows (to the extent no-one would *want* it as their default media player).

Re:Non-native widgets

[tepples]

The problems are (1) the UI for iTunes is not fully native

As I understand your post, "native" means "using one of the GUI toolkits included with the operating system distribution". In that case, KDE apps aren't "native" on Ubuntu, no full screen game is "native", and Microsoft Office 2007 isn't "native" either.

Re:Alternate solution

[Khyber]

Wow, had to stop and fap to your own fanfiction.

And when...

[Anonymous Coward]

...the tool from Geek Squad copies your music while your box is in the shop?

I realize this is Slashdot and your head probably just exploded at the thought of *you* going to Geek Squad - but there's a simple and glaringly obvious problem with watermarked media files:

They are, ultimately, completely useless in terms of actually determining wrongdoing.

TreeWalk

[macraig]

Use your own ISP's DNS servers instead or run your own resolving DNS server.

The first suggestion is just no longer an option, for so many reasons, all of them based on lack of trustworthiness in this climate of corporate dominance and machination. I was using OpenDNS for several years, but recently I started using TreeWalk to host my own modest DNS server. Seems to work fine, and I don't even notice it's there.

Re:TreeWalk

[Anonymous Coward]

Use your own ISP's DNS servers instead or run your own resolving DNS server.

The first suggestion is just no longer an option, for so many reasons, all of them based on lack of trustworthiness in this climate of corporate dominance and machination. I was using OpenDNS for several years, but recently I started using TreeWalk to host my own modest DNS server. Seems to work fine, and I don't even notice it's there.

The only way you can hit a local Akamai cache server is to use your ISP's DNS server .Akamai sells their service as a method to reduce bandwidth costs, and so the local cache only takes traffic from that ISP's customers.

If you're going to run your own DNS then you can setup the best of both worlds- relay the requests from your server to your ISP's, and if you see their search page's IP come back then either refer it to a 3rd party instead or just replace it with a negative response entry yourself.

Where are the torrents?

[louarnkoz]

Load balancing based on the DNS resolver is so 1999! Even when it works, it works by chance, and does not test the actual speed between your PC and the potential servers. Compare that to Bit Torrent, which actually tests the speed of the downloads. You really wonder why Apple, and Akamai, would not use some kind of torrent technology!

Re:Where are the torrents?

[Anonymous Coward]

You really wonder why Apple, and Akamai, would not use some kind of torrent technology!

Because BitTorrent is a free open protocol which Akamai would not be able to charge money for.

Re:Where are the torrents?

[thegarbz]

Some of us have metered uploads and our net also slows to a crawl when we send a mere 75KB/s upstream even though we can download at 2MB/s. I like bittorrent as much as the next guy, but I actively disable it where possible for services which charge me money AND leech off my bandwidth which also costs me money.

Re:Where are the torrents?

[Jon Stone]

Akamai charge for their distribution servers around the Net. If they were to use torrents, those distribution servers would become the seeds and Akamai would still be able to charge a small fortune for them.

Re:Where are the torrents?

[louarnkoz]

I was not so much think of getting the bits from fellow users as getting the bits from several Akamai servers -- or similar. Instead of blindly following some dumb "closest IP" rule, the download could test a couple of candidate servers and get the bits from those with the best bandwidth -- much like torrents do.

Re:Where are the torrents?

[docblack]

You really wonder why Apple, and Akamai, would not use some kind of torrent technology!

I don't think you understand how this works, Akamai is just a web proxy: Akamai gets free rack space from an ISP, in return Akamai serves content (Itunes, MS, Farmville ;) to the ISP customers thus saving the provider from using their backbone for this cached content. It's a simple system. If you are using OpenDNS Akamai doesn't know which servers to use for your session. Since there is no client software beyond a web browser that is requesting files there isn't a whole lot you can do without adding complexity. Measuring speed is kind of pointless as I can guarantee the cached content at your ISP's datacenter is the best place to download from :) -DRB

Re:Where are the torrents?

[Anonymous Coward]

You really wonder why Apple, and Akamai, would not use some kind of torrent technology!

Because BitTorrent is a free open protocol which Akamai would not be able to charge money for.

HTTP is a free open protocol which Akamai do change money for.

Seriously, Akamai sell access to their infrastucture; they'll put your content on hundreds of geolocated servers around the globe. Whether they act as web servers or torrent seeds is your choice.

Re:Where are the torrents?

[thegarbz]

Ahhh gotchya. Would work well except for services requiring low latency. Torrents actually take quite a while to ramp up to speed. If the difference is 20 seconds with the DNS trick vs 1 min of negotiating and testing the waters with a variety of servers, the DNS solution would still win. But for large file transfers it sounds like a good idea.

Re:Where are the torrents?

[Shin-LaC]

Among other things, because BitTorrent has very poor latency (ie it takes a long time from the user's request to when you actually start receiving data).

Re:Where are the torrents?

[magus_melchior]

Not necessarily-- they could tell their clients "Hey, we've got this new BitTorrent technology to make content distribution faster and more reliable", and charge a premium for the service contract.

WTF

[droidsURlooking4]

is iTunes?

Re:WTF

[dontmakemethink]

WTF is iTunes?

It's the virus that is installed when you update Quicktime.

Re:WTF

[Anonymous Coward]

It's the shit bloatware you have to install if some malicious relative gifts you an apple portable device and you want to try it out...

Re:WTF

[Phrogman]

If Microsoft can be investigated for tying IE to their operating system in an attempt to dominate the browser market, why can't Apple be investigated for tying iTunes to Quicktime updates. Every time my browser or OS decides it needs to update Quicktime, I get prompted to install fucking iTunes.

Apple: I got news for you: I don't listen to music. I have no need of iTunes, ever, period.

It does act like a fucking virus in some regards.

Re:WTF

[iainl]

Since you can install Quicktime without iTunes, but not vice versa (on the admittedly sensible grounds that the latter uses the former to actually decode the files), I'd say you've got that backwards.

Re:WTF

[jo_ham]

Just untick the "install iTunes" checkbox.

How very virus like!

Stupid technology, aka thinko

[VincenzoRomano]

One question: do you think there's more information in the IP address of the incoming HTTP GET request or in the IP address of the incoming DNS query?
While everyone is using a browser, very few are running a DNS server. Provided that it's properly configured.

I blame Akamai

[fearlezz]

There are dozens of free dns services. Akamai knows this problem. But for some reason, they don't take appropriate measures.

Their DNS can serve an IP based on the geo-location. If visitors are using a dns server that is known for hiding the actual location, I would suggest serving the IP of a redirect-only HTTP server. The client connects to this redirect-only HTTP server and the server returns a "301 Location:" header based on the clients actual IP/location.

This will make the initial connection for users of Google DNS/OpenDNS a little bit slower, but then allows the available bandwidth to be used optimally.

4.2.2.{1-6}

[Anonymous Coward]

I'm surprised no one mentions the above free/public/fast DNS resolvers :-p

Re:4.2.2.{1-6}

[realperseus]

I'm surprised no one mentions the above free/public/fast DNS resolvers :-p

(Nodding head)

Steven J. Vaughan-Nichols' take on this

[Anonymous Coward]

Changing DNS probably won't help your Video Streaming

how come

[MadMaverick9]

this website http://ip-address-lookup-v4.com/lookup.php and this one http://ipxml.info/myip/?ip=213.251.189.203 are able to figure out my location correctly no matter what DNS server I use?

maybe time for akamai and company to change the way they figure out an ip address's geo location.

Re:how come

[Leebert]

Because those sites are determining your location based on an HTTP request directly from your browser, not a recursive DNS query through your DNS server.

Re:how come

[MadMaverick9]

Yeah - I know that.

And maybe that's what akamai, itunes and company are doing wrong ... duh.

Re:how come

[CaptSlaq]

Yeah - I know that. And maybe that's what akamai, itunes and company are doing wrong ... duh.

Maybe it's time to read up on Akamai (and other CDNs) and perhaps learn that "they don't just serve up HTTP content".

Re:how come

[MadMaverick9]

oh gee ... I didn't know ...
http://en.wikipedia.org/wiki/IPv4#Header
Every IP packet has a source IP address that tells the server daemon where the packet came from.
So unless somebody is using a proxy server, some other anonymizer or NAT, that source IP address is the location of the user. The source IP address of the NATing machine should be close enough for all intents and purposes.

Re:talk's cheap fucker

[thetoadwarrior]

Amazon's MP3s are plain old MP3s which require no proprietary software that could delete your MP3s.

DNS Benchmark

[Anonymous Coward]

GRC DNS Benchmark
http://www.grc.com/dns/benchmark.htm
find the best DNS servers for YOUR location.

My W2003 router-PC runs DNS server, which allow me to
specify many DNS Forwarders to spread any potential
privacy, performance, security, risks.
(Next re-install already planned for Linux)

Btw, put potential spyware like OpenDNS and GoogleDNS
where they belong, in PeerBlock or hosts file
OpenDNS:38.99.20.0-38.99.21.255
OpenDNS:38.103.65.96-38.103.65.97
OpenDNS:38.103.65.148-38.103.65.149
OpenDNS:38.104.56.48-38.104.56.51
OpenDNS:38.104.74.40-38.104.74.43
OpenDNS:38.104.124.60-38.104.124.63
OpenDNS:38.104.128.128-38.104.128.131
OpenDNS:38.104.140.44-38.104.140.47
GoogleDNS:8.8.4.4-8.8.4.4
GoogleDNS:8.8.8.8-8.8.8.8

Wait...

[Charliemopps]

People still use iTunes?

Re:Wait...

[jo_ham]

I would say at least 100 million people do.

However many iPhones and iPods there are.

Re:Alternate solution

[rjstanford]

That's true. However they do require you to install quicktime in order to get the codecs, unless I'm missing something. And for whatever reason Apple insists on not using any native widgets. Which means that not only are you installing crapware, but it also looks ugly on top of that.

The widget thing is just stupid. Hell, last time I checked (a long time ago) Safari was doing its own text rendering on Windows. Bleah.

Of course, Firefox annoys me for the same reasons. Oddly enough, most /. users don't seem to include its GUI in their rants.

Also we found out why the chicken crossed the road

[Anonymous Coward]

Yes, this "news" was exactly that informative.

Re:Alternate solution

[Leebert]

Just in case you didn't realize, on Windows you *can* uninstall Bonjour. iTunes will happily re-install it every time you upgrade, but it's not necessary for it to stay installed.

What about using pdns-recursor?

[FedeTXF]

I always use a local dns recursor server so I point my dns settings to 127.0.0.1. I can only see advantages privacy and performance-wise. The kind of problem described in this article seems to be another advantage to my apporach over using an external DNS server, but at the same time I rarely see anybody recommending it. What are the disavantages of using things like pdns-recursor?

Re:What about using pdns-recursor?

[cpghost]

What are the disavantages of using things like pdns-recursor?

It all boils down to the way you bootstrap your name server. As long as you use your local ISP's servers to bootstrap named, that's perfectly okay, and even desirable. But if you bootstrap from a public root server, this is only okay as long as not too many people do it as well. Think about it this way: if hundreds of millions of little SoHo-Routers were to bootstrap their DNS from the same public DNS server each time they are rebooted, this public DNS server will sooner or later stop replying to all those queries by consumer devices. This is similar to the NTP D-link fiasco.

better solution

[Ephemeriis]

Better solution - How about Akamai watches where the actual HTTP/FTP request comes from, rather than the DNS? That should get you closer to the client.

I'll start using my ISP's DNS servers as soon as they figure out how to properly configure/maintain/run them. Until then, OpenDNS and GoogleDNS it is.

Re:better solution

[Rockoon]

Better solution - How about Akamai watches where the actual HTTP/FTP request comes from, rather than the DNS?

Because when the request comes, its time to deliver data, not a redirect. The machine that receives that request either delivers its content, or the user experiences gets no content at all.

Remember that Apple (and Hulu/NetFlix/etc) is running the web pages and clients.

Akamai (and other CDN's) is only serving up the large video and music streams. Akamai is not in a position to modify Apples pages or client content, nor would Apple be open to that sort of thing.

Any other system than what they are doing would place demands on Apple/etc to provide 100% assurance that 100% of the clients can redirect/etc (hence, could only work for HTTP, and still not on 100% of HTTP clients either .. so Apple would also have to start white-listing clients before even revealing the akamai link)

Re:Alternate solution

[jo_ham]

You missed the huge box notifying you that all your music would be deleted because the iPhone wasn't paired with that machine? You just clicked the continue button, labelled "restore iPhone" (the buttons are marked by task, not with "ok").

iTunes is very clear about when it will delete the music on your iDevice, and asks you clearly with a detailed warning when you attempt to sync an iPhone with a copy of iTunes that is not it's "home" base.

If you missed that, then you just don't read warning boxes that require user action. iTunes will not delete your music without user authorisation.

Old issue / no longer an issue

[Anonymous Coward]

A quick Gogle shows OpenDNS has been aware of issues with Geographic caching since at least 2008:

http://ideabank.opendns.com/story.php?title=exceptions_to_permit_geographic_caching_download_sites_to_work

Also, Apple claims to have resolved the issue for AppleTV in the US:

http://www.cultofmac.com/opendns-we-offer-fast-appletv-streaming-in-north-america-but-international-performance-is-akamais-fault/74342

Re:Old issue / no longer an issue

[MikeDataLink]

Not true! I just had this issue the other day. My AppleTV was telling me it was going to take 3 hours to download an episode of The Office. I rebooted it twice. Same thing. I manually set the DNS to my own Ubuntu Box (instead of OpenDNS it was getting from my Cisco 891), rebooted and the download took 2 minutes and 50 seconds (50/50 FiOS). This is clearly still an issue.

OK, so we then have....

[Anonymous Coward]

MIL - Mother-In-Law
MILF - Mother I'd Like to F--k.
MILILF - ???

Sorry, this is wrong

[sjvn]

That's not how Google DNS or the other open DNS sites work with the Content Delivery Networks. Here's how the process really works:

http://www.zdnet.com/blog/networking/changing-dns-probably-won-8217t-help-your-video-streaming/467

The bottom line is that changing your DNS is unlikely to help with your video-streaming, and if it does, it's pretty much a matter of you lucked out.

Steven

Re:Alternate solution

[MobileTatsu-NJG]

You mean it isn't required I sync with it just to get Podcasts onto a device that already has internet connectivity?

I've never required a sync to download podcasts... heck I've done it from 3G.

Re:Alternate solution

[MightyYar]

I'm just amazed that a slashdot reader isn't savvy enough to backup their music to something more reliable than a mobile phone.

It can help to solve problems too...

[Anonymous Coward]

When iTunes fails to download a song, because it is corrupted on your local Akamai server, you can switch to OpenDNS instead. It worked for me once.

A hosts file can do the same, for less

[Anonymous Coward]

Hardcode the domain/host name of the one that performs best for you right into your hosts file, thus:

72.215.224.16 ardownload.adobe.com

That way you don't even need to waste CPU cycles on DNS servers, period. It will resolve to the fastest one you find that way, via hardcoding that into your local HOSTS file!

APK

P.S.=> Sure you could use the other IP addy too, 72.246.30.19, but from your tracert? Looks like more "HOPS" to me as well as more ms travel time also... just an idea, you probably know about HOSTS but, I thought I'd throw it out there for you - better than eating up CPU cycles & RAM running a daemon/service in DNS servers, that you may NOT really need (the article suggests doing it, but this is a cheaper/faster work-around imo)... apk

auto-detect always needs a fallback

[Anonymous Coward]

This is why automatic detection of things like this should never be absolute. If you do geolocation, and it makes a big difference on download times, SHOW THE LOCATION you're assuming. Then, allow the user to select another location if the one your program has assumed is wrong.

Re:Alternate solution

[devilspgd]

Technically this is true, but I'm more annoyed by QuickTime's stream of vulnerabilities and the potential instability of having unneeded kernel modem drivers.

Constantly uninstalling something every time iTunes has an update isn't worth the hassle, nor is pulling apart the iTunes installer and beating it into installing just iTunes without Bonjour every time.

Bonjour is the one component that doesn't really annoy me, if only because it occasionally gets used (or at least some iPhone apps can use it to find a PC, and it's sometimes easier/lazier than entering IPs)

Re:Your post is plagiarized

[Anonymous Coward]

Karma whoring alert: parent post is plagiarized wholesale from http://news.ycombinator.com/item?id=2051206.

I thought parent post was being inflammatory until I read the link it provided. The timestamps make it really obvious. Parent post is telling the truth, so why is it modded down? Truth hurts so bad you have to silence it? I thought this place liked free speech.

Re:Alternate solution

[Leebert]

For the record, I loathe iTunes/Quicktime as well.

Re:Alternate solution

[devilspgd]

Me too, but iTunes is a requirement to use an iPhone (call me whatever names you want, but I have WM6, iOS, BlackBerry and Android here. My iPhone is the only one I actually carry and use)

QuickTime isn't nearly as obnoxious as it used to be. It does still steal a few file associations upon installation but once you tell VLC to take back what it supports, QuickTime is mostly a "install it and forget it exists".

Gone are the days where QuickTime would run in the tray and take up CPU and memory despite the fact that you'd never actually used it.

Still, I'd happily give up iTunes' lackluster media playback capabilities entirely if I could lose QuickTime in the exchange.

Re:Alternate solution

[dakohli]

I agree. I usually don't bother to read all of the dialog boxes, because I see so many of them when I'm setting up a computer. It is like the uac in windows. You get used to seeing so many, that you just click through.

My point was, I'm tired of the idiosyncrasies of the Mac world. Why do I have to use iTunes? Why do I have to jump through hoops to sync a damned mp3 player with a new computer? Why can't I sync it with more than one computer? I'm pretty sure there are work-arounds, however I thought Macs and Apple products were just supposed to work?

Re:Alternate solution

[dakohli]

Ahhh, do as I say, not as I do?

seriously, I didn't really lose anything truly. Maybe a couple of songs. I do back up, in fact I have a backup server here at the house. I just didn't like syncing my phone all of the time. If apple allowed wireless sync, perhaps then it would be more convenient. Of course I really was just annoyed at how my stuff behaved. In fact the last time I lost some songs, Apple reset the download flags and let me re-download them. I suppose I could chalk up the wretched, twisted policies to the Music Industry, who really want us to buy multiple digital copies of the same song to play on multiple devices.

Third-party DNS resolvers in the Wild

[gsmaragd]

In our recent study, that involves vantage points in more than 50 commercial ISPs and content requests for around 10,000 hosts, we observed that the location of DNS resolvers break the assumption made by CDNs about the vicinity of the end-user and its DNS resolver. Moreover, we observed that third-party DNS resolvers do not manage to redirect the users towards content available within the ISP, contrary to the local DNS ones. We do believe that this problem is not limited to iTunes but may effect the end-user experience when downloading CDNized content that is already a significant fraction of Internet traffic. You can find more about our comparison of DNS resolvers in the Wild here: http://www.net.t-labs.tu-berlin.de/papers/AMSU-CDRW-10.pdf You can find more about our study on the effect of third-party DNS resolvers in content delivery here: http://www.net.t-labs.tu-berlin.de/papers/PFASF-ICDUPADI-10.pdf To better understand DNS and its performance, we would like to scale up the experiments and for this we are seeking your help. If you are willing to participate in this effort, please go to the following link: http://www.fg-inet.de/ Download the script that can be found in the download section of the website, and run it from an Internet connection provided by a commercial ISP, e.g., at home. The typical duration of the experiment is around six hours. All major operating systems are supported (Linux, Mac OS, Windows etc.). Once the experiment is done, please upload the traces on our website: http://www.fg-inet.de/upload.php Our script performs DNS queries for a number of predefined hosts. This list is included in plain text in the download packages. The traces collected with our program do not interact with any of your browsing or download history or activity. The additional bandwidth consumption and CPU load due to the experiment are negligible. The traces collected on this website will be kept confidential within the project and will not be distributed to any third party, nor shared with any third party. You also have the option to make them accessible to the research community if you wish so.

Re:Alternate solution

[MightyYar]

I just didn't like syncing my phone all of the time. If apple allowed wireless sync, perhaps then it would be more convenient.

I agree, and I have to wonder why they don't allow this. The only thing that I can fathom is security, or maybe speed. Come to think of it, it is probably speed - even a 16GB iPhone would take something on the order of 4 hours to do a complete sync. That much use of the wireless radio would probably destroy the battery time :)

I suppose I could chalk up the wretched, twisted policies to the Music Industry, who really want us to buy multiple digital copies of the same song to play on multiple devices.

They're all in it together :)

Re:Alternate solution

[MightyYar]

Why do I have to use iTunes? Why do I have to jump through hoops to sync a damned mp3 player with a new computer? Why can't I sync it with more than one computer? I'm pretty sure there are work-arounds, however I thought Macs and Apple products were just supposed to work?

That's the rub... there is a real engineering tradeoff between "choice" and "just works". Apple is not magic, but they are pretty good at making a limited set of choices do a respectable job for a large number of people. The tradeoff is that it gets frustrating when you stop drinking the kool-aid.

Windows is the other side... almost infinitely configurable and the ecosystem is damned cheap - but you pay for it by being the family IT guy. All that choice comes at the expense of complexity.

Re:Alternate solution

[dakohli]

Well, there is a third choice. Linux+Android. Not pretty at times, but boy does it work. Considering the alternatives........

Re:Alternate solution

[MightyYar]

I didn't mean to slight Linux - it is of course a choice, and it is even more complex and configurable than Windows.

Re:Alternate solution

[dakohli]

No offence taken. I'm not really a raving looney. But I gotta say, that this particular problem has been a bit of a bugaboo for me. I've used all of the platforms, and each of them has held a particular attraction at a particular time. I'm starting to feel though, that I need options, and I don't want the options to limit me down the road, because then, I feel the need for change.

to use a car analogy:

Windows: Get a pretty good basic car, and start adding up the options, each of which will cost.

Mac: Get a really good car, but everything is pretty much standard. No Choices.

Linux: Here's a pile of parts, and a bunch of tools. Go ahead, make your own damn car.

Re:Alternate solution

[MightyYar]

The "solution" that I've settled on at the moment is to use a combination of systems. I have a dual-boot Windows/Linux homebuilt for the wifey and for that occasional program/file that you simply must run on Windows. I have an Apple laptop and desktop - the desktop mostly used as a server. I'm in the process of setting up a freebsd (well, FreeNAS) file server to replace the Apple desktop, which is now about 6 years old.

I'll probably wire SOMETHING up to the TV again, but it obviously won't be the FreeNAS box :)