Providers Ignoring DNS TTL?

cluge asks: "It seems that several large providers give their users DNS servers that simply ignore DNS time to live (TTL). Over the past decade I've seen this from time to time. Recently it seems to be a pandemic, affecting very large cable/broadband and dial up networks. Performing a few tests against our broadband cable provider has shown that only one of the three provided DNS servers picked up a change in seven days or less. After turning in a trouble ticket with that provider - two of the three provided DNS servers were responding correct - while the third was still providing bad information more than two weeks after that specific change. What DNS caches ignore TTL by default? Is there a valid technical reason to ignore TTL?" "This struck me as odd, and I decided to run a few tests using my own domain. Lowering the TTL to twenty four hours, and making changes and then checking to see when a change was picked up. I queried twelve outside DNS servers/caches that I had access to (Thanks to my friends and relatives with dial ups and DSL who put up with me and my requests to reboot their machine daily!). Checks performed against these outside DNS servers indicate that it may take as much as four to five weeks before a DNS change is picked up! Most DNS servers picked up the change within 48 hours. A small number did not (three out of twelve - that's a quarter of them!)

This merits more study, and prompts a few questions. So, before I begin with a more serious broad study, I'd like to get some feedback on the problem as I've seen it. I know the tin foil hat crowd will see the failure to propagate DNS correctly as censorship, and the OS/bind/djb/whatever zealots will simply see this as an argument for their particular religion.

Based on the responses I get, I will then setup and test a couple of domains with different DNS servers for 6 weeks and report back the findings. [volunteers welcome!]"

Comments

For non geeks

[Nurseman]

from Wikipedia

Time to live (TTL) is an 8-bit field in the Internet Protocol (IP) header that indicates how many more hops this packet should be allowed to make before being discarded or returned. It is the 9th octet of 20 in the IP header.

and

Shorter TTLs can cause heavier loads on an authoritative nameserver, but can be useful when changing the address of critical services like web servers or MX records, and therefore are often lowered by the DNS administrator prior to a service being moved, in order to minimise disruption

Hope thaty helps someone beside me :-)

Yes there is

[NicolaiBSD]

Is there a valid technical reason to ignore TTL?

Yes there is, the other way around though. I used to maintain DNS servers for ISPs in the Netherlands and turned of caching completely. That way you prevent update problems and cache poisoning.
DNS caching was invented when bandwidth and CPU time were expensive. Not the case anymore. Caching is silly.

Re:TTL is useless, so who cares?

[Kupek]

I get a flamebait for a response to a troll? Nice.

[OT]Strange problem with Google's dns server.

[DJ_Goldfingerz]

Just as a side note, since we're talking about DNS. Over where I work, we have a check with nagios that verifies if our DNS server is up by asking it to resolve www.google.com.

Everyday for about a minute or 2, google's dns server will return a SERVFAIL error. Another symptom is that the problem re-occurs every 24 hours and 2 minutes. So everyday the problem re-occurs with an increment of 2 minutes.

We also run the check with a few more host and only www.google.com gives us an error.

Re:DNS practices

[TTK Ciar]

There is a good, high quality, low cost alternative to buying expensive load-balancing hardware. You can run LVS/IPVS on a linux box and turn it into an intelligent load-balancing router.

At The Archive we have dedicated LVS servers, but if you don't want to spend any extra $$ you can use a machine that is already providing some other service. You can use keepalived to make multiple LVS servers failover for each other.

I wrote a (very brief) HOWTO for setting up LVS/Keepalived. It is Archive-centric, but should be somewhat useful outside The Archive too. Just use rc.local or rc.inet2 or whatever instead of rc.final.

LVS/Keepalived (which is both, free and Free) has worked very well for us thusfar. Our www farm typically handles 30 to 60 http requests per second, with intermittent spiking above 250 http requests per second, and lvs01 sits at 99% idle all day.

-- TTK

Moderators please read!

[SiliconEntity]

The whole thing about the web is it's based on trust... it's amazing it works as well as it does... I can see providors not obeying TTL simply to keep their DNS servers from being stuffed (or whatever the word du jour is)... It's all about trust, and the lack of it nowadays on the web... get used to this sorta thing happening more and more.

What a trivial comment... and yet it's rated 5. The author has a habit of posting two and three line comments within the first 5 minutes after a topic appears, and many of them end up with ratings of 5. In this case, the comment was posted at 8:32 AM on a topic that appeared at 8:30.

Please, moderators, use your judgement more carefully. Don't give a 5 rating to these generic and nearly content-free postings. The web is based on trust... well, duh. That's not insightful or interesting! It's only because it was posted early that moderators gave it a boost, and then other moderators followed suit.

Let's save high level moderation for comments that actually offer some unexpected information or unusual insights, not ones that just restate platitudes that anyone could see are true.

Re:DNS practices

[Jailbrekr]

Alternatively, you can use freeVRRPD and pound. freeVRRPD for the failover, and pound for the SSL authentication and load balancing. An added bonus is that pound logs all hits and misses (in an apache format), so the logging is centralized. While the CPU utilization is higher due to the SSL authentication, it makes things much simpler as all you need in the web farm is a relatively simple HTTP server (be it apache or, ugh, IIS) with no need to worry about SSL authentication.