Providers Ignoring DNS TTL?

cluge asks: "It seems that several large providers give their users DNS servers that simply ignore DNS time to live (TTL). Over the past decade I've seen this from time to time. Recently it seems to be a pandemic, affecting very large cable/broadband and dial up networks. Performing a few tests against our broadband cable provider has shown that only one of the three provided DNS servers picked up a change in seven days or less. After turning in a trouble ticket with that provider - two of the three provided DNS servers were responding correct - while the third was still providing bad information more than two weeks after that specific change. What DNS caches ignore TTL by default? Is there a valid technical reason to ignore TTL?" "This struck me as odd, and I decided to run a few tests using my own domain. Lowering the TTL to twenty four hours, and making changes and then checking to see when a change was picked up. I queried twelve outside DNS servers/caches that I had access to (Thanks to my friends and relatives with dial ups and DSL who put up with me and my requests to reboot their machine daily!). Checks performed against these outside DNS servers indicate that it may take as much as four to five weeks before a DNS change is picked up! Most DNS servers picked up the change within 48 hours. A small number did not (three out of twelve - that's a quarter of them!)

This merits more study, and prompts a few questions. So, before I begin with a more serious broad study, I'd like to get some feedback on the problem as I've seen it. I know the tin foil hat crowd will see the failure to propagate DNS correctly as censorship, and the OS/bind/djb/whatever zealots will simply see this as an argument for their particular religion.

Based on the responses I get, I will then setup and test a couple of domains with different DNS servers for 6 weeks and report back the findings. [volunteers welcome!]"

Comments

Re:DNS practices

[TTK Ciar]

There is a good, high quality, low cost alternative to buying expensive load-balancing hardware. You can run LVS/IPVS on a linux box and turn it into an intelligent load-balancing router.

At The Archive we have dedicated LVS servers, but if you don't want to spend any extra $$ you can use a machine that is already providing some other service. You can use keepalived to make multiple LVS servers failover for each other.

I wrote a (very brief) HOWTO for setting up LVS/Keepalived. It is Archive-centric, but should be somewhat useful outside The Archive too. Just use rc.local or rc.inet2 or whatever instead of rc.final.

LVS/Keepalived (which is both, free and Free) has worked very well for us thusfar. Our www farm typically handles 30 to 60 http requests per second, with intermittent spiking above 250 http requests per second, and lvs01 sits at 99% idle all day.

-- TTK

Re:DNS practices

[Jailbrekr]

Alternatively, you can use freeVRRPD and pound. freeVRRPD for the failover, and pound for the SSL authentication and load balancing. An added bonus is that pound logs all hits and misses (in an apache format), so the logging is centralized. While the CPU utilization is higher due to the SSL authentication, it makes things much simpler as all you need in the web farm is a relatively simple HTTP server (be it apache or, ugh, IIS) with no need to worry about SSL authentication.