Mirror Listings Though TXT DNS Records?

mackman asks: "I was wondering if anyone has ever though about using their DNS servers to provide mirror information? A specially formated TXT-record could easily provide a DNS-cache-friendly mirror listing. A TXT-record would just need a list of servers and paths, or perhaps a more complicated mapping for servers which only mirror a subset of the original site. This would allow for much more flexibility than a basic round-robin A-record scheme. For example, instead of pounding the Red Hat web server to get a mirror listing (or relying on Slashdot posts for that matter), why not do a 'dig -t txt mirrors.redhat.com'? Of course we could build this into download managers like wget."

Comments

How about ENRP?

[brianjcain]

Instead of shoehorning in functionality into a free-form field (or so I assume, by its name), why not use something more appropriate? Why not write an I-D about how DNS should include "MIRROR" records? I'll leave it to the mailing lists to debate whether DNS should know so much about a name, and the specifics of what exactly is meant by "mirror" (what content, what protocols, etc.).

The IETF has some great things cooking. RSerPool, e.g. -- though it's not designed primarily for load-distribution (rather uptime, AFAIK). Perhaps I'm suggesting you use a hammer to crack a nut, but this might abate the need for posting mirrors, like you said.

Open Source is such a beautiful thing. It's unfortunate that so much of its energy is spent writing competing implementations of various similar protocols. But I digress.

Already done, of a sort

[themo0c0w]

Here at Georgia Tech, the primary campus Unix machines (acmex, acmey, and acmez) go through a "round-robin" DNS resolution process. Each time someone digs to acme.gatech.edu, they get one of the three machines, depending on conditions. Helps load balance the servers and protect against one of them being down, say by a systems student "accidently" fork-bombing the server...

RFC 2052

[gehrehmee]

http://www.ietf.org/rfc/rfc2052.txt Doesn't do everything, but it definately accomplishes more advanced mirroring & balancing anyways. Combined with a little virtual-hosting on the mirror's side, all that's really missing is partial mirrors, as far as I can see.

An alternative

[csmiller]

One alternative way is for the DNS server to guess what country the DNS request is coming from (For most people, the request comes from their ISP's/company's DNS server*) and then to automatically redirect the request to the nearest mirror. E.g. a request from the UK for ftp.debian.org is redirected to ftp.uk.debian.org, which may be round-robined across the UK mirrors. Of course, for HTTP requests this can be handled at the HTTP level.

The exact method of the guessing is tricky, a rDNS can be used, but it migt be slow, and not all non-US ISPs (or companies) use their country TLD in thier canonical or alias names. The visual-traceroute style locactions might be more accurate, but the lookup time might be prohibitivly expensive.

* This generally true unless the program does its own lookups (some versions of Netscape), or the machine is running its own DNS server.

Why not use HTTP?

[jbrayton]

I think HTTP is a better place for this.

Consider the following:

• When a web server receives an HTTP request, it knows the IP address of the client. That "Client IP Address" may be a proxy server or a NAT Public IP Address, but that Client IP Address will still usually be the one performing the download. This can be used (with some additional work/expense and questionable reliability) to better determine where the user is coming from.
• Web browsers can send a language and even a sub-language, such as "en-us" for US English. That gives the server information about the location of the client.
• Web servers have the ability to issue redirects to web browsers.

So, you could have a server "downloads.mycompany.com", which takes a request for file "/dir/file.tar.gz", and then issues a redirect to an appropriate download server ("http://useastcoast.downloads.mycompany.com/dir/f ile.tar.gz") based on a number of variables including HTTP headers, Client IP Address, and server load.

Since the functionality can be implemented on an HTTP server transparent to browsers, I would not be inclined to avoid changing the DNS (or any other) protocol to implement this.

As a side note, I do think that sites should implement this transparently, whenever possible. I get annoyed when I am asked to "pick a mirror"; I think that is the server's responsibility.

Just my two cents.