Slashdot Mirror
Mirror Listings Though TXT DNS Records?
mackman asks: "I was wondering if anyone has ever though about using their DNS servers to provide mirror information? A specially formated TXT-record could easily provide a DNS-cache-friendly mirror listing. A TXT-record would just need a list of servers and paths, or perhaps a more complicated mapping for servers which only mirror a subset of the original site. This would allow for much more flexibility than a basic round-robin A-record scheme. For example, instead of pounding the Red Hat web server to get a mirror listing (or relying on Slashdot posts for that matter), why not do a 'dig -t txt mirrors.redhat.com'? Of course we could build this into download managers like wget."
This is a very nice idea.
But for more usability the record should include the location (country and continent) of each mirror, so your favourite download manager can try the nearest mirrors first.
Another question is how quick such lists would be replicated/cached by your ISP's DNS server...
-- I love the smell of Blue Screens in the morning.
It's an interesting idea, especially for the mirroring information, but I really don't think that kind of functionality belongs in a DNS.
There's alot to be said for keeping something that basic to the working of the internet as we know it as simple as possible.
Also, if I understand correctly, that kind of thing would increase the load on the domain servers, especially with that sub-site idea. And I just can't see any widespread adoption of that kind of thing anytime soon.
"The worst tyrannies were the ones where a governance required its own logic on every embedded node." - Vernor Vinge
Instead of shoehorning in functionality into a free-form field (or so I assume, by its name), why not use something more appropriate? Why not write an I-D about how DNS should include "MIRROR" records? I'll leave it to the mailing lists to debate whether DNS should know so much about a name, and the specifics of what exactly is meant by "mirror" (what content, what protocols, etc.).
The IETF has some great things cooking. RSerPool, e.g. -- though it's not designed primarily for load-distribution (rather uptime, AFAIK). Perhaps I'm suggesting you use a hammer to crack a nut, but this might abate the need for posting mirrors, like you said.
Open Source is such a beautiful thing. It's unfortunate that so much of its energy is spent writing competing implementations of various similar protocols. But I digress.
Here at Georgia Tech, the primary campus Unix machines (acmex, acmey, and acmez) go through a "round-robin" DNS resolution process. Each time someone digs to acme.gatech.edu, they get one of the three machines, depending on conditions. Helps load balance the servers and protect against one of them being down, say by a systems student "accidently" fork-bombing the server...
ph34r teh p0w3r 0f th3 c0w
Just check out the capabilities. It might be the kind of thing you're after.
http://www.ietf.org/rfc/rfc2052.txt Doesn't do everything, but it definately accomplishes more advanced mirroring & balancing anyways. Combined with a little virtual-hosting on the mirror's side, all that's really missing is partial mirrors, as far as I can see.
"You know, Hobbes, some days even my lucky rocketship underpants don't help" -- Calvin
This idea wouldn't work because people would not implement it. It would take time to set up mirrors in a DNS and then to update them. If you are talking about FTP, some downloaders already search for and accept multiple sites. They also check for proximity of downloads.
If you are talking about websites like the slashdot effect, good luck on getting anyone to implement and maintain the table. A viable option for slashdot effect would be to mirror it in your journal. Or you could use kazaa.
void
where are the damn moderators??
In the Yellow Pages?
Look, if you think you can get a job without some serious man-on-man greasy action, I'd like to see it.
IDC has a lot of information on their site referring to general cases and specific case, adoption raes vs. current market share, etc., Just quoting this stuff out of context is meaningless.
It sounds like you're quoting accurately, but years ago I was talking to a guy at a Microcenter who seemed to know what he was talking about. Then he insisted that Apple had gone out of business. I couldn't convince the guy that there was still an operating business at 1 infinite loop. But he _sure_ sounded like he knew what he was talking about.
How about _specific_ URLs to back up your quotes?
the clock on the wall says 4 til 7
DNS is designed to be an extensible, flexible, distributed, and massively scaleable lookup database. It is quite suited to what you propose, and has certainly been used for much lamer ideas already.
I wouldn't use TXT records, though. SRV records might be more appropriate, but people are apparently using SRV for LDAP nowadays so you're probably better off just defining a new type. Use the procedure laid out in the RFCs for designating a new type of "experimental" DNS record, so that everyone will be compatible with you (well, everyone that follows the RFCs, anyway, which might leave out MS and DJB) or figure out how to use one of the obsolete record types.
You can look at some of the other types of records in RFCs 1035, 1183, 1706, 1886, 2672, 2782, and 2874. Most of the important ones (well, other than IPv6 stuff) are in RFC1035.
Incidentally, you're going to get dozens (if not hundreds) of posts here that will say "You shouldn't do that with DNS!". Pretty much any question you ask around here that even mentions DNS will get the same result, so just ignore them. I think these people are a lot more interested in shooting down ideas than in actually testing them, which is the only real way to find out if they will work.
The exact method of the guessing is tricky, a rDNS can be used, but it migt be slow, and not all non-US ISPs (or companies) use their country TLD in thier canonical or alias names. The visual-traceroute style locactions might be more accurate, but the lookup time might be prohibitivly expensive.
* This generally true unless the program does its own lookups (some versions of Netscape), or the machine is running its own DNS server.
It has become appallingly obvious that our technology has exceeded our humanity. --- Albert Einstein
I think HTTP is a better place for this.
Consider the following:
So, you could have a server "downloads.mycompany.com", which takes a request for file "/dir/file.tar.gz", and then issues a redirect to an appropriate download server ("http://useastcoast.downloads.mycompany.com/dir/f ile.tar.gz") based on a number of variables including HTTP headers, Client IP Address, and server load.
Since the functionality can be implemented on an HTTP server transparent to browsers, I would not be inclined to avoid changing the DNS (or any other) protocol to implement this.
As a side note, I do think that sites should implement this transparently, whenever possible. I get annoyed when I am asked to "pick a mirror"; I think that is the server's responsibility.
Just my two cents.
Search the RFC/IETF sites for URs or Universal Resource Names.
.. transparent. But then this requires caching servers everywhere and thats a thread I don't really want to get into right now.)
;-)
Squid implemented some basic URN stuff a while back - It'd run a cgi script at a URL which returned a list of sites in a specific format. Squid would then ping the sites and return them to you w/ the RTTs. You could then choose the 'best one'.
Not very effective - and I don't think it (the squid side) ever progressed past the initial stages - but maybe its the beginning for someone to tackle this issue again.
(IMHO it could be made more
Note - there's RFCs covering URNs for various things (IETF documents, ISBNs, etc)..
No, DNS isn't really designed for this. Yes, DNS has been perverted enough, so I guess this'd just be another extension. Write an RFC, get it published..
I was wondering if anyone has ever though about using their DNS servers to provide mirror information? A specially formated TXT-record could easily provide a DNS-cache-friendly mirror listing.
Akamai already do something like this, and they didn't need to extend the DNS protocol in any way. They even provide the mirrors for you, if you use their service, and take care of ensuring that users get redirected to the nearest mirror. Sure beats having to scroll through a long list like you have to on sites that mirror the old-fashioned way, particularly since a site physically close to you might actually have less bandwidth than a faraway one.
Drop a note to the IETF, get yourself a proposed RFC number, and go for it.
where are the damn moderators??
stfu and change your threshold you cry babie.
The sort of information the file would contain would include (5 second analysis):
If the browser sees that the current server's address is not the same as the listed orginal then it knows than it is a mirror. Also, the mirror file would have the ability to list all the sites that are mirrored on the server.
This file could then be be read by a search Engine and then when it shows the site in the search results it could list the associated mirrors. The file would probably be called mirrors.txt or mirrors.xml, depending on the format used.
This is my 5c worth. If you like the idea, then consider it free for the taking
Jumpstart the tartan drive.
I agree with this statement totally. You don't want to use an unstructured field, but the idea that this is bad beacuase it's an application domain is not valid. Email has used DNS for routing for a long time, and that is most certainly an application level protocol.
The larger problem is how to model the information. DNS already supports the concept of storing geo-positional information. The problem is that the raw location isn't indicitive of the best mirror. The network topology needs to be understood and properly recorded, never mind the security implications.