Securing DNS From The Roots Up

jeffy124 writes: "This article at ComputerWorld tells the story of how ICANN would like to replace the root DNS systems with secured servers. Lars-Johan Liman, one of the root operators, spoke about the concept at ICANN's annual meeting today. He discussed how the world's current redundant DNS system is vulnerable to DDOS attacks and yet-to-be-discovered root holes in bind that can ultimately undermine the entire Internet by taking away the name-IP mappings that are relied upon by just about everyone."

Comments

News flash!

[Teancom]

Bind may be vulnerable to security exploits! Sendmail may *not* be as secure as qmail! Walking through harlem with $100 bills hanging out of your pockets isn't smart! Sky is blue!

Some people just never get the news....

My entire world is running amok

[Tsar]

The Internet is depending on unsecured servers for DNS? Now how am I going to sleep at night? Next you'll be telling me the earth isn't sitting snugly atop a giant turtle! Is nothing certain any more?

And once you lock those down...

[dave-fu]

...then malicious intruders will just go after the core routers, saturate lines, do things of that nature. Not that locking down DNS is a bad thing, but you can't defend everything all the time.

Why still running on BIND?

[kc8apf]

I have yet to find the great reason of why everyone uses BIND. I've been working on my own DNS server just for kicks. The protocol itself is trivial. It can be handled so easily, but yet, if you look at BIND's source code, you can't tell what is going on at all. So, why does everyone continue to use it? Or better question, why hasn't someone written a better alternative?

Re:Why still running on BIND?

[fanatic]

Already available is djbdns, written by D. J. Bernstein with security as a design goal. In fact, he offers rewards to anyone who can find a vulnerability.

Re:Why still running on BIND?

[DaSyonic]

djbdns, and other stuff written by him, (including qmail) is all under a restrictive license. He essentially prevents any vendor/distribution to release it, as any vendor would need to make minor changes, but a vendor can't even change the pathnames to certain files... that's not acceptable.

Read his license and see for yourself.

Re:Why still running on BIND?

[astrashe]

I don't know if it needs to be an SQL database, but it needs to be some system that will let you update the zone files dynamically and instantaneously.

The official BIND 4.x and 8.x trees are full of bad code that will probably yield serious bugs down the road. OpenBSD's audited BIND is based on 4.x, and as such lacks some important features that you really need. Theo and the gang have said that they have no intention of auditing the 8.x code, because it's so sloppy, and because they don't feel it adds enough to the 4.x tree.

I'm paranoid about the MySQL enabled version of BIND, because I'm not confident that it would be patched quickly in the event that an exploit was discovered.

I've been meaning to try to tie djbdns into a web application that could modify zone data -- the command line programs seem like they could be called easily by php, although I haven't actually installed it yet, so I could be all wet. If you don't do many modifications, it might be a reasonable compromise between a full blown SQL system and something more lightweight and secure.

A fully featured SQL database could probably keep itself in sync with djbdns with triggers, so long as you weren't pounding on the data and changing it frequently. That would eliminate SQL bottlenecks on the serving side, although I have no idea how it would stand up under heavy zone editing.

I wish the government would throw a couple of million bucks at this problem, and open source all the results. It would make the world a much better place.

Re:Why still running on BIND?

[Tet]

Why does everyone thing DNS zone's must be contained in flat text files? I would like to see a nice SQL backed system.

What on earth for? SQL is a general purpose query language designed to maximize flexibility over performance. SQL lets you do all sorts of complex nested subqueries and joins which simply aren't needed for DNS, so why have the overhead? It all comes down to using the right tool for the job. And in this case, a fast non-SQL database (such as Berkeley DB, for example) is far more suited to the job. Too many people equate the term "database" with "SQL", when that's just one of the options. Often it's the right choice, but sometimes it isn't, and this is one of those times.

Re:Why still running on BIND?

[shub]

Uh, yeah. Right. The first of what I'm sure will be many people to recommend djbdns. I've got a long list of reasons why djbdns is inherently bad, and I'll share some of them with you:

• By default, tinydns does not hand out referrals to questions it is asked about zones it does not control. I believe that this violates the spirt of the RFCs, if not the letter.
• By default, tinydns does not support the use of TCP at all. This most definitely violates the spirt of the RFCs, as well as the letter (if a DNS query via UDP results in truncation, you're supposed to re-do the query using TCP instead).
  Indeed, if you want to support TCP under tinydns, you have to configure an optional program called "axfrdns", which was intended to handle zone transfers, but also happens to share the same database as tinydns, and can handle generic TCP queries.
• The suggested method for copying contents of DNS zones is rsync, scp, or other remote copy tools. The DNS standard method of zone transfers (query type "axfr") is only supported as an additional, disrecommended method.
  The problem is that if you make a mistake and munge the database and then rsync or rcp that to the backup servers, you're totally hosed. Contrariwise, if you use the standard zone transfer mechanism, then the zone transfer should fail if the master is munged, and the slaves should keep a good working copy for a while and give you time to notice that the master is munged and needs to be fixed.
• Without a patch from a third party, tinydns does not listen to more than one IP address. If you have a multi-homed server, you have to apply a patch from someone other than they author, before you can get it to listen on more than one address/interface.
• Without a patch from a third party, tinydns does not support the standard "NOTIFY" protocol of informing secondary nameservers that the zone has been updated, and that they need to check the SOA serial number and download a new copy (if they don't already have it).
• Without a third party patch, tinydns does not support standard SRV records (which are intended to ultimately replace MX records, as well as perform similar functions for services other than mail).
• Like tinydns, dnscache will not bind to more than one IP address without a third party patch.
• Because they are separate programs, you can't have both tinydns and dnscache listening to the same IP address(es) on the same server.
  While this is not the recommended mode of configuration, some sites don't have the luxury of having separate authoritative-only and caching/recursive-only server(s), and need to mix them both on one machine (or set of machines). With the BIND 9 "view" mechanism, this is relatively easy to do. With djbdns, this is impossible.
• There aren't even any patches that can get djbdns to implement TSIG, Dynamic DNS, or DNSSEC, nor are they ever likely to be created (my understanding is that the author is strongly opposed to them).
  Unfortunately, as time goes on and more and more people are doing things like IPv6, VPNs based on IPSec, or people just care about being able to cryptographically prove that their servers are handing out the only correct information and that the clients are able to cryptographically verify this fact (think: electronic banking), these kinds of features are going to become ever more commonplace.
  Note that, with the advent of BIND 9, you can create a caching-only server that will validate cryptographically signed records, and all clients can benefit even if they do not themselves implement any of the new DNSSEC features.
• There are a number of things that djbdns does which I believe to be outright bugs. However, the author of this package simply refuses to accept that his code could be anything less than 100% perfect, and while he claims to have a "bounty" that he will pay for any bug that is found, in reality he is the one that gets to define what he accepts as a "bug", and has repeatedly demonstrated a tendancy to openly refuse to accept some purported bug, but then to quietly fix the code with future releases.
  So, let's look at some of these bugs:
  • When an IQUERY is sent to a djbdns server, it will respond with opcode set to QUERY. (it should simply copy the opcode, not make something up).
  • DNSCACHE (the caching server) does not respond to queries with the RD bit clear in the query. (Instead of simply answering from cache without recursing the dns-tree).
• One argument frequently used to support the use of djbdns over BIND is performance. Upon further investigation, this claim simply does not hold water.
  Benchmarks published by Rick Jones have clearly shown that BIND can scale up to at least 12,000 DNS queries per second, and there is every indication that BIND 9.2 will be able to go considerably higher. The best benchmarks available for tinydns indicate that it can handle at least 500 queries per second, but that is the highest number reported. Other people on the bind-users mailing list have indicated that they have performed their own (as yet unpublished) benchmarks of tinydns, and that it had notable performance problems that BIND did not suffer.
  The best published benchmarks from the author for dnscache report a query handling rate of less than one million records over a 4.5 hour period of time, which works out to an average of less than sixty-two queries per second. Even if you look at numbers of queries per CPU second, the best numbers they can provide are 13.7 million queries over a four week period of time with 128 minutes of CPU time used (an average of slightly less than 1784 queries per CPU second).
  Compare this with the requirement from RFC 2010 "Operational Criteriafor Root Name Servers" (since obsoleted by RFC 2870 "Root Name Server Operational Requirements") is that the machine and software in question be able to handle at least 2000 queries per second, and be scalable to levels higher than that. Indeed, recent reports have indicated that a.root-servers.net (considered by many to be the "primary" root nameserver) is currently handling around 12,000 DNS queries per second at peak.
  Preliminary benchmarks published on the bind-users mailing list have indicated that, on the same hardware, there is little or no performance benefit to using dnscache as opposed to BIND 9.1.2, and when these tests are re-run with BIND 9.2, I'm sure that it will come out even faster.
• Unfortunately, a lot of the reasons the author gives for running djbdns instead of BIND are related to problems in older versions of BIND which have been fixed or are largely non-issues in later releases of BIND 9.
  For example, he makes a big point of tinydns being better than BIND, because while the process is starting up, it still answers queries. While previous versions of BIND would not answer queries during startup, this is no longer a problem with BIND 9.
  Dan also makes a great deal of the fact that the djbdns tools run as a user other than root, and in chroot() environments. While the "monolithic setuid root" situation was an issue with older versions of BIND, even more recent releases of BIND 8 could be easily run as a non-priviledged user in a chroot() environment, and this is the preferred method of running BIND 9.
  Contrariwise, one of the legitimate big complaints about older versions of BIND is that they implemented zone transfers in a separate program. If the database was large, then the fork()/exec() overhead was large, and the system could seriously thrash itself to death as it copied all those pages (for systems without copy-on-write), only to immediately throw them away again when it fired up the named-xfer program. With BIND 9, this problem is solved by having a separate thread inside the server handling zone transfers, and no fork()/exec() is done. However, tinydns/axfrdns goes back to the fork()/exec() model that was so greatly despised.

And this doesn't even begin to get to the more core philosophical issues that Dan gives for running djbdns instead of BIND. I can easily rip him apart in these areas as well, but this takes more space than I think that I should devote here.

Suffice it to say that there is absolutely nothing that djbdns does that I believe can't be done at least as well (or considerably better) with BIND, and there are no security benefits it provides that cannot be provided at least as well (or much better) by a proper installation of a modern version of BIND.

I believe in the "security through diversity" scheme as much as anyone, but I'd take root nameservers running a program written in Bourne shell over djbdns. Hell, I'd rather fall back to using HOSTS.TXT than use djbdns.

Unfortunately, the other alternative of DENTS is also unsuitable for use as a production nameserver.

Show me something that is sufficiently better than BIND (and open source), and I'm sure that everyone will quickly gravitate towards it. Until then, BIND is the best we've got.

Re:Why still running on BIND?

[gorilla]

Here we have our internal DNS running off some Microsoft based system, with MSSQL as the datastore.

I've had to write a program which validates that all our hosts are still properly resolving, as every few weeks it drops a bunch of records. The adminstrators of the DNS have no idea why it happens, all they can do is manually re-add the addresses through the GUI system.

register.com's nameservers

[po_boy]

from the article:

Vendors at the conference offered their own security solutions. Register.com Inc. in New York, for example, has created its own propriety DNS software. The company continues to deploy BIND as well as its own software because diversity improves security, said Jordyn Buchanan, who worked on the team that developed the system.

Is there anyone here knowledgeable about this who can comment on a few things?

• Can I get the source to that in any way?
  
• Does it use a SQL database backend?
  
• Any chance of licensing it out even without the source?
  
• Does it support dynamic updates?
  
• Anything else cool about it?
  
• Are you hiring?
  

I'd love to see (more closely) another implementation of the DNS system other than the 3 or so commonly found.

Re:register.com's nameservers

[davidu]

I can't comment on register.com but at EveryDNS.Net we found bind to be too much of a risk to run for our servers. In the long run, DJBDNS has proven to not only be secure but also far easier to setup, administer as well as write parsers for.

Just my $.02,
davidu

Re:register.com's nameservers

[po_boy]

Would you really look at the source if it was given to you? Would you commit to fixing at least X bugs if they gave you the source?

I would probably look. I've leafed through the source to BIND to see how they do certain things. I doubt I would commit to fixing X bugs, but if I found any, I would submit patches. I have in the past to other people's work.

Does it really matter one way or another, or are you just throwing out buzzwords with SQL and backend?
Any chance of licensing it out even without the source?

yeah, it would be neat if it would. I've been looking for a good nameserver that holds its zone information in my database. I haven't found a good one for the hooks in BIND 9 yet, and I would be willing to ditch it if I found a descent replacement. I am also curious why they wrote their own, and I can see this being one of the reasons.

Since it's going to be used on root servers, I hope not.

I didn't realize they ran it any of the root servers. I just figured they ran it on some of the authoritative servers that they run. Those are the same servers that I update through their web pages rather regularly. Actually, I didn't even realize that register.com ran *any* of the root servers.Which ones do they run?

No need to comment on the last two.

Yeah, trolls aren't known for their stamina.

DNS? Ha!

[tang]

Real men surf the net using ip addresses. (And NOT in base 10)

djbdns and opennic

[SuperDuG]

djbdns states "I offer $500 to the first person to publicly report a verifiable security hole in the latest version of djbdns." ... and no one has claimed the $500 yet.

Also OpenNIC is an ICANN indepent root system ... why not just use them instead of ICANN?

Re:djbdns and opennic

[mutende]

And then there's the Open Root Server Confederation who has instructions for setting up your own root zone using DJBDNS or BIND. Highly commendable! And you get a whole slew of new TLDs in the process...

DNS in inherently flawed...

[Jeremy Lee]

Don't get me wrong. It's a great system, it's worked for a very long time, it does it's basic job admirably. My single main issues with it are it's centralization, and increasing politicization.

I've given this a little thought over the years. There's a few fundamental issues with the centralized DNS system.

I've tried kicking around a few replacements ideas, like a peer-to-peer exchange system carrying certificates that act a little like resource search records.

The FreeNet project actually gives a good model for how to distribute and search for these 'domain certificates'.

I'd like to see a system that you essentially 'anonymously' submit namespace entries to. Conflicts are resolved based on context. If a dozen people want "money.domain", fine. If you try to browse to it without any context, you have to choose which one you want based on other information in the certificates (full name, location, nickname etc) and once you've chosen, that context sticks. URL's would need to be extended to also carry this context, which probably need to be a cryptographic signature to prevent abuse.

It constantly amazes me that people are willing to pay $50 to 'own' a record in a database. The domain land grab was just stupid... in virtual space, you can always just make more land. As .info proves.

DNS will obviously persist for decades, (simply because of the financial and general mindspace investment in 'dots') but hopefully as only one of a plethora of address resolution systems. Name resolution needs to be a pool, not a tree.

"For as long as the DNS system exists, the Internet will never be free" - Morpheus, while very Drunk

Re:DNS in inherently flawed...

[Elwood P Dowd]

I'm sorry, I want DNS to work instantly. FreeNet gives a great model for how to solve this problem if it were ok for DNS to take between 3 seconds and 3 minutes to resolve. 3 seconds is too long. Centralization is necesary. Redundant is good, but it should still be centralized. If anyone can tell me how a decentralized DNS system would allow fast lookups of uncommon names, then I'll change my mind.

Re:DNS in inherently flawed...

[Webmonger]

.Info proves nothing. Our company just registered some .biz and .info domains, and I've advised against using them for anything important.

.info and .biz basically turn into blackmail, of the form: "What if someone typed in your domain name, and they didn't get your site? It could happen to you, if they type in acme.biz and someone else has registered it. So pay us money and it won't happen." The domains themselves are fairly worthless, because you get funny looks if you use a .info or .biz tld.

Web addresses should be memorable names. "yahoo" is easier to remember than "www.yahoo.org". And with www.*.com names, all people do remember is is "yahoo". The rest quickly becomes standard.

For humans, "yahoo", "cnet" and "amazon" are all top-level domains. .info just makes things harder to remember. And a lot of the .info names are the same as their .com equivalents.

Instead of creating new tlds that are mostly duplicates of existing tlds, we should be restricting domain ownership, so no legal person can own more than one domain. That should prevent people and companies from spamming DNS, so that good names remain available.

Starting to back it up.

[miguel]

This time I will be prepared.

I am downloading as we speak all the DNS records in the planet into my /etc/hosts file so I can be immune to the attacks

I encourage others to do the same.

Re:routing != DNS

[Zeinfeld]

The bottom line is this. Don't be too worried about DNS going down. Unlike www.microsoft.com or www.whitehouse.gov, there is little incentive for a malicous script kiddie to attack DNS.

Untrue, DNS is like www.whitehouse.gov under permanent attack. The article is based on a number of assumptions that are not true of all the root servers.

Steve Bellovin is somewhat inaccurate in his statement about BIND. While it is true that most of the Root servers run code that originated in BIND most of the heavy lifting is done by a few servers that sit on the fattest pipes that run a stripped code base. The code paths of that code base have at this point been as near to completely tested as anything gets.

The real problem is that most of the root servers are still maintained by the ad-hoc volunteer network of the 1980s Internet. As a result many of the 'root servers' are hosted on drinking straws rather than pipes.

There are 13 servers however and all have to go down to take out the Internet. Even then the effect would take some time to be felt. The root servers only manage the top level domains. These tend not to change very often and so the TTL on the root records can be made very long without causing operational difficulty.

A much more serious problem would be if someone brought up a fake root server. DNS does not provide authentication.

Rather than obsess about the code base problem ICANN should be either deploying BIND or telling the IETF the characteristics of the security protocol it really needs.

DJBDNS doesn't obey many RFC's, not OSS either

[dido]

You can't do zone transfers using djbdns for one thing. DJB thinks that zone transfers are evil, and has his own method for doing the task (rsync over ssh I believe), but whether they're evil or not is beside the point. Like it or not, zone transfers are a part of the core DNS protocols and any proper successor to BIND must implement them all. Starting a standards war with the IETF is not something I want to have along with a name server I deploy. Let Bernstein write an RFC for publication describing his idiosyncratic methods and get the IETF to ratify it as a core standard if he wants, if he truly thinks his way is the better way. The way he operates reminds me more of the way Microsoft handles standards than anything else.

Besides, djbdns is also deficient in a far more important way (for me and to a lot of people here on Slashdot anyhow, I hope): it's actually proprietary software with a limited license for gratis use. It's not Free Software or even Open Source, not by any reasonable definition of the term. There is no license along with his programs, and absent a license you have NO RIGHT to share, study, or change Bernstein's code!

axfrdns, part of djbdns does zone transfers

[bbk]

check here:
http://cr.yp.to/djbdns/axfrdns.html

this supports outgoing transfers. Incoming are a possible security risk (NO authentication happens in most cases, other than IP address checking, IIRC), making this a prudent decision, IETF or no.

BBK

ICANN does something useful

[kingdon]

Is it my imagination or is ICANN actually working on getting their job done rather than horribly complex politics (more complex than needed to solve the problem), or trademark/legal craziness? There's some background at the page of the ICANN DNS Root committee.

Now, I'm pretty skeptical that a closed source DNS server from Register.com is going to be a big part of the solution, but even that I don't really mind so much. Having a few alternatives is good if for no other reason than helping to keep BIND from stagnating.

The article didn't talk much about DNSsec (or this older page) which has got to be part of the solution (to try to give the 10 second summary, when a client makes a DNS query and gets a response, it is kind of tricky to ensure that the response is really from the correct server, and DNSsec uses crypto to solve this and other problems).

Want to solve all the BIND security problems???

[evilviper]

The answer is simple, just ask the author of IPF how he did it...

Change the BIND license to make it much more restrictive, then sit back as the OpenBSD developers build their own simpler, better, more stable, and much more secure, replacement.

SSH.
IPF.
BIND?

Security, reliability and the like

[Jordy]

Reading this article, I have to start wondering if maybe I'm misunderstanding the problem.

The actual root servers are only queried for the top-level domains and while they have rather massive databases, the types of queries they get is limited.

Now, I'm going to assume that given all the money collected for domains, there somewhere exists a nice pot of money available for running root DNS servers. If there isn't then something is seriously wrong with the administration of DNS.

Segmentation of the actual root servers from the world by utilizing a front-end dns cache that would rewrite the actual DNS queries would solve a lot of problems.

First, rewriting queries would allow an amazing amount of sanity checking to be done on the query itself and should prevent exploiting the back-end root servers directly.

Second, as front-end dns caches can be extremely simple and require almost no configuration, the OS installation can be absolutely minimal excluding even shells. You could go as far as to use an OS that allowed you to revoke system privledges such as certain syscalls (fork, exec, open, etc aren't all that necessary once everything is running) and even make the caching DNS server run as init (though you must have something to bring up networking interfaces.)

Physical segmentation is obviously important as well so a private backbone strung between all core root servers and a seperate interface on each front end cache to access them would help quite a bit.

Of course then comes the issue of DoS attacks which again should be rather easy to solve considering what we are talking about. Just buy a lot of front-end cache systems. You would think given how important root servers are and how much money domain revenues generate, buying a thousand or even ten thousand machines and sticking them in every major network access point wouldn't be all that big of a deal.

Now you still have to deal with the fact that most DNS servers still have a static list of root server IPs. Thankfully, the simple DNS queries that hit root servers can be done with a single UDP packet request and response (until you have to work up the hierarchy) making them prime targets for one of the many clustering solutions out there from simple IP sharing virtual servers to routing protocol tricks.

Of course, I may be oversimplifying the problem.

/etc/hosts!!!

[chrysalis]

100 Gb hard disks are cheap nowadays, and almost all OS support > 2Gb files. So securing the DNS from the roots up is simple : have a local /etc/hosts file with all existing hosts.
Then, subscribe to a mailing list that sends daily changes, so that you can keep your /etc/hosts file up to date.
Ehm... yeah. You first have to secure mail to do this.

Re:/etc/hosts!!!

[sharkey]

You first have to secure mail to do this.

Actually, using secure mail would get tiresome, don't you think? What is needed is a mail user agent that will simply take the incoming mail, run it as root, and modify/add whatever files neccesary without admin or user intervention. Now THAT would be a time-saver, huh?

Re:Homogeniety is bad

[A Masquerade]

I understand that at least one of the root servers is running an alternative DNS implementation produced as commercial licensed software by Nominum (who also produced and maintain the Bind 9.x implementation under contract to the ISC).