Slashdot Mirror

← Back to Stories (view on slashdot.org)

DNS Lib Underscore Bug Bites Everyone's Favorite Init Tool, Blanks Netflix (theregister.co.uk)

Posted by msmash on from the duh dept.

Reader OneHundredAndTen writes and shares a report: Systemd doing what it does best. From a report on The Register: A few Penguinistas spent a weekend working out why they can't get through to Netflix from their Linux machines, because when they tried, their DNS lookups failed. The issue emerged over the weekend, when Gentoo user Dennis Schridde submitted a bug report to the Systemd project. Essentially, he described a failure within systemd-resolve, a Systemd component that turns human-readable domain names into IP addresses for software, like web browsers, to connect to. The Systemd resolver couldn't look up Netflix's servers for Schridde's web browser, according to the report. In his detailed post, Schridde said he expected this to happen: ipv6_1-cxl0-c088.1.lhr004.ix.nflxvideo.net gets resolved to 37.77.187.142 or 2a00:86c0:5:5::142. When in reality, that wasn't happening, so Netflix couldn't be reached on his box. His speculation that libidn2, which adds internationalised domain names support to the resolver, was at fault turned out to be accurate. Rebuilding Systemd without that library cleared the problem.

17 of 292 comments (clear)

  1. Not a bug by arth1 · · Score: 5, Insightful

    Underscores are not allowed in domain names. Some resolvers allow them for historical reasons, because they were common in Microsoft environments that defaulted to converting a space to an underscore when entering the hostname on initial configuration, back when Microsoft thought that everybody would be using Microsoft Network and not Internet.

    But they're not legal, and should NOT resolve. My DNS servers do not have the ancient msdos compatibility turned on, and reject them as they should.

    libidn (internationalized domain names, punycode) do not use them either, and if it rejects them, all the better.

    1. Re:Not a bug by aardvarkjoe · · Score: 4, Insightful

      But they're not legal, and should NOT resolve. My DNS servers do not have the ancient msdos compatibility turned on, and reject them as they should.

      Although apparently the behavior that it has is to strip out the offending characters and then try to resolve the result, which doesn't make a whole lot of sense either.

      From the bug, it looks like the problem is caused by linking with libidn2, and support for that was marked as "experimental" in systemd, so this really doesn't matter much. You shouldn't be enabling experimental features in software unless you're willing to deal with potential problems.

      --

      How can we continue to believe in a just universe and freedom to eat crackers if we have no ale?
    2. Re:Not a bug by arth1 · · Score: 4, Insightful

      Don't expect the hostname to match functionality. One of the companies I have to download patches from every now and then have their ftp server named wwwonly.

      That said, and back to topic, underscores can be used in DNS, but not for hostnames, only for other services. Hostnames are restricted by rfc1123. So if it returned an SRV record or similar, it would be fine.
      But don't name a host with an underscore.

    3. Re:Not a bug by dgatwood · · Score: 5, Informative

      Disallowing underscores violates RFC2782.

      Nope. You misread it. That RFC says:

      An underscore (_) is prepended to the service identifier to avoid collisions with DNS labels that occur in nature.

      Which is to say that legal DNS labels may not include underscores. They are exclusively allowed for non-hostname types, such as service records, and they specifically chose that character for this use to ensure that it cannot conflict with any legal DNS name.

      --

      Check out my sci-fi/humor trilogy at PatriotsBooks.

  2. Re:Blanks Netflix for a userbase edge case by Anonymous Coward · · Score: 4, Insightful

    I guess you expected the headline to explain everything to you in full detail and with absolute accuracy, that's a pity.

    But users with systemd is NOT an 'edge case' really. In fact it's becoming more like users WITHOUT systemd would be the edge cases, within *nix.

  3. So reading between the lines... by Balial · · Score: 5, Funny

    "A Gentoo users ... recompiled a component... everything is working OK now".

    How is this not working as designed?

  4. The problem is systemd breaking unexpectedly by Anonymous Coward · · Score: 5, Insightful

    The real problem here isn't that a handful of Linux users couldn't use Netflix.

    The real problem is that, yet again, systemd has been involved in critical functionality breaking in an unusual and unexpected way.

    It doesn't matter if it was an external library that systemd used that's responsible. Systemd is responsible for the problem because it uses this flawed library.

    There's no reason for systemd to be involved with resolving domain names. Linux got by just fine throughout the 1990s, the 2000s, and even a big part of the 2010s without systemd being involved. Yet now that systemd is involved, things are going to hell.

    Long time Linux users will be very aware of how problematic systemd so often is in the dumbest of ways.

    Maybe somebody who just started using Linux in the systemd era thinks it's acceptable for their system to sometimes not boot properly, or for the domain name resolution to break unexpectedly. But long time Linux users know it wasn't like that before systemd was forced on the Linux community, and they know that such breakage is just not acceptable.

    This is just the latest in a long chain of problems involving systemd. It has gotten to the point where Linux's reliability is below that of the BSDs, of macOS, and as much as I hate to say it, even modern versions of Windows!

    Systemd needs to go, at least from important distros like Debian and Ubuntu. If Fedora wants to screw around with systemd, then so be it. But the other distros should remove it immediately.

    1. Re:The problem is systemd breaking unexpectedly by squiggleslash · · Score: 4, Insightful

      No, the real problem is that a library, Libidn, that's used by resolver libraries including that apparently shipped with systemd has a bug in it. The library dates back to 2002, it's not even as if systemd was relying upon some bleeding edge library written specifically for it. And yes, it's best practices, when implementing something like international domains to use a respected third party library rather than trying to roll your own, so they haven't made an error in relying upon it.

      This has nothing to do with systemd except for the fact the user happened to be using systemd at the time, and systemd happens to use this library. What next? A kernel bug gets blamed on systemd because systemd uses the kernel?

      The submitter is trolling.

      --
      You are not alone. This is not normal. None of this is normal.
    2. Re:The problem is systemd breaking unexpectedly by dgatwood · · Score: 4, Informative

      The real problem is that, yet again, systemd has been involved in critical functionality breaking in an unusual and unexpected way.

      No, the real problem is that Netflix violated RFC 1034 section 3.5 and RFC 1035 section 2.3.1, which both explicitly say that hostnames must still conform to the old ARPANET restrictions, which allow only letters, numbers, and hyphens. Underscores have never been legal in DNS hostnames, and in spite of the pain this spec-compliant behavior has caused for some users, the systemd behavior is correct, and Netflix needs to fix whatever broken software they have that incorrectly created an invalid hostname containing an underscore.

      The remarkable thing, frankly, is that any DNS resolver resolved that address, and more significantly, that the DNS servers actually responded to the request.

      --

      Check out my sci-fi/humor trilogy at PatriotsBooks.

    3. Re:The problem is systemd breaking unexpectedly by fahrbot-bot · · Score: 4, Funny

      What next? A kernel bug gets blamed on systemd because systemd uses the kernel?

      Wait! Who uses who now? :-)

      Sorry, I'm from the future, where there is no kernel, only systemd.

      Fun facts, after subsuming the kernel, the last non-systemd user land utility remaining is Emacs. Lennart and his (remaining) crew started the battle to absorb Emacs in 2040, five years before his death, and it's still raging many years after that. There have been casualties on both sides. Lennart died in 2045 when the experimental "systemd-elisp" module controlling his cold, robotic heart turned out to contain sleeper code, rumored to have been committed to GitHub by a radical faction of the FSF. He was found dead at his keyboard late one evening by the hooker he had ordered from Amazon Premiere earlier that day. The police report says he had been watching Trump porn.

      --
      It must have been something you assimilated. . . .
    4. Re:The problem is systemd breaking unexpectedly by zdzichu · · Score: 4, Informative

      Actually, the bug is not in libidn, but in libidn2. Or rather was – it got fixed rather quickly – https://gitlab.com/libidn/libi...
      As for systemd, it uses libidn by default. libidn2 support is marked as experimental – reasonable decision as this bug shows.
      The submitted article is pure flamebait - this is not a bug in systemd suite, but in 3rd party library; to experience this (already fixed) bug, distribution would have to have enabled experimental option. No sane distro does that.

      Nb. The Register articles with even a passing mentions of systemd are terribly misleading and often blatantly false.

      --
      :wq
    5. Re:The problem is systemd breaking unexpectedly by Anonymous Coward · · Score: 4, Informative

      Apparently you didn't read the RFCs, which do not say at all that "hostnames" "must" conform to anything. What they both say is that compatibility will be maximized if you use the host name syntax. RFC 2181 is also painfully clear that a DNS owner name may contain any octets at all. There is nothing remarkable about servers responding to such host names: they're supposed to.

      Indeed the "underscore name" convention is so important that it is how SRV records even work.

      _But_, and this is the key point, such names are not legal LDH names, which is what libidn2 is expecting. LDH names contain only letters, digits, and hyphens, and it's a foolish sysadmin who attempts to use some kinds of names (things that resolve directly to A or AAAA and probably CNAME or maybe DNAME and so on) that do not conform to LDH. This fact is what led IDNA to be invented: there's nothing preventing just looking up UTF-8 names in the DNS except that there's a lot of stuff that will probably break.

      And there remains the question of why in the heck systemd is involved in all of this. Systemd is the Windows registry of the Linux world.

    6. Re: The problem is systemd breaking unexpectedly by Anonymous Coward · · Score: 4, Insightful

      ...which is an utterly retarded design.

      Unix is a bunch of components by different authors, most with competitors, that use well-defined protocols to communicate. Unix works because stuff that sucks gets replaced, and no one person's vision defines what happens.

      Systemd and Windows are defined by one small man's vision, not by protocols and competition. And when that man doesn't think usernames should have certain forms, well, fuck everyone else, right?

  5. Yes, it is a bug by mrsam · · Score: 5, Informative

    The systemd fan club's response is that underscores are not allowed in DNS, and that this is ultimately a libidn2 bug.

    Both of these excuses are claptrap.

    Underscores are not valid in hostnames. They are valid in DNS labels.

    It is not the DNS resolver's job to translate internationalized domain names. It is the application's job to do so. The DNS resolver's job is to resolve the request. Full stop. Ten year old versions of bind will happily process, and pass on, internationalized domain name. This is because internationalized domain names gets transcoded into ASCII-compatible encoding and THAT's what in DNS.

    The way that it should work is as follows: an application, such as a web browser, translates an international domain name into ASCII-encoded hostname, and then looks it up in DNS. It would be the application's responsibility to use libidn2, or some other equivalent, to do the translation.

    A typical systemd fail.

  6. So let me get this straight by thegarbz · · Score: 5, Insightful

    A bug was noted in an optional library that wasn't default for any release of systemd.
    The following release of systemd downgraded support of the optional unused library libidn2 to experimental.
    A pull requested was put in the bug tracker by the maintainer (not Poettering) to fix this in the future.
    Some dude compiles a piece of software with an experimental library and ... wait for it, this is the best part ... he notices a bug.

    It makes front page news and Slashdot users start frothing from their mouth in their stupor.

    And you wonder why complaints aren't taken seriously by developers. *golfclap*

  7. Re:Blanks Netflix for a userbase edge case by Highdude702 · · Score: 4, Funny

    People read headlines on Slashdot? I just look at comment numbers and pop in, I really think this crypto currency stuff is getting dangerous. We need more Net Neutrality, because it will fix the problem with congress leaving too many tweets for Kaspersky to hack the elections.. appy apps? O.o

  8. READ THE FUCKING COMMENT! It addresses that! by Anonymous Coward · · Score: 4, Informative

    it was libidn2 that had the bug.

    NO SHIT! Did you even bother to read the comment before replying to it, and before wrongly criticizing it?! OBVIOUSLY NOT! The comment you didn't read, yet still replied to, contained the following:

    It doesn't matter if it was an external library that systemd used that's responsible. Systemd is responsible for the problem because it uses this flawed library.

    By choosing to use this foreign library, the foreign library code effectively becomes part of systemd. If a user invokes systemd to perform some action, but systemd does the wrong thing because it uses a broken library, then it's both the library that's broken and it's systemd that's broken. Systemd can't be excused just because it uses a broken library. It's a problem with systemd as much as it is with the foreign library.