Slashdot Mirror
DNS Lib Underscore Bug Bites Everyone's Favorite Init Tool, Blanks Netflix (theregister.co.uk)
Reader OneHundredAndTen writes and shares a report: Systemd doing what it does best. From a report on The Register: A few Penguinistas spent a weekend working out why they can't get through to Netflix from their Linux machines, because when they tried, their DNS lookups failed. The issue emerged over the weekend, when Gentoo user Dennis Schridde submitted a bug report to the Systemd project. Essentially, he described a failure within systemd-resolve, a Systemd component that turns human-readable domain names into IP addresses for software, like web browsers, to connect to. The Systemd resolver couldn't look up Netflix's servers for Schridde's web browser, according to the report. In his detailed post, Schridde said he expected this to happen: ipv6_1-cxl0-c088.1.lhr004.ix.nflxvideo.net gets resolved to 37.77.187.142 or 2a00:86c0:5:5::142. When in reality, that wasn't happening, so Netflix couldn't be reached on his box. His speculation that libidn2, which adds internationalised domain names support to the resolver, was at fault turned out to be accurate. Rebuilding Systemd without that library cleared the problem.
Underscores are not allowed in domain names. Some resolvers allow them for historical reasons, because they were common in Microsoft environments that defaulted to converting a space to an underscore when entering the hostname on initial configuration, back when Microsoft thought that everybody would be using Microsoft Network and not Internet.
But they're not legal, and should NOT resolve. My DNS servers do not have the ancient msdos compatibility turned on, and reject them as they should.
libidn (internationalized domain names, punycode) do not use them either, and if it rejects them, all the better.
Does anyone know if they've settled on a timeline for pulling all SSH into systemd as well?
fencepost
just a little off
I guess you expected the headline to explain everything to you in full detail and with absolute accuracy, that's a pity.
But users with systemd is NOT an 'edge case' really. In fact it's becoming more like users WITHOUT systemd would be the edge cases, within *nix.
Lennart: CLOSED. WONTFIX. ..b-but its a bug!!
Slashdot:
Lennart: well yes I see how you could think that but once you use OpenRC it becomes very apparent that this bug disappears and is resolved, so of course, its not a bug.
Good people go to bed earlier.
"A Gentoo users ... recompiled a component... everything is working OK now".
How is this not working as designed?
The real problem here isn't that a handful of Linux users couldn't use Netflix.
The real problem is that, yet again, systemd has been involved in critical functionality breaking in an unusual and unexpected way.
It doesn't matter if it was an external library that systemd used that's responsible. Systemd is responsible for the problem because it uses this flawed library.
There's no reason for systemd to be involved with resolving domain names. Linux got by just fine throughout the 1990s, the 2000s, and even a big part of the 2010s without systemd being involved. Yet now that systemd is involved, things are going to hell.
Long time Linux users will be very aware of how problematic systemd so often is in the dumbest of ways.
Maybe somebody who just started using Linux in the systemd era thinks it's acceptable for their system to sometimes not boot properly, or for the domain name resolution to break unexpectedly. But long time Linux users know it wasn't like that before systemd was forced on the Linux community, and they know that such breakage is just not acceptable.
This is just the latest in a long chain of problems involving systemd. It has gotten to the point where Linux's reliability is below that of the BSDs, of macOS, and as much as I hate to say it, even modern versions of Windows!
Systemd needs to go, at least from important distros like Debian and Ubuntu. If Fedora wants to screw around with systemd, then so be it. But the other distros should remove it immediately.
The explanation is that input validation shows that Netflix is using illegal server names, and so this is really a Netflix issue, and is not a problem with systemd at all. In fact systems that access their illegally named servers are the ones with the bug.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
systemd = not-invented-here anti-UNIX botnet trash
Any explanation for this piece of shit problem, asshole?
Because he's technically correct, which is the best kind of correct... The DNS specification expressly prohibits the use of the underscore character in domain names. It's netflix that's at fault here, more than anything else.
...si hoc legere nimium eruditionis habes...
This is not a comment to malign the horros of systemd. Rather, I would like to point out that a significant subset of /. readers DO expect the headline to explain everything so that reading the article becomes unnecessary.
Just sayin'. But you know it's true.
One has to wonder what other subtle bugs are in systemd. Purely unintentionally, of course. No TLAs would want an opportunity to widely disseminate new bugs into vast numbers of systems.
I'll see your senator, and I'll raise you two judges.
But users with systemd is NOT an 'edge case' really. In fact it's becoming more like users WITHOUT systemd would be the edge cases, within *nix.
I believe the edge case is Netflix viewers running systemd, not just users with systemd. Sure many people view Netflix via Linux, but I doubt it is a significant portion of all Netflix viewers, thus an edge case. Offended by being referred to as an edge case? Perhaps "edge case" is a bit too much troll as the parent post is getting modded, "relatively minor case" may be more accurate.
Any yeah, systemd still sucks, but doesn't warrant sensationalized headlines.
I assume the poster wanted to be funny, right ?
Or is it one of those "black is white", "up is down" orwellian thing ?
Living in interesting times....
systemd network manager also does not do server stuff to well like bonding / bridging / etc.
Any explanation for this piece of shit problem, asshole?
Yes. libidn2 is not a default and is marked as experimental and not ready for use. Also libidn2 isn't maintained Poettering.
Now what would interest far more people is, do you have an explanation for being an unbearable cunt?
The systemd fan club's response is that underscores are not allowed in DNS, and that this is ultimately a libidn2 bug.
Both of these excuses are claptrap.
Underscores are not valid in hostnames. They are valid in DNS labels.
It is not the DNS resolver's job to translate internationalized domain names. It is the application's job to do so. The DNS resolver's job is to resolve the request. Full stop. Ten year old versions of bind will happily process, and pass on, internationalized domain name. This is because internationalized domain names gets transcoded into ASCII-compatible encoding and THAT's what in DNS.
The way that it should work is as follows: an application, such as a web browser, translates an international domain name into ASCII-encoded hostname, and then looks it up in DNS. It would be the application's responsibility to use libidn2, or some other equivalent, to do the translation.
A typical systemd fail.
A bug was noted in an optional library that wasn't default for any release of systemd. ... wait for it, this is the best part ... he notices a bug.
The following release of systemd downgraded support of the optional unused library libidn2 to experimental.
A pull requested was put in the bug tracker by the maintainer (not Poettering) to fix this in the future.
Some dude compiles a piece of software with an experimental library and
It makes front page news and Slashdot users start frothing from their mouth in their stupor.
And you wonder why complaints aren't taken seriously by developers. *golfclap*
Why in the FUCK is your init system messing with this type of shit?
What's next? Will you add an email client?
People read headlines on Slashdot? I just look at comment numbers and pop in, I really think this crypto currency stuff is getting dangerous. We need more Net Neutrality, because it will fix the problem with congress leaving too many tweets for Kaspersky to hack the elections.. appy apps? O.o
Read the rest of it idiot.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
It's abundantly clear that systemd-resolved has quickly become a train wreck. It's inclusion in Ubuntu 16.10 was widely lamented and many folks have pointed out huge concerns for several different assumptions that it makes for fallbacks and erroneous configurations. That's not including the several different bugs that have plagued systemd-resolved thus far. Granted many of them are fixed but with the breakage what have we bought? Something that's a pretty basic task now requiring patch after patch. Additionally, what has this solved? Now we can make DNS configuration a bit easier to integrate across the board?
The bad rep that systemd especially resolved has obtained isn't just simply one where grey breads say "it's too different". It is one that time and time again, ignorant assumptions, bloated egos, and hasty code have led to a general distrust, especially when tools that have always worked are suddenly not working or worse still, become methods for exploits. I still think systemd is a vast improvement over the "ye olde init scripts", but while the idea is commendable, it's execution has been somewhat lack luster to put it mildly. There needs to be a serious "Come to Jesus" moment for the systemd team. You need to build trust if your going to build something that's rewriting the books. This is just another example of how that trust is being chipped away. Complexity of the task at hand aside, either the team is up to delivering or they are not. This ostinato where breakage just keeps happening needs a serious all hands or something to restore trust in the team guiding this project. Poettering, you are doing no favors to yourself nor your team by these stories. Deliver us from the hell of bad init if that's what you seek, but don't plunge us deeper into a different hell of your making and say that it's alright because you're the one who built it.
Underscores are not allowed in top level domains names, for example you can't register example_domain.com.
However, in sub-domains they are perfectly legal. For example: my_subdomain.example.com is perfectly valid.
We were bitten by this a few months ago when Microfocus (Novell)/SLES updated some dns libs. We had been using underscores since, well, forever and it worked file right up to change. It took a while to figure out what had happened. A few record/config changes later and all was well.
Conservative, mod down for violating
Nor does it deserve the title Everyone's favorite init tool
Personally, I read that as sarcasm. I still presume it was intended that way.
I think we've pushed this "anyone can grow up to be president" thing too far.
Does systemd recognize IPv6? Can that be the issue?
I would like to point out that a significant subset of /. readers DO expect the headline to explain everything so that reading the article becomes unnecessary.
So the 140 character generation is becoming the (what?) 80 character generation?
In the words of a now (in)famous former, #Sad
[ And the irony of this quip is not lost on me. ]
It must have been something you assimilated. . . .
NO SHIT! Did you even bother to read the comment before replying to it, and before wrongly criticizing it?! OBVIOUSLY NOT! The comment you didn't read, yet still replied to, contained the following:
It doesn't matter if it was an external library that systemd used that's responsible. Systemd is responsible for the problem because it uses this flawed library.
By choosing to use this foreign library, the foreign library code effectively becomes part of systemd. If a user invokes systemd to perform some action, but systemd does the wrong thing because it uses a broken library, then it's both the library that's broken and it's systemd that's broken. Systemd can't be excused just because it uses a broken library. It's a problem with systemd as much as it is with the foreign library.
This is the issue on systemd's github. It actually notes that they are aware of this and downgraded support for libidn2 to experimental.
This issue isn't newsworthy. As others have noted in the comments, underscores are not supposed to be in hostnames (they can be in other DNS RRs) and is about a bug in an experimental feature in a release of systemd that is not in any stable distros. People running rolling distros using the latest versions of everything are going to experience bugs. That's not news.
It's getting more and more difficult to respect the anti-systemd arguments when issues this trivial make headlines. Add to this that many of the arguments raised against systemd are disingenuous or plain ignorant.
I have been using systemd-networkd and systemd-resolved on Debian 9 and so far I like it. It's easy and clear to configure, just like using systemd service units. The integration with systemd-nspawn is very handy. And it introduces new features such as domain name routing.
There's nothing "sensationalized" about the headline, you simply misunderstood it and took its vagueness (a caveat to brevity) to the x-degree of extrapolation.
Sorry, that's your mistake as the reader.
If you can't parse headlines and get 100% of the information, try reading the article to ease your confusion in the future. It does often help. YMMV.
This is completely false:
https://www.freedesktop.org/so...
Why do you lie?
:wq
RFC 2782 talks about SRV records, which are a different beast than A or AAAA records. SRV records deliberately use the underscore character to emphasize that they should not be resolved by the normal DNS resolution libraries. As per the RFC:
An underscore (_) is prepended to the service identifier to avoid collisions with DNS labels that occur in nature.
RFC 2181 talks about other record types (MX, SOA, NS, PTR, CNAME, and so forth), and just says that the DNS server shouldn't prohibit those types of records.
...si hoc legere nimium eruditionis habes...
Do Linux users who use SysVinit encounter this issue?
It is pitch black. You are likely to be eaten by a grue.
Here you go McFly:
" Note however, that the various applications that make use of DNS data can have restrictions imposed on what particular values are acceptable in their environment. For example, that any binary label can have an MX record does not imply that any binary name can be used as the host part of an e-mail address."
IOW, you cannot have underscores as part of the domain name, and this document doesn't change that fact. (Since, again, you clearly can't understand what you read and need someone to interpret it for you)
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
No, my Android phone does fucktard; I wasn't in front of a PC at the time. At least you aren't stupid enough to try to contradict what I said. Good for you; I accept your apology.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
systemd network manager also does not do server stuff...
[satire_mode = ON]
That's because apparently the systemd crew thinks Linux is only used in laptops and the occasional desktop, but never on a device with more than one network port.
[satire_mode = OFF]
Go on, citizen, stamp the vote card. R or D, your choice.
If this was a completely different situation, it wouldn't be the same. I couldn't agree with you more. You are a genius.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
Sure...in software there's no one to say that you have to program an elegant way to fail when you have what seems like garbage data coming in...except for the fact that the "garbage" data is really what the endpoint is expecting and other, more user oriented systems, handle without any issue. Systemd is behaving like a damn government bureaucracy that is completely detatched from the way the world works.
Also...as has been brought up several times in this post but has yet to be answered: WHAT THE FUCK IS AN INIT SYSTEM DOING NAME RESOLUTION FOR? There is no sane reason that name resolution should be available at boot time, unless you're doing a network boot; but in that case it's the pxe boot on the network card that's handling it. The system itself has no need for name resolution until the network interface is brought up, which should be well after the init system has confirmed the system is stable and handed off control to the Kernel.
Because if he told the truth he would have no ability to complain, of course :^)
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
It doesn't, nor do you have to use it with systemd. It is an option.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
systemd was initially an init system (excuse the pun.) It now affords much more functionality, however this added functionality is optional. That being said, plenty of C and C++ compilers will compile broken source code and, as you call it, "fail elegantly". This does not mean a new compiler that properly adheres to the spec is wrong. It means that the other compilers were broken. This might surprise you, but specifications exist for a reason. You violate them at your own peril. When you do, expect something that happened to give the results you were hoping for now to "suddenly break" later, because the reality is your design was broken the whole time.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
I did read the article, which is how I knew it wasn't about systemd taking down all of Netflix, rather a select group of users who couldn't get to Netflix because they were running a distribution that used systemd in conjunction with a dns resolver library that couldn't deal with underscores (either rightly or wrongly so.)
There are so many articles (here and in other places) that I don't have time to read them all and I have to rely on headlines to help make the cut. Call it "sensationalized" or just call it "click bait", in my opinion the headline was written to imply something bigger than the story was in order to get folks to read it. It was enough to get me to look and my post was my way of complaining. I get that sensational / click bait headlines are now common in other forums, it is sad that it is happening here with the frequency that it does. It is also sad that like other places the editors don't really do much editing.
I read the headline and thought, "what fucking idiot wrote this nonsensical headline" ? And then started reading and continued to wonder how something written so poorly got posted to /. Then I remembered how shitty the editors are here.
Comment removed based on user account deletion
The issue emerged over the weekend
Gentoomen will get the joke. BTW, systemd is not used by default in Gentoo.
Escher was the first MC and Giger invented the HR department.
Back in the 1990's I was asked if .COM and .NET should continue to accept underscore in domain registrations. This was after I added "check-names" to BIND to prevent address and MX records with non-LDH names being accidentally added to zones in contravention of RFC 952 and RFC 1123 (still the current host requirement specification). I pointed out that if underscore was permitted that people would be continually having to explain why address lookups for names like "a.label_with_underscore.com" would not work reliably. The requirements for registration were tightened to only allow LDH.
Enforcing LDH for host names allows us to use prefix labels with underscores without running the risk of colliding with valid host names. It allows software to pick out host names from free form textual contexts. When you mail client automatically creates a link that is what it is doing.
Netflix need to fix their provisioning system. Their use of underscore in a hostname is wrong.
That doesn't says that at all....
It in fact says, "you can have underscores as part of the domain name, but it's possible browsers may not bother to try to resolve it."
You are such a manipulative shit.
Nope. The spec specifically says underscores are a good choice elsewhere BECAUSE they are NOT allowed in A and AAAA records.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
IETF RFC actually, et toi?
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
Did you read his comment before you replied, motherfucker? He expressly stated why your logic is fucking stupid: "This has nothing to do with systemd except for the fact the user happened to be using systemd at the time, and systemd happens to use this library. What next? A kernel bug gets blamed on systemd because systemd uses the kernel?" And now, as the facts come in, it wasn't even close to systemd's fault as this was experimental and no way default. When you have to go out of your way to bitch like you do, you just weaken your argument as being a whiny little bitch.
systemd-resolved is an optional component of systemd. I run a lot of systems with systemd as init and none of them run systemd-resolved (or systemd-timesyncd for that matter).
64 characters ought to be enough for everyone!
Nor does it deserve the title Everyone's favorite init tool
Personally, I read that as sarcasm. I still presume it was intended that way.
Agreed. It's like Microsoft's famous 'Where do you want to go today?'
I always read that as the first part of a conversation with your evil jailer: "Where do you want to go today? 'Cos this trains going to Hell, with stops in Dis and the Lake of Fire. If you upgrade to First Class, we'll take the pitchfork out of your ass.... and put it somewhere else."
Then Microsoft dropped the slogan. And I kicked ketamine.
Crumb's Corollary: Never bring a knife to a bun fight.
why don't you check to make sure? accuracy is everything.
"The hands that help are better far than lips that pray." - Robert Ingersoll (1833-1899)
Your logic only gets used when systemd is involved, any other software that fails because of an external source will not get the same crap, they will defend the software involved and say "xxx is not at fault, its an external xxxx fault".
"The hands that help are better far than lips that pray." - Robert Ingersoll (1833-1899)
I believe the edge case is Netflix viewers running systemd,
No, it's Neflix viewers who use systemd-resolved. I use Debian Stretch with systemd and, despite Neflix using illegal hostnames including underscores, it works ok.
Watch this Heartland Institute video
Nope, the library with the "bug", libidn2, is a GNU project, not part of systemd.
systemd-resolved which uses it is an optional part of systemd. (By default Debian doesn't use it for example).
And it's debatable whether it's a bug or not -- DNS hostnames are not supposed to include underscores.
Watch this Heartland Institute video
network manager is not-invented-here anti-UNIX botnet trash
ifconfig is stable, works, does everything needed. it did not need a replacement. Expanding it with new functionality if needed should have been the proper path. Not re-writing it with a billion dependencies into a 45% functional pile of trash.
Which is all true.
NOTABUG -- underscores are illegal
WONTFIX -- the problem was in the experimental libidn2, not in systemd code
RESOLVED -- libidn2 has been fixed.
HTH. HAND.
Watch this Heartland Institute video
What kind of record is that? SRV or TXT -- OK. A or AAAA -- nope, try again.
Watch this Heartland Institute video
Nah, your phone doesn't run systemd. It runs the joke android init system.
My phone does run systemd.
Watch this Heartland Institute video
You haven't seen my custom ROM repo :^)
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
But here's a question: Did you even bother to read the comment before replying to it, and before wrongly criticizing it?! OBVIOUSLY NOT! The comment you didn't read, yet still replied to, contained the following:
So, before you go on, perhaps you can tell us why the systemd maintainers were wrong to adopt best practices in this case? Because I could have SWORN, sworn up and down, that the major criticism of systemd by its haters is that the authors insist on rewriting everything.
They haven't in this case. They've done exactly what the haters told them to do. And you're still going to criticize them?
How about just filing a bug ticket with the libidn team, and then shutting the fuck up?
You are not alone. This is not normal. None of this is normal.
How is that different? This is the undesirable behaviour you just listed.
Because the editors here couldn't hack it at a high school newspaper. There's no respect for us and they have no integrity, they are click bait assholes.
Also, the problem is in libidn2, not systemd. I like to bash systemd as much as the next person, but it's libidn2 that has the problem.
Hehehe -- criticism of Netflix for not following RFC's is now seen as trolling on slashdot.
Watch this Heartland Institute video