Slashdot Mirror
Multiple Linux Distributions Affected By Crippling Bug In Systemd (agwa.name)
An anonymous reader writes: System administrator Andrew Ayer has discovered a potentially critical bug in systemd which can bring a vulnerable Linux server to its knees with one command. "After running this command, PID 1 is hung in the pause system call. You can no longer start and stop daemons. inetd-style services no longer accept connections. You cannot cleanly reboot the system." According to the bug report, Debian, Ubuntu, and CentOS are among the distros susceptible to various levels of resource exhaustion. The bug, which has existed for more than two years, does not require root access to exploit.
I got a bad case of loving you!
I love systemd!
and been around for 2 years and doesn't require root access??
If this happened on Windows, I & many others would be scornful of it.
Pain is merely failure leaving the body
Putting this level of complexity at such a low level of the system is going to cause show stopping bugs. And, with every new release, more complexity is added.
This is not a bug, it's a feature. This is the systemd way and you've all being doing it wrong!
Regards,
The Systemd Developers
Fortunately, we have so many different Linux distributions that this is not a problem! (...right?)
Ezekiel 23:20
Comment removed based on user account deletion
Hot-buttered Karma; so delicious.
All the people that were telling you that this init system called Systemd was overly complex, unaudited and insecure had warned you that this was coming. All the "Troll -1" modding on people that posted such warning here did not prevent the inevitable.
Not convinced? Here's a graph of the number of issues opened/closed since systemd moved to github last year.
Anons need not reply. Questions end with a question mark.
Nope. Never bothered with systemd. Can't really claim foresight, just never felt like rebuilding all that plumbing.
Lacking <sarcasm> tags,
requiring a powercycle. Doesn't endear systemd to me in the least.
Everyone who mocks these distributions for not toeing the Debhat line can all enjoy my "told you so".
I strongly urge people to RTFA for a detailed description of some of the technical problems with systemd.
It feels surreal that the most senior people in the Linux community, after decades of attempting to put out a credibly secure client and server platform, suddenly almost all decided to switch to this product.
How long are systemd proponents going to evade accountability to crying about detractors, greybeards and positoning opponents as anti-change.
Any criticism of Systemd and out come a hoarde of Redhat supporters and astroturfers to change the focus swiftly from the technical to the political
So this is less a technical struggle than a political one?
slashdot: A failed experiment.
https://medium.com/@davidtstrauss/how-to-throw-a-tantrum-in-one-blog-post-c2ccaa58661d
Can't have anyone criticizing any aspect of the holy systemd.
Whole thing boils down to:
"Following security practices in an init system is hard, and you've never done it so leave us alone."
Completely ignoring the fact that the only reason they patched this thing is because he made a big deal out of it.
And on what planet is testing for corner cases like empty strings the domain of fuzz tools?
That seems like a pretty standard test case to me.
I can understand if you don't test for a 1MB string, but empty seems like a no brainer.
If you're dissatisfied with systemd and you don't need any of its fancier capabilities (which as an end user I'm assuming would be Docker stuff), please consider switching to a non-systemd distro as soon as possible and (if you can afford the time or money) contributing to their development. The more support systemd alternatives can garner, the more likely it is that projects to will resist unnecessary systemd dependencies and it might even be that systemd itself will eventually become more modular and moddable.
I'm not a hater. I cringe every time I see +5 comments claiming that systemd didn't fix anything. Declarative syntax is (at least in principle) a massive win, especially for distro builders. And LXC is amazing stuff, and I certainly cannot fault Red Hat for wanting containers to behave perfectly. Unless something like Genode scores a major coup, containers are definitely the future of secure and robust computing.
But the actual details of systemd's course have been hair-raising. It needs to be more UNIX-like and less draconian in its requirements and less toxic in its effects on the FOSS ecosystem and unfortunately (given Red Hat's behavior over the past decade) it appears that pushing alternatives hard is the only way they can conceivably be convinced to change their ways or reform anything moving forward.
I encourage all of the haters here to try and put your money where your mouth is. Install, use, support and help promote a distro like Devuan or even better: go and find one of the multiple OpenRC distros available. OpenRC can't be the all-in-one automagic solution systemd endeavors to be, but it doesn't hide tons of stuff in huge C binaries and it's addressed most of the common frustrations people have with SysV. Arch Linux has an OpenRC variant (the standard install uses systemd), Gentoo was the distro that started OpenRC years ago, and Alpine linux uses it (which isn't an ideal easy desktop distro, but it's amazing for those wanting a secure minimal distro to build on and last time I checked it does run XFCE and Firefox.) There are probably others.
No its a technical struggle.
The UNIX philosofy is to make many smaller programs that does one thing and does it well. From a bug point of view that been godsend; smaller programs are easier to debug and test.
Large complex programs will always be a problem. Like webb browsers and systemd. The more complex a program becomes and the more it does the more complex is it to write secure code for all situations.
Just saying it like it are.
Yet more proof that Harry fucking pottering and co are not even remotely qualified to design the Linux init system.
The continued utter incompetence displayed by these idiots is utterly fucking astounding.
Yes. Let's keep adding more and more shitty, ill considered crap, to a poorly designed steaming pile of shit.
Surely the systemd cabal can't continue with their bullshit that systemd is modular and well designed.
Fuck pottering red hat Debian for forcing this shit on all of us.
Also, dbus. It's a message bus AND an rpc mechanism. What the fuck? Who the fuck would do that? Who thinks its a good idea to implement a message bus and rpc together?
Dbus needs to die.
Replacement: something like nanomsg - a system level message bus.
If we need rpc, don't we have a bunch of things in the kernel already? And only if those won't do, let's have a separate local only rpc with multicast.
Seriously, dbus, kdbus, bus-1, and all fucked, and designed by utter fucking noobs who clearly don't have any right designing OS level components.
"Debian, Ubuntu, and CentOS are among the distros susceptible to various levels of resource exhaustion."
Whew! Thank goodness I run Red Hat!
#DeleteChrome
In the meantime you may avoid using systemd as init in Debian by installing sysvinit-core or in Ubuntu by installing upstart-sysv in your transition to a systemd-less distro such as Devuan.
If you are using Debian Jessie, you can switch to Devuan by simply changing repositories. Its still in beta so don't do it on production servers yet. But do plan your migration, before this gets out of hand.
Artix
Your Linux, your init.
It's the only way to be sure.
The developers haven't stopped at what systemd needs to do and have gone on to what they want it to do, favoring the latter over the former.
It must have been something you assimilated. . . .
Yes.
Since office politics at RedHat decided this was the way to go and because they are putting in a lot of resources then that's the way it's going.
Also the way Lennart lobbied the gnome people to made things depend on systemd was very political. If you want the current gnome you need systemd or an extremely complicated workaround to make it multi-platform again.
This is why the mass standardization of Linux is a bad thing
Slackware user here.
What bootstrapping if any does it use?
I haven't really played with WSL much but I liken it to a chroot. On my linux box I used to mount a USB drive containing another distro and chroot worked for the most part, except certain processes seemed to run better with some form of pre-run init and for simplicity I ended up using systemd-container to start the necessary services!
Hence the windows environment is emulating linux syscalls behind the scenes but I would think there's some sort of init system magic occurring when the bash terminal is started up.
There are no non political struggles when humans are concerned. You should have learned this in kindergarten.
Seven puppies were harmed during the making of this post.
No, see the highly rated post above. There is no bug. The fact that there are problems after issuing that command is a coincidence. The server just coincidentally was hit by cosmic rays around the same time. Other reports of the same problem are similar coincidences. The systemd OS is just as perfect as the Microsoft operating system it takes its inspiration from.
Yes.
You should be careful with this. I don't disagree at all with the broader criticism of Red Hat's politicking; in fact in many ways I think the criticism has been far too limited. For example, their involvement in GNOME is inexcusable given not just the attitudes of the devs but also the explicit stated goals of the project.
But the longer you deny that there is any point whatsoever to systemd, the more damage you do to your cause, because it *does* do some stuff that it tricky to do in SysV. Want to change peoples' minds? Show them how to do something similar in OpenRC, or explain some other workaround. Emphasizing only the bad and pretending that the good doesn't exist usually does not end well, even if the good is overwhelmingly outweighed by the bad.
For example: containers are not fucking niche usage cases. Containers are the future of secure, robust, high performance computing. In the future, they will be widely used regardless of whether or not the user realizes that they are being used. Calling them niches that most users don't care about makes me think that he is a clueless, dogmatic fool. It makes me less likely to pay much attention to anything else he says.
init. Systemd is a bunch of Bernie Bros who think they can will away basic arithmetic to get something for nothing.
I am not an expert coder or sysadmin, but a quick read of SOLARIS SMF feature it would seem to address all the needs of a robust init system, without the many concerns of a too powerful/critical PID 1.
Since SMF is CDDL (I think), and an init process is not part of the kernel, why is it not possible to use a well developed and debugged (since 2006?) alternative to the legacy SYSV init method?
pgmer6809
when grey-haired conservative fuddy duddies warn of something, you should PAY ATTENTION even if you disagree.
In this case, the "conservative" is certainly not in the political sense, it's in the technical sense. The core philosophy of UNIX was: small dedicated programs doing discrete things (which can be easily developerd, debugged, tested, and yes... replaced/substituted. Many warned that systemd was the polar opposite and would inevitably invite this very sort of issue. The warnings were ignored because they were not consistent with what the cool kids wanted. It was much more cool to create a whole new gluttonous monster, than to do the hard work to fix a bunch of long-standing and not glamorous basic usability issues that might actually help Linux take over the desktop.
In the political sense a similar thing happened with Obamcare, where conservatives kept pointing out that the basic plan did not pass the economic "smell test", and that inevitably the rates would rise and the markets would fall apart because of the poor planning.
In both cases, the hard-charging progressives (in the technical sense for the former and the political sense for the latter) ranted and raved against the cautious conservatives flinging insults about being backwards, stuck in the mud, opposed to progress, etc rather than facing the actual criticisms, considering that thier opponents might have serious and valid concerns, and then addressing those concerns. In both cases, when the inevitable "I told you so" comments arise, the advocates of the changes get angry and complain and propose moving even further in their chose direction, without facing that the now proven problems are real and were real - they want to solve a real problem with politics and name calling.
Incidentally, before some partisan hack rates this "Troll", I'll point out that this is a trait of human nature and applies to the political right and technical conservatives as well. Some right-leaning "fiscal" conservatives love to propose reductions in social spending while ignoring left wingers who suggest some might be harmed, instead of facing the problems suggested. Some technical conservatives, particularly in places like the FAA, can actually suppress the increase in safety that modern systems could provide out of excessive fear of the risk of "new" (AOA indicators on small aircraft, and the typecerts required to put new avionics into older small aircraft come to mind)
I've made several requests for systemd proponents to supply a use case that SysV initd could not support and haven't received a satisfactory reply to this purely technical question. I was interested in what systemd could offer over initd. I find systemd proponents are overly veherment in their criticisms of initd proponents.
I sense this comes from an inability to address the issues raised and, perhaps a mindset that anyone who has an appreciation for initd's elegant power will simply be bulldozed into irrelevance. I think systemd's criticism of the rc scripts that starts a linux based system is valid criticism however we have to keep in mind that they were devised by Red Hat. It is dealing with rc shell scripts that are the brunt of the justification for systemd.
In that sense the unitd solution is tidy but also reveals the justification to replace initd is not based on a full understanding of its capabilities, or even an understanding of was it is, a process manager. rc scripts are only meant to prepare the system for entries in /etc/inittab, yet everyone tries to get everything done in rc, which serializes the Linux boot process. A parallell boot is completely achievable by using initd properly. I know there is more to it, like events and messaging, I'm just citing one example.
Yet I've never seen a Linux distro that's utilized initd's /etc/inittab file properly. Especially a Red Hat system. They don't use initd properly, the rc scripts are bloated with rewrites of what initd already does, and now we're replacing initd, keh? initd has yet to be utilized fully on modern linux systems.
Criticisms of sco the company aside: sco *as a distribution of unix* had an interesting adjunct feature to initd, the 'enable' and 'disable' command that managed entries in /etc/inittab, where you would configure the characteristics of the system you were running. Franky I think this is functionality is essentially
I think initd would make a lot more sense to more people if this functionality had been available in Linux from the beginning. It is true that initd is beguiling in terms of it's simplicity wrt its power, but it is also very worthwhile. It is supposed to be small as that is where the skill is expressed.
initd is where you design the characteristics of the system, it is not an event manager and all the other things systemd is supposed to be. Something that does all the functionality systemd has, belongs as an inittab enty, not as the first process the kernel runs.
The point of a bug like this is not that it is a big deal itself, the big deal is the failure mode systemd has been revealed to have due to its complexity. This the type of concern I have about systemd, what else can trigger such a failure mode. I have seen initd in a variety of failure modes and not once has it ever consumed all system resources and disconnected running processes.
Now we've seen systemd do something that initd can't.
My ism, it's full of beliefs.
Red Hat security relies on a system where components are constantly fixed but rarely updated. It's like using Ubuntu 08, but with almost no issues... Not sure it's worth it.
Slashdot, fix the reply notifications... You won't get away with it...
Do you really need to ask?
The problem is best outlined here:
https://twitter.com/systemdsuc...
Just the basic facts...
Large complex programs will always be a problem
Like a monolithic kernel?
The bug -IS- systemd.
Use init and all your problems will disappear.
And on the Eighth Day, Man created God.
The whole "FreeDesktop" Movement seems to be about making Linux more and more incomprehensible.
My theory for why this is is like this:
There are lots of people now growing up when Windows kinda worked (since about 2000). At the same time, involvement in "Open Source" software is seen as a good career move. So they churn out some shitty badly designed code as potential recruiters cannot tell good from bad code. Also they take part in design processes without the experience necessary for this. The result are overcomplex buggy solutions which suck in manpower to maintain them.
Take a look at the *BSD people. The team maintaining OpenBSD is probably smaller than the SystemD team, yet they manage to maintain a whole operating system.
they didn't land on the moon either and the earth is really flat, here are a few sites for you to feed on, http://www.theflatearthsociety... http://listverse.com/2012/12/2...
"The hands that help are better far than lips that pray." - Robert Ingersoll (1833-1899)
thats another pile of crap
"The hands that help are better far than lips that pray." - Robert Ingersoll (1833-1899)
go back to your homework reading your book for 8 years olds.
"The hands that help are better far than lips that pray." - Robert Ingersoll (1833-1899)
"Multiple Linux Distributions Affected By Crippling Bug Called Systemd "
aaaaaaa
Systemd is new and, as people wrote it in C without proper software engineering, it has a lot of flaws. However, this is the case in many software components. Unfortunately, this bug has existed for 2 years. Really, do they have no test regiment at all? In case the bug existed without prior knowledge, then this is just unfortunate, if it existed and the systemd developers knew about it, then this is irresponsible. In any case, I expect a fix in days or better in hours.
Voodoo programming can produce working software. Why not working hardware too?
You appear to be conflating Linux with the shit that gets bundled with it.
Systemd is not a large, monolithic, program. It is multiple smaller program that share the same interface and can communicate with each other. the original init had its own bugs, too. Systemd is more modular and flexible than init, since it uses a modular approach and allows more flexibility with schedularing start up of programs.
[tinfoil hat]
The linked article makes interesting reading [even to a non-tekkie like me, when it describes how systemd is bloating with all sorts of additional [and buggy] functionality that takes it far beyond being an init replacement.
I am reminded of Edward Snowden's disclosures from the NSA, in which we learned that the NSA had deliberately submitted a weakened PRNG (Pseudo-Random Number Generator) to an encryption scheme with the deliberate intent to weaken it so that they could crack it easily.
When you look at the sprawling scope-creep, the poor testing and bugs like this, you have to start wondering if this wouldn't be a perfect trojan to be able to subvert an otherwise robust, secure OS...
Just sayin'
[/tinfoil hat]
So issuing this command makes cosmic rays strike our planet?
Linux and systemd are way more powerful than even I could have imagined!
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Worst feature. Ever. systemd saw this "feature" in windows, and have tried to implement it in linux as journalctl.
Semantic logging as implemented in the windows NT kernel, is a very complex framework kernel component but also very low performant, of course claims the contrary in the documented overview, but actually admits the subpar performance and non-robust implementation in other, not easily found, parts of the official msdn documentation.
For an inexperienced novice, heavily typed log messages can appear to solve problems, and such novice developers might think of untyped text causing problems. On the contrary, semantic logging, which journalctl is an obvious implementation of, do increase the complexity considerable, and many people before me have pointed out this critical design flaw. This complexity is easy to disregard in the name of heavily typed logging, complexity in software is hard to quantize, you need to accumulate experience before you realize complexity is very dangerous in software.
Logfiles that cannot be parsed by simple text based generic tools any longer is of course too complex. Instead heavy weight tools are now needed to be able to retrieve any information from systemd's behaviour. The "advanced query" functionality is said to require semantic logging. This is wrong (a lie actually), advanced querying tools for text based logfiles did already exist long before systemd was hacked together, no need to reinvent the square wheel.
Symptoms of too much complexity in software can appear in many forms, and TFS is a valid example as good as any. The bug in TFS is not unnatural, it's even expected to happen more frequently compared to more simple systems.
KISS FTW.
Download Slackware 14.2 iso
boot dvd of Slackware 14.2 iso
install Slackware 14.2
edit host file, change default host name to "Linux Mint"
sit back, enjoy your secure, stable OS install
dreaded scurrilous bit-twiddler from Oklahoma
I take it you never used Unix type systems lately? Because that philosophy isn't actually used at all - everything from kernel to drivers to commands (etc.) do much more than one thing.
If that philosophy would actually be used we'd have a microkernel system.
It seems that the bug was fixed then, according to the people who did not manage to reproduce the bug on theirs systems (please read trough the comments in the linked bug description to get a impression).
The thing with systemd is that you want the executable that remains after initializing the system is done, to be super-lean and provable. Ideally, init would replace itself with another executable image after all the complex, hard work is done. And ideally, because of its complexity, the initial hard work should be done by a scripting language.
What 'su' does inside systemd is completely beyond me. That should be a separate system call.
Religion is what happens when nature strikes and groupthink goes wrong.
Because you can't reproduce, the bug doesn't exist?
I'd be looking for a commit that specifically says it patches this bug, or a closure of all related bug-reports, before I started walking off in disgust.
I'd check your computer first.
Nothing here.
According to the linked article this bug affects Debian. I'm running both testing and stable versions on different architectures. They are all unaffected. I've compared the claims of "some person on the www who hates systemd" to what I can test in the real world on actual systems which include systemd (and run reliably without issues).
His claim is exaggerated at best. Hence I don't believe the bug exists *as it is described by the author*. The author apparently has an axe to grind. I don't.
I hope that's clear enough for you.
Because the distribution is the only possible factor?
Maybe you have hardware that allows things to use a different path? Maybe you have speed that the author's computer doesn't. There are LOTS of bug-confirmations in the bug report, from independent people.
Did you try it inside the infinite loop? Because even in the bug report they have to do that to get it to trigger and it can takes minutes to do so.
This is why bug-reporting, bug-detecting, and de-bugging are HARD. "It doesn't happen for me" is not a solution to a bug. Finding out where the path was that hit the bug, why that path was taken, whether that path is reasonable, and what happens when that path is taken is HARD and may bring things crashing down for some and nothing for others.
Reminds me of a guy who runs a software company I deal with a lot. I submitted screenshots of their website totally fucking up on a range of devices - Android, iPhone, tablet, smartphone, Chrome, Safari - that made it unusable. He sent me a back a screenshot of his Samsung tablet where it worked as expected.
My reply was less than polite. The bug report confirms that it DOES EXIST, AS IT IS DESCRIBED, and has independent confirmations.
Rather than make excuses, how about thinking yourself lucky and still pressing people to fix confirmed problems?
Anyone else try and run the code? NOTIFY_SOCKET=/run/systemd/notify systemd-notify ""
"....how about thinking yourself lucky ..."
thanks for the condescension, always a winning strategy and a great argument too.
I have Intel Atom 32 and 64 bit, Core-i#, and AMD64. I don't experience the bug. Systemd hasn't caused me any problems since trying daily image testing installers last year, and yes I filed the appropriate bug report and it got fixed.
I hear that systemd is a cause celebre amongst the disenchanted and is supposedly very bad and wicked and probably the product of a secret conspiracy hatched between Red Hat, Microsoft, Osama bin Laden and Cruella de Vil. But as an end user I find it useful and easy to customise to my preferences and have no particular objection to it. I also don't care if it gets replaced by something else that does the same job. Nor do I care about the personas of those who write it.
When I encounter bugs that bother me I file bug reports (if they have not already been reported). When I don't find bugs I don't.
When someone tells me the stuff I'm using is broken, and what is more it is inherently BAD, made by BAD people for BAD reasons, and I test it and it works fine on test as well as in daily use then to me that person's credibility is diminished.
Have a nice day.
Maybe it's time to use simple dedicated systems for things...?
the Microsoft of Linux - although Microsoft is trying to be that now. What they've done is no surprise.
You're right. I don't know how I could have thought that Obama's doubling of the debt from 10 trillion to nearly 20 could have been anything other than fiscal discipline.
Trump's a Democrat through and through. Also, I don't recall init having a feature that goes 'chmod -R 777 /; for f in `ls ~/*`; do mail -s "do not distribute, wink wink" -a $f me@hillary.com; done'. That must have been in the latest patch that I missed.
. The main difference between LXC and Docker containers is LXC launches a minimal init process in the chroot/pivot root while Docker launches the app process directly.
I know containers don't depend on cgroups or any other stuff that the systemd folks hype, but it's something that systemd appears to pander to--and it's something that Poettering is specifically catering to. The issue is that different types of criticisms are being conflated:
1. Some people say that cgroups suck (in implementation or design.) Not having used it, I couldn't say, but if there are concrete reasons for saying so then OK, that's fine.
2. Some people don't like that it's a hard requirement for systemd. I'm fully on board with this criticism--it locks out the BSDs unnecessarily and doesn't appear to jive with systemd's core purpose, or rather what should be its core purpose. This is where the modularity complaints come into play: whatever Poettering wants to have cgroups for, he should bloody well keep it out of our base init system.
3. Some people say "who needs cgroups?" (or the much worse "who needs containerization?") and this makes me cringe. There are so, so many clusterfucks in this world that can be gracefully sidestepped with virtualization/containerization and anything that improves the use or control of containers is by definition a win, at least on paper (but see #1.)
#3 is quite bad for the anti-systemd cause; the response "why would you want to do that?" / "We're not concerned with that use case" is one of the most annoying things to say to anyone.
#1 may or may not have merit.
#2 is entirely appropriate, and if you can offer an equivalent or better solution on an OpenRC system (either implemented or an in-principle solution) then that would also be a great point to raise.
I don't want Windows slickness because you don't get slickness from Windows. Surprise surprise you don't get it from Systemd either. I have upgraded some of my boxes to Systemd. But the rest will not be upgraded. I'm still hoping for a popular alternative. Why hasn't Devuan seen wider adoption? Keep It Simple Stupid!
Paul Beardsell
Ad hominem. Play the ball not the man. Feel free to make a substantive point any time you wish.
Paul Beardsell
For my next install, I think I may move over to (one of the variants of) BSD Unix.
There's an actual Linux Mint with sysvinit, it's LMDE 2 (debian jessie with Mate or Cinnamon)
It may change with an LMDE 3 based on debian stretch, and I think that may be likely, but I don't know afterall.
I suggest you actually try it with something and you will see that you will have to either greatly modify either the environment or the script unless the script is incredibly trivial.
Then consider that I'm referring to closed source software where I don't really understand what the scripts are doing and have no way to find out other than hoping a support person will relay messages from me back and forth to one of the software developers.
It's entirely impractical unless it's for something you've done yourself or something that is very well documented.
My point here is the vendors have not got it going with systemD.
If it's so trivial why do they mandate running on RedHat6 or similar for their 2016 release?
It's not just libraries since you can drag all the libraries from RedHat6 into RedHat7 without any drama if you want to run older applications.