Slashdot Mirror
Does Systemd Make Linux Complex, Error-Prone, and Unstable? (ungleich.ch)
"Systemd developers split the community over a tiny detail that decreases stability significantly and increases complexity for not much real value." So argues Nico Schottelius, talking about his experiences as the CEO of a Swiss company providing VM hosting, datacenters, and high-speed fiber internet. Long-time Slashdot reader walterbyrd quotes Nico's essay:
While I am writing here in flowery words, the reason to use Devuan is hard calculated costs. We are a small team at ungleich and we simply don't have the time to fix problems caused by systemd on a daily basis. This is even without calculating the security risks that come with systemd. Our objective is to create a great, easy-to-use platform for VM hosting, not to walk a tightrope...
[W]hat the Devuan developers are doing is creating stability. Think about it not in a few repeating systemd bugs or about the insecurity caused by a huge, monolithic piece of software running with root privileges. Why do people favor Linux on servers over Windows? It is very easy: people don't use Windows, because it is too complex, too error prone and not suitable as a stable basis. Read it again. This is exactly what systemd introduces into Linux: error prone complexity and instability. With systemd the main advantage to using Linux is obsolete.
The essay argues that while Devuan foisted another choice into the community, "it is not their fault. Creating Devuan is simply a counteraction to ensure Linux stays stable. which is of high importance for a lot of people."
[W]hat the Devuan developers are doing is creating stability. Think about it not in a few repeating systemd bugs or about the insecurity caused by a huge, monolithic piece of software running with root privileges. Why do people favor Linux on servers over Windows? It is very easy: people don't use Windows, because it is too complex, too error prone and not suitable as a stable basis. Read it again. This is exactly what systemd introduces into Linux: error prone complexity and instability. With systemd the main advantage to using Linux is obsolete.
The essay argues that while Devuan foisted another choice into the community, "it is not their fault. Creating Devuan is simply a counteraction to ensure Linux stays stable. which is of high importance for a lot of people."
Oh my! I'm going to hang back, make some popcorn, step into some Tyvek coveralls, grab a hard-hat and safety goggles and enjoy the show!
If you want news from today, you have to come back tomorrow.
The BSDâ(TM)s and Illumos. There is no reason to use the tire fire that is Linux. You have options!
Here's a list of actual problems that should have been solved instead of introducing the nightmare of systemd upon the Linux (Debian specifically) world:
- Forceful, unconditional kernel operations. When I say "unmount this filesystem," I'm not asking a question. When I say "terminate this process," I expect the process to be removed from memory and the runqueue, regardless of consequences.
- When I say "reboot" I mean "reboot." Hangs are not okay, ever.
- Actual, real soft NFS failures. Do not hang during boot for any reason unless that share is marked hard,nointr. Do not hang during shutdown/reboot, either.
- Enforce GPL-standard syntax on new incoming utilities. If you want into the package tree, use a GNU parsing library and use it correctly. Perhaps a standardized syntax wrapper available for package maintainers.
- Bolt simple parallelization, triggers and flow control onto init/rc.
- Drop this selinux shit. It's too complicated and causes more problems than it solves. Vulnerabilities come from bad code, not a lack of complex call ACLs. Security is a process, not a feature.
- Standardize and fix bluetooth support ffs.
My $0.02, as a 25-year Linux admin.
A government is a body of people notably ungoverned - AC
A big ol ball? My init.d was about 13 scripts big which were readable and editable. Ever tried to edit systemd files? Depending on systemd version you have to create overrides, modify symlinks or edit systemd files straight up which can be in about 5 different locations and on top of that, systemd can have overrides on any changes either with an update or just inherited.
Systemd makes every system into a dependency mess.
Remove/fail a hard drive and your system will boot into single user mode, not even remote access will be available so you better be near the machine just because it was in fstab and apparently everything in fstab is a hard dependency on systemd.
Custom electronics and digital signage for your business: www.evcircuits.com
How are they going to get in and stay in if admins have real logs and can detect persistence?
Domestic spying is now "Benign Information Gathering"
So the short answer is: Yes, systemd makes things unnecessarily complex with little benefit.
That matches my experience - losing a lot of time trying to figure out why things don't work. The improved boot time is lost several times over.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
1. You've moved having a basic understanding of the boot process, and the ability to fix things, from having a decent knowledge of bash to being a C wizard.
2. You've broken decades of understanding the boot process.
3. It breaks KISS, as it doesn't simply do startup. Hell, it does ntpd.
It breaks a lot of the *concept* of unix. Maybe to something preferred by a lot of people - but it also turns it into an alien mess to a lot of other people.
yet another rant with zero details.... "all the problems caused by systemd" and yet not a single one listed.
Use "journalctl" to see the details log.
It must have been something you assimilated. . . .
it turns out that, on arm embedded systems at the very least, where context-switching is a little slower and booting off of microsd cards results in amplification of any performance-related issues associated with drive reads/writes when compared to an SSD or HDD, sysvinit easily outperforms systemd for boot times.
Do one thing, and do it well. Systemd has eaten init, udev, inetd, syslog and soon dhcpd. Yes, that is getting ridiculous.
Systemd creates a dependency mess which means it cannot be replaced by simpler things, which wasn't the case before systemd.
I don't think there's a problem with the idea of systemd. Having a standard way to handle process start-up, dependencies, failures, recovery, "contracts", etc... isn't a bad, or unique, thing -- Solaris has Service Manager, for example. I think there's just too many things unnecessarily built into systemd rather than it utilizing external, usually, already existing utilities. Does systemd really need, for example, NFS, DNS, NTP services built-in? Why can't it run as PID 2 and leave PID1 for init to simply reap orphaned processes? Would make it easier to restart or handle a failed systemd w/o rebooting the entire system (or so I've read).
In short, systemd has too many things stuffed into its kitchen sink -- if you want that, use Emacs :-)
[ Note, I'm a fan and long-time user of Emacs, so the joke's in good fun. ]
It must have been something you assimilated. . . .
This is Slashdot, any ill you can imagine can be traced back to systemd.
“Common sense is not so common.” — Voltaire
code contribution: /code/systemd
rm -rf
So the short answer is: Yes, systemd makes things unnecessarily complex with little benefit.
That matches my experience - losing a lot of time trying to figure out why things don't work. The improved boot time is lost several times over.
I completely agree. Troubleshooting is really a bitch with systemd, much more time-consuming. For instance, often systemctl reports a daemon as failed while it's not, or suddenly decides that it didn't start because of some mysterious arbitrary timeout while the daemon just needs some time to run a maintenance tasks at startup time. And getting anything of value out of the log is a pain in the ass.
Quite often I end up writing control shell scripts specifically to be called by systemd, because this junkware is too fragile and capricious to work with actual daemons. That says a lot about the overal usefulness of systemd.
Nothing has been gained with systemd, at least not on servers.
lucm, indeed.
Troubleshooting is really a bitch with systemd, much more time-consuming. For instance, often systemctl reports a daemon as failed while it's not, or suddenly decides that it didn't start because of some mysterious arbitrary timeout while the daemon just needs some time to run a maintenance tasks at startup time.
Not to mention that the damn logs are not plain text, which in itself complicates things before you even have the chance to start troubleshooting.
#DeleteChrome
it's a fine OS, the only thing it's missing is a really good init system.
I disagree on SELinux, not because its interface is well-designed (it is not), but because it is needed for some things.
On the rest, I fully agree. And instead, systemd solves things that were already solved and does it badly. The amount of stupidity in that decision is staggering.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Yeah, yeah I know the history of its development and how log files are binary and the whole debug kernel flag fiasco. And I don't care. By the time I used systemd, that had already long passed.
I switched from Squeeze to Jessie a couple years ago, had some growing pains as I learned how to use systemd... but that was it. No stability issues, no bugs. Can't say whether things run better, but they definitely don't run worse.
I had only really been using Linux for a few years before the onset of systemd, and honestly I think that's part of the problem. People who complain about systemd the most seem to have been using Linux for a very long time and just "don't want to change". Whether its nostalgia or sunk-cost fallacy, I can't say, but beyond that it seems much more like a philosophical difference than a practical one. It just reminds me of people's refusal to use the metric system, for no better reason than they are unfamiliar with it.
If systemd is so terrible, then why did a lot of the major distros switch over? If they didn't, it would just be a footnote in the history of open source: "Hey remember when they tried to replace sysV and init with that stupid thing with the binary log files? What was it called? SystemP?" The fact that Devaun has not overtaken Debian in any real way (at least from what I've seen, feel free to correct me if I'm wrong) indicates that my experience with systemd is the norm, not the exception. The market has spoken.
I read TFA, there is not one single specific bug or instability mentioned about systemd. What is the "tiny detail" that split the community? I have no idea, because TFA doesn't say what it is. I know that part of the philosophy behind Linux is "figuring it out yourself", but if you don't explain to me these low level kernel details (if that's even what they are; again, I have no idea), then don't expect people like me to be on your side. Linux is just a tool to me, I don't have any emotional attachment to it, so if things are working OK I am not going to start poking around under the hood just because someone posts an article claiming there are problems, but never specifying what those problems are and how they affect me as a user.
Honestly TFA reads like "We are having development problems, therefore systemd sucks." I get that when major changes to the platform happens there are going to be issues and annoyances, but that's the way software development has always been and will always be. Even if systemd was perfect there would still be all kinds of compatibility issues and new conventions that developers would have to adapt to. That's what I would expect to happen whenever any major change is made to a widely used and versatile platform like Linux.
Even Linus doesn't really care:
"I don't actually have any particularly strong opinions on systemd itself. I've had issues with some of the core developers that I think are much too cavalier about bugs and compatibility, and I think some of the design details are insane (I dislike the binary logs, for example), but those are details, not big issues."
I'm not saying systemd is "better" or "the right answer". If you want to stick to distros that don't use it, that's up to you. But what I am saying is, get over it.
Only crack the nuts that crack. You don't put the ones that don't crack in the sack.
I started out using Slackware over twenty years ago. Biggest reason I left it was the libc5/glibc2 debacle. It became nonviable for me to remain on Slackware at that time. Bounced around through several package-managed distros and eventually ended up on Debian. I've now watched Debian become increasingly complex, sometimes needlessly, and sometimes because different major components have features that other major components lack. This means even if one is running something like xfce for the windowmanager one has to have a lot of Gnome or KDE stuff installed for basic stuff to work.
It's gotten worse since Systemd entered the picture. Honestly pulseaudio is still not mature, not sure how the person who didn't get that working right was entrusted to replace init.
Do not look into laser with remaining eye.
Windows is a very complex system. Not necessarily because it needs to be complex, but rather because of "wouldn't it be great if we could also..." thinking. Take the registry. Good idea in its core, a centralized repository for all configuration files. Great. But wouldn't it be nice if we could also store some states in there? And we could put the device database in there, too. And how about the security settings? And ...
And eventually you had the mess you have now, where we're again putting configuration files into the %appdata% directory. But when we have configuration in there already anyway, couldn't we... and we could sync this for roaming, ya know...
Which is the second MS disease. How many users actually need roaming? 2, maybe 3 out of 10? The rest is working on a stationary desktop, never moving, never roaming. But they have to have this feature, needed or not. And if you take a look through the services, you'll notice that a lot of services that you simply know you don't need MUST run because the OS needs them for some freakish reason. Because of "wouldn't it be great if this service did also...".
systemd now brought this to the Linux world. Yes, it can do a lot. But unfortunately it does so, whether you need it or not. And it requires you to take these "features" into account when configuring it, even if you have exactly zero use for them and wouldn't potentially not even know just wtf they're supposed to do.
systemd is as overengineered as many Windows components. And thus of course as error prone. And while it can make things more manageable for huge systems, everything becomes more convoluted and complicated for anyone that has no use for these "wouldn't it be great if it also..." features.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Can I ask, why don't you and other admins/devs like you start to contribute to systemd?
Lennart Poettering has specifically said that he will not accept many important kinds of patches, for example he refuses to merge any patch that improves cross-platform compatibility.
And what's the reason, because people on forums are complaining? Because binary log files break the UNIX philosophy?
Here is my analysis of systemd, spread across multiple posts (links towards the bottom). It's poorly written software (the interfaces are bad, you can read through my links for more explanation), and that will only get worse over time if an effort isn't made to isolate it over time. This is basic system architecture.
"First they came for the slanderers and i said nothing."
Journald makes logging simple and convenient, right?
journald has been known to run out of memory and stop responding.
Due to the design of "oh just connect stdout of the process to journald" if you restart journald it closes all of those file descriptors and you silently lose all further logging from already running processes.
Journald, by design, will only log so much per process, meaning that if it's logged too much since startup/an error you're interested in, you've now lost it.
Why 'they' had to go for demonstrably broken binary logging using a new interface I don't know. They could have just extended the syslog format to make it mandatory to pass along program name and process ID in the message. Then they could have made it "easier" to find the logs by having a per program/facility directory under /var/log and then stuck to simple, plain text logging that the existing *nix tools can search with ease. But, nope, they had to go with this shitfest instead.
And that's only one component of the whole systemd shitfest. On a very simple Debian install I've had it exhibit issues with shutdown, hanging on something that is simple or ignorable.
Sadly I had to abandon using Devuan after a while. The only really supported version is the jessie (Debian old-stable) version, and I'm not sure even that gets timely security updates. Their equivalent of Debian stable (Stretch) is 'ascii' and got next to no updates during the few months I used it. Boot up was both nice and fast (a major systemd selling point) and reliable (unlike systemd). I guess they just need more in the way of human resources so that they can nail down which Debian packages have problematic systemd tentacles involved, then they could pass through other Debian updates as soon as they're available.
>To me, the fact that the major distros have adopted systemd is strong evidence that it is probably better
"Better" is a subjective term. Software (and any product really) does not have some absolute measurable utility. It's utility is specific to an audience. The fact that the major distros switch is probably strong evidence that systemd is "better" for distro developers. But the utility it brings them may not apply to all users, or even any particular user.
A big part of the reason people were upset was exactly that - the key reasons distros had for switching was benefits to people building distros which subsequent users would never experience. These should not have trumped the user experience.
All that would still have been fine - we could easily have ended up with a world that had systemd for those who wanted it, and didn't have it for those who didn't want it. Linux systems are supposed to be flexible enough that you can set them up to whatever purpose you desire.
So where the real anger came in was the systemd's obsessive feature-creep made it go into a lots and lots of areas that have nothing to do with it's supposed purpose (boot process management), in that area it's biggest advantages are only useful to people building distributions (who have to maintain thousands of packages and ensure they reliable handle their bootup requirements regardless of what combination of them is actually installed- systemd genuinely did make that easier on them - but no user or admin ever experiences that scenario). But that feature creep itself wasn't even the issue, the issue was that - as it entered into all these unrelated areas (login was the first of many) - it broke compatibility with the existing software to do those jobs. This meant that, if you built a system to support systemd, that same system could not use any alternatives. So now, you had to create hard dependencies on systemd to support it at all - for distros to gain those benefits, they had to remove the capacity for anybody to forgo them, or alternatively provide two versions of every package - even ones that never touch the boot process and get no benefit from systemd's changes there.
And the trouble is - in none of those other areas has it offered anything of significant value to anybody. Logind doesn't actually do anything that good old login didn't do anyway, but it's incompatible so a distro that compiles it's packages around logind can't work with anything else. Replacing the process handler... and not only did it not add any new functionality it broke some existing functionality (granted, in rarer edge cases -but there was no reason for any breakage at all because these were long-solved problems).
Many years ago, I worked as a unix admin for a company that developed for lots of different target unix systems. As such I had to maintain test environments running all the targets. I had several linux systems running about 5 different distros, I had solaris boxes with every version from 8 onwards (yep, actual Sparcs), I had IBM's running AIX, I even had two original (by then 30 year old) DEC Alphas running Tru64... and I had several HPUX boxes.
At the time, while adminning all these disparate unix environments on a day-to-day basis and learning all their various issues and problems - I came to announce routinely that Solaris pre-Version-10 had the worst init system in the word to admin, but the worst Unix in the world was definitely HPUX because HPUX was the only Unix where I could not, with absolute certainty, know that if I kill -9 a process - that process would definitely be gone. WIped out of memory and the process table with absolutely no regard for anything else - it's a nuclear option, and it's supposed to work that way - because sometimes that what you need to keep things running.
SystemD brought to Linux an init system that replicated everything I used to hate about the Solaris 8/9 init system - but what's worse than that, it brought the one breakage that got me to declare HPUX the absolute worst unix system
Unicode killed the ASCII-art *
Granted, I have never needed any kind of tampering or corruption mitigation in my log files over the last 20 years of Linux administration. So the value for at least my usage of journalctl has been sum negative because I don't see the value in a command that by default truncates log output.
So the answer for systemd is to workaround it by using a "legacy" service to restore decades of functionality.
SMF was the death knell for Solaris (along with the Oracle purchase), and it feels like systemd is going to be the anchor which drags Linux into the abyss.
--WooooHoooo--
... their only arguments are hyperbolic personal attacks. (Look, I can do that too!)
Hint: Your side is just as stupid as your opposing site. There is no sane or reasonable, let alone sensible side. Because that is how Americans are. At least it is beyond their *tiny* mental box.
Regarding systemd, I state *both* A and B:
A) Monolithic "frameworks" have always been a stupid idea. Because they disable you from plugging them into *your* system, and force you to plug into *theirs*. Because they want to dominate you! And they are mutually exclusive as a result of that.
B) Traditional init systems are very limited and badly limiting nowadays. Like still using DOS as the underpinnings of your actual system. A more generic event/trigger system is much more sensible.
THE PROBLEM IS: That systemd throws away what's good about traditional init systems (like "everything is a file"; modularity; being able to do things with a simple file manager, text editor and maybe a script.).
It could have done the event/trigger thing *without* sacrificing modularity (tools that do *one* thing, and do it right!).
It could have acted less like a dominatrix on a power trip, swallowing everything.
The base ideas were good. The personality of the way it was implemented, was that of a complete egocentric psychopathic asshole with a god complex.
Give me a sane eventd, and I will ditch the old init system before you can blink.
I don't agree. Systemd is the most visible part of a clear trend within Red Hat, consisting in an attempt to make their particular version of Linux THE canonical Linux, to the point that, if you are not using Red Hat, or some derived distribution, things will not work. In essence, Red Hat is attempting to out-MS MS by polluting and warping Linux needlessly but surely. The latest: they have come up with the 'timedatectl' command, which does exactly the same as 'date'. The latter is to be deprecated. Red Hat, the MS wannabee. They will not pull it off, but they are inflicting a lot of damage on Linux in the process.
Using text processing skills to process a generic text file isn't any harder than using journalctl. The difference is that the former is generically applicable to just about any other software on the planet, and the latter is for journald. It's not that complex to confer the journalctl benefits without ditching *native* text log capability, but they refuse to do so.
Using ForwardToSyslog just means there's an unnecessary middle-man, meaning both services must be functional to complete logging. The problem is the time when you actually want logs is the time when there's something going wrong. A few weeks ago was trying to support someone who did something pretty catastrophic to his system. One of the side effects was that it broke the syslog forwarding (syslog would still work, but nothing from journald would get to it). The other thing that happened would be for the system to lock out all access. I thought 'ok, I'll reboot and use jornalctl', but wait, on CentOS 7, journald defaults to not persisting journald across boot, because you have syslog to do that.
Of course the other problem (not entirely systemd project fault) was the quest to 'simplify' console output so he just saw 'fail' instead of the much more useful error messages that would formerly spam the console on experiencing the sort of problem he hit (because it would be terrible to have an 'ugly' console...). This hints about another source of the systemd controversy, that it's also symbolic of a lot of other design choices that have come out of the distros.
XML is like violence. If it doesn't solve the problem, use more.
I really struggle to reconcile the Slashdot view that systemd is total crap and the fact that every major Linux distro has switched to it.
The Linux ecosystem is not sane. Redhat wanted more control of Linux so they pushed systemd. GNOME developers are easily distracted by shiny things (as proof I submit GNOME 3) so they went ahead and made GNOME dependent on it. And then Debian (which most Linux distributions are based upon) adopted systemd because GNOME depended on it. There were some other excuses, but that's the biggest reason. You can blame Redhat and Debian for this clusterfuck, and really, only a small handful of people in the Debian community are actually responsible for Debian's involvement. Debian's leaders were split almost down the middle on whether they should go to systemd. This is why major changes should require a 2/3 vote (or more!)
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"