Slashdot Mirror
Does Systemd Make Linux Complex, Error-Prone, and Unstable? (ungleich.ch)
"Systemd developers split the community over a tiny detail that decreases stability significantly and increases complexity for not much real value." So argues Nico Schottelius, talking about his experiences as the CEO of a Swiss company providing VM hosting, datacenters, and high-speed fiber internet. Long-time Slashdot reader walterbyrd quotes Nico's essay:
While I am writing here in flowery words, the reason to use Devuan is hard calculated costs. We are a small team at ungleich and we simply don't have the time to fix problems caused by systemd on a daily basis. This is even without calculating the security risks that come with systemd. Our objective is to create a great, easy-to-use platform for VM hosting, not to walk a tightrope...
[W]hat the Devuan developers are doing is creating stability. Think about it not in a few repeating systemd bugs or about the insecurity caused by a huge, monolithic piece of software running with root privileges. Why do people favor Linux on servers over Windows? It is very easy: people don't use Windows, because it is too complex, too error prone and not suitable as a stable basis. Read it again. This is exactly what systemd introduces into Linux: error prone complexity and instability. With systemd the main advantage to using Linux is obsolete.
The essay argues that while Devuan foisted another choice into the community, "it is not their fault. Creating Devuan is simply a counteraction to ensure Linux stays stable. which is of high importance for a lot of people."
[W]hat the Devuan developers are doing is creating stability. Think about it not in a few repeating systemd bugs or about the insecurity caused by a huge, monolithic piece of software running with root privileges. Why do people favor Linux on servers over Windows? It is very easy: people don't use Windows, because it is too complex, too error prone and not suitable as a stable basis. Read it again. This is exactly what systemd introduces into Linux: error prone complexity and instability. With systemd the main advantage to using Linux is obsolete.
The essay argues that while Devuan foisted another choice into the community, "it is not their fault. Creating Devuan is simply a counteraction to ensure Linux stays stable. which is of high importance for a lot of people."
Oh my! I'm going to hang back, make some popcorn, step into some Tyvek coveralls, grab a hard-hat and safety goggles and enjoy the show!
If you want news from today, you have to come back tomorrow.
The BSDâ(TM)s and Illumos. There is no reason to use the tire fire that is Linux. You have options!
Here's a list of actual problems that should have been solved instead of introducing the nightmare of systemd upon the Linux (Debian specifically) world:
- Forceful, unconditional kernel operations. When I say "unmount this filesystem," I'm not asking a question. When I say "terminate this process," I expect the process to be removed from memory and the runqueue, regardless of consequences.
- When I say "reboot" I mean "reboot." Hangs are not okay, ever.
- Actual, real soft NFS failures. Do not hang during boot for any reason unless that share is marked hard,nointr. Do not hang during shutdown/reboot, either.
- Enforce GPL-standard syntax on new incoming utilities. If you want into the package tree, use a GNU parsing library and use it correctly. Perhaps a standardized syntax wrapper available for package maintainers.
- Bolt simple parallelization, triggers and flow control onto init/rc.
- Drop this selinux shit. It's too complicated and causes more problems than it solves. Vulnerabilities come from bad code, not a lack of complex call ACLs. Security is a process, not a feature.
- Standardize and fix bluetooth support ffs.
My $0.02, as a 25-year Linux admin.
A government is a body of people notably ungoverned - AC
A big ol ball? My init.d was about 13 scripts big which were readable and editable. Ever tried to edit systemd files? Depending on systemd version you have to create overrides, modify symlinks or edit systemd files straight up which can be in about 5 different locations and on top of that, systemd can have overrides on any changes either with an update or just inherited.
Systemd makes every system into a dependency mess.
Remove/fail a hard drive and your system will boot into single user mode, not even remote access will be available so you better be near the machine just because it was in fstab and apparently everything in fstab is a hard dependency on systemd.
Custom electronics and digital signage for your business: www.evcircuits.com
So the short answer is: Yes, systemd makes things unnecessarily complex with little benefit.
That matches my experience - losing a lot of time trying to figure out why things don't work. The improved boot time is lost several times over.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
1. You've moved having a basic understanding of the boot process, and the ability to fix things, from having a decent knowledge of bash to being a C wizard.
2. You've broken decades of understanding the boot process.
3. It breaks KISS, as it doesn't simply do startup. Hell, it does ntpd.
It breaks a lot of the *concept* of unix. Maybe to something preferred by a lot of people - but it also turns it into an alien mess to a lot of other people.
yet another rant with zero details.... "all the problems caused by systemd" and yet not a single one listed.
Use "journalctl" to see the details log.
It must have been something you assimilated. . . .
it turns out that, on arm embedded systems at the very least, where context-switching is a little slower and booting off of microsd cards results in amplification of any performance-related issues associated with drive reads/writes when compared to an SSD or HDD, sysvinit easily outperforms systemd for boot times.
Do one thing, and do it well. Systemd has eaten init, udev, inetd, syslog and soon dhcpd. Yes, that is getting ridiculous.
Systemd creates a dependency mess which means it cannot be replaced by simpler things, which wasn't the case before systemd.
I don't think there's a problem with the idea of systemd. Having a standard way to handle process start-up, dependencies, failures, recovery, "contracts", etc... isn't a bad, or unique, thing -- Solaris has Service Manager, for example. I think there's just too many things unnecessarily built into systemd rather than it utilizing external, usually, already existing utilities. Does systemd really need, for example, NFS, DNS, NTP services built-in? Why can't it run as PID 2 and leave PID1 for init to simply reap orphaned processes? Would make it easier to restart or handle a failed systemd w/o rebooting the entire system (or so I've read).
In short, systemd has too many things stuffed into its kitchen sink -- if you want that, use Emacs :-)
[ Note, I'm a fan and long-time user of Emacs, so the joke's in good fun. ]
It must have been something you assimilated. . . .
So the short answer is: Yes, systemd makes things unnecessarily complex with little benefit.
That matches my experience - losing a lot of time trying to figure out why things don't work. The improved boot time is lost several times over.
I completely agree. Troubleshooting is really a bitch with systemd, much more time-consuming. For instance, often systemctl reports a daemon as failed while it's not, or suddenly decides that it didn't start because of some mysterious arbitrary timeout while the daemon just needs some time to run a maintenance tasks at startup time. And getting anything of value out of the log is a pain in the ass.
Quite often I end up writing control shell scripts specifically to be called by systemd, because this junkware is too fragile and capricious to work with actual daemons. That says a lot about the overal usefulness of systemd.
Nothing has been gained with systemd, at least not on servers.
lucm, indeed.
Troubleshooting is really a bitch with systemd, much more time-consuming. For instance, often systemctl reports a daemon as failed while it's not, or suddenly decides that it didn't start because of some mysterious arbitrary timeout while the daemon just needs some time to run a maintenance tasks at startup time.
Not to mention that the damn logs are not plain text, which in itself complicates things before you even have the chance to start troubleshooting.
#DeleteChrome
it's a fine OS, the only thing it's missing is a really good init system.
systemd turned me into a newt!
#DeleteFacebook
A newt that can type on a KEYBOARD??
"He got better!
Perhaps you have had no problems with systemd because you aren't trying to use it to do much.
Lots of people, myself included, have had issues trying to get things which are trivial in pre-systemd or on other OSes to work properly and consistently on systemd. There are many, many, many examples of issues. If someone asked me for examples, I'd have a hard time deciding where to start because so many things have been gratuitously changed. If you really think there aren't examples, just read this thread.
On the other hand, I have yet to see real technical discussion about problems that systemd apparently is fixing. I honestly and openmindedly am curious about what makes systemd good, so I've tried on several occasions to find these discussions where good technical reasoning is used to explain the motivations behind systemd. If they exist, I haven't found any yet. I'm hoping some will appear as a result of this thread.
But you bring up the idea that the "market has spoken"? You do realize that a majority of users use Windows, right? And people in the United States are constantly electing politicians who directly hurt the people who vote for them more than anyone else. It's called marketing. Just because something has effective marketing doesn't mean it doesn't suck.
systemd fails silently if something is wrong in /etc/fstab. It just doesn't finish booting. Which is moderately annoying if you have access to the system console and you can guess that an unchanged /etc/fstab from before systemd that worked for a while with systemd is now suddenly toxic
If you do not have easy access to the system console or you are not blessed with divine inspiration, that is quite a bit more than annoying. Thanks to the binary log files you cannot even boot something random and read the logs, but at least you aren't missing anything, because nothing pertinent to the error is logged anyway.
The problem is that one camp won't admit that old init is a pile of shit from the 80's whose only virtue is that the stench has faded over time, and the other camp won't admit that their new shiny toy needs to be understandable and debuggable.
A proper init system needs dependencies and service monitoring. init + monit does not cut it today. Systemd does that bit rather impressively well. It's just terrible at actually booting the system, all the early boot stuff that you could depend on old init to get right every time, or at least spit out aggressive messages about why it failed.
Finally! A year of moderation! Ready for 2019?
Systemd is nothing but a thinly-veiled plot by Vladimir Putin and Beyonce to import illegal German Nazi immigrants over the border from Mexico who will then corner the market in kimchi and implement Sharia law!!!
People who complain about systemd the most seem to have been using Linux for a very long time and just "don't want to change".
no, that's not it. people who have been using linux for a long time usually *know the corner-cases better*. in other words, they know *exactly* why it doesn't work and won't work, they know *exactly* the hell that it can and will create, under what circumstances, and they know *precisely* how they've been betrayed by the rail-roaded decisions made by distros without consulting them as to the complexities of the scenario to which they have been (successfully up until that point) deploying a GNU/Linux system.
also they've done the research - looked up systemd vs other init systems on the CVE mitre databases and gone "holy fuck".
also they've seen - perhaps even reported bugs themselves over the years - how well bugs are handled, and how reasonable and welcoming (or in some sad cases not, but generally it's ok) the developers are... then they've looked up the systemd bug database and how pottering abruptly CLOSES LEGITIMATE BUGREPORTS and they've gone "WHAT the fuck??"
also, they've been through the hell that was the "proprietary world", if they're REALLY old they've witnessed first-hand the "Unix Wars" and if they're not that old they experienced the domination of Windows through the 1990s. they know what a monoculture looks like and how dangerous that is for a computing eco-system.
in short, i have to apologise for pointing this out: they can read the danger signs far better than you can. sorry! :)
Can I ask, why don't you and other admins/devs like you start to contribute to systemd?
Lennart Poettering has specifically said that he will not accept many important kinds of patches, for example he refuses to merge any patch that improves cross-platform compatibility.
And what's the reason, because people on forums are complaining? Because binary log files break the UNIX philosophy?
Here is my analysis of systemd, spread across multiple posts (links towards the bottom). It's poorly written software (the interfaces are bad, you can read through my links for more explanation), and that will only get worse over time if an effort isn't made to isolate it over time. This is basic system architecture.
"First they came for the slanderers and i said nothing."
>To me, the fact that the major distros have adopted systemd is strong evidence that it is probably better
"Better" is a subjective term. Software (and any product really) does not have some absolute measurable utility. It's utility is specific to an audience. The fact that the major distros switch is probably strong evidence that systemd is "better" for distro developers. But the utility it brings them may not apply to all users, or even any particular user.
A big part of the reason people were upset was exactly that - the key reasons distros had for switching was benefits to people building distros which subsequent users would never experience. These should not have trumped the user experience.
All that would still have been fine - we could easily have ended up with a world that had systemd for those who wanted it, and didn't have it for those who didn't want it. Linux systems are supposed to be flexible enough that you can set them up to whatever purpose you desire.
So where the real anger came in was the systemd's obsessive feature-creep made it go into a lots and lots of areas that have nothing to do with it's supposed purpose (boot process management), in that area it's biggest advantages are only useful to people building distributions (who have to maintain thousands of packages and ensure they reliable handle their bootup requirements regardless of what combination of them is actually installed- systemd genuinely did make that easier on them - but no user or admin ever experiences that scenario). But that feature creep itself wasn't even the issue, the issue was that - as it entered into all these unrelated areas (login was the first of many) - it broke compatibility with the existing software to do those jobs. This meant that, if you built a system to support systemd, that same system could not use any alternatives. So now, you had to create hard dependencies on systemd to support it at all - for distros to gain those benefits, they had to remove the capacity for anybody to forgo them, or alternatively provide two versions of every package - even ones that never touch the boot process and get no benefit from systemd's changes there.
And the trouble is - in none of those other areas has it offered anything of significant value to anybody. Logind doesn't actually do anything that good old login didn't do anyway, but it's incompatible so a distro that compiles it's packages around logind can't work with anything else. Replacing the process handler... and not only did it not add any new functionality it broke some existing functionality (granted, in rarer edge cases -but there was no reason for any breakage at all because these were long-solved problems).
Many years ago, I worked as a unix admin for a company that developed for lots of different target unix systems. As such I had to maintain test environments running all the targets. I had several linux systems running about 5 different distros, I had solaris boxes with every version from 8 onwards (yep, actual Sparcs), I had IBM's running AIX, I even had two original (by then 30 year old) DEC Alphas running Tru64... and I had several HPUX boxes.
At the time, while adminning all these disparate unix environments on a day-to-day basis and learning all their various issues and problems - I came to announce routinely that Solaris pre-Version-10 had the worst init system in the word to admin, but the worst Unix in the world was definitely HPUX because HPUX was the only Unix where I could not, with absolute certainty, know that if I kill -9 a process - that process would definitely be gone. WIped out of memory and the process table with absolutely no regard for anything else - it's a nuclear option, and it's supposed to work that way - because sometimes that what you need to keep things running.
SystemD brought to Linux an init system that replicated everything I used to hate about the Solaris 8/9 init system - but what's worse than that, it brought the one breakage that got me to declare HPUX the absolute worst unix system
Unicode killed the ASCII-art *
... their only arguments are hyperbolic personal attacks. (Look, I can do that too!)
Hint: Your side is just as stupid as your opposing site. There is no sane or reasonable, let alone sensible side. Because that is how Americans are. At least it is beyond their *tiny* mental box.
Regarding systemd, I state *both* A and B:
A) Monolithic "frameworks" have always been a stupid idea. Because they disable you from plugging them into *your* system, and force you to plug into *theirs*. Because they want to dominate you! And they are mutually exclusive as a result of that.
B) Traditional init systems are very limited and badly limiting nowadays. Like still using DOS as the underpinnings of your actual system. A more generic event/trigger system is much more sensible.
THE PROBLEM IS: That systemd throws away what's good about traditional init systems (like "everything is a file"; modularity; being able to do things with a simple file manager, text editor and maybe a script.).
It could have done the event/trigger thing *without* sacrificing modularity (tools that do *one* thing, and do it right!).
It could have acted less like a dominatrix on a power trip, swallowing everything.
The base ideas were good. The personality of the way it was implemented, was that of a complete egocentric psychopathic asshole with a god complex.
Give me a sane eventd, and I will ditch the old init system before you can blink.
Using text processing skills to process a generic text file isn't any harder than using journalctl. The difference is that the former is generically applicable to just about any other software on the planet, and the latter is for journald. It's not that complex to confer the journalctl benefits without ditching *native* text log capability, but they refuse to do so.
Using ForwardToSyslog just means there's an unnecessary middle-man, meaning both services must be functional to complete logging. The problem is the time when you actually want logs is the time when there's something going wrong. A few weeks ago was trying to support someone who did something pretty catastrophic to his system. One of the side effects was that it broke the syslog forwarding (syslog would still work, but nothing from journald would get to it). The other thing that happened would be for the system to lock out all access. I thought 'ok, I'll reboot and use jornalctl', but wait, on CentOS 7, journald defaults to not persisting journald across boot, because you have syslog to do that.
Of course the other problem (not entirely systemd project fault) was the quest to 'simplify' console output so he just saw 'fail' instead of the much more useful error messages that would formerly spam the console on experiencing the sort of problem he hit (because it would be terrible to have an 'ugly' console...). This hints about another source of the systemd controversy, that it's also symbolic of a lot of other design choices that have come out of the distros.
XML is like violence. If it doesn't solve the problem, use more.
I really struggle to reconcile the Slashdot view that systemd is total crap and the fact that every major Linux distro has switched to it.
The Linux ecosystem is not sane. Redhat wanted more control of Linux so they pushed systemd. GNOME developers are easily distracted by shiny things (as proof I submit GNOME 3) so they went ahead and made GNOME dependent on it. And then Debian (which most Linux distributions are based upon) adopted systemd because GNOME depended on it. There were some other excuses, but that's the biggest reason. You can blame Redhat and Debian for this clusterfuck, and really, only a small handful of people in the Debian community are actually responsible for Debian's involvement. Debian's leaders were split almost down the middle on whether they should go to systemd. This is why major changes should require a 2/3 vote (or more!)
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"