Slashdot Mirror
Ask Slashdot: Can You Say Something Nice About Systemd?
ewhac writes: "I'm probably going to deeply deeply regret this, but every time a story appears here mentioning systemd, a 700-comment thread of back-and-forth bickering breaks out which is about as informative as an old Bud Light commercial, and I don't really learn anything new about the subject. My gut reaction to systemd is (currently) a negative one, and it's very easy to find screeds decrying systemd on the net. However, said screeds haven't been enough to prevent its adoption by several distros, which leads me to suspect that maybe there's something worthwhile there that I haven't discovered yet. So I thought it might be instructive to turn the question around and ask the membership about what makes systemd good. However, before you stab at the "Post" button, there are some rules...
Bias Disclosure: I currently dislike systemd because — without diving very deeply into the documentation, mind — it looks and feels like a poorly-described, gigantic mess I know nothing about that seeks to replace other poorly-described, smaller messes which I know a little bit about. So you will be arguing in that environment."
Nice Things About systemd Rules:
Bias Disclosure: I currently dislike systemd because — without diving very deeply into the documentation, mind — it looks and feels like a poorly-described, gigantic mess I know nothing about that seeks to replace other poorly-described, smaller messes which I know a little bit about. So you will be arguing in that environment."
Nice Things About systemd Rules:
- Post each new Nice Thing as a new post, not as a reply to another post. This will let visitors skim the base level of comments for things that interest them, rather than have to dive through a fractally expanding tree of comments looking for things to support/oppose. It will also make it easier to follow the next rule:
- Avoid duplication; read the entire base-level of comments before adding a new Nice Thing. Someone may already have mentioned your Nice Thing. Add your support/opposition to that Nice Thing there, rather than as a new post.
- Only one concrete Nice Thing about systemd per base-level post. Keep the post focused on a single Nice Thing systemd does. If you know of multiple distinct things, write multiple distinct posts.
- Describe the Nice Thing in some detail. Don't assume, for example, that merely saying "Supports Linux cgroups" will be immediately persuasive.
- Describe how the Nice Thing is better than existing, less controversial solutions. systemd is allegedly better at some things than sysvinit or upstart or inetd. Why? Why is the Nice Thing possible in systemd, and impossible (or extremely difficult) with anything else? (In some cases, the Nice Thing will be a completely new thing that's never existed before; describe why it's good thing.)
We will assume out of the gate that systemd boots your system faster than ${SOMETHING_ELSE}, so no points for bringing that up. Bonus points are awarded for:
- Personal Experience. "I actually did this," counts for way more than, "The docs claim you can do this."
- Working Examples. Corollary to the above — if you did a Nice Thing with systemd, consider also posting the code/script/service file you wrote to accomplish it.
- Links to Supporting Documentation. If you leveraged a Nice Thing, furnish a link to the docs you used that describe the Nice Thing and its usage.
By
eldavojohn
Systemd has a nice ring to it. The way the syllables roll off my tongue pleases me greatly. It could be the title of a great anime series. It could even be the lost name of an ancient forgotten god-king. It might even be the name I give my first born. It sounds much more authoritative and genuine than sysvinit or upstart or inetd. For instance from my non-technical fourth grade perspective this is what I interpret the others to mean:
Contrary to your base assumptions, systemd does not actually boot faster on my Pentium II (Intel inside) system. I just like the way it sounds.
My work here is dung.
I've done migrational upgrades to System-d with Arch Linux with zero problems in addition to using it with new installations. It works fine, and I'm still really confused about the jihad-level hatred it seems to engender in some people.
AntiFA: An abbreviation for Anti First Amendment.
Betteridge's law of headlines is an adage that states: "Any headline which ends in a question mark can be answered by the word no.
Is this the right headline to apply it on?
I like that in the future when the integration is more complete, I'll be able to install a database or a webserver and then once in a full moon when a cosmic ray hits the process and kills it, systemd will just restart it.
Yes, you can do that with other tools too once you've learnt your lesson (many years ago I had 1.5 year uptime with Apache and then it suddenly crashed) and I am using one of those at the moment. What I like is that this will just work out of the box for newbies and veterans alike with no clunky configuration/interfacing.
If you have a service called 'foo', then 'systemctl status foo' not only shows you whether the service is running, it also shows you the last 10 log entries created by that service. This is great when the service failed; usually the error message will be right there.
How does it do this? Well, because all processes created by the service are in the same cgroup, all of the log messages (and even anything they print to stdout, which would have been lost otherwise) can easily be tagged with the service name (and a bunch of other metadata). You can use journalctl to query the journal for the logs from a specific service, and systemctl status does this for you.
Using the information gleaned from this year old bug I was able to create a Tower of Hanoi solver using the dependency resolver's attempts to determine whether a NFS filesystem should be mounted after the network comes up or not. Every attempt to start Apache (which depended on the filesystem) represents moving a disc to the middle tower.
My problem with things like systemd is not that they have nice features - lots of things have nice features. Windows' Shadow Copies is a lovely feature that's a lot easier to configure that some alternative equivalents, etc.
It's that they put those nice features into some new paradigm of configuration when you could have just added them to the existing system.
Reliable servers don't just crash. This is the Windoze (C)(TM) method of reliability by restarting. Defensive nanny processes are a hack and hardly unique to systemd. They invariably result in trying to restart a broken configuration 1000 times in a row. Death to systemd.
Seriously, I've been a Linux Guy since the 90ies and I honestly couldn't care less.
Anything I know about init is about runlevels - and those are a really neat thing. I mean really cool. You can fiddle with those using mc (Midnight Commander) and debian has a stack of 4-6 of those preconfigged and set up by default - last time I checked, some 7 years ago or something anyway.
Point is, my grandma can set up a runlevel that ex- or includes the LAMP stack in it's 'launch', 'init' or whatever-it's-called sequence and I can set my box to it by typing "init [simple Int here]" for my box to go there.
Again, that is pretty neat and cool and the best working solution I've run into so far.
Way better than anything in the Windos world, that's for sure.
If this "systemd" thing - whatever that is - doesn't break this or offers a neater improvement on that runlevel stuff or a way better concept that's worthwhile moving into, perhaps like the SVN vs. Git thing in which Git comes out on top IMHO - without requiring some bullshit GUI tool to be usable, that's all very fine and dandy with me.
If, on the other hand, you're going to push this new fad and hurt me wile doing so, I'm coming for you some time in the future. With a baseball club and my mafia friends. Other than fucking around with one of the best filemanagers ever - Konqueror - and replacing it with an inferior dolphin - this isn't some GUI toy you should fiddle with. This is Linux at a level where it's actually *the* industry standard. As in 'no other even comes close to this level of reliability and quality". Fucking that up would be a really stupid idea.
Otherwise I really positively couldn't care less - and that's how it should be, no? Except for, maybe, if I were a System Developer or Distro Release Manager or something.
My 2 cents.
We suffer more in our imagination than in reality. - Seneca
'systemd' needs to stay optional, and I mean that explicitly. optional. Not "default", and not "optional, in the sense that you then have to do the maintainer's job for them because they are too lazy to consider people not using systemd, because systemd is the default, and the maintainer does not want to consider the people that dont want systemd, regardless of the reasons or circumstances." kind of way.
Systemd is potentially useful for only a subset of the linux ecosystem, and forcing this kind of change is a bad thing.
Please allow me to explain why:
Systemd seeks to be a "Be all, end all solution to system initialization", which ultimately means that it will itself try to cover every possible thing that its maintainers believe needs to or should happen during system init. That in and of itself means that it will be large and cumbersome; exactly the things that embedded linux should avoid, where ultra-minimalism is king. (We are talking systems that have just a few dozen megabytes of memory, and just a few hundred megahertz of processor power at the most. Having all that gobbled up by the init system as soon as power is applied is not going to win you any trophies, and boldly asserting that embedded devices need to obey a desktop paradigm is throwing the baby out with the bathwater.)
This is especially true with "reference distributions", like debian. Debian "console only" deployments with tools like debootstrap are reasonably common with embedded devices, as are deployments that make use of chroots for specific sandboxed services. A chroot does not need a full blown init like systemd. It is best served with a simple init script. Building a distro with the intention of killing simple init, and replacing it with a monolithic solution like systemd will make service daemons much more difficult to control in this way, and will actually rob core functionality away from the distro that goes that route--- exclusively in favor of desktop flavored deployments.
Linux is more than desktops.
Linux is routers.
Linux is home automation systems.
Linux is servers performing specialized functions.
Linux is so much more.
Please be more considerate about trying to force systemd into debian. Optional is OK. Optional like "gnome vs kde vs xfce vs $ManagerHere". Not "optional" like "unity on ubuntu". Debian is a reference distro, upon which many other distros are based. It has already found its niche in the linux ecosystem. Please dont try to reinvent it.
The idea of booting services in parallel is nice, but the problem is that apparently it doesn't have a way to specify that you need to wait for a dependant service to hold off until the initialization of the dependancy is complete. Systemd considers it "booted" as soon as it launches, which causes people problems with unreliable network initializations and such that have been resolved for Sys-V style init scripts for years (if not decades.)
I do not fail; I succeed at finding out what does not work.
http://uselessd.darknedgy.net/ProSystemdAntiSystemd/
This comes from an "anti-systemd" source, but tries to cut through a lot of the controversy and hostility shown on both sides. Bear in mind, you only see the anti-systemd view on Slashdot, but you get just as much idiocy on the other side as well. For example, the Poettering "death threats" were actually a joke made by a bunch of people in an IRC channel. Here, read the log (ctrl-F "hitman"):
http://logs.nslu2-linux.org/livelogs/maemo/maemo.20130215.txt
What the fuck, you think you can set rules for discussion on Slashdot?
Sadly he can't force the rules on anyone, no matter how useful those rules would be for such a debate. So, instead of actually doing what the poster wants, we instead get comments like yours which add absolutely nothing to whether or not systemd is any good.
And yes, it's a question I was considering posting on too. It would be nice to see some useful information.
So thanks for clogging the top level posts up with useless, pointless crap. You have done your fellow slashdotters a great service showing you can be angry on the internet.
SJW n. One who posts facts.
I've been starting to migrate many of my services at home to containers to make them a bit easier to maintain (a bit of a tangent - having 5 containers instead of one host with 5 services means that you have to do 5x as many updates, but each update can at most break one thing at a time). This was trivial to do with systemd-nspawn.
With a command line that barely fills a terminal line I can launch a container, have it boot systemd inside the container, have a few bind mounts, and have it get its own IP like a lightweight VM. Within the container systemd just does whatever it is told to do, like launch ssh so that I can get in, configure the network, and launch whatever services the container was intended to provide. The container journal logs are symlinked back to the host log directory, so they're really easy to look at from the host.
Sure, you can do similar things with docker, but doing it with systemd involves less tooling in general.
Also, for simpler situations systemd-nspawn makes a VERY good substitute for chroot. In addition to doing everything chroot does, it starts a separate process namespace so you don't see outside processes from inside the container. It also automatically sets up /dev for you, sets up resolv.conf, etc - it can do all this while just spawning one program inside just like chroot does (so no need to run systemd inside). It can also set up bind mounts if you ask it to. When you exit it cleans up - no lingering bind mounts, or tmpfs, or /proc and such inside. Also, any mounts inside the container aren't visible outside, so you can run a backup on your chroot and not have it follow bind mounts, or try to save /proc/kcore or whatever. In fact, you could spawn 5 containers inside the same directory and they can have private /tmp and /dev and /proc while seeing changes each other make in the files in the actual chroot.
With more people jumping to FreeBSD, we might get better hardware support over here.
Prediction for end of Universe #42: Fencepost error in Quantum_bogosort.cpp
One thing I really like about systemd is that when you stop a service, it actually stops.
I used to run monit with openrc and when you wanted to restart a service you had to play games to ensure that it was really killed, and that the service state was cleaned up, and so on. Just telling openrc to stop the service just wasn't reliable at all - it worked well when nothing was wrong, but if nothing was wrong chances are monit wouldn't be doing anything.
Systemd is very effective at containing processes and their children and when you stop them, they are all gone for good. If you want to restart a service, systemctl restart service will get the job done 100% of the time, assuming the configuration/etc lets it restart. It does support graceful shutdown of individual services, followed by process genocide.
This also applies to things like cron jobs you launch through it. When the parent process ends, anything left gets cleaned up.
Systemd was forced down my throat by Arch Linux. I didn't know anything about the controversy back then, so I just thought: "There's probably a good reason for this, let's get to work".
I read some docs and I liked the security features a lot! You can tighten services easily with a declarative syntax.
Here's a snippet from my ntpdate.service file. You don't need much systemd knowledge to guess at what each line does:
PrivateTmp=true
ReadOnlyDirectories=/
InaccessibleDirectories=/boot
InaccessibleDirectories=/root
InaccessibleDirectories=/etc/ssh
LimitNPROC=1
DeviceAllow=/dev/null rw
DeviceAllow=/dev/urandom r
User=nobody
Group=nobody
CapabilityBoundingSet=CAP_SYS_TIME
NoNewPrivileges=true
I ended up enjoying that work and tightened things so much that I hit a bug, which was resolved in just a few days: https://bugs.freedesktop.org/s...
But I still don't know how to configure the network properly T_T
Just installing systemd over sysv speed booting up from a minute to 10 seconds, and shutting down from half a minute to under one second. I always hated Linux couldn't handle shutting the fuck down efficiently, now it does.
Or even better : The nice thing is systemD is not portable. It will infect only linux.
aaaaaaa
A small thing I've come to appreciate with systemd is that it actually cares about exit codes. This applies to any unit, including timer units (the equivalent of cron jobs). I ported most of my cron scripts over to systemd and suddenly started noticing scripts which had been having non-zero exits for ages, but fcron just didn't care about exit codes.
You can tell systemd to ignore exit codes for a process, or specific exit codes. However, I've found that in general using systemd I have a lot more awareness of abnormalities in my services.
Sure, you can often get away with ignoring exit codes, just as you can often get away with ignoring compiler warnings. However, in getting rid of them I fixed a few problems ranging from trivial to important, and my system is more robust for it.
You are NO Linux guy! A Linux guy cares about all things Linux, however slightly.
Wrong. A Linux guy cares more about Linux than any other OS and is one who's judgement perhaps has a little weight.
Like, if he's been programming since '85 or something like that and has been using *nix during the times when the only usable editor on it was Emacs or Vi.
You may not believe it, but I, and quite a few others who do computing for a living, actually have a life outside of computers and fiddling with init-scripts and xconfig. Partly because I've done that to death already back in the day when there was nothing else to do and making Gnome 1.x , Nautilus and GKrellm look like Star Trek was a cool way to spend your time.
*the old rooster ruffles his feathers*
We suffer more in our imagination than in reality. - Seneca
If you think you can add this nice thing to syslog, then please do. Quite a lot of people will sing your praises, in fact.
Writing service files for my own daemons or modifying existing ones is pretty close to trivial. The files are short, easy to understand, and there isn't any risk of runaway child processes like there is with a sysvinit init script making them close to trivial to write and maintain. If anything I would say that's why so many distros are jumping on board.
I had to write service files as an early adopter, but it would also be useful for anyone rolling out their own daemons or that needed to tweak the behaviour of an existing service for their own needs. I imagine it would also lead to fewer packaging bugs.
In which case a nanny process restart is useless. Thanks for making my point, idiot.
Many hardware failures are transient, and a process restart is a very effective fix, at least in the short term. In the longer term, you'd better have monitoring in place so you know that the restarts are happening and can decide when to fix the hardware.
In addition, many process crashes are caused not by hardware failures, but by obscure software defects, and a process restart is not only effective at getting the production system back online, but arguably is a complete solution to the problem if the defect is sufficiently obscure that it's very rarely triggered, and hence not worth the large amount of effort required to identify and fix it.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Systemd is modular:
- It's broken up into multiple independent processes, each of which handles one major thing well.
- It's broken up into libraries so that commonly used code (such as parsing config files) is implemented once and shared among the services, saving memory (because you know how shared object libraries work, right?) and ensuring that there's only one implementation of any one thing that needs to be tested and debugged.
- Interdependence among services is minimized, although as with any real, complex system, there are chains of dependencies.
- Dependencies on and among services are handled on-demand so that only the services you need are running (often started well after boot). As a side effect, boot time is shortened.
- Process 1 (init) is very small, with minimal functionality, in order to minimize the chances that it will crash.
The above are all true, or at least they are consistent with claims made by the developers. Sure, I have negative things to say, which are also true, but I don't want to add to the noise of all the false negative claims floating around.
Why do we have to mention Bennett in the non-Bennett posts? can we just have a break please?
(stay with me here...) Once upon a time there was a community. In the community were lots of different opinions - Slackware, Redhat, Debian, the weird *BSD folk - we all worked together, despite being of different religions. We'd yell at each other, and to an outsider we'd look as though we hated each other, but we were yelling at each other at the same bar while buying each other drinks. We yelled at each other because that's just what we liked to do. We had a certain set of rules that we all followed - and those rules were our real religion. We contributed code upstream. We filed bug reports. We did code review. We contributed. We Kept It Simple, Stupid. RMS was one of our major prophets - maybe even a god (though, we often started rolling our eyes and heading home for the night if he showed up at the bar to drink with us). We laughed at people who would declare, year after year, that this would be the Year of The Linux Desktop.
Then, along came two things - Ubuntu, and modern capitalism/culture/media/whatever - a mindset where there should be no plan, just go go go new feature new feature new feature go go go (I'm looking at you Agile, facebook, google...). Suddenly, the highest and best praise your project can get became whether it was "disruptive."
The *NIX/FOSS community would not have been a place for this to take hold, were it not for Ubuntu. Ubuntu decided they would break all our paradigms - they'd refuse to contribute patches upstream, they'd take simple processes that worked well and left tremendous power in the user's hands, and replace them with very broken messes of stuff. (In contrast to what we had...) they'd make an experience that mostly worked for complete novices - to be distinguished from most other distros that rarely worried much if even their initial installer failed because meh, you should know enough to know how to fix it yourself. They'd ignore religious ideals like only using OSS. And last but most certainly not least, they replaced init.d.
Problem is, when a lot of new people started in on the scene via Ubuntu (and the like), the established distros decided that they had always wanted their distro to be the desktop featured in The Year of the Linux Desktop, and realized they were losing overall "market" share (@#$%@ for those nitwits thinking of people as a "market," when we had been a "community" for ages), even though the number of users of each of the major distros was still increasing. So they looked around at what Ubuntu was doing to become popular, and tried to decide what to adopt from it. Unfortunately, this new crop of people included the likes of Lennart Poettering, who would have ideas such as this one, regarding systemd. Instead of seeing diversity and differences as good things, those of his ilk decided to destroy (yes, a harsh word...but it's pretty much accurate) the FOSS community. An entire set of ideals just...disappeared. No longer are simple things kept simple, no longer is "Do one thing and do it well" followed, no longer do we try to let open inter-connectivity organically solve problems of integration (instead, we just birth a giant Rock Biter to mow our laws).
Systemd came from a new set of ideals where solving problems that don't exist is great, so long as the big bad Establishment is taken out. I actually saw it as a bit of agism - where youth expected to be peers to those who had been around for ages, and when they weren't immediately accepted as experts they just co-opted the entire environment and left us old farts without any toys anymore. Oh wait...you wanted something good about systemd. Um, well, my laptop now boots 0.5 seconds faster than it otherwise would have, even if I no longer know why and can no longer really do anything about it. That's good, right?
I like it because it hasn't been proven to cause Tourette syndrome. The swearing it seems to cause is just a coincidence.
Systemd during a normal boot is doing some detailed logging of the boot process that let's you do a simper version of bootchart. This allows you to find out how long each service took to boot and also do a graph of them.
It's concerned complimentary to the actual bootchart...
[1] http://0pointer.de/blog/projec...
Back in 1993 I worked on a network process controller for Inmarsat-A. It had it's own custom init process group leader which erected unix domain IPCs between certain children, some shared memory segments, and performed a waitpid to relaunch any child process that died. I was new to the project and new to unix so when I suggested we use inittab to take over this task, the team lead just laughed. Init was not up to the task. The arrival of systemd finally addresses this use case: a process group leader that can care for of a hierarchy of tightly coupled processes. Systemd may be complicated, but so is the intended use case.
The reality is that before systemd showed up, there wasn't really any project with an active upstream that tried to solve the plumbing problem (I'm not talking about init in isolation here). Each distro had to invent their own hacks, some of them good, some of them not.
The fact is that the community that is beginning to form around systemd is much more healthy than the scattered bits and not-quite-fitting pieces we had before. Maybe that's sad, I don't know. I think that in the end, the unification around systemd will allow competitors to form (just implement the interesting subset of the systemd interface and you can integrate with all distros!). So long term we'll end up with a much more vibrant plumbing for Linux.
Holden: You're in a desert, walking along in the sand, when all of a sudden you look down...
Leon: What one?
Holden: What?
Leon: What desert?
Holden: It doesn't make any difference what desert, it's completely hypothetical.
Leon: But, how come I'd be there?
Holden: Maybe you're fed up. Maybe you want to be by yourself. Who knows? You look down and see a server, Leon. It's serving web pages ...
Leon: Server? What's that?
Holden: You know what a computer is?
Leon: Of course!
Holden: Same thing.
Leon: I've never seen a computer... But I understand what you mean.
Holden: You reach down and install Microsoft Windows on it, Leon.
Leon: Do you make up these questions, Mr. Holden? Or do they write 'em down for you?
Holden: The server lays on its back, its case baking in the hot sun, thrashing its hard drive trying to boot up, but it can't. Not without your help. But you're not helping.
Leon: What do you mean, I'm not helping?
Holden: I mean: you're not helping! Why is that, Leon?
[Leon has become visibly shaken]
Holden: They're just questions, Leon. In answer to your query, they're written down for me. It's a test, designed to provoke an emotional response... Shall we continue?
[Leon nods]
Holden: Describe in single words only the good things that come into your mind about... Systemd.
Leon: Systemd?
Holden: Yeah.
Leon: Let me tell you about Systemd.
[Leon shoots Holden with a gun he had pulled out under the table]
In which case a nanny process restart is useless. Thanks for making my point, idiot.
Your logic is flawed. Random bit flips sometimes happen. They don't necessarily take down the whole machine.
Having read the myths page I largely believe it was the right thing to do. Linux is a living operating system and sometimes it has to be dragged kicking and screaming away from things that may have been acceptable in 1990 but not when going against other modern operating systems. Wayland is another ongoing example of that and I'm sure that once it becomes the default choice in some dists that we'll see people being extremely vocal about that too.
Poettering get to pretend that he's Linus. That, and it manages to start all of the services correctly on my system without hanging about 9 times out of 10. Aside from that? as previous posters have said, if a service stops/crashes for some reason? I want it to stay stopped until i can figure out what is wrong. MySQLD? out of disk space? I don't want that to keep on yo-yoing up and down until i have it fixed. I agree, SystemD should be optional, you want it on your phone/IOT device? fine. My server that I reboot about once every 2-3 months? I want init.
Background: I've professionally administered Unix and Linux machines for >25 years, including various BSDs, Linuxes, Irix, HP-UX, Solaris/SunOS, AIX, etc. I've been certified by several vendors or distributions, including, since 1999, Red Hat (which gives me quite a bit of background on their specific implementations over the years). I don't work for a company doing development of any OS or platform. heck, other than random 401K type aggregate ownership, I don't own stock in any company that cares about this issue at a deep level, to the best of my knowledge :)
Personal Bias about this thread: You pretty much lose all the credibility possible with me when you start of with "I ... dislike systemd because ... it looks ... like a poorly-described, gigantic mess I know nothing about ... ." (It's the "disliking something you know nothing about" that bugs me). Otherwise, I don't care much about this debate on a personal level. I currently admin boxes using systemd as well as everything else, and nothing about systemd has caused me anywhere near the heartache that it seems people who haven't used it much seem to feel about it.
Seriously, there're thousands of pages of documentation about what it is, how it works, and what most if not all of the design basis decisions are/were. I'll link you to a few of them because hey, you can get to slashdot and post, but you can't seem to use Google ;) (tongue in cheeck, of course). There're plenty of folks who DO have great detailed reasons on why they don't like bits and pieces of it, and you should be able to compare them to the various info I'm linking below.
Systemd has tons of upside and tons of downside. Most are pretty well detailed, although many of the gut reactions people seem to have to it are based on a lack of understanding about how it works and what it's compatible with, to wit "I can't use shell scripts for anything at startup anymore!" , "All of my old chkconfig or SysV scripts can't be included at all!", "It kills off syslog!", "The only reason it exists is to make laptops boot faster and in the server world we don't care", etc. Those are easily researched and the actual basis (or lack thereof) pretty easily found.
So, for why the systemd setup looks like it does, you can go back 4.5 years to where the announcement and rationale is described. Speed is part of it, as is device changability, as is double-forking, resource limits, and service state checking and recovery. Yes, it's a load of stuff. Definitely a system-wide approach VS a semi-random collection of various ways to do things all tacked together (which is, frankly, what most Unix and Unixlike systems are, through survival of the fittest).
http://0pointer.de/blog/projec...
http://0pointer.de/blog/projec...
http://0pointer.de/blog/projec...
http://0pointer.de/blog/projec...
http://0pointer.de/blog/projec...
Since RedHat's obviously the largest major proponent and arguably the source of the most production users, here's their documentation:
https://access.redhat.com/docu...
Here's the project page with loads of links about the software and uses cases:
http://www.freedesktop.org/wik...
And of course so many questions have been raised the developers have posted their rebuttals to myths or misunderstandings.
http:/
OK I'll try this. Traditionally Init services were about starting processes. They didn't have capacities for keeping processes running. Which meant that for processes that need to be kept running and for which there was a real possibility of failure (most of them) the init had to start a process management system which then started the functional process. This has been the status for years. With systemd there is a process maintenance component standard in the init system. Which means that processes not only start but are capable of being kept in a stable state easily and automatically. Process management stops being something system admins work hard at and instead becomes something that happens out of the box.
Bennett provides a service.
Mosquitoes provide a service.
systemd provides a service.
Let it be.
It little behooves the best of us to comment on the rest of us.
Sometimes, availability really is critical. In that case I want to take the risk of an automatic restart before the cause is investigated. However, it is important to appreciate that the approach is risky . The restart can cause cascading errors that change a reasonably simple issue into a multi day recovery operation.
No, that's not what I mean. You've removed the new "useful" features that I'm okay with - and might want - but in doing so you've told me to replace everything I have with systemd.
Much more useful to me would be to get that listing, with those log tails, on a sysVinit. And that is far from impossible.
What we have here is bundling issues. Want the nice feature that does something quite simple? Then change everything you have to our system.
That's systemd's problem (and your response is their attitude) in a nutshell.
That's all well and good, but if Linux developers say that they want to take the desktop then they will need to come to terms with the fact that users don't care about anything other than usability.
"There are lies, there are damn lies, and there are statistics"
I worked on space rated instruments at NASA Goddard for a while, and talked to a very old dude who claims to have seen this happen once in his very long career.
I am very small, utmostly microscopic.
Something nice about it....okay, Windows 8.1 didn't implement it :-P
Caveat: I am a server admin.
With systemd, one can't even remotely log a journal natively, which is par for the course in many server environments. You can't make this stuff up, please see bug #1098132 (https://bugzilla.redhat.com/show_bug.cgi?id=1098132) At the time I'm writing this, that functionality still just *doesn't exist* in systemd/journalctl. It was almost pushed into the last revision, but the bugs were show-stoppers so was pulled. Even if it does go into the next systemd revision, how long until it's really kosher for solid/production environments? 1 year? Two? Three? Yet it's being pushed as the default *now*.
To explain why this is important: if someone cracks in, the log files are going to be one of the first things they muck with. You have locked down syslog/et al, so you can tell if they muck with the binaries, but the log files themselves are considered compromised. Logging remotely at the same time before the data even hits the disk creates another layer of safety, which you simply can't do with systemd/journalctl. One day you will, assuming you want the binary journal and journalctl, but even if you wanted those now you couldn't. Yes, you could rsync over a copy of the files, but there's a reason why people aren't just doing that for regular logs and this is going long.
Over the last decades, people have worked out tried-and-true systems for documenting and verifying logging. So from an audit perspective, with journalctl you can't lock down syslog-ng and that process, because you've introduced an untrustworthy one above it. They know the current strengths and limitations -- to the point that there are legal/liability issues involved for many if their logging goes wonky. Journalctl adds a layer between the systems they have that work (and even have protocols in place for in terms of auditing), and as of right now adds a *wonky* layer that's very buggy. And by buggy I mean prone to corruption and simply not doing as its told.
Here's the real kicker: most of this issue would go away if systemd was willing to allow the user to not use journalctl and let things like syslog-ng have that data in the raw. It is choosing not to allow that as a tactic, as opposed to what the users and tech itself wants, and so here we are.
Also, this entire proposition by samzenpus is inane. When one thinks backwards from what the motivations might be, none of them are good and make me lose that much more respect for the site.
Have gnu, will travel.
Who said it had to be quietly?
So, the nicest thing you can say about systemd is an ad-homenim attack on people who don't like it?
And I wonder why systemd threads end in massive flame fests.
SJW n. One who posts facts.
I came here to post something similar. We've seen a load of switchers over in FreeBSD land as a result of systemd, including one company that's currently a moderately large RedHat customer and plans to migrate before their current support contract expires.
I am TheRaven on Soylent News
i'm a small time admin, running about 100+ wireless nodes for artisans, private and commercial users. the -majority- of users running debian on desktops (autopilot upgrades rolled out), the rest are smartphones etc. windows is -actively- banned from the network. all routers, servers, firewalls are running debian, handrolled with lenny, wheezy and jessie, but all configs by the book, and audited by an experienced 2nd set of eyes. i'm 35 years in computing, telecoms and electronics. systemd is the worst single time-waster event in my professional career. although not yet enabled by default in jessie, it's causing a -lot- of disruption with basics like remote rebooting becoming an economical hazard (petrol costs), remote desktops not cranking up, auto logins not working (previously reliable packages like gdm3 break on upgrade due to forced dependencies to systemd related packages), endless hangs on startup/shutdown, many users complain about slow shutdowns, or machines not shutting down at all, services not starting. and yes: i understand how the beast works, and i can see the beast is doomed. the logging is a ---fucking--- nightmare, works fine on paper, but is making it impossible to get basic stuff done when in the real world things break, and taking the unreadable logs with it into the ether. i'm yet to see a machine that actually booted faster, or indeed shut down faster. agressive parallelization seems to apply ony for a subset of computing equipment i'm unfamiliar with. i can already hear the smart arse comments from corporate armchair admins: 'do this, do that ...' but the reality is that the problems caused by the yet flaky systemd is creating such an extra workload that is buries any hope of getting to grips with it. driving to each customer and wipe the systems fucked by systemd and re-install them with wheezy or carefully pruned jessie is cheaper, and we can all have our lives back. i never thought i'll be wading through backported software again ...
for regular user systemd brings no obvious advantages, at least not for their ancient harddrives on single core desktops - but instead a few ended up with trashed installs, e.g. by old non-critical errors being dragged into an upgrade and then breaking on (remote) reboot.
if i hadn't left the core machinery on lenny and wheezy i'd be in deep shit now, and likely going out of business.
stefan zalewski, burren.org
To date, there is no evidence that systemd causes ebola.
Also, not coincidentally, quietly allowing hardware problems to persist until data structures and the filesystem to be corrupted before anybody notices.
Hence the importance of monitoring so that the failure is not quiet, as I already pointed out. Please try reading and fully understanding posts before responding.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Wow. The usual complaint is the myth that systemd is monolithic. It's not. But now you're complaining that it's broken up into too many services? And how is this any different from sysvinit, which also starts any number of different binaries?
All they've done in systemd is write C code to start up services that used to be started instead by shell scripts and added the ability to make dependency resolution automatic so that services are only started when they need to be. So basically, they made it smarter and faster. The complaint that it's got too many binaries is moronic and a complaint just as well against sysvinit.
Systemd is modularized into a number of different binaries, each of which handles a different service. Thus, different functions are isolated from each other. This enhances stability and improves startup performance when not all need to be running first thing at boot time.
Oh, and all of systemd's config files are written in ASCII, in the traditional UNIX way. One cool thing they've done is made a single parser implementation (in a shared object library) so that all of the config files have the same syntax. Also, when you debug a problem with parsing for one service, you automatically debug it for all others at the same time.
If so, the name choice was brilliant (regardless of whether the project itself is.) Thanks for the info.
:)
Also: Wikipedia suggests that the 'D' in "Système D" can stand for "démerde"
HSJ$$*&#^!#+++ATH0
NO CARRIER
Not to mention crashes caused by rare, hard-to-reproduce race conditions.
Indeed. That is one sub-category of the obscure software defects I mentioned. It's probably the best example, actually.
It's interesting to describe the approach used by many systems at Google (where I work; I'm now in Android but used to work on Google servers): a common pattern at Google is to crash immediately upon detection of any error. This is actually just a logical extrapolation from Google's long-used approach of building reliable systems on top of cheap, unreliable, commodity hardware, applying the same notion to individual software components. System designs assume that anything can fail at any time, and are built to recover gracefully, possibly with some degradation. So there is extensive infrastructure to fail a request over to to another process instance, and to automatically restart any failed processes. And of course there is extensive and detailed monitoring, with lots of statistical analysis of failure modes plus charting of everything to enable patterns to be recognized and various forms of alerting, ranging from automated bug filings, to e-mails, to pages delivered to on-call engineers.
Given that approach to fault tolerance, it's often very reasonable, at least for non-Java processes, to simply abort/crash whenever anything goes wrong. Restarts are automated and fast (for non-Java processes; JVMs are a bit slower to start) and monitoring and alerting take care of letting people know what happened and how often. This includes both hardware and software-related failures. Monitoring also pays special attention to processes that fail repeatedly (called "flapping") upon restart and generates high-priority alerts. The restart infrastructure will also slow and even stop the restarts of flapping processes.
Anyone who's used the googletest unit test framework for C++ may have wondered about the extensive support for and documentation of so-called "death tests", which allow you to verify that your code crashes when it's supposed to, and in the right way. This is a consequence of this particular approach to fault tolerance; if crashing is part of your reliability plan, you need to test that your code crashes when and how it's supposed to.
None of this has anything to do with systemd, of course. The fact that a strategy is effective in Google's environment is utterly meaningless in single-server contexts. In this case, though, auto-restart plus monitoring and flapping control seems like something that could usefully work in many contexts, perhaps even as the default.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
systemd's logging system is a huge improvement over old legacy style text logs. It has more early boot logs since it can log while still in initramsfs, and with kdbus it will probably get even earlier and late logging; the goal is "metal-to-metal" logging.
Having structured and indexed logfile (called journal) is a huge improvement in many ways; it allows for rich meta-data like monotonic timestamps, high precision time stamps, UUID's, exe file names and paths, and incredible easy and powerful sorting and filtering with tools like "systemctl".
If you try to add meta-data like monotonic timestamps in flat text files, they become very long and cluttered, making them hard to read for humans.
Another problem with flat text files are the fact that watch scripts depends on the exact wording of the daemon output strings; if the developers changes the wording, third party scripts will fail. In a perverted sense, the exact wording have become a API that cannot easily be changed or extended.
Not so with systemd; it has a stable API and lots of language bindings. It is easy to make a watch script that targets the stable field's.
Some CLI examples: I have used full length options for readability and since systemd have excellent bash completion for everything, it doesn't involve much more typing.
journalctl --boot -1 --priority err
Show all log entries from previous boot only, that have log level "error".
journalctl --since -5m
Show log entries generated the last 5 minutes.
journalctl --follow --unit smartd.service
Follow (tail) the smartd daemons log output.
journalctl _KERNEL_SUBSYSTEM=usb --priority warning
Show only messages generated by the kernel USB subsystem that have log level "warning"
journalctl --field _PID
The "--field" option makes systemctl show all values of the following journal field. In this case it will show a list of all PID's that have ever written to the journal. Want to see every executable that have ever generated log output, substitute _PID with _EXE. Notice the underscore. Field values with underscores have kernel guarantee for being correct.
It is also easy to track two services that you suspect are causing each other problems:
journalctl --output short-monotonic --boot -u NetworkManager.service -u chronyd.service
This will show the log entries for NetworkManager and a sNTP client from the current boot, and show the output with monotonic timestamps. Nice.
The only good thing I can see about systemd is the exposure of some Linux system APIs that were not exposed via the POSIX subsystem. Nice - but not required by most of us - and could be added to existing standards based initialization daemons without totally rewriting the rules.
Otherwise it seems to be more likely a thinly veiled (actually not veiled at all...given comments of the principles) attempt to fragment the POSIX world - and forcing projects with limited resources to make a Hobson's choice of whether to support systemd based Linux or POSIX standards exclusively. It breaks write once - compile/run anywhere - that was generally available for those who made sure their applications were POSIX compliant. This means that a lot of software that was available across Linux, and Unix flavors (BSD) will now be exclusively available on one or the other - thus fragmenting the *nix world.
Software is not separate from the ethics that surrounds it. This approach and apparent rabid anti-interoperability view is arrogant and self-serving at the expense of cooperation and choice. Furthermore, the monolithic architecture, obfuscated binary logs, and centralized configuration are antithetical to the Unix way - and makes a linux system as difficult to deal with as a Windows system from an automation and management perspective, and raises concerns in terms of security (the greater the complexity in a system, the greater the opportunity for bugs - and thus the greater the attack surface).
Finally it throws away many many years of experience/knowledge acquired by system admins, developers, and users about how a *nix system operates and is configured. This fragmentation of the human factors aspect will by its very nature cause faults/issues during operation.
So - for a host of reasons, I believe it is technically - and more importantly - ethically wrong.
There is actually one more good thing I can think of: it will spawn new distros, software projects to provide alternatives of various applications in the stack, and perhaps new operating systems altogether - with a renewed focus on design simplicity (KISS) and all of the benefits that come from that. Once a system becomes too complex to understand - are you sure you can trust it? So to recap: systemd has two things going for it; exposure of Linux APIs, and the power to breath life into the further exploration of alternatives in the OS/application layer.
Lodragan Draoidh
The more you explain it, the more I don't understand it. - Mark Twain
Did you ignore the first paragraph?
Yep, but I fooled around with hotplug a while back. Bsically it resulted in a shell script being called. So you could execute arbitrary code on a hotplug event. Sure it might be nicer in systemd, but doing it at all (what you mentioned) was certainly -possible- before.
So since what you wrote was not actually an advantage of systemd the way you said it, I decided to ignore it.
That left a massive blob of ad-homenim attack on people who don't like systemd because they hate normal users and want linux to be hard so they can be super elite or some other bullshit.
In other words the mythical RTFM n00b linux neckbeard. I've been "in the community" for over 15 years and I don't think I've ever actually seen one in the wild.
No sane person wants Linux to be hard to use. And I doubt you can actually find any instance of anyone anywhere advocating sysvinit because it's harder than systemd and therefore better.
I think you're massively biased to the point of being bigoted against people who you simply do not understand. Your lack of understanding leads to hostility.
As I said, no sane person wants to have a system that is hard to use. People like me certainly don't count as "normal users". A system that is easy for me to use is one that is transparent and malleable. Any system that does to much "automagically" is harder for me to use because it is harder for me to bend it to my will. Changing the behaviour involves chasing down all sorts of levels of automatic systems to find who processes what. For example, for the first time in 10 years under systemd I can no longer get the power button to function properly under FVWM.
It so happens that that is the polar opposite of what "normal users" want where they would rather have a system that does as much automatically as possible. To them my system is hard to use. To me, their system is hard to use.
See, none of that involves me hating normal users, nor does it involve me feeling superior or wanting to squeeze them out in order to feel elite. It just involves me wanting to get my laptop up and running with the tools I like using working the way they worked before.
But according to YOU, I'm some RTFM n00b user hating neckbeard who uses linux only to feel superior to the smelly unwashed and despicable masses, and all because I don't like systemd. The fact that no one I asked on a variety of fora could tell me which component was supposed to deal with the power button and other ACPI events and how to override the default action has *nothing* to do with it. It's all because I hate users.
SJW n. One who posts facts.
I like the fact that systemd's wholesale acceptance by all major Linux distros means much less fragmentation in both the way the Linux OS is managed and the easy that upstream projects are now able to support advanced features in a distro agnostic way.
I also like the fact that it exposes advanced kernel features like "cgroups" and "Capabilities" and make them easy to use; just add a keyword in a text config file and restart the service.
Add ProtectSystem=full in a service config file, and the service and all processes it makes can only "read" /usr and /etc enforced by capabilities. So even if the service is compromised, the hacker can't modify these directories, even if the hacker are able to execute code with root privileges.
http://www.freedesktop.org/sof...
Also "ProtectHome=" makes the service unable to read /home, so that no information stealing can take place.
Perhaps the best thing about these features and the many other systemd features like those, are that they can be enabled by either upstream or the distro makers, so that the end user doesn't have to anything in order to improve the system security. It will simply mean improved security and "defense-in-depth" as deafault on systemd distros.
I have an RHEL 6 system with multiple commercial Java applications. For some reason, every commercial Java application out there seems to bundle half of a linux distro with it: they insist on using their own instance of a web server, their own application server, their own database, their own mail daemon, etc. Starting those things serially is true insanity and takes forever.
With all of this stuff migrated to RHEL 7, tweaked so that there's no init + rc.d craziness left, a 4 minute boot-up turns into 1:30 boot-up. This is with spinning drives. I have another iteration of tweaks where I exposed all internal service dependencies inside each of those monolithic app monsters to systemd, and we can be up and running in 1:15. Looking at I/O statistics, with further judicious use of preloading we should cut it down to 1:00-1:05 after a clean shutdown. The shutdown times, usually well over 2 minutes long, are down to 30 seconds.
This is all measurable, no-bullshit, experimental data. We even managed not to have to touch any of the commercial app's rc scripts, I simply coded up the requisite systemd configurations for those services, based on the rc scripts. If there's any question as to systemd's effect on the application, or if we need to deal with vendor support, we can always start it up using the rc scripts.
So, I don't care about ideology behind systemd, and how it fits with someone's ideas about what Unix or a Linux distro should or shouldn't be. All I know is that it'd have taken me 5-10x as long to tweak the system rc.d scripts to parallelize them than it took me to "port" the commercial apps we use over to systemd. Of course the distribution's own services already use systemd, so I didn't have to touch that, only tweak a few things slightly to further leverage parallelism. It has saved us time and money, and the system availability is much better in face of the rare reboots. Everyone is happy. We're a small shop with a single server, and we use no virtualization nor any other enterprisey stuff. Just a plain old RHEL 7 running directly on hardware, with selinux turned on, with custom policies written for each of the commercial apps.
A lot of the "recommendations" I've got from "veteran" admins essentially reduced to throwing money and resources at the problem. That's IMHO a rather direct vindication of systemd: if it takes a SAN, multilevel storage and VMs just so that the damn init/rc.d-based system will boot in a reasonable amount of time, and all of that is taken care of by systemd, it'd be rather irresponsible for me to try and ignore systemd. In the name of what? Nothing I can think of. It's not like I can tell the management "hey, RHEL 7 comes with systemd, but I can keep using RHEL 6, not use systemd, and spend $25k+ for a SAN, VM and OS licenses, and the time to set it all up and document it all".
<rant> It takes some chutzpah for commercial vendors to claim RHEL 7 "support" in their solutions, if they support neither selinux nor systemd.</rant>
A successful API design takes a mixture of software design and pedagogy.
Caveat: I am not a sysadmin. But I have read up on SystemD.
With systemd, one can't even remotely log a journal natively
Why not? SystemD offers its own logging system, but does nothing to prevent you from installing a more capable logging daemon such as rsyslog.
Note that before Fedora 20, rsyslog was installed by default, along with the SystemD logging. In the announcement it says:
Emphasis added by me.
http://fedoraproject.org/wiki/Changes/NoDefaultSyslog
You stated "one can't even remotely log a journal"... well, one can if one is able to type: yum install rsyslog
So IMHO your whole argument fails. Not only is it not impossible, it's not even difficult.
The story was submitted by a user named "ewhac". Unless you are accusing "ewhac" of being a sock puppet fro samzenpus, this whole mini-rant seems rather pointless.
lf(1): it's like ls(1) but sorts filenames by extension, tersely
Wrong, services are considered "booted" when they are ready to process requests. For most services, the kernel kan be made to "buffer" request, so we don't care in which order they are started. For those rare cases when we do wan't to hold off a service, the unit-files can specify exactly what to wait for.
Oh, and by the way Sys-V init scripts never solved this. They depended on the daemons to solve this, which they virtually never did. Or sprinkle the init-scripts with strategically placed sleeps. This resulted in either the boot taking to long, or fail.
Try to do a 'grep sleep /etc/init.d/*' Every sleep you find is a potential boot failure.
You can read more about it in the systemd-documentation. You'll find it on the systemd homepage: http://www.freedesktop.org/wik...
It happens about 40 times a day on you average PC
No, it doesn't. I operate lots of machines with ECC RAM. I've seen ECC correction. Bits flip, but nowhere near that frequently. It's very rare.
For those who think SysVinit style init systems is what Linux should be using the next 30 years, there is Slackware. It is a nice general purpose distro that is very traditional. So nobody is forced to use systemd if they don't want to.
Until some key functionality used by people is no longer available in that distro due to decisions made upstream to no longer support the code base, or other dependencies.
If I use KDE - which I do - then packages for that become unavailable at some point in Slackware given the above. That means I will be forced to use systemd if I want to continue using KDE; which also means I will have to change distributions, assuming Slackware remains systemd free, as well.
Not trivial. Not easy. Not freedom of choice.
That is simply a lie.
Lodragan Draoidh
The more you explain it, the more I don't understand it. - Mark Twain
I have all sorts of high reliability services running on quite a few different servers, and I have plenty of them that will restart themselves, monitor their own crashrate and terminate completely if they crash N times in M minutes, etc. This is not rocket science and doesn't require to be built into PID 1 or something like that.
Frankly I think its a bad idea to create dependency chains onto huge complicated subsystems that often aren't appropriate, aren't needed, or are simply overkill.
"Malo periculosam, libertatem quam quietam servitutem." -- Jefferson