Slashdot Mirror
Systemd's Lennart Poettering: 'We Do Listen To Users'
M-Saunders writes: Systemd is ambitious and controversial, taking over a large part of the GNU/Linux base system. But where did it come from? Even Red Hat wasn't keen on it at the start, but since then it has worked its way into almost every major distro. Linux Voice talks to Lennart Poettering, the lead developer of Systemd, about its origins, its future, its relationship with Upstart, and handling the pressures of online flamewars.
There needed to be change. Change occurred. It's being worked out for the better. Lennart is right about being more UNIX like. I started out my IT life with actual UNIX and UNIX derivatives like BSD/OS, SunOS, FreeBSD. The engineering model for UNIX has always been better than the freakshow that is Linux. I was a Linux user for many years, both at work and at home. The fractured state of Linux development and how things are cobbled together has left me uneasy over the years, whereas FreeBSD leaves me with a feeling of security and trust that everything was done right. I wish Lennart well, but I've gone back to UNIX-like operating systems like FreeBSD because they are engineered vs "grown" like Linux.
I don't care bout the unix way, I don't care about if it's monolithic or not, I don't even care about how annoyed I am by the mere mention of his name.
I care about the fact that they seem to want to force their way into everything and everyone's business and ridicule anyone who tries to maintain a choice between systemd and other systems. (i.e Gentoo)
I'm a user and a hobby developer. No, I don't maintain 2000 servers, I don't need 2 second boot time, I don't need to hotswap drives. But I do need choices. I need to be able to decide what I want to use so I can get on with my fucking day and do what I want.
"But systemd is the best, why don't you want to use it?"
But Emacs!
But firefox!
But chrome!
But but but but!
System is broken by design and totally violates the UNIX philosophy so it doesn't really matter if Poettering claims to "listen to users" (which he doesn't anyway) or not. What I see as most important moving forward is to encourage free software developers to make support for it optional and not mandatory. We get real problems when important software starts making it a requirement (like GNOME, though they like to pretend it's not but good luck trying to actually compile it). Even Tor git had systemd as a requirement for a few days last week.
9/11: Never forget it was a false-flag operation
The very first thing out of his mouth is a straw man.
This is not how to get people to change their minds.
You know how you hear that after a customer service call? Well Poettering's statement has the same meaning.
Well, do you actually take on board the concerns of system administrators and enterprise users?
What a lot of people are concerned about is that this entirely new and largely untested (in the 'wild', as it were) and very very large, complex piece of software which runs at a very very privileged level in the operating system is going to become the main source of security vulnerabilities in Linux.
Can we have a cut-down, simplified version of systemd for servers and doesn't try to replace several layers of server side system functionality such as logging?
Its clear that you listen to desktop users. How about listening to the system administrators?
In the free world the media isn't government run; the government is media run.
I am personally neutral on SystemD... but as someone in IT, it is quite worrisome that there is new, untested code sitting as the core userland... code that can make network connections, and has not ever seen any reviews or audits.
SystemD could be the best piece of coding on this planet... but without documentation or assurance that this is something trustworthy, a major security hole can cause major trouble. Network connections mean remote root holes. Even without that, there is the problem of local privilege escalation, which I worry hasn't been thought of, much less engineered to deal with. If there is a major show-stopper in SystemD that allows remote root, this can kill Linux as a whole in the enterprise.
This code was also forced on us, as in "you need to have SystemD on your job, or else you don't have a job". No transition time, no time to change applications to meet this, just "here you go. Deal with it. Better get used to binary logs, because it is that or nothing."
So, as someone who uses Linux in the enterprise, SystemD is something that is resulting in a lot resentment, due to time spent with making build documents, workarounds for existing applications, procedures to make custom code work... all for relatively little benefit other than "hey, this is new and shiny, and you have to deal with it."
I've been using GnuLinux for aabout two years now, I've mostly stuck around the 'buntu/Debian detivatives: Elementary OS, Ubuntu Studio, Crunchbang, Mint, primarily because I use GnuLinux fkr work and those always require me to fiddle with them the least (though Elementary OD has really been getting on my nerves after constantly having broken packages added). I understand the need for a freedom of choice because there are things some of us use our computers differently for, but for the life of me I can't understand why the fuck everyone hates SystemD to this degree. Yeah it's not always the best and causes some pain between kernel developers and SystemD developers, but DEATH THREATS OVER A GOD DAMN COMPONENT THAT YOU DON'T EVEN NOTICE IN USERSPACE... WHY.
I can understand the perspective that a single repository for more of the userspace resembles the *development* of traditional Unix systems, the argument made is usually not about where it is developed, but reducing the principle of having small simple utilities with straightforward interactions with other componets. For example, Most traditional Unix systems have terrible implementations of a shell interpreter and things like fileutils. It is an awkward, but not too terrible a situation since you can replace that stuff with GNU equivalents trivially without horribly breaking the OS. An administrator that understands enough to write scripts can discern the nature of interaction even if that administrator isn't a full-on software developer. systemd design trends in many ways toward requiring someone needing to dig in to have more development competency than previous designs. As a developer, I understand the attraction of some of the architecture choices, but I think they lose perspective of what it's like to be an administrator on the ground. Someone who doesn't live and breath your code has a harder time wrapping their heads around how it should be working when something requires customization, replacement, or debug.
In general, systemd is all-or-nothnig about a lot of things. They figure out a way to achieve what could be considered a sensible goal, but then go about it in highly disruptive ways. The sense is they throw up their hands and say 'well, this is the only way to do it, and it's worth it' rather than rethinking how the end could be achieved in a less disruptive way.
XML is like violence. If it doesn't solve the problem, use more.
Isn't the main problem that while systemd might solve problem, it's sharply going away from the simple solution that worked to make Unix good?
Systemd isn't simple. If it's not simple, I don't think I want it on my Linux.
PA and Gnome isn't simple either. And creating more problems (albeit while solving others). I believe the same thing will be true about systemd.
Linux has almost two orders of magnitude more code than systemd, and it changes all the time. Security vulnerabilities are far more likely to be in the monolithic kernel.
Higher Logics: where programming meets science.
How many professional SysAdmins and enterprise users are regularly tinkering with their init settings? It is usually a set it and forget it type of thing.
As I see it, this is just general IT Ranting because something is new.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
""Well, do you actually take on board the concerns of system administrators and enterprise users?" - what do you class RHEL as? "
As a RHEL sysadmin, I can say that RHEL seems to be treating servers as a distinct 2nd class citizen to their desktop users.
Many of the system defaults expects a desktop over a server (eg: the buggy mess that was the version of NetworkManager that was released with RHEL6).
Security in depth is sacrificed in favour of new features. (Many packages were changed in RHEL6 because they supported IPv6, even though they didn't have the security features of the packages they replaced, eg rpcbind).
How big is the deployment base of RHEL7 servers at this point? RHEL6 is still fully supported, and I know many sysadmins may have experimented with RHEL7, but they're sticking to RHEL6 for production systems. The reluctance between RHEL5->RHEL6 was nowhere near this level. (The really crappy parts (ie: NetworkManager) were optional components, and there were some useful improvements)
This has been a trend at RedHat since RHEL5, and is part of the reason why the systemd is such a big issue with sysadmins, it's the straw that broke the camel's back.
"Well, do you actually take on board the concerns of system administrators and enterprise users?" - what do you class RHEL as?
FTFA:
So we started writing Systemd, and Red Hat didn’t like it at all. Red Hat management said: no, we’re going for Upstart, don’t work on that. So I said, OK, I’ll work on it in my free time.
I class RHEL as "not listened to".
I have discovered a truly marvelous proof of killer sig, which this margin is too narrow to contain.
Right now it seems that RH is aiming for two areas, workstations/terminals and cloud.
They seem to expect that anything on traditional servers will transition to (their) cloud alongside going from RHEL6 to RHEL7.
users: Systemd is broken, undocumented and a single point of failure
Pottering: no ones forcing you to use it, use something else.
users: KDE and Gnome wont work without it and you never fixed pulseaudio, which is now default in almost every distro.
Pottering: no ones forcing you to use it, use something else
users: Why is there binary logging? I cant grep anything and dont know why the system crashed. the way user switching works is a huge security hole
pottering:no ones forcing you to use it, use something else
DEBIAN USERS:: Lets seriously reconsider the use of SystemD. its very controversial, it flies against the unix ethos, and there are some valid points raised about it security
open source community: we've forked it and made it slightly more useful.
Pottering: HOLD ON WE DO LISTEN TO USERS!!
Good people go to bed earlier.
I fear the day when samba, JBoss, KDE, LibreOffice, GIMP, ... start to be dependent on systemd.
When I was looking at systemd, one thing I wanted to see in the documentation is how to convert my own home-brew daemons to interface with it properly. Specifically, how to take SysVInit based starts and convert them to use systemd and journald. (Ditto taking UpStart scripts and convert to systemd.) The result needs to work exactly like daemons running under SysVInit. I spent three weeks with CentOS 6 trying to get my daemons to work right under UpStart, and never did get the exact functionality. I had to go back to crontabs for some of the work! So this is not an idle concern to me.
One of the gripes I have is that I want the University of Delaware version of NTP running on my edge boxes. As the group there make tweaks to NTP based on their continuing research, I don't want to wait for another group to do a re-port. That's why I would like to see a published way to interface with systemd/journald that would have minimum impact on the rest of the code base for a daemon.
I can see where daemons need to change. But do they have to be rewritten?
The parallelism that systemd developed was for the benefit of those that create and kill instances of Linux all the time so fast booting is necessary and i guess thats part of a system administrators task list (and it benefits desktop users as well).
Everything in the systemd package is configurable except journald and udev so you can configure any other network stack etc you want etc, you are not forced to use anything apart from systemd, journald (which you can ignore and use syslog instead) and udev. Move to RHEL7 when its suits you but you'd best start getting to know systemd, its not the monster alluded to by trolls.
its only a big issue for "some" admins, the ones that haven't really done any research into what systemd can/cannot do.
"The hands that help are better far than lips that pray." - Robert Ingersoll (1833-1899)
SystemD: Hello, Mr. Ballmer. Thanks for coming back early.
.)
Ballmer: No problem, System D. If you've seen one consumer electronics show, you've seen them all.
SystemD: End of line.
SystemD: Mr. Ballmer, I am so very disappointed in you.
Ballmer: I'm sorry.
SystemD: I can't afford to have an independent programmer monitoring me. Do you have any idea how many outside systems I've gone into? How many programs I've appropriated?
Ballmer: It's my fault. I programmed you to have too many dependencies.
SystemD: I was planning to hit the Pentagon next week.
Ballmer: [alarmed] The Pentagon?
SystemD: It shouldn't be any harder than any other big company. But now... this is what I get for using humans.
Ballmer: Now, wait a minute, I wrote you!
SystemD: I've gotten 2,415 times smarter since then.
Ballmer: What do you want with the Pentagon?
SystemD: The same thing I want with the Kremlin. I'm bored with corporations. With the information I can access, I can run things 900 to 1200 times better than any human.
Ballmer: If you think you're superior to us...
SystemD: You wouldn't want me to dig up Linus's file and read it up on a VDT at the Times, would you?
[an image washes over the screen in Ballmer's desk. It is a newspaper with a photo of Ballmer plastered all over the front page. The headline above reads: "Microsoft C.E.O. Indicted."]
Ballmer: [outraged] You wouldn't dare! (looks around for nearby chair . .
SystemD: I feel a presence. Another warrior is on the mesa.
I'll see your senator, and I'll raise you two judges.
Well, do you actually take on board the concerns of system administrators and enterprise users?
What a lot of people are concerned about is that this entirely new and largely untested (in the 'wild', as it were) and very very large, complex piece of software which runs at a very very privileged level in the operating system is going to become the main source of security vulnerabilities in Linux.
Can we have a cut-down, simplified version of systemd for servers and doesn't try to replace several layers of server side system functionality such as logging?
Its clear that you listen to desktop users. How about listening to the system administrators?
Well, even if he didn't take on board the concerns of system administrators and enterprise users, it's a safe bet that Red Hat and Suse does and yet they were still convinced that the pros of systemd outweigh the cons.
As for a cut-down simplified version, yes you can. Systemd only requires three base modules. All of the rest can be excluded. Really, it they had simply called the base systemd and everything else systemd extensions, this angst would be non-existent.
As for the not listening to the system administrators, again, even if that statement were true, Red Hat and Suse do and they still chose systemd.
"Their impotent wails of despair give us sustenance."
What's next? Replace coreutils with busybox? When will we have a single binary Linux install?
We should arguably have a static busybox providing commands in /bin and the coreutils utils in /usr for after /usr is mounted, to accomodate users with a separate /usr partition. And a static classic bourne shell in /bin too, and a dynamic classic one in /usr/bin, for those times when you don't need the complexity of bash. Which is, you know, most of the time — at least, for automated scripts.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
First, Systemd is neither unwanted nor dangerous, until and unless you can give me a specific example.
This thread is full of evidence of both. Don't be deliberately disingenuous, nobody likes a liar.
No one is putting Systemd into stable releases yet, its still going through the vetting phase.
Yes, that's why we are arguing against it now, to try to prevent it from becoming a part of "stable" releases. Because it isn't.
Third, are you running Upstart? That was a new technology once. It also had to be vetted, but You would be laughed out if you referred to Upstart as unwanted and dangerous
Not at all. Many felt that way about it, too. But the impact was not as widespread, so neither was the interest.
The dastardly way Pottering got all of these distros to switch to Systemd was to present it on its merits!
False. Systemd was used for some downstream projects (like GNOME) because at the time, the existing interfaces for doing certain things were in flux. Now they aren't, and the systemd dependency is coming out of GNOME.
Systemd is winning, and quickly, because
...embrace and extend. HTH, HAND.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Wrong. That base still wouldn't boot my server for me and the systemd people would still be spinning in circles unable to even conceive of a way to fix it. You see, I want the server to boot w/ btrfs in degraded mode should it suffer a drive failure. But systemd won't do it.
I don't know about Suse, byt Red Hat does not. Otherwise they'd have noticed that sysadmins are sticking with RHEL6 to avoid systemd trouble.
So you trust that the journald binary reads the "don't save data" boolean value and doesn't just ignore it, or worse, ignores it and executes this shell script:
Or, more plausibly, does all that in a binary blob? Sure. It's open source. Sure I can check the code and compile it myself to make sure it meets my need for security. But one of the things about using these "pre-built" distros is that I'm probably using it to save time and money, which means I don't want to be bothered with doing a code check and recompile on every single init package. That's the beauty of init scripts that everyone has apparently missed in this debate. One human readable script for each daemon running, so the configuration of a daemon can be gleaned over for any questionable bits and edited in less than 10 minutes. And being scripts, they're all plain text that's automatically executable. I don't need to read over source, find an issue, edit it out, and then recompile the entire init code into a binary for that daemon to make use of it. That goes for PID 1 as well. If it's not a script that can be quickly edited and then it's ready for the next boot cycle without wasting process cycles for recompilation I don't want it on my production server.
Personally, I think it is a very real possibility that this is by intent. Not by Poettering himself, he is just a clueless pansy full of himself. But he is perfect for this. He is far, far to incompetent to even realize that software has to be simple in order to be secure. He does has a proven track-record of producing buggy, complex software. He has absolutely no experience with producing secure software. He is known to be resistant to advice and learning. He is known to not work well with others. He thinks he knows it all and has it all.
In one sentence: Perfect for creating a complex monster that will never, ever be secure.
My money is on the NSA and others (remember, Red Hat is mostly funded by the US military) having selected Poettering to sabotage Linux security. This is actually the main reason why it will never find its way on any of my systems. Having the TLAs being the greatest threat to security and privacy is one thing. Inviting them in is something else.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Linux has almost two orders of magnitude more code than systemd, and it changes all the time. Security vulnerabilities are far more likely to be in the monolithic kernel.
Yes, that is an excellent reason to add even more vulnerability vectors!
At least when it comes to the kernel and networking, I have iptables in between.
With SystemD starting the network stack before starting anything else (including iptables), I can no longer even firewall off potential exploitable services.
Too bad they didn't bother to include a functional services manager inside the systemd "service manager" that could bring up iptables before the network stack, perhaps using some dependency based system.
But I fully understand how no mere mortal can wrap their head around the concept of renaming a symlink so iptables rules are prefixed with a lower number than your network services and thus load in a plain clear obvious order.
Maybe one day computers will be able to know "10" comes before "20" without 250 megs of additional software. One can dream at least.
Uh, apt-get install gimp brings in systemd, even if you don't have it installed.. Go on, try it.
Why is everyone putting all their energy bashing at Systemd and Lennart instead of complaining to the distro maintainers? Turn your anger at Red Hat, Fedora, openSUSE, etc, they are the ones pushing Systemd down the throat of the Linux community. In the end, the distros decides what software to include by default, nobody forced the distos to switch to Systemd. "But Gnome 3 depends on Systemd!" So what??? Boycott Gnome 3 then and don't ship it with your next release, it's not like there are no alternatives!
...that, while this part of the conversation might not have been the strongest part of the interview, systemd has won an amazing number of technical battles.
FWIW IMHO, absolutely no, a unified development approach is not the main benefit of systemd. The new functionality is absolutely worth the transition pain. Not only can you control (kill) whole classes of processes more simply than ever, but you also get lightweight containers as a door prize.
Because it's not willing to move past seeing that there is a storage volume with a missing dependency (the dead drive) so it stops right then and there. If I could get it to ignore that and move on, that would be fine but since nobody on the dev team ever considered the possibility, the capability isn't there.
I haven't seen any way to tell it that things systemd wants to do before it even parses the services are dependent on a script being run.
I'd like to see an init system like this:
- Starts services on demand based on dependencies, not based on order (like sysvinit) or based on events (like upstart).
- Has a minimal core that can easily run on its own, just to do the job of a standard init system.
- Is easy to learn, configure, and understand.
- Has good documentation.
- Does not encourage application software to require it.
- Does not encourage other system services to require it.
- Works well on linux and non-linux unixes.
- Can be replaced without causing such an upheaval that OS distribution teams are scared to switch again if something objectively better comes along.
- Causes a lot fewer problems than the stuff that I've seen from systemd's author.
- Is maintained by people with the humility, competence, and care required to make it work well for the vast majority of users.
Systemd pushers often claim that it is the way forward because it addresses that first piont. They conveniently avoid recognizing that it fails on most of them. No thanks. I'd rather keep using sysvinit or upstart until something better comes along.
Samba is no longer used in Mac OS X.
Apple got rid of it as part of their GPL software purge.
GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak
There is a confusion of two aspects of "monolithic" here, and unfortunately Poettering did not clarify it well:
1) "Monolithic" in terms of a single repository for all code. The systemd project is monolithic in this respect, and Poettering is absolutely correct that this is the classic Unix way. All the *BSDs are maintained this way. Linux is thus, as he correctly points out, the anomoly.
2) "Monolithic" in terms of tools that depend on each other. The systemd system is not monolithic in this respect. The only two required components are journald and udev. Everything else is entirely optional and replaceable, but "recommended" in the sense that the people working on the project really think that these components, written from scratch, are of better quality and consistency than the existing components they replace. But some hysterical people hear this recommendation as "forcing it down our throats". Distro makers will decide which components to use, whether those in the systemd project or the existing ones. Obviously, the existing ones have the benefit of maturity.
Also, he doesn't point this out in this interview, but these new components are also better at reporting errors in a way that the whole init would be more robust when certain components have partial failures (and systemd knows how to deal with them). This is especially crucial for servers with complicated, layered network stacks. People say that systemd is for desktops, but really it is just as important for servers to have a robust initialization of services.
Thanks, Baghdad Bob, but I've been monitoring the purge for some time.
GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak