New Year's Resolutions For *nix SysAdmins (cyberciti.biz)

An anonymous reader writes: A new year, with old systems. It is time to break bad old habits and develop good new ones. This list talks about new years resolutions for Linux and Unix sysadmins. List includes turning on 2FA on all services, making peace with systemd, installing free SSL/TLS certificates, avoiding laptops with horrible screens or wireless whitelist in BIOS, building Linux gaming rig and more. What resolutions are on your list regarding sysadmin or IT work in 2016?

Comments

My new years resolution...

[Anonymous Coward]

is 4k baby!

Re:My new years resolution...

[greenfruitsalad]

this is the year ipv6 will take off (again)

Re:My new years resolution...

[Anonymous Coward]

It's goign slowly.
A lot of resources are already on IPV6, its just most people don't know it/care enough.
My Ireland based ISP tried to force IPV6 on me already.(and failed, since i do need my ipv4 address for NAT services that don't accept IPv6)

Re:My new years resolution...

[John Da' Baddest]

Let's see if Perl 6 will beat IPv6 into takeoff mode.

Re:My new years resolution...

[Bert64]

They're not trying to force exclusive ipv6 on you, they're trying to make you go dual stack which you absolutely should.

Re:My new years resolution...

[Hognoxious]

Screw it, let's do a Gillette and just jump to 8.

Re:My new years resolution...

[Opportunist]

And Linux on the Desktop. Let's not forget about that.

Re:My new years resolution...

[Anonymous Coward]

It has been on my desktops for several years now - what's the problem?

Re: My new years resolution...

[Anonymous Coward]

Yeah. When people talk about that they mean you and nobody else.

Re: My new years resolution...

[Anonymous Coward]

Hooray for sudo and centralized syslog!

dnssec

[greenfruitsalad]

maybe i should finally do dnssec. i've been planning to do it for about 5 years.

Re:dnssec

[Lennie]

It's better to do it now than 5 years ago. Because it's easier to so now.

Also for mailservers like Postfix they now support the use of DNSSEC+DANE-TLS-certificates:
http://www.postfix.org/TLS_REA...

This means: encrypted SMTP connections between mailservers and man-in-the-middle is not possible.

Re:dnssec

[alantus]

The big problem is the lack of support from so many TLDs.

Re:dnssec

[AmiMoJo]

I aim to be fully encrypted, and offer secure versions of all services. Even local NTP on my LAN.

That includes texts and phone calls.

Re:dnssec

[Anonymous Coward]

1010 out of 1184 tlds are signed. The stranglers seem to be aero. , travel. , int. , mobi. , pro. , tel. and some 2 letter country codes. So avoid those domains until they get signed.

Re:dnssec

[laffer1]

Is it automated to generate new keys yet? My biggest issue with setting up DNSSEC was remembering to update it before the old keys expire. If I forget on a webserver, I could just install a new cert and go on. If I forget on a domain, no one with a cached entry can access it.

Re:dnssec

[Lennie]

Yes, you can do that.

For example with PowerDNS nameserver supports it:
https://doc.powerdns.com/md/au...

I believe for example Bind and https://www.knot-dns.cz/ also has a system for something similar:
https://www.knot-dns.cz/docs/2...

Maybe: http://ddiguru.com/blog/133-bi...

HTTPS

[Anonymous Coward]

find a way to cache HTTPS !

Re: HTTPS

[Anonymous Coward]

Turn in your IT lifer badge because the solutions have existed for 5+ years and aren't likely to change

Propaganda

[Anonymous Coward]

"making peace with systemd" Might as well make peace with terrorists.

Re:Propaganda

[Z00L00K]

No peace until systemd is dead.

Re:Propaganda

[greenfruitsalad]

i don't know how to mod you "off topic" AND +1, so i won't mod you at all.

Re:Propaganda

[Anonymous Coward]

i don't know how to mod you "off topic" AND +1, so i won't mod you at all.

I didn't even considered what subject I was posting under, maybe that was my mistake. I caught on completely on the claim of "making peace" with "terrorists."

Because really. What are terrorists even? Don't you have to be in war with them for them to kinda be terrorists? If you had a peaceful relationship then how could the be terrorists? So what's the problem?

If you want to have a war with terrorists then you'll of course have war and terrorism, because YOU'RE IN WAR WITH THEM!

Now I'm not suggesting to let them be and fuck over, but US clearly is acting about the same even if they may view their leadership as more legit than the other guys, but the behavior? If USA haven't spend the last decades fucking up the region consistently maybe just maybe it would had been more stable and people would had been more happy with their lives there.

It's my fucking country which get to take in the majority of both actual legit refugees and all other sort of free-loaders and I definitely don't want them here and destroy my own people, culture, history and risk my own safety and freedom for them!! That doesn't mean I really hate them or what not. I'm just fine with Iraqis and Syrians I guess - I just don't want them in my country and being denied having the right to it, that Swedes built it and made it what it was, that we have our own culture and history. I want Swedish schools to teach my culture and history - not to wipe it out!

I would grant the Iraqis and the Syrians and the Yazidis and the Kurds and the Jews and so on the same! Fuck multi-culturalism and cultural-marxism!

Re:Propaganda

[Anonymous Coward]

Nothing is safe unless you write your own machine code. "As long as you can trust the hardware"

Systemd on slashdot

[Anonymous Coward]

Systemd on slashdot? That might go over like a lead balloon from what I've seen herer with some. The idea is sound enough in it and has merits in it though imo. The bad talk around it here is possibly not real. I say that since it's possible that the 'bad press' generated here about it is generated by some fool with a few sockpuppets to make it appear he's many people so much of it that it influences weak minds even by ac that tends to influence herds of them as if he were an expert which some will assume he indeed is since he posts here and they want to be part of the crowd and the winning team yet they never back their bad press with anything factual when they do it. I've seen this technique before and yes I know this goes on. I used to work for a marketing firm that specialized in such heinous garbage. Anything to make a buck in other words. Of course it could also be coming from sysadmins that don't want to give up their custom scripts they worked on that yes do work for them (buggy or not) and are loathe to change to something that may be better. Anything to be on the 'winning team' and their team is what they use. Not what's really better.

Re:Systemd on slashdot

[wierd_w]

This is a problem with "old vs new".

sys V init is old. So are the old, genuine unix wizards.

SystemD is new. So is Pottering and Pals.

The divide comes from "old culture" vs "new culture." The old unix culture adores simplicity, sparseness, and adaptability. The new culture adores easiness, one-stop shopping, and cohesive wholes.

This argument will never really die. The old culture will point out the endless littany of security problems with software like systemd, simply because of the complexity and scope-creep of the project. (this increases its attack surface, and makes it attractive to malware authors and hackers.) The new culture will point out the endless littany of hair pulling and hours spent dealing with obtuse scripts and strange scripting behaviors for things they feel should be solvable with a mouseclick.

These two will never live under the same roof. Both are right, and both are wrong. There are systems and applications where sysVinit makes sense, and is desirable. There are systems and applications where systemd makes sense, and is desirable.

The real sin here is not heresey by either group.

The real sin is the decision of the foss community to pick a side, and in so doing, remove that choice from other people, by choosing to make systemd a hard requirement, solely for their own convenience.

Re:Systemd on slashdot

[Anonymous Coward]

The main arguments against systemd are that it's monolithic, rapidly becoming non-optional and was quickly shoved into every distro that matters.

The fact that some of its components are poorly designed and implemented is just icing on the shitty systemd cake.

Re:Systemd on slashdot

[dwywit]

I'd mod you up if I hadn't used all my points.

Re:Systemd on slashdot

[Princeofcups]

This is a problem with "old vs new".

sys V init is old. So are the old, genuine unix wizards.

SystemD is new. So is Pottering and Pals.

The divide comes from "old culture" vs "new culture." The old unix culture adores simplicity, sparseness, and adaptability.

The "old culture" knows that a server is just part of a bigger process, and reliability and maintainability are more important than simplicity, sparseness (whatever that means in this context), and adaptability. Without something like systemd, Linux cannot be enterprise ready. "Rolling your own" scripts for failover and redundancy is the worst idea when more than one admin has to diagnose problems at 2:00 AM. You want something supported by the vender, with standard configuration options, that can be easily understood by everyone on the team. Sometimes the "most elegant" solution is not the best for the business.

Re:Systemd on slashdot

[kthreadd]

The main arguments against systemd are that it's monolithic

Here is the output of ps -ef | grep systemd on a Ubuntu 14.04 desktop which uses upstart and not systemd as its init system:

root 387 1 0 2015 ? 00:00:00 /lib/systemd/systemd-udevd --daemon
root 701 1 0 2015 ? 00:00:00 /lib/systemd/systemd-logind

It's clearly not completely monolithic if you can run two fairly central components without even the init daemon present.

rapidly becoming non-optional

Yes, in some distros. It's entirely up to the distro maintainers what options they want to support. That they choose to focus on supporting systemd is hardly systemd's fault.

was quickly shoved into every distro that matters.

Systemd was included in Fedora 15 which was released almost five years ago and we still have distros that are going to switch. This is one of the slowest tech migrations ever to take place in the GNU/Linux community.

Re:Systemd on slashdot

[AmiMoJo]

I don't buy that init scripts are simple or even particularly adaptable. I've hand crafted a few in my time and they are always a pain in the arse to manage. When you get to the point of having multiple scripts bring called which then call sub scripts and all of them somewhat unique to that machine... It's time to look for something better.

I have not played with systemd extensively, but it seems to have the right idea. While in theory scripts give you more control, in practice they just make it harder to admin the system and make you waste time hacking them. Having it all tied together and controllable from one place is much better.

I used to have a highly customised Firefox install. Eventually Firefox sucked enough to drive me to Chrome. At that point I realised it was better to just adapt my workflow to something better some times, instead of proclaiming it sucks because it doesn't do things the way I did a decade ago.

Re:Systemd on slashdot

[Anonymous Coward]

> The bad talk around it here is possibly not real. I say that since it's possible that the 'bad press' generated here about it is generated by some fool with a few sockpuppets to make it appear he's many people so much of it that it influences weak minds even by ac that tends to influence herds of them as if he were an expert which some will assume he indeed is since he posts here and they want to be part of the crowd and the winning team yet they never back their bad press with anything factual when they do it

Was that supposed to be coherent or 5 sentences your drug-fueled brain melted together? "The bad talk"? Seriously, morons like you are part of the problem.

Re:Systemd on slashdot

[Anonymous Coward]

> It's clearly not completely monolithic if you can run two fairly central components without even the init daemon present.

That's because it's not related to init. Init is one subsystem. systemd, being a process manager, doesn't care if init is included or even when it is started. Once it's started, the communication between the parts are within the systemd communication channels (increasingly). This is what makes it monolithic. SMH

> This is one of the slowest tech migrations ever to take place in the GNU/Linux community.

Probably because it's bad.

Re:Systemd on slashdot

[kthreadd]

That's because it's not related to init. Init is one subsystem. systemd, being a process manager, doesn't care if init is included or even when it is started. Once it's started, the communication between the parts are within the systemd communication channels (increasingly). This is what makes it monolithic. SMH

Please correct me if I'm wrong here but what process manager are you talking about? Pid 1 on Ubuntu 14.04 is upstart and it owns both of those two services. They are as far as I can see running completely independent.

Re:Systemd on slashdot

[Z00L00K]

If just systemd really was easiness, it's definitely just like fishing in muddy water.

Re:Systemd on slashdot

[Anonymous Coward]

> This is one of the slowest tech migrations ever to take place in the GNU/Linux community.

Probably because it's bad.

That makes no sense. If it was bad then it wouldn't just be slow. It would not be adopted at all.

Re:Systemd on slashdot

[Kjella]

The real sin is the decision of the foss community to pick a side, and in so doing, remove that choice from other people, by choosing to make systemd a hard requirement, solely for their own convenience.

Did you just try to make writing software to scratch your own itch sound like a bad thing? If Poettering wants to write systemd-only software, nobody's going to stop him. Nobody should be able to stop him. Nobody's going to force him to create or maintain SysV init scripts either. And if other projects find that depending on systemd is convenient, so can they. Open source is not a democracy. Developers do what developers want, regardless of what their users think and by users I mean everybody from other projects to distro packagers to system administrators to end users. The conflict was lost when systemd was voted in as the default, trying to bend the Debian system to force developers to preserve the old ways was a fool's errand. If it had passed, all that would have happened is that Debian would have lost them. Nobody can make that kind of committee decision stick.

Re:Systemd on slashdot

[drinkypoo]

I don't buy that init scripts are simple or even particularly adaptable. I've hand crafted a few in my time and they are always a pain in the arse to manage.

If a daemon needs a complex init script to manage it, then systemd won't be able to manage it at all, because it can't do everything a script can do. If systemd can manage a daemon, then it didn't need a complex init script, and you did it wrong.

When you get to the point of having multiple scripts bring called which then call sub scripts and all of them somewhat unique to that machine... It's time to look for something better.

If you need that much complexity to run a daemon, then either your daemon sucks or you do. Unless, of course, you're just talking about simple things like script libraries. If those confuse you, perhaps Unix is not for you. Shell scripting is a core Unix feature.

If you could explain cogently what you meant, perhaps you would get different responses. But so far, you're talking bananas. Systemd's unit files are merely a handful of name-value pairs, they don't even need sections because all of the names are unique (even between sections) but they put sections into them anyway because systemd is all about obfuscation, and needless effort. Systemd's "additional functionality" (unit files, cgroups support) could be replaced by a relatively small shell script, based on existing init scripts. Indeed, any distribution which uses script libraries in its init scripts could have done this cheaply. For example, cgroups are a kernel feature manipulated via short, common commands, like creating directories.

I used to have a highly customised Firefox install. Eventually Firefox sucked enough to drive me to Chrome.

I used to use Chrome for Google services. Eventually Google fucked Chrome hard enough to break Youtube ad-blockers. Then I realized that people who use Chrome are Part Of The Problem.

Re:Systemd on slashdot

[gweihir]

Nice propaganda piece, full of lies and slander. My guess is you still work for that marketing company and got paid to write that.

Re:Systemd on slashdot

[gweihir]

The real sin here is not heresey by either group.

The real sin is the decision of the foss community to pick a side, and in so doing, remove that choice from other people, by choosing to make systemd a hard requirement, solely for their own convenience.

I completely agree. If I could just easily ignore systemd, because whenever I install a new Linux machine, I get offered both alternatives by default, I would not care at all. This way, I have the thing forced on me (or nearly so), which is just not acceptable.

Re:Systemd on slashdot

[Z00L00K]

Wrong - it's with systemd that Linux will have a very hard time to be enterprise ready.

Re:Systemd on slashdot

[drinkypoo]

The "old culture" knows that a server is just part of a bigger process, and reliability and maintainability are more important than simplicity,

That is why Systemd is fail. Shell scripts have reliability and maintainability. Systemd complicates troubleshooting of the boot process, and requires that we trust code written by a known lame. Remember pulseaudio? I sure do. I spent hours fighting with it.

Without something like systemd, Linux cannot be enterprise ready.

Why?

"Rolling your own" scripts for failover and redundancy is the worst idea when more than one admin has to diagnose problems at 2:00 AM.

Only if your goal is to hire sysadmins too stupid to comprehend shell scripts. The problem is that people that dumb are also too stupid to troubleshoot problems. Try hiring some of the many out-of-work qualified sysadmins who actually know Unix, instead of an H1-B or some kid just out of school.

You want something supported by the vender, with standard configuration options, that can be easily understood by everyone on the team.

And that is how vendors have been using Unix since forever. With shell scripts, supported by the vendor, with standard control flow, that can be easily understood by anyone intelligent enough to be a systems administrator to begin with. Your complaint is that the point and click Microsoft-or-Apple-fanboy-wannabe-sysadmin cannot point and grunt his way through poorly implementing Unix configuration. No one who knows what they are doing will have sympathy for that idea, because there are so very many qualified professionals going unemployed right now. It's not hard to get people who know what they are doing, if you try to hire them. You don't want to do that. You want to hire idiots and you want magically good results. But someone incapable of systems administration without systemd will simply fall flat on their face when a problem crops up, and because they're using systemd it will be harder to troubleshoot the problem.

TL;DR: A monkey who can't understand a shell script can't be a real sysadmin anyway. All they can be is a button puncher. They should work in a NOC, and you should hire someone who knows Unix to administer Unix.

Re:Systemd on slashdot

[gweihir]

Completely bogus argument. Linux has been "enterprise ready" 10 years ago. You probably also think Solaris was not "enterprise ready" before it got SMF, (the
"systemd" of Solaris) yet most enterprise sysadmins hate its guts and Solaris did run a lot of mission-critical things long, long before SMF ever made an appearance.

Incidentally, in actual enterprise-setups, you do fail-over via network elements, not on the machine itself. But I guess you do not know that because you actually have no clue what you are talking about.

Re:Systemd on slashdot

[Anonymous Coward]

Structural and semantic deficiencies in the systemd architecture for real-world service management, a technical treatise

Re:Systemd on slashdot

[wierd_w]

Uh huh.

And how did that work out for Gnome3, Ubuntu's choice of Unity, or MS's choice of Metro?

Oh, right. It resulted in a user revolt.

It's a two-way street, even if you don't like that fact. Developers become irrelevent without user satisfaction.

Re:Systemd on slashdot

[Anonymous Coward]

Except of course, that the old strategy doesn't work without constantly having sysadmins grooming and spoiling their pets (servers), something they seldom do other than as an afterthought after a long and hard failure. Heck, at some companies I've been to, nobody can reboot any servers without all the admins being alert and starting services manually.

If we are to compete with cloud solutions, we need to treat servers and services like cattle. Slaughter the ones that have problems, diagnose later and bring up new ones to replace bad parts of the whole system. This requires something like systemd on the infrastructure/platform side, and then it makes sense to have similar strategies on each server also - to support hot-swappable parts and a rapidly changing platform. To do this manually with scripts is really not feasible, as it doesn't work in practice and you're often dealing with race conditions that always can bring new surprises. The next OS update might change everything again as well, and does not support your assumptions, prayers and hopes.

What we need to support is private cloud solutions, or we will be out-competed by more efficient public cloud solutions. For this to happen, some of the grumpy old grey beards need to retire. You have planned for that, right? No, I think most of the old-timers have planned for nothing but their own benefits, and deserve absolutely no sympathy in this regard. Fuck you, for making the workplace a miserable place to be at!

Captcha: nestling

Re:Systemd on slashdot

[thegarbz]

If a daemon needs a complex init script to manage it,

Define complex. 99% of init scripts are handling of the state of the daemon, PID files, configuration etc. Or to put it another way: Every daemon on my system can be started with a single line in the console, why do I need a script to manage it at all?

You are right. Init scripts don't need to be complex. Yet on every system they are still longer than 1 line, actually they are typically longer than 100 lines. I've written systemd scripts (or whatever they are called). I've written upstart ones. I've copied and pasted and modified a sysv init script and would have probably cut myself if I were forced into a position to write one from scratch.

Eventually Google fucked Chrome hard enough to break Youtube ad-blockers.

That is news to me. I haven't seen a youtube ad on my computer in many years.

Re:Systemd on slashdot

[thegarbz]

This has been this way for a long time. Redhat maintainers actually wrote a long post about "choice" a decade ago. Their efforts for maintaining a distribution go up exponentially far more than simply maintaining 2 packages with 2 startup scripts if they give the user choice. Their summary was you have choice as far as what distribution you use. Don't like what is being provided, then fork or move to another distribution.

Quite frankly given the poor state of many distributions with regards to unresolved bugs and very slow moving adoptions of features I fully support them not wasting time on trying to please everyone.

Re:Systemd on slashdot

[Anonymous Coward]

Your logical falicy is: black-or-white

https://yourlogicalfallacyis.c...

And some old use neither sysv or systemd. They use something newer and less broken.

They don't see systemd and its advocates as a new culture but as exactly the corporate driven nightmare they escaped from a couple of decades ago that spawned the Free software revolution.

Further, they don't see both as 'right',

They see one as a small component that can (and has been by many 'old culture') be replaced by other components in a staged and robust fashion.

They see the other as not that. That systemd, pottering and pals are just wrong and wrong minded.

Re:Systemd on slashdot

[wierd_w]

Keeping track of the PID is important if you want to kill the process reliably when it hangs up. (because invoking the executable to tell it to down wont work reliably with a hung process.)

That is, unless invoking ./etc/init.d/foo shutdown is harder for you than doing
ps -A
to find the process ID then doing
sudo kill FooID

I suppose you might feel it safe to use
killall foo

but what if you have multiple instances of the daemon running? (say, different ssh server daemons on different ports)

Taking the time and enduring a little pain so that you can do ./etc/init.d/foo#x shutdown

to shutdown instance x of the daemon saves lots of time and effort later.

Re:Systemd on slashdot

[thegarbz]

I agree. I just disagree that you should replicate a 100 line script for every bloody instance and every daemon over and over again when that should be handled by the parent part of the system.

Re:Systemd on slashdot

[Anonymous Coward]

That's why systemd keeps track of the pid, even pids of multiple processes when using cgroups so that you can reliably know that every child process has been killed as well.

Re:Systemd on slashdot

[drinkypoo]

What we need to support is private cloud solutions,

No, junior. That's called a cluster. Cloud computing is when you spin up instances as you need them, and get billed appropriately. We don't need systemd for clusters. They predate it dramatically. And we're already using virtualization, which also doesn't require systemd. If there's anything else old that's new again that you would like explained for you using only short words, let me know. I'm here for you.

Re:Systemd on slashdot

[AmiMoJo]

If a daemon needs a complex init script to manage it, then systemd won't be able to manage it at all,

I see...

If you need that much complexity to run a daemon, then either your daemon sucks or you do.

So what you are saying is that some daemons suck and are better avoided anyway. Okay.

Eventually Google fucked Chrome hard enough to break Youtube ad-blockers.

Strange, they work fine for me. There is an issue if you install the YouTube app because, for security reasons, extensions can't screw with apps. Just uninstall the YouTube app and it will go back to loading as fast as in other browsers, but your ad-blocker will work.

Re:Systemd on slashdot

[Bert64]

I like many people, have used init scripts for years and very rarely have i had to customise any of them or write my own, and almost always use the standard init scripts that come with the package.
However i do understand shell scripting, so i can see what the init script is trying to do and go through it manually if i need to debug something or if i need to run something in a non standard way (for instance recently i had a hardware failure, so we mounted the drive in another server and brought its services online temporarily inside a chroot).

I have a recurring problem on OSX where a service checks a pid file, but if that service failed to shut down correctly before a reboot the pid file is still there and if something else has started on the same pid the service won't start... This was a pain to debug with the launchd system on osx, but would have been easy with init scripts.

Re:Systemd on slashdot

[Anonymous Coward]

The world keeps moving forward. The enterprise world looked different ten years ago. Back then you treated servers like pets. Now everything is about cloud and servers are seen more as cattle and that changes everything. Sysadmins used to keep the same installation up for years, even rebooting only when switching out the hardware. That era is going away. Bring up a new server instead, or 10 000 new ones. Rebuild your entire infrastructure every single day. Why install updates when you can just create new servers instead. Keeps the systems reproducible. That's where systemd fits it, because it handles all of this. Sysvinit made assumptions about what the world looked like, where it could write files and not. It does not lean well to making atomic systems, which is where the enterprise is going. That's why we need systemd.

Re:Systemd on slashdot

[Anonymous Coward]

That's why systemd doesn't use "PID files". It's a broken concept.

Re:Systemd on slashdot

[dotancohen]

Systemd on slashdot? That might go over like a lead balloon from what I've seen herer with some.

Surely you mean a lead (spelled led) zeppelin?

The article is rather insightful and one of the few that are worth reading. I disagree with a few points (2FA for ssh on his personal workstation?!?) but there are some good points in there:
* Make peace with systemd
* Learn the new tools: ip, ss, iw (not on his list, but on mine)
* Learn ZFS (on my list for half a decade)
* Use tools such as Lets Encrypt on all websites (good idea in theory, in practice not feasible due to 3 month certs)
* Learn Android
* Stop doing IT consulting for cheap businesses. Unfortunately, this is actually where the bulk of my bread comes from!

Re:Systemd on slashdot

[Hognoxious]

When I read that, I hear the voice from the "mongo db is webscale" cartoons.

Re:Systemd on slashdot

[Anonymous Coward]

And if other projects find that depending on systemd is convenient, so can they. Open source is not a democracy. Developers do what developers want, regardless of what their users think and by users I mean everybody from other projects to distro packagers to system administrators to end users.

You forget that most projects have received much contribution by many people, including non-developers. Many projects are community projects, even officially claiming they are. They're not one or a few developer's pet project. Sure you can fork them, but if you have to fork everything, you won't get much work done. Community project leaders are supposed to be responsible to the community at large.

This includes not forcing systemd down user's throats, both for the base installation, and for the various programs which were changed to stupidly depend on systemd alone.

Sure some of these people were in their right to exercise their freedom. But they were still arsehole moves.

Re:Systemd on slashdot

[Hognoxious]

exponentially far more

Not just exponentially, but far more exponentially?

Since I'm obviously not as clever at maths as you are, can you tell me the equation for that?

Re:Systemd on slashdot

[DeHackEd]

I've been using unix (ie. only Linux but we'll pretend that counts) for over 15 years now. Not quite the "old" you may think of but old enough.

I gave systemd a try. It actively fought me and I cannot accept that. It has too much of a "my way or the highway" mentality that you just can't fix without major C hacking and recompiling. If you don't like its way of doing things then too bad.

sysvinit scripts may be slower to boot and have fewer automatic/behinds-the-scenes features you want, but I can make any arbitrary change to them with minimal effort. I can run them with line-by-line tracing using "set -x" and find out exactly why it's hanging. I can rescue it with *any* install media even if it doesn't have systemd and '/etc/init.d/servicename start' will actually work.

systemd is fine for desktops run by people who think Firefox is the only app they really need to Surf The Net. sysvinit is designed for people who want control of their systems and want to be able to inspect what it's doing. And I'm sorry, I NEED the latter to do my job properly.

Re:Systemd on slashdot

[thegarbz]

You want equations look on the RedHat mailing list. The example they cited involved maintaining 2 programs in the distribution, doubling the testing cases for some 50 other programs that now rely on not one but two dependencies, and writing and maintaining a program to allow the user to switch between the two cases.

You can crap on my pre-morning-coffee post all you want, but that doesn't change the facts. The maintainers of a distribution themselves have come out and said WHY they don't provide more choice to the user. Don't complain to me, complain to them.

Re:Systemd on slashdot

[kthreadd]

* Use tools such as Lets Encrypt on all websites (good idea in theory, in practice not feasible due to 3 month certs)

Take it as a good incentive to learn automation.

Re:Systemd on slashdot

[Anonymous Coward]

Problem: linux had been successful in enterprise settings long before systemd existed. Something that one with your UID should know pretty well.

Re:Systemd on slashdot

[Anonymous Coward]

The world keeps moving forward. The enterprise world looked different ten years ago. Back then you treated servers like pets. Now everything is about cloud and servers are seen more as cattle and that changes everything.

Not for most of the servers I take care of. Most are of the use cases we deal with our servers have to be pets.

Certainly our HPC cluster compute nodes are cattle, but for a lot of the other stuff (MySQL, Postgres, Jenkins, Git, Subversion, JIRA, Confluence, etc.) they're one-offs.

Re:Systemd on slashdot

[Anonymous Coward]

One-off servers works really well until they fail, or until the load goes up just enough that the server is no longer enough to power it. Every service benefits from being able to scale up. There are so many cases where being able to automatically spin up even just a second server is valuable. Sure some applications will have to be redesigned to work this way, but it's worth it and in the long run will be inevitable.

Re:Systemd on slashdot

[Anonymous Coward]

Except of course, that the old strategy doesn't work without constantly having sysadmins grooming and spoiling their pets (servers), something they seldom do other than as an afterthought after a long and hard failure.

Wrong - good sysadmins don't fuck with anything that's working. Bad sysadmins are bad sysadmins, and systemd isn't a solution for that. It's hard to argue that the complexity of systemd helps.

Heck, at some companies I've been to, nobody can reboot any servers without all the admins being alert and starting services manually.

...

So they hired crappy sysadmins.

And crappy people do crappy work.

How can systemd fix that? How does deliberatly violating KISS help anything?

Re:Systemd on slashdot

[EmeraldBot]

The real sin is the decision of the foss community to pick a side, and in so doing, remove that choice from other people, by choosing to make systemd a hard requirement, solely for their own convenience.

Did you just try to make writing software to scratch your own itch sound like a bad thing? If Poettering wants to write systemd-only software, nobody's going to stop him. Nobody should be able to stop him. Nobody's going to force him to create or maintain SysV init scripts either. And if other projects find that depending on systemd is convenient, so can they. Open source is not a democracy. Developers do what developers want, regardless of what their users think and by users I mean everybody from other projects to distro packagers to system administrators to end users. The conflict was lost when systemd was voted in as the default, trying to bend the Debian system to force developers to preserve the old ways was a fool's errand. If it had passed, all that would have happened is that Debian would have lost them. Nobody can make that kind of committee decision stick.

I am a little suprised that someone with your UUID would have such a philosophy about the Debian project, but fair enough. I'd like to refer you to the Debian Manifesto, a document that was written when the Debian project was first founded. In particular, I'd like to refer you to section A.3:

thus, a distribution is created based on the needs and wants of the users rather than the needs and wants of the constructor.

, with "constructor" referring to the creator.

To be very clear, I am not advocating that Poettering shouldn't be programming what he pleases, nor that all work should be productive. However, I am saying that at least for the Debian project, the focus (used to be) on the users, and making a solid operating system useful to everyone. There's been a large resurgence in the "developers first" attitude the last couple years, and while it lends itself well to hobbyists, these kinds of people should not be working on a large and collaborative project with the goal of being useful to everyone - because, as you've made very clear, the center of that philosophy is all about your wants and needs, something which is directly contrary to the main goal of the Debian Project (and really, any large open source project whose goal is to be useful to more than the developers).

Re:Systemd on slashdot

[dotancohen]

Take it as a good incentive to learn automation.

You are 100% correct, and I will in fact address it that way!

Re:Systemd on slashdot

[nine-times]

Without something like systemd, Linux cannot be enterprise ready.

Why?

"Rolling your own" scripts for failover and redundancy is the worst idea when more than one admin has to diagnose problems at 2:00 AM.

Only if your goal is to hire sysadmins too stupid to comprehend shell scripts.

And there you've just answered your own question as to "why?" Not that their goal is to hire stupid sysadmins, but their goal is, to some extent, to avoid relying on people being particularly competent.

I see this as part of the disconnect between techies and business/manager types, and you're only illustrating it. For those who are pretty good at the tech stuff, they think, "That's ok, I'll just write a custom script, and if something breaks I'll figure it out." Meanwhile, the business guy has been burned by that too many times. For the handful of competent techs he's hired, he's also hired a couple incompetent guys, as well as some who were fairly competent but not as smart as they thought they were. He's seen a tech write that custom script, he's seen it work well enough for a while, and then seen it turn into a disaster as soon as someone else had to touch it.

It's hard to run a business relying on everyone to be a bunch of super-hero super-geniuses. Businesses want a solution that's drop-dead simple. They want things to be used in the way they're supposed to, and to be fully supported. They don't like customization unless it's something that was designed to be customizable, and supported in that customization. They want it all to be documented and clear, and they don't like relying on people to be good at their jobs, because no matter how hard you try to hire good people and how much you're willing to pay them, most people turn out not to be particularly good at their jobs.

I'm not saying these are unbreakable rules, but there is a sort of tendency. The point is, larger and established businesses don't really want a clever hack that saves them a few thousands of dollars at the risk of a failure that would cost millions. Whether systemd is relevant to that issue-- I don't want to get into that discussion. But you're asking why you need something "the point and click Microsoft-or-Apple-fanboy-wannabe-sysadmin" can figure out in order to be considered by some people to be "Enterprise ready", then that's why.

Re:Systemd on slashdot

[Anonymous Coward]

Problem: linux had been successful in enterprise settings long before systemd existed. Something that one with your UID should know pretty well.

and as Linux + systemd reliability drops, enterprise will move to more reliable *nix systems where reliabilty is more important than "easiness"

Re:Systemd on slashdot

[MrKaos]

I don't buy that init scripts are simple or even particularly adaptable.

Hang on a minute - are you talking about /etc/init.d scripts that get linked to a runlevel in /etc/rc or a script called from /etc/inittab managed by init? - because they are different.

Using /etc/inittab it's pretty easy to make something a system managed service using init this way, to be either respawned, run once, on or off, from a single file.

Customizing rc level 4 is also an option, for example to make the system boot in a more specific way for a media centre client or something more specific. There is no reason why you can't mangle runlevel 4 to make it do what you want and then save runlevel 3 and 5 for more general purpose modes. Following thew media centre example to make an operational rc4 and maintenance mode rc3 or rc5 which you might configure as a boot option.

I think the reason people think it is difficult is because of context. The rc script in /etc/init.d should exit, however the script called from /etc/inittab should not, so that init can act appropriately to manage the process. Scripts in /etc/init.d should only run once at start up or when called manually.

However you can make init maintain a process easily with a line like:

mc:4:respawn:/etc/runMediaCentre If the process is killed, init respawns it. I think people think it's some sort of secret sauce, however it is just a file and a line is similar in complexity to a unit file.

I've hand crafted a few in my time and they are always a pain in the arse to manage.

Yes, they can be however I found that was because of my thinking. I just remember scripts called from inittab don't exit.

When you get to the point of having multiple scripts bring called which then call sub scripts and all of them somewhat unique to that machine... It's time to look for something better.

It might help to think of the design pattern "Strategy". The runlevel scripts linked from /etc/init.d to /etc/rc.d set the context of the machine (like making vpn, db or other things avilable) and the entries in /etc/inittab define the aplication specific strategy that is dependant on that context.

Because init knows what the runlevel is, the strategies are only ever run in context. If there is something wrong with the context, then you have a problem (like no db or vpn).

I run systemd on desktops and test systems - mainly to see what the fuss is about and ensuring I'm prepared if I need to be. I just haven't seen a compelling argument for it. I ask why we are replacing something elegant and powerful.

If you are arguing for systemd, have you ever tried using inittab or customizing runlevels? It is powerful and demanding, but not impossible. As a core *ix paradigm it forces you to skill up to the next level of mastery over your systems and understand how your processes behave.

systemd seems to abstract away that knowledge of how processes behave, like wishing for less challenges instead of more skills.

Re:Systemd on slashdot

[MrKaos]

I agree. I just disagree that you should replicate a 100 line script for every bloody instance and every daemon over and over again when that should be handled by the parent part of the system.

No, you should have one script and 100 lines in /etc/inittab. Each line should have different arguments, ideally pointing to a configuration file for that particular instance of your daemon, i.e you have 100 differently configured instances of the same deamon. Then init manages them for you.

Re: Systemd on slashdot

[Anonymous Coward]

Not old vs new at all.
It is smart vs dumb and outright idiots.
In case you didn't get the memo, systemd folks are the idiots with Poettering being their lead senior idiot in residence.

Re:Systemd on slashdot

[MrKaos]

That's why systemd doesn't use "PID files". It's a broken concept.

You don't need a PID file if you use init properly.

Re:Systemd on slashdot

[MrKaos]

* Learn the new tools: ip, ss, iw (not on his list, but on mine)

ip and ss are massive improvements over netstat, well worth learning. These tools are certainly more evolved than netstat

Re: Systemd on slashdot

[Anonymous Coward]

Thank you.

Reading this I realized that syatemd was born out of a misunderstanding in the first place. Or rather inability to understand. It looks like Poettering boy ran into some sort of issue with init scripts at one miserable point of time in his meager existance then decided that he was too stupid to understand runlevels and all that. So he decided to "heal the world" once and for all, not even understanding what the world is about nor what it's actual problem is. Ever since the world has a problem aka systemd.

Re:Systemd on slashdot

[Anonymous Coward]

because...

Re: Systemd on slashdot

[kthreadd]

How can you become someone like him? Just make things up and force distros to use it...

Wait a minute, that's not really what happened didn't it?

Re: Systemd on slashdot

[Anonymous Coward]

If a system breaks because one dependency is replaced for another same-function dependency, that system was broken long before, and by design. Properly designed systems rely on layers and interface specifications being met, not on implementation. Thus a properly designed system does not need new or changed test scripts if a dependency changes. Zero increase in complexity.

I don't know the specific arguments by Red Hat admins you cite, but I doubt they meantsupporting mutiple init systems is a problem. If they did they already drank the cool aid of systemd which indeed makes it hard for any other initsystem in its place. However that is not because multiple init systems are hard that is because systemd is poorly designed to the point where it has deliberately thrown out the time proofen Nix* philosophy, do one thing only and do it good. If that's what Red Hat folks were talking about I can see where they are coming from but I can also see that they are on the wrong path.

Re:Systemd on slashdot

[i.r.id10t]

Same here. My biggest worry was about binary logging - I often will grep thru various logs looking for stuff, even if to just show students what is actually going on when something is happening.

But... my Debian 8 setup, with systemd, still has all the log files I'm used to, in the same places they've always been, with the same information in them. So I don't know if systemd is "double logging" or if Debian set this up to keep us old farts happy.

Re:Systemd on slashdot

[dbIII]

but their goal is, to some extent, to avoid relying on people being particularly competent.

Unfortunately that includes the people writing systemd.
A binary log that is written to by several things at once if an example of people who have either never heard of a race condition or do not care if the log gets corrupted for example - utter newbie mistake. A computer that won't boot because systemd can't work out what to do with a usb mouse dongle is one of the many examples of how immature the system is despite a decade of work - it's parallel when it's insane to be so and serial when it should be able to do several things at once instead of getting hung on what to do with a USB device when it's really none of it's business and it should let the kernel do that job instead.
Their goal is really just to bring a lot of separate things under the control of a single small group that does not have the patience to solve one problem before they expand to replace another bit of software.

Re:Systemd on slashdot

[Anonymous Coward]

If we are to compete with cloud solutions, we need to treat servers and services like cattle. Slaughter the ones that have problems, diagnose later and bring up new ones to replace bad parts of the whole system.

Here's a guy who gets it.

This requires something like systemd on the infrastructure/platform side

I'd argue that. systemd is pointless in this context. If your server is inflicted with mad cow, you don't restart its left eyeball and hope for the best. You shoot it in the head and burn the corpse.

Virtualization, containerization and configuration management technologies are far, far, far more important than whatever init system is running on your disposable nodes.

Re:Systemd on slashdot

[Anonymous Coward]

SystemD's outbursts are completely disgusting—so much so, that if there are any children or sensitive people reading this letter, I suggest that they stop now and not read what I am about to describe. Please note that many of the conclusions I'm about to draw are based on cogent and virtually incontrovertible evidence provided by a set of people who have suffered immensely on account of SystemD. I welcome SystemD's comments. However, SystemD needs to realize that its press releases are based on a denial of reality, on the substitution of a deliberately falsified picture of the world in place of reality. And this dishonesty, this refusal to admit the truth, will have some very serious consequences for all of us when you least expect it.

When I was growing up, we were taught that one should always try to strengthen our roots so we can weather the storms that threaten our foundation. Nowadays, it seems that more and more kids are being taught that defeatism is the key to world peace. You can thank SystemD for this disaffected pedagogical viewpoint, especially given that it appears to have found a new tool to use to help it muddy the word “succinylsulphathiazole”. That tool is academicism, and if you watch it wield it you'll indeed see why in some sense, its twisted dream of implementing a callow parody of justice called “SystemD-ism” has triumphed. Of course, this would better be called a nightmare, not a dream. In point of contrast, I'm one of those people who dreams about embracing the cause of self-determination and recognizing the leading role and clearer understanding of those people for whom the quintessential struggle is an encompassing liberation movement against the totality of tribalism. That's why I write that by writing this letter, I am indisputably sticking my head far above the parapet. The big danger is that SystemD will retaliate against me. It'll most likely try to force me to become increasingly frustrated, humiliated and angry although another possibility is that it managed to convince a bunch of the most incorrigible loons I've ever seen to help it create a factitious demand for its malefic “compromises”. What was the quid pro quo there? I have searched numerous sources for answers to that question. No two sources seem to agree on any given point except for one, that it's sad that SystemD's most full-throated claim is that it possesses infinite wisdom. One would think it could strive for a little more accuracy there. It could perhaps even admit that it twists every argument into some sort of “struggle” between two parties. SystemD unvaryingly constitutes the underdog party, which is what it claims gives it the right to place patronizing scofflaws at the head of a nationwide kakistocracy.

SystemD's protests stem from a combination of cognitive bias and inaccurate information, and everyone with half a brain understands that. If the past is any indication of the future, SystemD will once again attempt to lock all the exits from our present state to the world of constructive reason. If the human race is to survive on this planet, we will have to uplift individuals and communities on a global scale to protect our peace, privacy, and safety. It's somewhat tricky to help people see SystemD's drugged-out fairy tales for what they are, especially since the media in this country tend to ignore historical connections and are reluctant to analyze ideological positions or treat a fringe political group seriously. I like to challenge people to help you reflect and reexamine your views on SystemD. I realize that that's a tall order, but I realize that the tone of this letter may be making some people feel uneasy. However, even if you're somewhat uncomfortable reading about SystemD's oppressive projects, please don't blame me for them. I'm not the one using both overt and covert deceptions to put a clog on all attempts to limit its power. I'm not the one building a totalitarian death machine. And I'm not the one scorning and abjuring reason. As a fina

Re:Systemd on slashdot

[menkhaura]

My kingdom for a mod point :) Bravo, Mr. poo! Good way to start a year.

Re:Systemd on slashdot

[phantomfive]

It's insulting to call it "old" or "new" (it's also a mindless stereotype). Plenty of people can see the benefits of systemd, at the same time seeing the drawbacks of systemd.

For my own part, I've been working on a code review of systemd. I spent a lot of time trying to understand why it was getting adoption. I've delved into the code and looked at the interfaces.

I suggest reading my reviews, but the end result is this: systemd is a good attempt to solve problems that people have, but it isn't a very good solution.

Re:Systemd on slashdot

[drinkypoo]

I agree. I just disagree that you should replicate a 100 line script for every bloody instance and every daemon over and over again when that should be handled by the parent part of the system.

Systemd's daemon management system, which is based around unit files, is dedicated to the idea that you don't need a multitude of init scripts. And in fact, it's broadly true; most programs fall into one of a very small set of cases. But solving this problem doesn't require a new init daemon. All it requires is a new init script which parses unit files, or something like them. The files get a hashbang at the top which calls the script, which reads the unit file and does whatever needs to be done using the usual init script library used by the distribution, if any. You could, in fact, do this with unit files themselves, and if I ever download something that doesn't have an init script but does have a unit file and I don't feel like bashing out an init script based on some other init script, maybe I'll actually do that. But so far, thankfully, it hasn't come up.

I'm not saying that every program needs its own unique init script. What I'm saying is that the important functionality provided by systemd (nobody cares about saving a few seconds' boot time on the desktop when the POST takes aeons) could have been provided by a series of small shell scripts. Yet here we sit arguing about how horrible shell scripts are, because they're being misused. Meanwhile, shell scripts are still easy to troubleshoot, and certainly much easier to troubleshoot than systemd. I should not need to somehow connect a debugger to my system in order to find out why it won't boot, but that's what systemd brings to the table.

Re:Systemd on slashdot

[MrKaos]

because...

...anything that needs to know what the PID of a process to manage the process should not be managing the process. That is what init does, and it already knows the PID.

Typically, a PID file is scripted when you need to maintain a process. If you need to maintain a PID file, chances are you are using a script in /etc/init.d linked to a runlevel entry in /etc/rc.whatever. The use case here is that you want to shut the process down on shutdown *OR* change of runlevel. In that mode you're trying to figure out K ans S scripts and what order the script is started and shut down so that you can kill your process gracefully on exit.

The answer is you let init manage it. People seem to get confused about setting up the "context" of a runlevel (/etc/init.d) as opposed to the "strategy" employed in that runlevel (/etc/inittab) and then try to write an unnecessary script. If you are scripting an init process and you need to use a PID file, then you are addressing the strategy of the system from within the context and I suspect this is where many people get confused.

init will manage the execution of the system strategy by managing the lifecycle of the process for you. If you want it to stay up, you tell init to "respawn" it, then if you kill it or the process exits, init restarts it. If the system changes runlevel, say for shutdown or other, then init shuts the process down. Or, if you want to take the process down, you disable it in /etc/inittab and kill -1 1 when you are ready. I think there is a vim inittab command that does this for you.

This is where systemd tries to mix the strategy of the runlevel into the context, whereas what you actually want is to have the two concepts loosely coupled so they are easier to manage by isolation. People might be surprised how much raw performance potential init has to offer when you get your head around it especially when you discover that in a lot of cases you don't need a script at all.

Re:Systemd on slashdot

[drinkypoo]

It's hard to run a business relying on everyone to be a bunch of super-hero super-geniuses. Businesses want a solution that's drop-dead simple.

I get that. The problem that they don't seem to get is that we live in a complicated world, and there is no such thing as a simple solution to that which doesn't involve raining fire down from the heavens, a global flood or some other such reset button which is only available in god games. In the really real world, we simply have to deal with complexities. A computer which could solve all its own problems would be smart enough to go on strike.

Re:Systemd on slashdot

[phantomfive]

The Debian team seems to agree with you, at least, that's why they adopted systemd.

I've written about the things that I like, and that disturb me about systemd.

Re:Systemd on slashdot

[Anonymous Coward]

If you're rebuilding your infrastructure every day, your fucking doing it wrong, moron. And apparently have nothing else to do.

Re:Systemd on slashdot

[Anonymous Coward]

Which I've been doing for years, without this systemd crap.

Re:Systemd on slashdot

[Anonymous Coward]

The shell is extraordinarily powerful. Why look for poorly supported, maintained and often non performant libraries and api's when with the shell you can access a rich ecosystem of high performance tools written in c. For a certain class of software, orchestration and leveraging Linux capabilities, nothing beats the shell.

But the shell has become disrespected and often just not understood of late because a large portion of those who want to make a career in software have no incentive to learn the shell. Practically it makes more sense to learn Python, Ruby, Go etc that will pay, and this is not a bad thing. The problem is then some of these folks with little knowledge of shell turnaround and spread misconceptions, in places like hacker news etc and this then becomes received wisdom by repetition.

There is nothing wrong with init scripts. If someone abuses the shell to write bad code the problem lies with the specific script, and not the shell. This kind of misguided criticism can apply to any language and is not sincere in its intention. If some shell scripts become complex you will see the same complexity in systemd, its not going to suddenly go away and you can see systemd being used to call these complex scripts as the functionality is needed.

The problem with Systemd is politics and choice, its of the bazaar giving way to a cathedral in its midst and this newly built cathedral using the power of hired open source developers to push agendas. No one would worry much about Systemd as along as it was another alternative, but systemd does not want to coexist. All Redhat supported projects suddenly want to depend on systemd for various reasons. If this is not politics what it? The presence of systemd dissenters is easily understood, there is always dissent and the need for debate and that's healthy. But what explains the profusion of systemd supporters in every thread and discussion who arrogantly dismiss debate and frame the discussion as people against change. Since when is debate and discussion not normal and who frames this as anti change? Where does a relatively new and untested project with not many users get so many zealous supporters and advocates?

And notice the shift in discussion tactics. First it was shell scripts and init are old and inadequate with little technical discussion beyond just branding shell scripts as bad, and highlighting how systemd enables some niche add trivial functionality that few people need. Now its about how everyone has adapted systemd so it must be good, and to get with the program. This is pure politics with PR and smear tactics similar to those adopted for pushing through unpopular decisions like nuclear plants through a skeptical local population. Even if Systemd is the best thing since sliced bread I don't how any open source supporter can support it in good faith.

Debian arrogantly dismisses its own users and claims more accountability to developers which by proxy, is Redhat developers. So Debian is now more accountable to Redhat than its own users! The Linux foundation is driven by corporates as are nearly most open source bodies, there is not a single body that represents users and tries to address issues about funding and projects and represents the user voice in open source and the result is users now have no voice. Money has its own agenda, if users don't protect their interests others certainly will.

Re:Systemd on slashdot

[Anonymous Coward]

Me too. I have serious doubts that this anon knows how real System Admin works. My database servers reboot just fine thank you and I've always been able to delegate to juniors (to oversee the process just in case) without fear...before systemd showed up to save me.

Re:Systemd on slashdot

[pnutjam]

Lazy system admins don't mess with things. There are always improvements to be made and testing to insure things can fail and be restored. People like you are why nobody can reboot that 10 year old Solaris box that nobody understands.
A good system admin shouldn't be afraid to break something and use the opportunity to fix it and learn something new. They should always be on the lookout for ways to smooth processes and verify documentation.

Re: Systemd on slashdot

[Anonymous Coward]

Where is the space for debate in your narrative since opposing view points are either by ''sock puppets' or 'people who love their scripts'. So are we to conclude that is systemd is so perfect that anyone who has an alternative view is 'motivated?

it's this binary framing of the debate by systemd supporters that suggests perhaps the sock puppets have being employed liberally by the other side since logically systemd is too new and buggy enough not to have so many zealous supporters.

Re:Systemd on slashdot

[drinkypoo]

A good system admin shouldn't be afraid to break something and use the opportunity to fix it and learn something new.

Along with being willing to pay for a good system admin, a company has to be willing to pay for test hardware in order for IT to be successful. I've definitely worked in situations where I didn't have that, and was expected to do more with less, and then complained at bitterly when there were coughs. Sorry, kids, if you want it done right, you should pay for it to be done right. If you want it done wrong, I'll do my best.

They should always be on the lookout for ways to smooth processes and verify documentation.

And document processes — but it's easy to document your way right out of a job if your management is sufficiently short-sighted. I really enjoy writing documentation, and take some pride in it, but I'm leery of getting carried away any time it's not the point of a fixed-period contract. Even making too many templates can get you replaced with a mouse monkey. Now I know why so many design files don't have any guide lines left in them.

Re:Systemd on slashdot

[Anonymous Coward]

The former.

Re:Systemd on slashdot

[Anonymous Coward]

Dinkypoo, you hit the nail on the head.

Now all we need is a solution for the "monkey" which has already been hired. What use can they be? :|

Re: Systemd on slashdot

[Anonymous Coward]

Nothing comes close to Linux in data centers and cloud deployments. Genie's out of the bottle.

Re:Systemd on slashdot

[gweihir]

You seem to have zero experience with actual enterprise IT. What you say is straight from a "cloud services" ad, but it has no resemblance to what is going on in the real world, where an application has to work or 10'000 people can go for coffee. And yes, servers may still run many months at a time without reboot. For enterprise applications on server-side it is often a requirement that they can run for an unlimited time.

You also seem to have no actual system administration experience. If you install a new server, you still have to configure it and the configuration part is the major part of the work. Nobody brings up "10'000 new" servers unless they all follow a very carefully created pattern (and hence sysv-init is an advantage because of higher reliability).

Re:Systemd on slashdot

[gweihir]

HPC is easy: If a job is lost, you just re-schedule it. HPC is also not what most enterprise software does.

Re:Systemd on slashdot

[aaarrrgggh]

Where are these good out of work sysadmins you speak of? In the Los Angeles area, I have found exactly one (contractor). It is pushing me to need to migrate off Samba and onto Windows/AD for our office after a decade. Systemd also pushes me in that direction, as it becomes closer to a Windows type of solution anyway... just not as mature.

And before you suggest a full time sysadmin, we are a 40 person shop and there just isn't enough to justify a full time position. I would much rather pay $135-150/hour for the ~200 hours a year of work we have and let someone make a lot more money than they could as a FTE.

Re:Systemd on slashdot

[Anonymous Coward]

Nothing better than a boot process that randomly segfaults. SUPER COOL!

Re: Systemd on slashdot

[Anonymous Coward]

Thank you, that was awesome. I'm sick to death of these inexperienced self important idiots reinventing things we've had for years and then complaining when we don't worship at the altar of whatever they think is cool this week.

Re:Systemd on slashdot

[Anonymous Coward]

Bingo about shell scripts. The one thing about shell scripts is that they are at their core a collection of commands that the admin can run manually, wrapped in logic that codifies the thinking an admin do based on the output of said commands.

Thus if something breaks, a proper shell script is a guide that any admin can follow to reproduce the error.

Re:Systemd on slashdot

[goarilla]

When do you decide to have a system managed service (for example apache) or a /etc/init.d initscript ?

Re:Systemd on slashdot

[KGIII]

What you say is spot on and the bit about the changes in attitudes is also astute. I've made it this far down the thread while saying nothing. I probably should continue that trend but, alas, here I go...

Am I a systems administrator? Sure, I guess. Not so much now as I once was and that was quite some time ago but, today, I've got a ton of Linux installs - on desktops and servers alike. I even have remote co-lo hardware that runs Linux. I'd not /really/ call myself a sys admin but, for the sake of this conversation, I guess I might fit - even though absolutely none of what I run is important to anyone but me and a few other people who may do stuff like use my hardware for remote backups or hosting a small site.

I've tried to dislike SystemD. I really have. I mean, yeah, I love me a good hate fest and I love ranting about philosophical things like the Unix Way. Yet, I learned a few new commands (it's kind of neat to be able to check and blame things in the startup to see if you can eek out a few more milliseconds or find out what barfed and where) and I've found a few places where it comes in handy.

In other words, I've not really had any problems. Yes, I had to change a few things but I'm kind of used to that. Yes, I had to learn some new things and that's okay - I actually enjoy that part. I can still use init scripts if I want but I don't actually have any compelling reason to do so. I can even use a GUI and make stuff start on boot if I want - I can even stuff scripts in there.

So, I guess I've already made peace. I'm not saying you have to but I'm saying that my life is easier having done so. I get that the developers didn't listen to everyone but, how many users are not complaining? Keep in mind, it's the people who are unhappy that make the most noise. The people who are content say very little. They can't please everyone and make changes. I suspect that the people who don't like it are a small, but noisy, minority.

Re:Systemd on slashdot

[Hognoxious]

Lennart has poured his heart and soul into creating a solution, and you don't have the problem to go with it? You inconsiderate bastard!

Re: Systemd on slashdot

[Anonymous Coward]

That's precisely what happened.

Re:Systemd on slashdot

[Ensign Nemo]

"Without something like systemd, Linux cannot be enterprise ready."

You are a complete and utter dumbass. You don't even know how much you don't know. Why don't you actually listen to some of the greybeards, instead of shouting them down because you know better.

Do you realize how many mission critical enterprise systems run on Linux, and have since well before systemd was even thought of? You understand how massively large some of those systems are?

Pull your head out of your ass and get some perspective.

I don't hate systemd but dumbass comments like yours make me think systemd might actually be the colossal mistake that so many people are screaming it is. You jackasses think that those old greybeards have no idea how business works, what's actually important or how to do 'real' computer things. F*cking arrogant jerks.

Re:Systemd on slashdot

[phantomfive]

So I don't know if systemd is "double logging" or if Debian set this up to keep us old farts happy.

Systemd is configurable to do that, and the Debian team likes that style of logging enough to configure it that way (or so I've heard....I haven't verified the Debian setup).

Re:Systemd on slashdot

[Anonymous Coward]

I know the alternative. Systems keep running for years, eventually get stale and take weeks, months or even years to migrate away. We used FAI to automate everything at install time. But as the systems changed to FAI config had to be updated as well and they quickly came out of sync. Five years later when the version of Ubuntu we had deployed ran out of support we tried to install a new server and found that the config had major differences to what the old server was doing. What was the old server doing? No idea, years of different admins doing a patchwork of things over the years. Now everything is containerized. New servers are brought up when we need to change anything, the old ones are brought down and destroyed. I know that we can exactly recreate our entire infrastructure if we had to start over from scratch. Before I would have no idea. If a catastrophic event destroyed our data center we would probably take months before we were back in action. Now it takes hours.

Re:Systemd on slashdot

[Anonymous Coward]

> hours spent dealing with obtuse scripts

"abstruse"

"obtuse" means "dim-witted" (of a person) or an angle that is greater than 90 degrees.

"abstruse" means "difficult to understand"

(Sorry, I'm only correcting you because I don't think I've seen anyone on slashdot use "abstruse" correctly, always substituting "obtuse" for it.)

Re:Systemd on slashdot

[sjames]

Systemd can't even manage to bring a system up when a redundant drive has failed. Init scripts in the initramfs work around it for RAID by bringiong the raid up before systemd gets a chance to screw up. It's still just plain broken for btrfs.

For applications that have special requirements there, you want an init system that is truly modular. That is where you can just drop in replace the functionality on an as-needed basis. That implies a need for a simple and well documented API/. The APIs involved in systemd are neither simple nor well documented.

Re:Systemd on slashdot

[sjames]

Translation, the business guy has hired cheap monkeys who turned out not to be able to handle any difficulty.

That's the problem. Anybody can captain a ship in calm seas. It just doesn't take all that much to stand at the wheel and look important if the controls are dumbed down enough. The problem is, in rough seas it takes real ability and controls that haven't been dumbed down to get through it. If the controls have been dumbed down, even the old salts can't help you.

Lift the hood on systemd sometime. Look at the bazillion twisty little config files hidden away in the lib directories. You'll be horrified.

Re:Systemd on slashdot

[sjames]

Have a look at the hundreds of inscrutable little config files under systemd's hood and be horrified.

Somewhere in there and the thousands of lines of C code is the reason that systemd cannot handle a degraded raid or btrfs filesystem gracefully, but even the authors of systemd cannot find it for over a year.

If the init scripts have gotten that complex, it's surely self inflicted damage. Just rip it all out and go with a simple script that just does what is actually necessary.

Re:Systemd on slashdot

[sjames]

No, he made tossing out software that scratches other people's itches over their objections so you don't have to bother with co-existing in an ecosystem a bad thing.

Nobody would mind systemd if it didn't try so hard to preclude other solutions.

Re:Systemd on slashdot

[nine-times]

Lift the hood on systemd sometime.

I'm not advocating for systemd. Just pointing out why "enterprise ready" to some extent can mean, "doesn't require any special genius to operate".

Translation, the business guy has hired cheap monkeys who turned out not to be able to handle any difficulty.

That's a really dumb translation, and shows you missed the whole point. You might get how the tech works, but you have no idea how to run things if you think the solution is to hire more expensive people. Sometimes you hire very expensive monkeys on the idea that they're brilliant and "worth it", and then they still make a mess of things. If you've actually had to do hiring, you probably know that it's very difficult to find good people, and "expensive" doesn't always mean "good".

And really, it doesn't make sense to always hire very experienced people. To keep things running well, you need a mix. You get some senior people to do the difficult stuff, and then you get some junior people to handle the stuff that no senior person is going to want to waste their time managing. And then you train the junior people as you go, which means at some point, you're going to have to hand off some of the senior work to the junior people, even if they're not 100% "ready", because sometimes that's how you learn.

It may be true that anyone can captain a ship in calm seas, but part of the goal of a business is to engineer the seas to be calm. A lot of the "old salts" get off on being the guy who heroically and astoundingly manages to patch up the Titanic before it goes down, but the business-oriented guy just wants to steer around the icebergs.

Re:Systemd on slashdot

[gweihir]

In Enterprise IT, try "cannot be migrated away" as worst case frequently encountered in practice, as there is no spec or source, the people doing it have left and the system still is needed. As to FAI, I have used it and while it is very nice for HPC clusters with mostly or fully identical configuration, I do not think it is a good approach if each system is different. You problem with Ubuntu was not Ubuntu, but shoddy system administration and documentation practices. Technology cannot fix that.

I would also suggest that while your current approach works well now, unless you very carefully document what each application needs from its container and why, you are going to run into exactly the same problem as before in 5 years.

Re:Systemd on slashdot

[sjames]

Shell scripts are junior admin material already. To a degree, they're power user material. If your admins aren't even up to power user level then my point stands. They're not even really admins, why do you let them have root access to your boxes?

Perhaps more to the point, it's not like init scripts need to be constantly messed around with. Surely there must be an actual admin somewhere in the enterprise that can deal with it the once a year (if your enterprise is large enough) it is needed?

Re:Systemd on slashdot

[nine-times]

Shell scripts are junior admin material already. To a degree, they're power user material. If your admins aren't even up to power user level then my point stands.

Again, this is apparently just over your head. The question is not whether a theoretical smart, qualified, careful, competent person should be able to handle it. The question is whether you want to bet the future of your multimillion dollar business on the competent execution of a particular action of any single individual. The answer you'll get will often be, "not if I can avoid it." The idea is to make things as pre-built, standardized, and fool-proof as possible so that you're only relying on people to be competent as much as is absolutely unavoidably necessary.

Because if you've ever actually run anything in your life, you know that most "qualified" people are still not good at their jobs, and even those relatively rare individuals who are very good at their jobs make incredibly stupid mistakes every once in a while.

Re:Systemd on slashdot

[sjames]

So you really do think you can design a workplace for monkeys and somehow get a decent work product out of them.

You actually do prefer something too inflexible to be fixed in the event of a problem over something that requires a minimum of competence.

Do you also hire sales people with no social skills and accountants who flunked arithmetic?

Did your lawyers pass the bar?

Re:Systemd on slashdot

[Anonymous Coward]

TL;DR: A monkey who can't understand a shell script can't be a real sysadmin anyway. All they can be is a button puncher. They should work in a NOC, and you should hire someone who knows Unix to administer Unix.

Those monkeys often get MS certifications to go with their point-and-click configuration skills.

Maybe systemd is a means to bridge the gap in administration and will make it easier for Linux to be accepted into businesses. They gray beards can complain all they'd like, but it's still a focus on "the old ways". Part of the popularity of Windows is the ease in configuration for the enterprise. There are very few businesses that want to hire an army of Unix oracles to manage the few thousand machines they have... When they can get by with 4 or 5 kids just out of school.

Re:Systemd on slashdot

[dave420]

It is supposed to be automated, so that would be a sensible approach.

Re:Systemd on slashdot

[ookaze]

Along with being willing to pay for a good system admin, a company has to be willing to pay for test hardware in order for IT to be successful. I've definitely worked in situations where I didn't have that, and was expected to do more with less, and then complained at bitterly when there were coughs. Sorry, kids, if you want it done right, you should pay for it to be done right. If you want it done wrong, I'll do my best.

They should always be on the lookout for ways to smooth processes and verify documentation.

And document processes — but it's easy to document your way right out of a job if your management is sufficiently short-sighted. I really enjoy writing documentation, and take some pride in it, but I'm leery of getting carried away any time it's not the point of a fixed-period contract. Even making too many templates can get you replaced with a mouse monkey. Now I know why so many design files don't have any guide lines left in them.

This there is the true reason why so called "old sysadmins" despise systemd so much: it makes everything much more simpler and faster to do correctly, and allows them to be replaced far more easily by less paid individuals.
I never thought of that, but now it's clear. Because true sysadmins could never say so many absurdities about systemd if they really tried it.
Though I think these syadmins are thinking to much into this, you still need to be a competent sysadmin to manage systemd, the countless clueless mistakes made by very bad old sysadmins (if even you can call them that sometimes) seen in bug reports are there to prove it.
I'm not worrying at all. Systems were impossible to manage correctly with initscripts, one bad sysadmin could destroy everything or compromise the security of your boxes without you seeing anything, I've seen that countless times.

Re:Systemd on slashdot

[ookaze]

If you are arguing for systemd, have you ever tried using inittab or customizing runlevels? It is powerful and demanding, but not impossible. As a core *ix paradigm it forces you to skill up to the next level of mastery over your systems and understand how your processes behave.

systemd seems to abstract away that knowledge of how processes behave, like wishing for less challenges instead of more skills.

Do you realize that every single one of the anti-systemd crowd is not advocating what you are talking about (really using init), but the crazy crappy way with symlinks and everything used by every distro that used sysvinit?
You're right of course, but what you describe is roughly the same work that has to be done with systemd: trimming these insecure faulty initscripts to one invocation line.
Sometimes it's not possible and requires a shell script, but the work is the same with systemd and truly using sysvinit.
But sysvinit is really too limited compared to systemd, and would need lots of shell scripts to work properly, just to bott a Linux system, and most importantly, sysvinit needs a competent sysadmin custom configuring it.

Re:Systemd on slashdot

[ookaze]

Have a look at the hundreds of inscrutable little config files under systemd's hood and be horrified.

Not if you are used to the hundred of big initscripts under /etc . I know I never was intimidated by systemd config files.

Somewhere in there and the thousands of lines of C code is the reason that systemd cannot handle a degraded raid or btrfs filesystem gracefully, but even the authors of systemd cannot find it for over a year.

You are wrong, the problem lies in the Linux kernel and btrfs implementation, the developers explained this long ago, but you're too computer illiterate to understand the explanations given by developers. You surely didn't understand that systemd by default won't use a policy that can lead to data loss, no matter how loud you yell at it. And these problems are partly policy problems, managed everywhere (even with hardware RAID) by daemons or services, that won't let you boot by default, you have to configure them to do it.

If the init scripts have gotten that complex, it's surely self inflicted damage. Just rip it all out and go with a simple script that just does what is actually necessary.

"Just do it" is the answer of clueless people that have no experience or knowledge of what they're talking about, especially when they're saying this to people with lot more experience, like distro sysadmins.
Initscripts are inherently broken in a dynamic environment, just starting with race conditions.

Re:Systemd on slashdot

[sjames]

I am quite familiar with the scripts under /etc/init.d. The bazillion config files for systemd aren't intimidating, they're horrifying. They actually use "come from" (a popular programming language joke) style logic! I don't just mean the ones under /etc, I mean the ones under /lib (under the rug?).

The kernel and btrfs work just fine when systemd isn't there to muck it up. I tested that by setting up a system under sysV, testing, and then under systemd. The sysV system booted up fine in degraded mode (after I specified that I wanted that behavior). The systemd system refused to boot in degraded mode in spite of me specifying that I wanted degraded mode when necessary.

System tools are supposed to provide the mechanism to enforce the admin's policy. They are not to dictate policy to the admin.

Backups are supposed to prevent data loss. RAID either hard, soft, or implemented within the filesystem is helpful there but the primary purpose is to improve the machine's availability. Sitting at an emergency shell prompt in the initramfs is anathema to availability. The correct behavior is often mount it anyway and email the admins. If my policy for the server is that it shouldn't mount the degraded volume, I can certainly do that with or without systemd. If it should mount the volume anyway, I can only do it without systemd.

I doubt you want to use your experience in a dick measuring contest here. Just do it is also sometimes used by people good enough to actually just sit down and do it.

While I do recognize that sysV is limited in a dynamic setup, I also recognize that whatever the real solution is, systemd will hinder it's development and actively fight it's implementation through tightly coupled dependencies, poor documentation, and unstable APIs If the systemd project would put up real walls between "moduiles" such that it becomes truly modular and drop it's all things to everyone but it's our way or the highway attitude, parts of it might actually figure in the solution. It really needs to get divorced from freedesktop.org.

Re:Systemd on slashdot

[MrKaos]

Do you realize that every single one of the anti-systemd crowd is not advocating what you are talking about (really using init), but the crazy crappy way with symlinks and everything used by every distro that used sysvinit?

Which was *Red Hat's* idea. No, I didn't.

Red Hat layered that crappy rc system over the original, effective and very simple way init implemented things. Now you're telling me that this is the justification for systemd! Such irony, I can't see their second being any better.

It's a shame, init is one of the purest O.S paradigms there is, it really shows the genius of the people who designed UNIX and C in the first place. Simple elegance.

You're right of course, but what you describe is roughly the same work that has to be done with systemd: trimming these insecure faulty initscripts to one invocation line.

In systemd implementations the unit files will use those init scripts. People usually learn the minimum they can get away with and make do from there.

Sometimes it's not possible and requires a shell script, but the work is the same with systemd and truly using sysvinit.

So the point is?

Go back to init and see what it *can* do. It's simplicity makes it extremely flexible for a large amount of boot conditions. I doubt you have a use case it can't satisfy.

But sysvinit is really too limited compared to systemd, and would need lots of shell scripts to work properly, just to bott a Linux system,

How so? Be extremely specific please.

and most importantly, sysvinit needs a competent sysadmin custom configuring it.

What sort of argument for systemd is that? That you don't have to be competent to configure it or that it is necessary to dumb down an elegant, simple, powerful, flexible paradigm for a bloated one so that you understand it?

I've made my peace with systemd

[Anonymous Coward]

I've made my peace with systemd by switching all my machines to BSD variants - pfsense for my router, FreeNAS for my file server and OpenBSD for my desktop and laptop.

I'm one of many who have fled into BSD-land. 2016 will see an even further decline in the amount of Linux users. Linux can only shit on its userbase for so long (systemd just being the most well-known example of not listening to users) before the mass exodus kicks off.

Re:I've made my peace with systemd

[c-A-d]

I'm sort of right behind you. I can understand SystemD for creating the tools to make a good desktop. However, tools that make a good desktop do not necessarily make a good server. That's where I see the problem.

I run Linux Mint on my desktop and as long as it works, I don't really care what's under the hood. However, I also maintain servers where I care very much what happens under the hood. For example, I care about being able to read plain text logs and I don't understand what possible reason there could be for storing logs in a binary format. I also don't see the need to have a super complex system configuring my network interface simply because in a typical server environment your IP address doesn't change often, if at all.

I'm definitely looking at FreeBSD again (it's been 15 years since I touched any BSD outside of monowall/pfsense) and am using it for a new project at work that's currently in Alpha testing phase. I'm doing it because it's more in line with my KISS preferences but I will admit that I miss iproute2.

Re:I've made my peace with systemd

[Anonymous Coward]

First of all, what is "binary logs" really about? At its core it's just a file format. A well-defined file format for storing data. Nothing bad so far. But why use a custom file format for storing logs? Isn't it ok to just append lines to a set of files as they come in? No. There are many reasons why that's not enough. Since journal entries has an actual format we can find things much easier. Let's say all entries from a unit between two points in time. Now I'm sure there are sysadmins that can write bug-free regexes for time stamps and have it work even if part of the log has been rotated, that's awesome. But for the rest of us being able to just have journalctl do the right thing and do it fast and reliably is also awesome.

And you can of course choose to use rsyslog if you want to. Just enable syslog forwarding, disable the journal and keep using old plain text logs and have fun writing regexes, and that's OK.

Re:I've made my peace with systemd

[gweihir]

If you have to ask then you do not understand the issue. It is glaringly obvious though.

Re:I've made my peace with systemd

[Anonymous Coward]

Please explain. For me the entire "OMG the logs are binary" is just as good of an argument as arguing that PostgreSQL should store its data in a csv file instead of a binary database because plain text is easier to read with text processing tools installed on every unix box since the 1970s.

Re:I've made my peace with systemd

[Anonymous Coward]

I've moved to BSD also, this includes my desktop. It took a while for me to learn the ropes and get my systems running the way I wanted them but now that things are going smoothly I have no reason to go back to Linux and I hope it stays that way.

Re:I've made my peace with systemd

[drinkypoo]

And you can of course choose to use rsyslog if you want to. Just enable syslog forwarding,

...and pray that if there is a problem with systemd, the log entries still actually make it that far, which is not a foregone conclusion.

The single worst thing systemd did was fuck up logging. There was no need for it. They could have added binary logging to any existing syslogd. Instead, NIH, because Poettering. But based on experiences with pulseaudio, he can't code his way out of a nutsack. Now he wants control of the boot process, after fucking up the audio process for years. And you want to give it to him, because you forgot history.

The normal adoption process for something like systemd would be to roll a new distribution using it, and then once it is tested, debugged, and proven, for it to propagate throughout the community. That process was short-circuited, to everyone's detriment.

Re:I've made my peace with systemd

[gweihir]

No. Seriously, if you need to ask then you do not even remotely have the level of understanding and experience this needs. I cannot fix your deficient education and experience.

Re:I've made my peace with systemd

[thegarbz]

For example, I care about being able to read plain text logs and I don't understand what possible reason there could be for storing logs in a binary format.

ForwardToSyslog=yes in journald.conf

Everytime I see the complain about journalling in systemd I cry a little. The only thing journald does is allow logging earlier in the boot process before a syslog daemon has even started and give you options. One of those options include outputting everything back to your syslog daemon of choice if you're so deadset against having easy to browse journal entries which provide far more detail than the traditional syslog.

I also don't see the need to have a super complex system configuring my network interface simply because in a typical server environment your IP address doesn't change often,

You mean in *your* typical server environment. Systemd provides a lot of functionality that can expressly help in server environments such as actually intelligently knowing the state of the daemon not just relying on some PID file that may or may not accurately represent the state of the daemon, the ability to restart the sudden death of a daemon (configurable of course so you don't end up with a restart loop), and the above mentioned logging options which make it so much easier to track a misbehaving part of the system by cutting through the noise of logs.

Re:I've made my peace with systemd

[drinkypoo]

I'm sort of right behind you. I can understand SystemD for creating the tools to make a good desktop.

No. The boot process is just as important to a desktop as a server, and when systemd breaks the boot process, it also breaks troubleshooting the boot process. There's nothing about a desktop that magically makes you want to do this. Also, all of the functionality of systemd which does not exist in other daemons already extant on the system (i.e. cgroups support) would actually be far more useful on a server than on a desktop... but you can't trust systemd on a critical server, because the boot process is even more important there. So, who is it for? The only argument that even slightly makes sense are people spinning up short-running VMs. That tends to be horribly inefficient in the best case, though. It helps you do stupid things faster? That's the definition of computers, I guess. But I'm not sure the goal should be to do stupid things faster so that you can be more stupid.

Re: I've made my peace with systemd

[Anonymous Coward]

Every time I see a systemd fan miss the point about the problem with binary logs and suggest that forward from a binary log to a non binary log is the solution..... We want the fowarding to happen the other way around so when the binary log craps out we still have a readable log file rather that a partial unreadable binary turdlet.

Keeping binary logging before the syslog achieves nothing.

Re:I've made my peace with systemd

[wierd_w]

I have only modest experience working with *nix flavor boxes, and I fully understand the need for text based logs.

I see no real value in a binary based log, unless you want to attach some kind of diagnostic symbol metatdata to the log. (and if you did that, you had better have a dedicated storage array to store the logs...cause they will get big FAST.)

Basically, you need to be able to read the logs with the most minimal of tools, because you are going to be diagnosing it in a downed state most likely-- You cannot bank on having a full suite of binary manipulation tools on hand. You will be lucky if you have more than vi.

Also, text based logs compress REAALLY well for long term storage for audit purposes! Binary logs? Probably not so much.

Not to mention-- if the binary logs are in some stupid "easily damaged" format, then having a process suddenly die horribly from abnormal termination will result in corrupt logs, so good luck figuring out when or how the process died. Not so with good text logs. It can cut off right in the middle of the debug print, and the text file is still valid. Hell, the file chain can be damaged from FS corruption, and parts of the log will still be readable.

Text logs are just plain better in those regards.

Re: I've made my peace with systemd

[kthreadd]

What does that even mean? When you enable syslog forwarding the journal will send it to syslog directly without going into the journal. You can even disable the journal log if you don't want it and only want to rely on syslog. So there is no binary log that can "crap out". It just sends it off to the syslog, just like sysvinit handled syslog.

Re: I've made my peace with systemd

[Anonymous Coward]

Why do I have to have journald upstream of my logs even when I turn off the journal. Some of us don't want your points of failure.

Re:I've made my peace with systemd

[Anonymous Coward]

For example, I care about being able to read plain text logs and I don't understand what possible reason there could be for storing logs in a binary format.

ForwardToSyslog=yes in journald.conf

Everytime I see the complain about journalling in systemd I cry a little. The only thing journald does is allow logging earlier in the boot process before a syslog daemon has even started and give you options. One of those options include outputting everything back to your syslog daemon of choice if you're so deadset against having easy to browse journal entries which provide far more detail than the traditional syslog.

Wow. Way to miss the point.

Unix has run for decades without "far more detail than the traditional syslog", which is an asinine strawman anyway, as logging details are generated by the application/daemon and can be as detailed as that application/daemon wants them to be. Adding complexity and therefor bugs and unreliability to the logging process doesn't fix a damn thing.

So not only are you wrong about "far more detail", even were you correct all you have is a solution in search of a problem - which is pretty much all of systemd in the first place.

I also don't see the need to have a super complex system configuring my network interface simply because in a typical server environment your IP address doesn't change often,

You mean in *your* typical server environment. Systemd provides a lot of functionality that can expressly help in server environments such as actually intelligently knowing the state of the daemon not just relying on some PID file that may or may not accurately represent the state of the daemon, the ability to restart the sudden death of a daemon (configurable of course so you don't end up with a restart loop), and the above mentioned logging options which make it so much easier to track a misbehaving part of the system by cutting through the noise of logs.

Why does systemd need to "intelligently know the state of the daemon"? That's the daemon's responsibility, and if the daemon does that poorly systemd can't fix it anyway. If a daemon can't stay running properly and doesn't have the facilities to figure out why, piling a ton of bloat on top of it isn't going to help.

Again - a solution in search of a problem it can solve.

Re: I've made my peace with systemd

[kthreadd]

Something needs to respond to the syslog calls. I guess that could go into systemd init but I heard some people didn't like putting stuff into PID 1.

Re:I've made my peace with systemd

[Anonymous Coward]

Bingo.... I moved to FreeBSD when systemd hit arch linux, What a cluster arch linux has become... Never going back.

FreeBSD current and poudriere, cross builds for my two raspberryPI2 servers, Aye it just works.

Re:I've made my peace with systemd

[rl117]

As an example for the problems with troubleshooting, I've recently installed a few Ubuntu 15.10 systems (bare metal and in VMware VMs). In both scenarios, NFS filesystems fail to be mounted at boot. Why? I have no bloody clue. There's nothing in the logs. If I umount them (they are showing as mounted but give immediate I/O error on use) and remount by hand it all works just fine. No idea how to troubleshoot it. With sysvinit I could boot to an interactive shell in the late initramfs and step through every script by hand, and pinpoint exactly where something was failing. With systemd, even with correctly configured systems, I still experience occasional unrecoverable hangs at boot as it screws up its nondeterministic boot ordering and waits forever on something or other (who knows what it might be). systemd has absolutely not improved boot reliability. If I boot a FreeBSD system with exactly the same NFS configuration, it mounts everything perfectly at boot every damned time.

Another thing, is the broken compatibility with what came before. Example: I edit /etc/nfsidmap.conf to configure NFSv4 id mapping. Previously I'd reload/restart the nfs-common service with 'service' or 'invoke-rc.d'. Job done. What's the systemd equivalent? I don't know, and I can't see any obvious candidate. Since both these commands now forward to systemd, a diligent engineer would have made the old service names forward to the new target(s) to retain compatibility, maybe with a helpful message indicating what the new names were. There's nothing. I don't see any obvious nfsidmap or generic rpc or nfs units. Yes, it's partly me lacking familiarity with the new way of doing things; but the lack of care in having a smooth transition for admins makes for a terrible time, and the mess of units makes the whole thing difficult to comprehend as a whole and baroquely overcomplicated.

Re:I've made my peace with systemd

[kthreadd]

Journald does compression and the structure allows it to actually verify that the logs are sound. Remember that logs can be damaged for multiple reasons. Bugs are of course one reason but the more scary reason is tampering. Syslog has absolutely nothing that prevents tampering with logs. Everything that is submitted to syslog goes in, there is no verification that it's even the correct program. Play around with the logger command on a Linux box and you will be scared. Journald on the other hand actually verifies that the data that goes into the logs are from the correct sender and that once they have been entered you can verify that they have not been tampered with.

Re:I've made my peace with systemd

[Anonymous Coward]

Classic anti-systemd response. All FUD with zero substance. Actually this isn't even FUD, I don't know what this is. Ignorance maybe?

Re:I've made my peace with systemd

[drinkypoo]

Another thing, is the broken compatibility with what came before. Example: I edit /etc/nfsidmap.conf to configure NFSv4 id mapping. Previously I'd reload/restart the nfs-common service with 'service' or 'invoke-rc.d'. Job done. What's the systemd equivalent?

I would argue that this is actually the distribution's job, but someone certainly should have taken care of it if the goal is to be easy and convenient, which is the usual claim by systemd supporters.

Re:I've made my peace with systemd

[Anonymous Coward]

I made my peace too. I installed the latest Ubuntu, RTFM and really have no idea what all the fuss is about.

Re:I've made my peace with systemd

[Anonymous Coward]

The boot process is just as important to a desktop as a server,

Indeed. I use Linux on my home desktop PCs. I don't give a rat's ass about boot time: even before SystemD, the post-POST boot time was smaller than the POST time, and anyway I reboot the thing like once a month when there's a new kernel update. An extra 2 or 3 seconds doesn't matter, compared to the hours I've already lost with SystemD.

I have a laptop that's been happily running with Ubuntu releases since 8.something. I just installed 15.10 on it. It doesn't finish booting. It dumps me into a bash prompt with a message about using journalctl to see what's wrong. Which I try, but as far as I can tell, the output has fuck-all info about what the problem is.

So why doesn't this box boot? I have no clue. It's booted with every version of Ubuntu before this. Can I eventually figure it out? Maybe, who knows. But it's not making that process very easy, that's for sure. But boy, am I glad I saved 2 seconds of boot time!

Re:I've made my peace with systemd

[Lawrence_Bird]

I made my peace nearly 10 years ago when I bailed on Slakware (and I was one of its early adopters in the 90s). While I did like the underdog status of NetBSD, I settled on FreeBSD, have not looked back and use it on servers, desktops and even a laptop (gasp!). Perhaps I'm more easily satisfied because new shiny never was a big thing to me.

Re:I've made my peace with systemd

[present_arms]

I won't touch SystemD either, I'm lucky that the main maintainer for the distro I use has no interest in using it at all. If he is forced he may well just close the distro down, then I go to bsd :D

Re:I've made my peace with systemd

[Anonymous Coward]

I'm switching from Arch Linux to FreeBSD.
It just seems that FreeBSD is cleaner and more thought out than any Linux I've ever seen. Everything seems to have and be in it's place and there aren't all these layers upon layers of shims and widgets running. In my testing I've never had to reboot it. It's native module-less ZFS seems really useful and integrated, even in the installer and it's been in production for years, unlike BTRFS which just doesn't seem like it will ever make it. And the best is that, unlink the Linux distro-of-the-month game, I never have to worry about FreeBSD going away or going corporate or getting it twisted... FreeBSD has been there since the first Linux kernel, and in that time Linux distros have been churning and burning through their users. I seriously don't have time for that.
Anyway, testing is done and I'm happy so I'm off to do my official FreeBSD install now :-)

Re:I've made my peace with systemd

[rl117]

Sounds good... right up to the point when there's a problem. Then what happens? systemd notices that the log is corrupt, and... deletes it. No log of the problem. See: the long and angry (and unfixed) bugzilla tickets.

As for log compression, you don't want the actively written log to be compressed. If there's a problem, even as small as a single bit error, then the log will be unrecoverable. That's the tradeoff you make with compression. logrotate compromises by only compressing older logfiles. If there are any minor errors in the active log then you can still read it just fine.

As for tampering. It's of minor importance only. While the systemd people and their fanbois might harp on about it, they are catering for a problem which is of far less importance than hardware failure or power loss. Right now, all that foward hashing is so useless. If a simple power failure causes the checks to fail and the whole log to be discarded, it's a net negative. You threw away all my bloody logs! Like many of the systemd features, there's a whole fanfare about how essential it is and how everything else is awful and insecure. And as usual, there's some small credence to the claims. But it's massively overblown, and it has significant downsides. The "scary" things you mention are all of low likelihood--they only apply if your system has been compromised; there are rather more likely and less nefarious things to care about before these. I'd rather have a guarantee that my logs won't be deleted on a whim. Never, ever delete my logs!

Re:I've made my peace with systemd

[gweihir]

I have only modest experience working with *nix flavor boxes, and I fully understand the need for text based logs.

It is not a difficult insight to have. That is why somebody asking the question is either trolling or really has no experience with situations where you need the logs. In the second case it is very hard to enlighten them. You had to be there. In the case first they do not want to find out.

That said, you list some of the most important reasons to avoid binary logs. In particular the "easily damaged" aspect seems to be a real problem with systemd and it is an invitation to any attacker to just corrupt the logs instead of having to try to manually excise the traces of their break in. That makes binary logs with no resilience against corruption an exceptionally stupid design.

Let me just add one more problem (there are several more I can think of), because one of the bogus arguments for systemd is enterprise-readiness: Binary logs are not enterprise-compatible. In enterprise IT you need to process (read: add triggers for certain patterns), store and aggregate logs from a lot of different sources. The only way this can be made to work in practice is if you have line-oriented plain-text logs on which you can work with pattern-matching. Other things have been tried and universally failed. One reason is that the platform processing the logs is usually completely different from the multitude of platforms generating the logs. If you need some special tool to decode binary logs, you are usually screwed, because it may not even be possible to get that to run on the log analysis platform. Another reason is that binary file formats change and hence the are not robust. As soon as the log format changes a bit, you have to update all your analysis tools to be able to read them. With line-oriented plain-text logs you can usually just ignore the changes.

That said, on a meta-layer, binary logs in systemd are just one of the things that show that the systemd developers may be quite intelligent, but are inexperienced and have no appreciation for mistakes made long before by others and what the solutions to them were. Kind of like what Microsoft does. Making the same mistakes others did decades before is not good engineering. It is just arrogance coupled with incompetence. What the systemd developers lack is _wisdom_.

Re:I've made my peace with systemd

[gweihir]

Integrity checking is utterly worthless if availability is not ensured. An attacker on a systemd-binary-log box can just corrupt the logs or throw them away himself, no need even to fake a credible one. As systemd is known to do that from time to time by itself, this does not raise suspicion. On the other hand, credibly faking or completely excising traces of an attack from a text-log is quite a bit of work with a real risk of screwing up.

The level of non-understanding exhibited by this design is scary.

Re:I've made my peace with systemd

[gweihir]

As for log compression, you don't want the actively written log to be compressed. If there's a problem, even as small as a single bit error, then the log will be unrecoverable. That's the tradeoff you make with compression. logrotate compromises by only compressing older logfiles. If there are any minor errors in the active log then you can still read it just fine.

Excellent point and one that, again, requires just a bit of the respective real experience and understanding the systemd developers seem to be so sorely missing.

As to log-deletion on corruption, that is the most bone-headed stupid design. Flag it as corrupt, but let people read them! And keep them! My guess here is that the systemd developers are too inexperiences to come up with a robust binary log format, but are unwilling to go back to the sane choice, because they are too much in love with their creation. As to insecure, what is insecure is if you suddenly have a part of your of logs missing, but you have no clue whether an attacker deleted them, or whether systemd decided to have its 5 minutes of insanity. _That_ is insecurity.

Re:I've made my peace with systemd

[Anonymous Coward]

"Another thing, is the broken compatibility with what came before."

Try this:

# mkdir /mnt/whatever
# mount -t tmpfs -o nr_inodes=0,size=100m none /mnt/whatever

Let me know if that fails. It doesn't fail on _any_ of my non-systemd systems running similar kernel versions.

Re:I've made my peace with systemd

[gweihir]

You wish. But classical pro-systemd propaganda: systemd knows best and all other have to justify themselves, even on the most obvious and trivial things.

There is a large body of knowledge out there how to do these things right. Those that insist that body be quoted whenever somebody criticizes systemd are ignorant fools. And by the time-honored principle "if it is not broken, do not fix it" it is always the new thing that has to justify itself, not the old one that has been working for decades.

Re:I've made my peace with systemd

[Anonymous Coward]

Gotta keep them text files tiny.

Re:I've made my peace with systemd

[Anonymous Coward]

Systemd do only make token efforts to support NFS because devops cattle farms do not use NFS.

Same deal with how they do not support remote login on service errors, because said cattle farms use LOM setups.

You are expected to either be there 24/7 (desktop) or have dedicated hardware and software for remote management (devops cattle farm, aka IAAS).

Re:I've made my peace with systemd

[rl117]

I'm not sure what this has to do with NFS. Fails on Ubuntu 15.10 for me. But zero inodes doesn't look like a useful setting. Works with a positive count or if unspecified.

The NFS issue is likely it mounting the NFS filesystem before all the necessary RPC services are working, or something like that.

Re:I've made my peace with systemd

[KGIII]

I've poked at that distro (assuming it's the one in your signature). Back at the start of last year (2015) I finally decided to switch entirely to Linux. I'd always had Linux installed on a partition. In fact, I used to use Unix (AIX or Solaris, mostly) exclusively. Then, for whatever reason, I started using Windows and I was happy with it.

I noticed that my brain was turning to mush and I wasn't learning anything new. I was just passively consuming and learning little. As I age, I insist on learning new things - it's my way of keeping sharp. So, I tried pretty much every popular Linux distro on bare metal or in a VM.

IIRC, PCLinuxOS was a fairly small project but a fairly consistent one - like someone (or a few someones) kept working at it and just kept it going at a fairly constant rate. I'd read through a bunch of older posts, news articles, and whatnot. I'm pretty sure I'm thinking of the same one. They were fairly regular at keeping things up to date and they had a few dedicated users who were also keen on contributing. I kind of doubt that the project is large enough to where one person, let alone two, are able to make a living maintaining it to the exclusion of all other employment - but, I confess, I did not look into the financial information nor do I know if it is publicly available.

At any rate, I found that the group of users and maintainers seemed to be rather enthusiastic but the ecosystem was fairly small. Nothing seemed to be unfinished but it looked, hmm... Well, it looked unpolished. I'm not sure if there's a better way to express it.

I ended up going with the Ubuntu family or a derivative. I ended up with Mint Cinnamon and Lubuntu for the LXDE. I don't actually mind SystemD. I think I might have even sent the PCLinuxOS folks a small donation, even though I'd no plans on using their distro. Why? Heh... Well, to skip a novella - I enjoy sending something along to help people who are enthusiastic - even if I may never see any direct benefit from their work.

I suppose, I'll have some fresh hardware to be poking at (to keep me busy over the winter) and get to just play with it rather than having to rely on it., I might just have to give it another try. The hardware will be the hardware that was here at this house - I'm replacing it and putting Linux on it all so that it's here when I return. I'll then have that hardware, several years old, to poke and play with. I'll give it another shot and see if there's something I can do to make it work for me. It's always good to have multiple choices.

Re:I've made my peace with systemd

[walterbyrd]

I have switched to FreeBSD. There is a lot to like. But there are also weaknesses.

For example: FreeBSD seems to have poor support for NTFS. Also, FreeBSD does seem to be able to auto-mount a usb drive.

Re:I've made my peace with systemd

[Rutulian]

NFS filesystems fail to be mounted at boot. Why? I have no bloody clue. There's nothing in the logs.

Unlikely. If there was an error running the mount command, it was definitely recorded in the log. Did you use journalctl -u nfs or journalctl -b?

With sysvinit I could boot to an interactive shell in the late initramfs and step through every script by hand, and pinpoint exactly where something was failing

You can get a debug shell in systemd. The process is just a little bit different,
http://freedesktop.org/wiki/So...

With systemd, even with correctly configured systems, I still experience occasional unrecoverable hangs at boot as it screws up its nondeterministic boot ordering and waits forever on something or other (who knows what it might be).

I'm sorry, but that's bulls**t. If your system is randomly hanging, then it's not configured correctly. While the boot sequence of systemd can be characterized as non-deterministic, it is not random. If services are hanging because dependencies have not been met, then you need to specify your dependencies properly.

Another thing, is the broken compatibility with what came before. Example: I edit /etc/nfsidmap.conf to configure NFSv4 id mapping. Previously I'd reload/restart the nfs-common service with 'service' or 'invoke-rc.d'. Job done. What's the systemd equivalent?

Since you should be using a >3.5 kernel, nothing. The rpc idmapper has been replaced with the nfsidmap system call in the kernel. So there is no need to start/restart an idmapper service.

Yes, it's partly me lacking familiarity with the new way of doing things;

Understatement. Have you read any of the systemd docs or transition/setup guides available? There are a ton. Just do it.

Re:I've made my peace with systemd

[Rutulian]

systemd notices that the log is corrupt, and... deletes it

That's not what happens.

See: the long and angry (and unfixed) bugzilla tickets.

Which one? The one that says journald will rotate the log on detection of corruption, thereby preserving it in an un tampered state while still allowing the uncorrupted portion to be read?

Re:I've made my peace with systemd

[Rutulian]

Basically, you need to be able to read the logs with the most minimal of tools, because you are going to be diagnosing it in a downed state most likely-- You cannot bank on having a full suite of binary manipulation tools on hand. You will be lucky if you have more than vi.

This is the biggest load of canned bullsh**t argument. Where do you guys come up with this crap? We don't work on 1970s mainframes anymore. Why would vi be the only thing available? Why would an emergency shell not have the journalctl utility? Why would there not be serial logging capability? Why would you not be able to boot with a rescue disk (in case all other things fail)? The answer is, there is no reason. It is a made up scenario that doesn't exist.

Also, text based logs compress REAALLY well for long term storage for audit purposes! Binary logs? Probably not so much.

More bulls**t. There is no reason to think the journal won't compress well. Being binary, or not, has little if anything to do with compressibility. Remember that ASCII is itself a binary format! Also, auditing is one of the primary strengths of journald. It is designed specifically with auditing in mind.

if the binary logs are in some stupid "easily damaged" format, then having a process suddenly die horribly from abnormal termination will result in corrupt logs

That makes no sense at all. A process dying does not result in a corrupt log. There would be an error message, or at the very least a sigterm message, printed in the log just as it would be on the console if it had been run from an interactive terminal.

Hell, the file chain can be damaged from FS corruption, and parts of the log will still be readable.

The journal is designed to preserve as much of the log as possible when corruption happens. Anything written before or after the corruption event will be accessible by journalctl.

Re:I've made my peace with systemd

[Rutulian]

They could have added binary logging to any existing syslogd. Instead, NIH, because Poettering.

If binary logging was the sole intention, then yes. But, as you know, the journal does a lot more than just a binary output of rsyslog, and would not be easily achieved by patching rsyslog. One can argue that those features are unnecessary, but that is a different subject.

The normal adoption process for something like systemd would be to roll a new distribution using it, and then once it is tested, debugged, and proven, for it to propagate throughout the community.

Well, systemd was being used by Fedora and Arch for quite some time. RHEL was on the way to adopting to it already. Ubuntu already had Upstart, so maybe they were forced into systemd by the upstream changes, but if you ask me, I think systemd is an improvement over upstart. It sounds to me that enough people liked it that distributions started adopting it. Maybe it was a faster change than you would have liked, but it is not a conspiracy. It comes down to who is doing the work to maintain the distributions and dependencies, and they decided that systemd was the way forward. An alternate way may be shown to us by Gentoo. We'll see, but I doubt there will be much traction.

Re:I've made my peace with systemd

[Anonymous Coward]

It has nothing to do with NFS. It has everything to do with systemd coming along and poorly reimplementing parts of coreutils and friends.

"But zero inodes doesn't look like a useful setting. Works with a positive count or if unspecified."

So, I gather that you tried it and it failed? nr_inodes=0 (and nr_blocks=0 and size=0) is a *very useful* special value that has been documented for at *least* a decade. https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt (search for nr_blocks=0 and read the paragraph.) I expect that the Systemd Cabal thought as you did and made it an error to attempt to use tmpfs in such an configuration... rather than reading the kernel documentation to see if their assumption was wrong.

I've had reason to make use of this option when storing *very many* small files on a small tmpfs volume. It's niche, but necessary.

Re:I've made my peace with systemd

[drinkypoo]

It comes down to who is doing the work to maintain the distributions and dependencies, and they decided that systemd was the way forward.

Well, no. At least in the case of Debian, a minority of those people rammed the change through at a convenient moment. I don't know what it looked like in other distributions, but I do know about that one. Much of the internal community didn't want it, and virtually none of the external community, but they did it anyway.

Re:I've made my peace with systemd

[sjames]

In my case, I need my desktop to resemble a server because that's where I do my initial testing in rapid fire fashion. But since it's also a desktop, that means it needs to be a server with X and a suitable environment. That's why I object to the systemd cancer infecting desktop environments. If the DEs would just be sane and not depend on systemd OR if systemd would play nice and truly support picking and choosing bits of it, I wouldn't mind so much.

Fortunately, there remain non-Gnome DEs that work without systemd. Of course, the systemd people are advocating for new dependencies as hard as they can.

Re: I've made my peace with systemd

[sjames]

Something needs to respond to the syslog calls

Yes, and some people want that to be the system logger. They don't want to play telephone with journald standing in the middle. It's just one more point of failure.

Re:I've made my peace with systemd

[sjames]

Given that systemd's whole claim to fame as an init is that it automagically takes care of boot dependencies, that's a pretty big fail.

Re:I've made my peace with systemd

[sjames]

A lot of people with long experience regularly dump their databases into CSV files to maximize their odds of meaningful recovery of the data should something go wrong.

Re:I've made my peace with systemd

[rl117]

https://bugs.freedesktop.org/s... is certainly one of the issues. But there are other situations which *do* cause log data loss. Google shows up plenty with little effort.

Re:I've made my peace with systemd

[Rutulian]

Did you actually read that report? The very first comment:

The only way to deal with journal corruptions, currently, is to ignore them: when a corruption is detected, journald will rename the file to .journal~, and journalctl will try to do its best reading it.

Further down in comment #3, some more detail:

Journal files are mostly append-only files. We keep adding to the end as we go, only updating minimal indexes and bookkeeping in the front earlier parts of the files. These files are rotated (rotation = renamed and replaced by a new one) from time to time, based on certain conditions, such as time, file size, and also when we find the files to be corrupted. As soon as they rotate they are entirely read-only, never modified again. When you use a tool like "journalctl" to read the journal files both the active and the rotated files are implicitly merged, so that they appear as a single stream again.

Now, our strategy to rotate-on-corruption is the safest thing we can do, as we make sure that the internal corruption is frozen in time, and not attempted to be "fixed" by a tool, that might end up making things worse. After all, in the case the often-run writing code really fucks something up, then it is not necessarily a good idea to try to make it better by running a tool on it that tries to fix it up again, a tool that is necessarily a lot more complex, and also less tested.

Now, of course, having corrupted files isn't great, and we should make sure the files even when corrupted stay as accessible as possible. Hence: the code that reads the journal files is actually written in a way that tries to make the best of corrupted files, and tries to read of them as much as possible, with the the subset of the file that is still valid. We do this implicitly on every access.

(emphasis added)

So, in other words, there can be journal corruption, as there can be corruption in any data file. The way journald deals with that is by rotating the log, so that a new log can start in its place, and the contents of the old log preceding the corruption are preserved. The journalctl utility knows how to read the corrupted files and extract useful information from them.

Google shows up plenty with little effort.

Google shows up people ranting about something they don't understand. The way journald handles corruption is probably the best possible way any software can handle corruption.

Re:I've made my peace with systemd

[dave420]

Your position can be summed up in precisely the same way - "knows best and all other [sic] have to justify themselves, even on the most obvious and trivial things". Seriously. Just accept that some people like to do thing some ways, and others other ways. Should you find yourself in a gradually-disappearing minority, you might want to take the hint that your way of doing things is on its way out, and adapt. It's happened with every technology or way of doing things which was one common-place but now relegated to the archives and tales passed from generation to generation of admins. Technology comes and goes. Stamping your foot, making massive generalisations, and seeing things in very "black and white" terms is a sure-fire sign that you are not using your head in the argument, but your heart. Arguing will not get someone to feel the same emotion about something you hold dear, so stop. Just stop.

Re:I've made my peace with systemd

[dave420]

Why don't you post some of those links, Mr. Lazy-Pants? :)

simple list here..

[Anonymous Coward]

block windows 10 upgrades on all windows boxes.

turn off 'recommended' updates ('important' only) on those same systems. that's mostly how the upgrade and telemetry shit gets in atm.

that's it.

make peace?

[Gravis Zero]

systemd was thrust upon everyone and you want us to accept it just because? how about this: document the code, document the interfaces, solidify the ABI, make the code portable, do a security audit instead of trying to force me to use systemd!

my resolution is to uproot systemd's tendrils from an otherwise decent operating system.

Re:make peace?

[AmiMoJo]

Security audit systemd... Because init scripts are already secure and we wouldn't want to replace them with something that hasn't been audited!

Re:make peace?

[Anonymous Coward]

That's so 2015. As long as you avoid the recent redhat derivatives, debian, ubuntu and a few other less important efforts you can leave them with the brand and migrate with the good will. When their left holding just the name they'll donate the rest to apache. What it comes down to is that their just not our kind of people.

Re:make peace?

[phantomfive]

The problem is the systemd team doesn't understand a good way to write portable software.

Re:make peace?

[Anonymous Coward]

No, they understand perfectly ... THEY DO NOT WANT PORTABLE SOFTWARE !
SystemD is a Trojan Horse - it destroys Linux ... AND weakens FOSS -
Once Linux programs become dependent on SystemD, they
ARE NO LONGER EASILY PORTABLE !
This suits M$ (and others) PERFECTLY - no more challenges to
M$ Office on Windows, because Linux stuff WON'T RUN on WinDoze ANYMORE !! :-(

Help stamp out SystemD !!

Re:make peace?

[rl117]

Seriously, have you even thought about this in any depth?

What's more safe and secure, many tens of thousands of lines of C running in PID0 with root privs, or interpreted scripts each running as root or with reduced privs in their own separate process?

It's obviously the latter. The only compiled code is the shell interpreter, which is well tested and used all over the place with root privs already. And the shell scripts being run are trusted. Now, you can argue all the other points about the downsides of shell vs unit files. But as for security, this one is plain and obvious. All that C is just dying for a trivial buffer overflow or some other standard exploit to allow a full compromise of the system from some valid or even user-supplied untrusted input. Yes, all that code is long overdue for a thorough audit; we're living on borrowed time as it is. How long to you think it will be until the first major exploit surfaces? Maybe then we'll have a rethink about the appalling design practices the systemd "geniuses" have foisted upon us. Having so much code running in the most critical and vulnerable part of the system is idiocy of the first order.

Re:make peace?

[Anonymous Coward]

What's more safe and secure, many tens of thousands of lines of C running in PID0 with root privs, or interpreted scripts each running as root or with reduced privs in their own separate process?

Yep, because clearly one cannot run C code with reduced privs and systemd doesn't make use of any features to do so (like for example NoNewPriv or syscall filtering or network namespaces or private /tmp or ...) and makes it easy to enable these features for other daemons too (though syscall filtering is by its nature not quite so trivial to set up).

Re:make peace?

[phantomfive]

How long to you think it will be until the first major exploit surfaces?

Looks like it's already happened.

Re:make peace?

[Anonymous Coward]

I have made my peace with systemd. It is running on my laptop that came install with Ubuntu and I rebuilt with Debian - the system I've been using for 15+ years. Then came systemd. My peace came from a new Gentoo install that I'm warming up to before making it my primary solution. I regret that we've come to a parting of the ways, but I did not choose the new path of Debian and many of the distributions. At least Gentoo still allows me to work around it for now.

Re:make peace?

[Gravis Zero]

The problem is the systemd team doesn't understand a good way to write software.

ftfy. ;)

Re:make peace?

[sjames]

Init scripts manage that by not presenting much of an attack surface. They only run briefly and don't typically create a listening socket. The vast majority of them have already exited by the time that even local interaction with the system is possible.

Re:make peace?

[ookaze]

Seriously, have you even thought about this in any depth?

What's more safe and secure, many tens of thousands of lines of C running in PID0 with root privs, or interpreted scripts each running as root or with reduced privs in their own separate process?

It's obviously the latter.

Dangerous people like you must be discredited at all costs before young sysadmins believe your stupid sentence.
The Linux kernel proves you wrong, and obviously, the shell interpreter used to interpret your shell scripts proves you wrong too.
You're clearly not qualified to talk about this matter, and the obvious answer to your question is the opposite of the one you've given.
Shell scripts have always been unsecure, the countless workarounds that you can even find in the Linux kernel to try to mitigate the security nightmare that are shell scripts should be well understood by most sysadmins, or you're in for some surprises when you'll be a bit more seasoned.

The only compiled code is the shell interpreter, which is well tested and used all over the place with root privs already. And the shell scripts being run are trusted. Now, you can argue all the other points about the downsides of shell vs unit files. But as for security, this one is plain and obvious. All that C is just dying for a trivial buffer overflow or some other standard exploit to allow a full compromise of the system from some valid or even user-supplied untrusted input.

Oh man, you even contradict yourself, but you choose the path of nonsense anyway. Shell scripts most of the time (every time in distributions) need countless other binaries besides the shell interpreter, every single execution of which can be exploited, for example by changing the environment, even with race conditions, sth that is impossible to resolve with shell scripts if you need external ressources or even to write sth. Rootkits are very happy with initscripts actually, but they can't cope with systemd for now, if they ever can. Most of the dynamic state of Linux (disks, network, ...) need binaries anyway to be handled.
The only case when shell scripts work are with a well known customized static setup. Which breaks as soon as you change sth. Been there, done that.
Debian and its famous ifup/ifdown that never worked and will never work is another example: how many new sysadmins locked themselves out of their distant boxes, thinking that they could dynamically change their network configuration?

Yes, all that code is long overdue for a thorough audit; we're living on borrowed time as it is. How long to you think it will be until the first major exploit surfaces? Maybe then we'll have a rethink about the appalling design practices the systemd "geniuses" have foisted upon us. Having so much code running in the most critical and vulnerable part of the system is idiocy of the first order.

Systemd has already been audited and continues to be. Far more than sysvinit has ever been, and initscripts have never been audited.
A lot of initscripts are even full of very dangerious bugs or clearly unsecure, and can't be made secure because then they break and don't work anymore.
Most are impossible to secure. These initscripts need a sysadmin knowing their behaviour and constantly monitoring them to ensure they are actually running, as they just don't work.
No wonder every distro fled as fast as they could from initscripts.

Why is this blog spam on Slashdot?

[Anonymous Coward]

I have a hard time taking anything seriously when some of the major points consist of "I will start to use strong passwords and secure authentication methods" and "I'm not doing to use Godaddy to register domains anymore".

All the way

[Anonymous Coward]

1440P all the way, baby.

That's my resolution for '16.

all on my Raspberry Pi!

[Anonymous Coward]

using an rpi2 for my "desktop" machine. it does everything I need at a "good enough" speed. no systemd.

running all kinds of python and C code, lots of great apps including arduino development and ham radio VoIP stuff, and embedded hardware fun toys...

can't beat it! 2016 is the year of the RPi Desktop!!

(captcha homeless !!! how does it know I am living at my Mom's!??!)

BOFH

[Anonymous Coward]

Shouldn't there be a BOFH link here?

http://bearbin.net/bofh

Get paid.

[Anonymous Coward]

Waiting a week or two before moving on to unemployment. Was a great learning experience, no doubt about that.

Why would you make peake with the enemy?

[gweihir]

In particularly one that is trying to destroy all that is good and proper like systemd? Making peace with it would be stupid!

Punch in the face anyone who uses the word "rig"

[Anonymous Coward]

Seriously, I hate that shit. Your computer is not a "rig".

Just skip it

[Anonymous Coward]

Sysadmins "embrace" new stuff when *all* of their ecosystem runs the new stuff. Otherwise they are just multiplying the amount of their work. The "ip" command is for Linux as is "ss", and I'm quite sure not all *nixes run SystemD. People don't upgrade just because it's fun, that's why we are still seeing Python 2 in default distros. Upgrade to Python 3 would be "all the trouble for no gain", i.e. "if it works, don't fix it".

certifications

[Skapare]

... are not in my future.

Complete the deprecation and removal of systemd.

[Anonymous Coward]

Migrate from any distribution that has become a redhat derivative.

2016 the year of systemd free infrastructure.

2016 will be a much better year that 2015 for that alone.

wierd_w: You're "A-OK" by APK... apk

[Anonymous Coward]

See subject & this http://it.slashdot.org/comment... You're very fair (I hope you're always like that, YOU are like how /. used to be, fair & analytical - not trolling illogic logic bs!)

APK

P.S.=> That MAY also go for a few others here, being like wierd_w (cool) - others? WHY DON'T THE FUCKING DO NOTHING CRITICS try do BETTER than systemd?? THEY'RE BIG TALKING "ne'er-do-well" & troll shills largely, some are Garth & Wayne "WE FEAR CHANGE" types who can't adapt is another and DON'T HAVE THE SKILL TO EVEN BEGIN TO TRY in coding which minus coders? THEY WOULD BE HELPLESS - we make the code for them to USE, as "users with a better password" who use coder's work YET HAVE THE BALLS TO PUT IT DOWN minus having any to show for themselves... truer words were NEVER spoken on /., bank on it!

(SystemD: I see the shit they take here & whom I feel for actually AND respect - it's NOT EASY bucking the system, & one that's SO RIGGED to help the profits of the puppetmasters of sockpuppets PAID to further their agendas vs. better products they're afraid of (don't TRY tell me it doesn't go on either - it IS reality & anyone who doubts that? I can EASILY prove it & back it from reputable enough sources))... apk

Peace

[Anonymous Coward]

No.
Get rid of systemd.
My resolution for 2016

Resolve to be more open minded.

[jellomizer]

I am truly curious on how many systemd trolls would really know if their system had it or not, if it weren't for all they hype about it.

Re:Resolve to be more open minded.

[Anonymous Coward]

The most vocal opponents of systemd (whom you mislabel as "systemd trolls") tend to be professional system administrators and professional software developers with decades of experience.

Many of them have been responsible for hundreds of thousands of production servers and desktops. They've used more operating systems than you could probably count. They've dealt with nearly as many different init systems.

These decades of experience has taught them a thing or two. They've come to learn what works, and what doesn't.

Well, it turns out that systemd embodies just about everything that's known not to work well! It's rife with architectural problems, for example. Binary logging is a big no-no. Trying to do absolutely everything, and doing it all very poorly, is a big no-no. Not following the proven UNIX philosophy is a big no-no. These aren't just minor bugs that can be fixed with some code changes. These are deep flaws that require systemd to be thrown away.

It's no wonder we see Linux distribution mailing lists and bug trackers filled with so many reports of problems caused by systemd. Systemd has thrown out decades of accumulated knowledge and experience, and replaced it with approaches that are known to not work.

Systemd has caused a lot of inexcusable problems for a lot of its victims. We're talking about things as serious as Linux installations that don't boot properly. That's one of the worst things that can happen to a Linux installation; it renders it unusable!

Instead of labeling these professionals as "systemd trolls", you should be thankful that they've done the right thing and started moving away from Linux. Many of them have, or are in the process of, converting the systems they're responsible for from Linux to FreeBSD or OpenBSD. They just can't let their critical systems fall victim to systemd and the problems it has been shown to cause.

Re: Resolve to be more open minded.

[Anonymous Coward]

+1 thank you. #rageagainstsystemd

Re:Resolve to be more open minded.

[Sarten-X]

The most vocal opponents of systemd tend to be unprofessional system administrators and architects from an unrelated field, with decades of experience on unrelated systems.

Many of them have been responsible for configuring hundreds of thousands of systems in a non-systemd manner. The biggest variation in their system experience is usually whether they use pacman, apt, or yum. They've only dealt with one, or maybe two init systems.

These decades of experience have made them unable to consider anything different. They've come to reject anything that doesn't fit squarely in their particular interpretation of the Unix Philosophy.

Well, it turns out that it's extremely easy to mischaracterize systemd. Its architecture rejects the knee-jerk assumptions that we've used for so long. Rather than a "simple" storage backend, logs are viewed with a tool that filters down to relevant information. Systemd presents minimal interfaces to everything, intending them to be overridden with the usual system tools. It emphasizes the "do one thing and do it well" concept, where the "one thing" is "integrate programs into a coherent system". Systemd has different answers to the underlying question of "how do we turn a kernel into a productive system", that require sysadmins to work from a different perspective.

It's no wonder we see Linux distribution mailing lists and bug trackers filled with so many reports of problems caused by ignorant sysadmins. As a different approach to bringing up a system's components, when sysadmins blindly try to use their irrelevant experience and poor guesses to configure the system, things don't work.

Those poor sysadmin habits have caused a lot of inexcusable problems for a lot of their victims. We're talking about things as serious as Linux installations that don't boot properly. That's one of the worst things that can happen to a Linux installation; it renders it unusable!

Instead of taking the professional approach of understanding the new system prior to judging it, these sysadmins have blamed systemd. Many of them have, or are in the process of, spreading blatant lies and myths about systemd to distort public opinion. They just can't bring themselves to accept that a new perspective might ultimately serve their needs just as well, if not better.

Re:Resolve to be more open minded.

[Anonymous Coward]

You pro-systemd people come off as really crazy.

The GP provided us with well-reasoned, factual arguments.

You just made shit up.

What you're saying isn't even plausible.

You come off looking like a lunatic!

Re:Resolve to be more open minded.

[Anonymous Coward]

"a non-systemd manner."

That says it all. You're actually saying "Our religion is the only true religion". The rest of your post is equally egotistical and shallow.

You're attempting to claim the systemd problems are the fault of professional sysadmins with decades of experience? That's laughable. Don't be surprised if people don't take you seriously.

I for one am quite familiar with systemd and the "systemd approach". I have many hours of professional experience with it and alternatives and your claim that it is vastly superior is one of those porkies.you are so fond of laying on others. It's a mediocre experience at best with lot's of flat-out incorrect error messages, disorganized documentation (quantity does not equal quality and critical information is missing), unreliable booting (failed NFS mounts anyone?), heisenbugs (it's dependency management works, sort of, but there's a lot of handwaving going on), badly designed and inconsistent configuration languages, inconsistent use of directories, symbolic links and naming etc. In fact most of the things you are so fond of claiming of the alternatives.

The failure of egostical people like yourself to even admit you have a problem is just the icing on the cake.

Re:Resolve to be more open minded.

[Anonymous Coward]

Like you said, they tend to be useless old fucks set in there ways.

The world will be better when they are gone.

Linux gaming rig

[PsyMan]

LOL, Linux and Gaming Rig in the same sentence, who'd have thunk it eh. Microsoft should also port excel to the playstation 4 and Xbone so they will obviously take off in the financial industry as a direct replacement for Windows based PC's.

Re: Complete the deprecation and removal of system

[Anonymous Coward]

I am moving to antiX

systemd?

[Anonymous Coward]

Fuck you!

5# - Learn Go lang, swift and android programming

[Anonymous Coward]

Agree with android, but disagree with Go and Swift. They are not even close to "everywhere" and I am skeptical of more demand.

My resolution.

[Anonymous Coward]

Become a developer.

Better hours, astronomically higher salary. No need to deal with the fact that the old breed is retarded (your init scripts are shit and so is your face) and so is the new breed (binary logs are fucking stupid and so are you).

Re:My resolution.

[phantomfive]

{U|LI}

Re:My resolution.

[fahrbot-bot]

You realize that I was joking - right?

Re:My resolution.

[phantomfive]

Nah, I'm too tired to realize

Not "old" vs "new"

[walterbyrd]

So called "new" culture is just Red Hat trying to force their proprietary crap on everybody by name-calling and shaming.

MS does the same thing. The entire systemd scam is right out of MS's playbook.

Redundant

[PPH]

making peace with systemd, .... building Linux gaming rig

My resolution.

[fahrbot-bot]

New Year's Resolutions For *nix SysAdmins

After 30 years as an admin and systems programmer, finally find out what that damn asterisk stands for.

Shit list

[Anonymous Coward]

2: not everything needs to be forced to use a system insecure by design
5: golang is shit
6: buzzwords
7: disgusting
10: non-free software
11: old tools just work

My new years resolution...

[Anonymous Coward]

Disabling ssh for root and forcing the other admins to use thier own admin accounts instead of service accounts!

Simply Linux has no future

[t8z5h3]

Basically the idea of freedom in the o.s. Is dead because we build Linux as programmers not thinking about ease of use or support, we tree and split the code base between every flavor of Linux yet the code is tied to a disbusion then we fight over every little stupid thing unless there is agreement Linux is a failure as a network admin I want a turn key system that I plug a IP address in do some basic tranning with my staff and logs everything so I can watch it as it works away addressing things as they come up and is supported by staff of the product.

DNSSEC

[RR]

I strongly disagree with his recommendation for DNS. That’s because I want to spread DNSSEC.

The problem with services like Amazon Route 53 is they generate DNS records dynamically. That means they need the signing key to be online, on the DNS load balancer, and they don’t bother to do so. If you really need your DNS to be globally distributed (How many people actually look for your domain, anyway? How many times is the answer cached on Google public DNS already?), you should look into CloudFlare. CloudFlare uses a custom implementation of ECDSA to decrease the cost of DNSSEC signatures, making it practical to do online signatures and also very effective NSEC white lies.

Sex. Lots and lots of sex.

[Anonymous Coward]

Windows and systemd are much easier to cope with if you're getting laid.

Of course, frequent sex won't make Wndows/systemd any better, but you'll care less about how awful they are.

getting back in?

[Anonymous Coward]

I have 20 years of *nix sysadmin experience including several flavors of Linux as well as the usual Solaris and other *nix varieties. I got laid off several years ago during the Great Recession and took time to do other stuff. Now I'd like to get back in, but the employment gap since my last sysadmin job is apparently not helping so I've been taking install/repair contracts for phone companies to keep busy. While that's not a bad gig if the pay is good enough, the job does have its physical demands and I only have so many years left doing it. So ... as a New Year's Resolution, I want to figure out how to restart my sysadmin career. What could I do to make myself hireable? Volunteer on an open source project? Which one is looking for sysadmin types? Take a low level/mid level contract position? Learn systemd? Anything else?

go to hell systemd

[Anonymous Coward]

go to hell systemd made you die a painful horrible death!

My resolution

[Anonymous Coward]

This year I'm learning all about the BSDs and dropping Linux. It has become something I no longer want to use.

Pfft

[Anonymous Coward]

Fuck systemd.

there are no sysadmins...

[Anonymous Coward]

only devops.

systemd killed the linux server; long live *BSD

[dltaylor]

No competent administrator would run something as arcane, unreliable, and fragile as systemd on a server, given any sort of choice.

Goals for 2016?: remove those last few linux boxen and migrate the services to *BSD (Open is my choice, but it does have some lag on drivers; have to brush up on writing those, I guess).

Re:Complete the deprecation and removal of systemd

[Anonymous Coward]

2016 the year of systemd free infrastructure.

Good luck with your Enterprise GNU/Hurd installation..

Don't let the door hit you on the way out...

[Dawn Keyhotie]

... All you systemd haters. Flock to your precious *BSD variant, whichever you think will get more than a smidgen of server market share.

Wake me up when you can run Docker or rkt containers on any BSD. Or when there is any major outage that is attributable to systemd. Or when you can find a job at any hosting provider or IaaS vendor that doesn't require strong Linux credentials.

Just a bunch of whiny babies who spam any forum anywhere with their anti-systemd vitriol when any mention of systemd is made, no matter how tangential it may be to the discussion at hand.

Make peace or make tracks, just please shut the hell up already!

New SysAdmin duties

[Anonymous Coward]

>> building Linux gaming rig
Shit wot, building gaming rigs is now SysAdmin territory wholly smokes batman when did I miss this one???