Slashdot Mirror

It's already there, as systemd-timesyncd. http://www.freedesktop.org/sof...

...the bug that prevents me from having accelerated graphics in Linux, I'd be among that 1%. Until then? Reboot... reboot.... reboot... reboot...

1. my impression is that systemd is just more transparent about it, because syslog just drops the corrupted log entries silently.
2. that is just wrong. journald just rotates the log files, nothing is lost. journald shows all partials log files together as one journal:

http://www.freedesktop.org/sof...

If called without parameters, it will show the full contents of the journal, starting with the oldest entry collected. [...] Output is interleaved from all accessible journal files, whether they are rotated or currently being written, and regardless of whether they belong to the system itself or are accessible user journals.

So now you are going from saying that people do not use DB's for logging unless they are clueless; (Let me quote you on this:)

And yes, I have extensive experience with log analysis. You do not turn the logs into a database unless you are fully clueless.

..to that DB's can be a good thing?
And please remember that ACID doesn't give any protection against file corruption, nor does it prevent user space programs corrupting the logs by generating obviously impossible field values. But then again, neither does the journal nor plain text logs.

Again, for those who need to run a full DB for their logs, systemd's binary file format is great, because it allows the journal to be exported in industry standard formats like JSON. That way the remote log-sink database can receive and store rich meta data in a totally stable and structured way; changing hostnames, IP's, or even different wordings or even different and changing languages used by the daemon log output isn't a problem with the journal, since it is based on field values, not complex regex-ing of unstructured, undocumented, unstable, language specific words.

Oh, and tamper proof, cryptographically "sealed" logs too (FFS) if you want that.

If your remote log-sink solution isn't a full DB, you still gets all the benefits of receiving structured log entries with eg. full microsecond precision timestamps _and_ monotonic timestamps. It is trivial to convert eg. JSON output with defined field names to any other structured format, while converting and aggregating unstructured text files is a pain.

It is also great for those who only needs local logging; the journal has many of the advantages of a full DB, but without the complexity and overhead. Its append based file format is also much more robust against file level corruption than databases. Since the log files are structured and indexed with field values, they are easy to perform powerful, yet simple queries on.

How do you find all syslog entries with the priority level "error" generated by the previous boot only?
With the journal it is : "journalctl -b -1 -p err"

And how do you generate a full list of every executable, including their path?

Since you can "tab" trough the values in the journal this can easily be done: "journalctl -F _EXE "

And with the -x switch, the help database is activated, giving further explanation on what the log entry means, and gives direct links to upstream support, perhaps linking directly to a page that explains the error code etc:

Example:
# journalctl -b -x -u systemd-logind.service

mar 07 16:46:16 localhost systemd-logind[546]: New session 1 of user Peter H.S.
(log entry above, help database output below:)
-- Subject: A new session 1 has been created for user Peter H.S
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/m...
-- Documentation: http://www.freedesktop.org/wik...

There are seriously many people who just trawls through long log files (even using vi apparently), because it is hard to grep for anything unless you already know what to grep for, and because regex'es are well, regex'es; difficult to use, understand and to remember all the many variations. Basically, newbies can't read or filter Linux syslog log-files by other means than trawling through them.

They can't have a useful GUI either because it is impossible to make a distro-agnostic syslog gui. Again, a problem that systemd's journal solves.

The bottom line is, that the only virtue syslog files have, namely that they are human readable, is a serious hindrance for their use too: they can't add monotonic timestamps and micro-precision timestamps and other meta-data without being excessively difficult to read for humans.

With the journal you can give machine parsers exactly the structured log info they need and can benefit

So now you are going from saying that people do not use DB's for logging unless they are clueless; (Let me quote you on this:)

And yes, I have extensive experience with log analysis. You do not turn the logs into a database unless you are fully clueless.

..to that DB's can be a good thing?
And please remember that ACID doesn't give any protection against file corruption, nor does it prevent user space programs corrupting the logs by generating obviously impossible field values. But then again, neither does the journal nor plain text logs.

Again, for those who need to run a full DB for their logs, systemd's binary file format is great, because it allows the journal to be exported in industry standard formats like JSON. That way the remote log-sink database can receive and store rich meta data in a totally stable and structured way; changing hostnames, IP's, or even different wordings or even different and changing languages used by the daemon log output isn't a problem with the journal, since it is based on field values, not complex regex-ing of unstructured, undocumented, unstable, language specific words.

Oh, and tamper proof, cryptographically "sealed" logs too (FFS) if you want that.

If your remote log-sink solution isn't a full DB, you still gets all the benefits of receiving structured log entries with eg. full microsecond precision timestamps _and_ monotonic timestamps. It is trivial to convert eg. JSON output with defined field names to any other structured format, while converting and aggregating unstructured text files is a pain.

It is also great for those who only needs local logging; the journal has many of the advantages of a full DB, but without the complexity and overhead. Its append based file format is also much more robust against file level corruption than databases. Since the log files are structured and indexed with field values, they are easy to perform powerful, yet simple queries on.

How do you find all syslog entries with the priority level "error" generated by the previous boot only?
With the journal it is : "journalctl -b -1 -p err"

And how do you generate a full list of every executable, including their path?

Since you can "tab" trough the values in the journal this can easily be done: "journalctl -F _EXE "

And with the -x switch, the help database is activated, giving further explanation on what the log entry means, and gives direct links to upstream support, perhaps linking directly to a page that explains the error code etc:

Example:
# journalctl -b -x -u systemd-logind.service

mar 07 16:46:16 localhost systemd-logind[546]: New session 1 of user Peter H.S.
(log entry above, help database output below:)
-- Subject: A new session 1 has been created for user Peter H.S
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/m...
-- Documentation: http://www.freedesktop.org/wik...

There are seriously many people who just trawls through long log files (even using vi apparently), because it is hard to grep for anything unless you already know what to grep for, and because regex'es are well, regex'es; difficult to use, understand and to remember all the many variations. Basically, newbies can't read or filter Linux syslog log-files by other means than trawling through them.

They can't have a useful GUI either because it is impossible to make a distro-agnostic syslog gui. Again, a problem that systemd's journal solves.

The bottom line is, that the only virtue syslog files have, namely that they are human readable, is a serious hindrance for their use too: they can't add monotonic timestamps and micro-precision timestamps and other meta-data without being excessively difficult to read for humans.

With the journal you can give machine parsers exactly the structured log info they need and can benefit

contain rich meta-data....

What a load of claptrap. People log to log files for reasons of the lowest common denominator. We have things called 'databases' for this kind of stuff and there's perfectly good reasons we don't use them for logging which should be obvious to anyone with an ounce of sense.

Really, are you unaware how common it is to aggregate log files in databases? This is a major selling point for Rsyslog that it is able to do so.
In fact, Rainer started Rsyslog exactly in order to overcome the many deficiencies with syslog(3), including the severe limitations of unstructured text files.

systemd's journal is pretty much a stroke of genius in that it overcomes all the limitations of unstructured, unindexed text logs, while not being a full blown DB either (the journal files are basically appended text files with a different line delimiter and an index in front).

Log files keep on growing every year, because people log more and more, and systems are running more and more daemons. Analysing such logs means machine parsing, and structured and indexed log formats like the journal have a huge advantage for this kind of work.

Since the journald binary logging file format is stable and fully documented....

I sincerely hope Red Hat and Lennart is paying for this piece of contrived PR.

Ah, so now facts and reality is propaganda. Take a look here
http://www.freedesktop.org/wik...
and here (about the file journal file format)
http://www.freedesktop.org/wik...

The arguments against it are based on contrived scenarios

Corrupted logs on a perfectly running system isn't a contrived scenario.

It is when the corruption means squat for the ability to actually read the logs.

For some reason you seem to think that "corruption" discovered by "journalctl" means the logs are unreadable, but as explained, they are often marked "corrupted" just because a single field value in a single log entry was discovered as impossible. This doesn't mean that the log file can't be read.

....through the Linux/Unix concept of piping.

Piping eh? Wow. You give the impression you haven't heard of it before ;-).

I don't care what you think, as long as you agree that the whole notion of not being able to use standard Linux text tools like grep together with systemd's journal is just plain wrong and a total non-issue.

There just aren't any good arguments against structured, indexed log files that can be programmatically accessed and has rich meta-data.

contain rich meta-data....

What a load of claptrap. People log to log files for reasons of the lowest common denominator. We have things called 'databases' for this kind of stuff and there's perfectly good reasons we don't use them for logging which should be obvious to anyone with an ounce of sense.

Really, are you unaware how common it is to aggregate log files in databases? This is a major selling point for Rsyslog that it is able to do so.
In fact, Rainer started Rsyslog exactly in order to overcome the many deficiencies with syslog(3), including the severe limitations of unstructured text files.

systemd's journal is pretty much a stroke of genius in that it overcomes all the limitations of unstructured, unindexed text logs, while not being a full blown DB either (the journal files are basically appended text files with a different line delimiter and an index in front).

Log files keep on growing every year, because people log more and more, and systems are running more and more daemons. Analysing such logs means machine parsing, and structured and indexed log formats like the journal have a huge advantage for this kind of work.

Since the journald binary logging file format is stable and fully documented....

I sincerely hope Red Hat and Lennart is paying for this piece of contrived PR.

Ah, so now facts and reality is propaganda. Take a look here
http://www.freedesktop.org/wik...
and here (about the file journal file format)
http://www.freedesktop.org/wik...

The arguments against it are based on contrived scenarios

Corrupted logs on a perfectly running system isn't a contrived scenario.

It is when the corruption means squat for the ability to actually read the logs.

For some reason you seem to think that "corruption" discovered by "journalctl" means the logs are unreadable, but as explained, they are often marked "corrupted" just because a single field value in a single log entry was discovered as impossible. This doesn't mean that the log file can't be read.

....through the Linux/Unix concept of piping.

Piping eh? Wow. You give the impression you haven't heard of it before ;-).

I don't care what you think, as long as you agree that the whole notion of not being able to use standard Linux text tools like grep together with systemd's journal is just plain wrong and a total non-issue.

There just aren't any good arguments against structured, indexed log files that can be programmatically accessed and has rich meta-data.

Its out according to the Mesa-dev mailinglist and are available at the regular FTP. They probably just haven't gotten arround to publish a news item on their site.

Its out according to the Mesa-dev mailinglist and are available at the regular FTP. They probably just haven't gotten arround to publish a news item on their site.

Here is one of many:

https://bugs.freedesktop.org/s...

The TL;DR resolution: journalctl --fsck after rotation or pump it into a traditional syslog

Lennart Poettering's comments about it:

"our strategy to rotate-on-corruption is the safest thing we can do, as we make sure that the internal corruption is frozen in time, and not attempted to be "fixed" by a tool, that might end up making things worse"

Totally sensible solution, why would you edit log files? They are not meant to be changed.

Besides that, the log entries that are mangled are in them self important info; How can you detect a pattern of a misbehaving program that sends wrong field values if you delete the entry every time it happens?
Or what if the log corruption is caused by a failing disk where the bad blocks are spreading like wildfire? If you silently delete such entries and files, you delete the evidence for that something is wrong.

read this, it might help http://www.freedesktop.org/sof...

Here is one of many:

https://bugs.freedesktop.org/s...

The TL;DR resolution: journalctl --fsck after rotation or pump it into a traditional syslog

Lennart Poettering's comments about it:

"our strategy to rotate-on-corruption is the safest thing we can do, as we make sure that the internal corruption is frozen in time, and not attempted to be "fixed" by a tool, that might end up making things worse"

1) Vulkan does away with OpenGL state so it's easy to wrap.
2) It exists, see: piglit ( http://piglit.freedesktop.org/ ).
3..5]) Good idea.

It would be a really horrible binary format if there wasn't an included newline equivalent.
http://www.freedesktop.org/wik...

Because the format is designed to be mostly append-only you should be able to skip over most corruption in the reader, and the writer should rotate the log file if any corruption is found. Additionally there is a mechanism that lets you know with high certainty weather a section of the file has been corrupted since it was tagged. "Tag objects are used to seal off the journal for alteration. In regular intervals a tag object is appended to the file. The tag object consists of a SHA-256 HMAC tag that is calculated from the objects stored in the file since the last tag was written, or from the beginning if no tag was written yet"

YES

THERE

ARE

YES

THERE

ARE

YES

THERE

ARE

This is lennart at poettering.net himself announcing systemd 219, just yesterday:

> http://lists.freedesktop.org/a...

"Note that this version is not available in Fedora F22/F23 yet. The
linker on ARM segfaults. Since the i386 and x86_64 versions built
fine, I decided to release 219 anyway."

By all means this corroborates all the points gweihir gave us above.

Why in the hell should I be forced by any ditro maintainer out there to use a piece of software from a dev/pm that just "release anyway"?

Systemd doesn't even have a stable feature set yet, and yet you say the world has moved on? Maybe you are on a different world?

It also has a bug list that is huge, and the number of critical bugs embarrassing.

> moderated my posts down.

I think three of my last four systemd posts were marked as trolls even though I gave specific examples of bugs. The systemd community is simply toxic.

Last night, I created a bug at:

https://bugs.freedesktop.org/e...

With a script I found from:

http://www.reddit.com/r/linux/...

that I was able to use to reproduce two different systemd bugs with on a Red Hat 7 and a CentOS 7 system. It is a well written and very self-explanatory example. I can no longer find the bug. It looks like they deleted it.

Sure it is useful on the server. In no particular order:

1. The journal digitally signs each entry with the entry contents and the hash of the previous entry, so that for an attacker to insert a spoofed entry or remove a valid one they have to alter the signature on every entry after that point or else the signature mismatch will be detected. And you can still also send logs to rsyslog if you want.

2. Faster boot time does matter for a server - when you need to move physical boxes, add hardware that requires a power cycle, and so forth less time to restart is helpful.

3. You don't need to run "piles of things" with systemd. It's modular, so you only need to compile in the services you want. http://freedesktop.org/wiki/So...

4. Systemd lets you set limits on resource usage by each service: memory limits, CPU limits, Block IO limits, etc... which is useful on the server.

5. On-demand socket-driven service start is useful, so ssh is available 100% of the time but sshd isn't actually consuming resources until it receives a connection attempt.

6. Per-service private /tmp directories, configurable read-only access to some directories, so a hacked service can't access information it should not or make invalid writes.

7. Because of the use of cgroups in the Linux kernel, when you halt a service you can be confident there are no uncleaned resources - threads, forked processes, file handles, etc... left in use.

8. systemd is compatible with services written in any programming language you want, including shell scripts, so you don't have to rewrite your custom SysV init service in systemd. Just spent a few hours in the documentation to make the text file you need, and you're set.

The autostart specification defines a method for automatically starting applications during the startup of a desktop environment after the user has logged in and after mounting a removable medium.

http://www.freedesktop.org/wiki/Specifications/autostart-spec/ Obviously, desktops could still do with a better way of starting services and making sure they stay running. Because that wouldn't be at all useful on servers or anything.

I'm aware the register spec was open up -- sadly performance still sucks, and there are tons of bugs.

i.e.
http://dri.freedesktop.org/wik...

They released all the docs for the GPU, drivers are on their way http://dri.freedesktop.org/wik... https://www.youtube.com/watch?...

From Poettering himself :
http://lists.freedesktop.org/a...

The Gallium drivers for AMD have experimental OpenCL support. Southern Islands and Sea Islands GPUs are the best supported: Gallium Compute

Lennart already announced that systemd will include it's own replacement for glibc, the most widely used gnu software.

http://lists.freedesktop.org/a...

The only real fault I could give it is that in my experience, if you're so tightly wired into external things that you become an essential part of them, that's probably not the optimal system design.

That really isn't the case with systemd; it basically depends on glibc and util-linux for "mount" etc. just like most other init-systems, but everything else is optional.

That upstream projects actually make use of systemd features like systemd-logind can hardly be a problem either; all the non-systemd platforms are free to provide their own similar services that upstream projects can use, so there are no problems there either.

I DO have a lot against the binary logging for very good reasons. I've worked with a lot of products that did their logging in proprietary binary formats over the years, and even when the internal data formats are well-published, the mere fact that intermediary services are required to access and format them has made their respective products harder to work with.

Some facts first; systemd's journal isn't proprietary. It is a well documented standard with a stable API and language bindings:
  http://www.freedesktop.org/wik...
There are several independent journal readers already. I can't think of any non-contrived user case where it is a problem that the logs are binary.

In fact, I can't think of any non-contrived user case where old style flat text file logging is better than systemd's journal; it scales better in every direction and allows for powerful filtering in a much better way than using humongous regex lines, is much more secure, allows for a single collated system log and much better debugging, etc. etc.
Not forgetting that it is now possible to have reliable, robust logwatch scripts and even make a log viewing GUI that isn't just a glorifed "less" with windows decorations.

Just like you I have been bitten by (proprietary) binary log files (and much worse; binary config files). And just like you I was really, really skeptical about systemd's binary log format. But the more I used it, and the more documentation I read, the more I became really glad about it: systemd's journal just solves so many problems I have had over the years with syslog files. It is a really well thought out and well designed logging system.

We're nearing the point where you can buy a graphics card, plug it in, and it "just works." The main issue is that it could take months for the bleeding edge to make it into the latest kernel, so brand new GPUs could be problematic.

+1. All my latest Radeon installs have been: buy it, plug it in, it works. That is because I always check this matrix before deciding which card to buy. Note that it is now green all the way to the right on nearly every hardware feature you care about. A notable exception is OpenCL, which is WIP. If I wanted to develop with it right now, that would move me back to fglrx, and only that. Notice how the latest chipsets are the ones with OpenCL closest to prime time.

In addition to which, you replace it with something that requires more typing, and does not give any feedback. service nfs restart stopping nfsd starting nfsd etc.

systemctl restart nfs

Here you go; put this in your .bashrc:

service() { systemctl $2 $1; }

And the entire concept of *BINARY* logfiles,

Install a syslog daemon and put this in your /etc/systemd/journald.conf?

[Journal] ForwardToSyslog=yes

Or just install syslog-ng 3.6.1+, which requires no additional journald configuration?

xml configuration files [are] insanely stupid when X won't run, or isn't installed (like on a server).

Where does systemd use XML configuration files? (And what does XML and X have to do with eachother?)

And telling me that oh, it boots up *so* much faster means something *only* if I'm on a laptop.

Or a virtual machine, or a container...

@Anonymous Coward: "You know how you hear that after a customer service call? Well Poettering's statement has the same meaning."

Have you tried subscribing to the systemd Development Mailing List?

From http://www.catb.org/esr/writin...

(ii) Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.

My biggest problem with systemd is the excessively tight coupling that seems inherent in its design philosophy. I wouldn't have any real objection for using systemd for my init, but I can't use my syslogger - logs always have to go through journald first. If you disable journald, even if you have another logger running, daemons started with systemd will produce no logs. I don't use it myself, but apparently the same is now true of logind (whatever that does), it used to be able to run seperately but it apparently can't be done any more. I think the same is scheduled to happen to udev.

The interfaces between these components also seem to change very frequently, making implementing a third-party tool to replace functionality very difficult. From http://www.freedesktop.org/wik... - "A number of systemd's APIs expose Linux or systemd-specific features that cannot sensibly be implemented elsewhere"

As a friend put it; the linux kernel uses a monolithic architecture but a modular construction, systemd uses a modular architecture but a monolithic construction.

Here's the source code. Just go and have a look.
http://cgit.freedesktop.org/sy...

So it's C instead of shell, the same programming language that probably most of the software you're running is written in anyway.

My only worry is with the AMD/ATi video, but it's been a long time since I had been burned by the ATi 7500 All-in-Wonder...

That stopped being a worry a few years back. I've been running Radeon graphcs exclusively and continuously now for years. I switched from fglrx to exclusively Xorg drivers a few years back, at first with some noticeable loss of functionality and performance, but now... no worries, it is way past good enough. BTW, that is not just normal desktop loads but a lot of serious OpenGL hacking. For a coulple of years I put up with jaggies... no antialiasing support (though I hacked some myself with buffer accumulation, only applicable to my own code.) Now the Xorg drivers have MSAA antialising for all recent hardware and MLAA up to Evergreen with the last few generations apparently just needing testing. (MLAA is full screen smoothing just of detectable edges, slightly crappier but way faster than multisampling.)

By way of illustration:

lspci | grep Radeon
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Park [Mobility Radeon HD 5430]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300 Series]

$ uptime
  18:17:28 up 169 days, 4:44, 25 users, load average: 0.62, 1.18, 1.40

The 25 users are all me, last reboot was a power failure that outlasted my UPS. I guess you could call that stable :) I know, I know, 5430, the machine beside it has a much better card. But the 5430 is surprisingly good and - most important to me - completely, utterly silent. And I can push a million triangles per frame through it at video rates, not bad for an ancient card that set me back all of $50.

I haven't tried many 3D games, but I can report that Civ V works just fine with Xorg drivers, the only serious problem being the risk of getting permantly sucked into the game and losing touch with real life.

Putting it in pid1 is mostly driven by cgroups (the Linux kernel's hierarchical process-grouping/resource-management system). The initial kernel design for cgroups was that it was a shared resources managed via a pseudofilesystem (cgroupfs), but the developers of that subsystem seem to have decided that design was unworkable, and are moving towards a design where there can be exactly one userspace controller of the cgroups system at any given time. That more or less has to also be the process supervisor, or else you can't really do sensible things with tying resource-management to services (and increasingly, containers). And that all has to happen when the system is brought up, too. So either it needs to be in PID1, or it needs to be in several PIDs that are tightly coupled via an IPC mechanism. The systemd designers consider the second design more complex and error-prone. See e.g. here, plus a third-party comment here.

Wait, but devuan is the fork, so systemd must be the spaghetti.

Doubts?
man journald.conf and appreciate systemd's implicit mandate for backup suffixes that won't fill your hd undetected as an exercise.

As for me, my init is sh: Still a better log story than systemd.

I was just quoting the (ex-)maintainer of the systemd, from his e-mails from the CTTE discussion.

Without source or citation. I think your representation of what was said is rather biased.

Debian feedback would be submitted to mainline - but if it is rejected, he wouldn't even carry a custom Debian patch for it, because he doesn't want to deviate from the mainline. And he, as the maintainer of the systemd, would not consider it a bug. As such somebody else would have to fix somewhere else.

If it isn't a bug, why patch it? Sure, some people have tried to drop some turd patches into systemd, eg. ripping out security features in order to support some obscure glibc variant. The right thing of course is to patch the glibc variant to support the proper security functions, not patching systemd.

No package maintainer wants to support non-trivial, non-mainline patches without very good reasons. The whole point of open source software, that as many people as possible can share and enjoy improvements, so patches should go upstream as fast as possible. Maintaining non-trivial, non-upstream patches can also be a real problem when backporting security fixes, and may introduce patch specific bugs too.

If you are willing to grep through the 1K emails - you would definitely find that being repeated several time.

I have actually read most of them at the time, and I still think you are misrepresenting the systemd maintainers.

Here is a Debian specific patch that predates Debians adoption of systemd as default init-system:
http://cgit.freedesktop.org/sy...

It's obviously not Debian specific.

It is very obviously a distro specific part of systemd: Debian was added to the list where Fedora and Arch were already present.

Huh? The main point is that systemd mainline accepts Debian (and distro) specific patches if it is unavoidable. Despite the many claims to the contrary, the systemd developers do care about feedback and have many different distro developers with commit access. If you got a good user case, chances are good that a distro specific patch will be accepted. And having the patch going into the upstream repo is much better than carrying it as a separate distro patch.

In short, Debian developers where taking part in creating systemd long before Debian discussed making systemd the default init system. So they knew what they where doing when selecting systemd over Upstart (the CLA was enough to discount it), and the latest GR have confirmed that the vast majority of Debian developers firmly backs systemd as the default init-system.

Sure, the main branch of systemd wants to have as few distro specific patches as possible, but they do accept them if there is no other solution.

I was just quoting the (ex-)maintainer of the systemd, from his e-mails from the CTTE discussion.

Debian feedback would be submitted to mainline - but if it is rejected, he wouldn't even carry a custom Debian patch for it, because he doesn't want to deviate from the mainline. And he, as the maintainer of the systemd, would not consider it a bug. As such somebody else would have to fix somewhere else.

If you are willing to grep through the 1K emails - you would definitely find that being repeated several time.

Here is a Debian specific patch that predates Debians adoption of systemd as default init-system: http://cgit.freedesktop.org/sy...

It's obviously not Debian specific.

It is very obviously a distro specific part of systemd: Debian was added to the list where Fedora and Arch were already present.

.... The systemd devs and maintainers committed to literally to nothing, basically saying "if it is good for Fedora is should do the job for Debian too; no Debian specific patches are going to be accepted even into the Debian systemd package".

That is simply wrong. Please notice that several long time systemd developers with commit access to the systemd git tree, are in fact Debian Developers. So there is a lot of "Debianism's" in where files placed etc.

Sure, the main branch of systemd wants to have as few distro specific patches as possible, but they do accept them if there is no other solution.

Here is a Debian specific patch that predates Debians adoption of systemd as default init-system:
http://cgit.freedesktop.org/sy...

All million lines of it ;)

Seriously, I'd really love to go in myself and fix the bug that's currently preventing me from using GLX, but I wouldn't even know where to begin. I think Xorg is seriously understaffed in terms of volunteers compared to the scale of the project - it looks like most bug reports don't get responses for months or years, if ever.

What's the bug#?

Is this the same as Chris Murphy | 19 May 02:54 2014
problem with degraded boot and systemd

aka [systemd-devel] timed out waiting for device dev-disk-by\x2duuid

Why do KDE and Gnome require Systemd as designed?

Because they need someone to handle the login stuff.

That used to be consolekit.

But consolekit seens to be dead. systemd provides logind which does what the DM's want. If someone would write an equivalent for sysvinit or upstart, or provide a way of running logind unser sysvinit or upstart then whole bunches of whining could stop.

It absolutely is true for system utilities (like init). Vim is a text editor. It edits text. It does not edit graphics, init the system, act as a login daemon or multiplex your shell. But then, it's not a system utility. Look at awk, sed, grep, less, etc etc. Look at getty and login. Look at screen.

Exactly. But look at all the systemd "system utilities" like systemctl, journalctl, machinectl; they all work exactly like any other first class Linux system tools; they all only do one thing but do it well, they can be piped, they aren't chatty when successful (and actually care a lot about exit codes), aren't interactive etc, care about text output formatting (turn off legends etc) so they are perfectly scribtable.

The point is that all the systemd tools are doing everything expected by system tools according to "Unix philosophy".

Did you know that in Debian Wheezy there are TWO init systems that work at the same time? They weren't designed to do that but because they do things the Unix way, they don't 'mind' either.

I must say I can't see a reasonable use case for this. Sounds racy in all circumstances.

But here's a "funny" where systemd is most definitely wrong (yes, I have actually been giving it a chance, I just don't like what I see). I have a VM where I have yanked a virtual disk out from under btrfs. My fstab states that I want it to mount in degraded state if necessary (such as if a disk is missing). systemd *REFUSES* even though I explicitly commanded the action. How is that the Unix way? How is that supposed to help uptime? Thank the gods it's not a production box! Then I google as to why that might be and first post I find is some someone claiming IT'S A FEATURE! So there we are, the admin and owner of the box says just do it and damn the consequences and it refuses like a Windows box.

Take a look on these discussions;
http://www.spinics.net/lists/l...

http://lists.freedesktop.org/a...

Basically, systemd requires manual intervention to allow to boot btrfs arrays with both /a missing disk/ and in /degraded mode/
Not a bad default really.

Anyway, in order to allow btrfs to automatically boot in degraded mode with missing disks, and doing it /correctly/ you need some extra module/script/daemon to handle it, since neither the kernel nor systemd (udev) have any knowledge about the internal state of btrfs. Nothing new in that, raid etc. have always been handled by such a daemon. I think that if you use mdadm with btrfs raid, you can automatically mount degraded mode arrays. The critical point is the timeout timer; a crude method that needs to be set according to the particular array in question.
Bringing up a degraded array as RW risk killing the whole array, so it is not something to be done just because a drive is late at appearing.

http://git.neil.brown.name/git...

Now, just to complete the picture, do you know what journalctl told me about what was failing and why? It said the mount timed out. THAT IS ALL. Is this the system I am supposed to trust in production? The one designed by people who KNOW what they're doing?

Isn't that all you need to know to find the error?
Also, use the "-x" with journalctl, it may give further info to generic error messages and even link to more info.

Anyway, systemd have excellent debugging facilities; try to turn on debugging ("kill -56 1" from the CLI, or by setting "MaxLevelKMsg=debug
MaxLevelConsole=debug" in "/etc/systemd/journald.conf" and restart (journald or the VM)

Digging in to it, I fin

All this makes me wonder why someone can't build some other piece of software which would perform the required session management in this particular case? It's not dependency for nefarious means, it's dependency which provides a specific feature that a programmer would like to use. Why can't that be coded separately from systemd by someone other than the systemd maintainers?

Systemd is also somewhat know for having no stable interfaces between its parts; combined with the increasing large number of parts it has, one would need to implement quite a lot to handle the required functionality. Systemd does have a chart of its API stability promises, but it looks to be last updated a year ago.

FWIW, I ran Debian Jessie with sysv for a while - and for about a month it was impossible to login via gdm (workaround was to switch to kdm or lightdm), so there's already been (testing-)user visible impact.

This is all very frustrating, especially since I really like the init/process-management parts of systemd, but not the glomp-up-all-the-projects and explicitly not working with others parts...

And the criticism from those who are against systemd is extremely important to consider. The complaints are very sound, from a technological perspective. They're also based on decades of real world experience, which just cannot be ignored.

I'm not a total fan of every design feature of everything systemd has done but gave you actually read their supporting references? I'm most of the cases boycottsystemd has rephrased events to make the systemd folks look as bad as possible in ways that would make a Fox news reporter feel proud. A good example is their comment about requiring "bug for bug" compatibility with glibc was instead a use of a certain non posix flag needed for thread safety and complaining that it is tightly tied to Linux is about as helpful as complaining that udev is tightly tied to Linux.

At any rate, I find it very telling that they don't actually mention any of their supporters.

Systemd will forward to syslogd if you want it to so you can still use your standard tools to view the logs if you want.

Not an answer.

Why not?

Systemd will forward to syslogd if you want it to so you can still use your standard tools to view the logs if you want.

Not an answer.

I was kind of neutral about systemd until I realized that the only way to get centralized logging out of systemd boxes is to turn on syslog mode (journald has no concept of network transport).

At that point, I realized that the systemd developers aren't actually server admins.

It is trivial to export systemd log files to such a centralized logging server by using "systemd-journal-gatewayd", it can even export the logs in JSON to preserve the rich metadata.
The details are here:
http://www.freedesktop.org/sof...

So to sum up; it is easy to make systemd work with centralized logging servers, and it support a wide range of formats and encryption and security features too.

For me, it's keyscript in crypttab which completely stopped my systems from booting. They're not keen to ever implement becase apparently shell scripts are intrinsically racy (for what it's worth my own keyscripts have a 10s timeout and fall-back to askpass if the crpyt token doesn't become available. I've never had one of my servers reach this time-out, the hardware config rarely ever changes). I wrote about some of the infinite permutations possible to support the use-case of just having a 4-line shell script, but it just seems that systemd is religiously opposed to shell scripts. Eventually, someone pointed out that I could pass keyscript args in the kernel boot parameters which seems to be a partially satisfactory solution for now.

For what it's worth, I do like the declarative nature of systemd for starting processes, socket activation etc. and I have migrated most of my stuff to systemd. It bothers me that debugging dependency issues is still so hard (ever tried "systemd-analyze dot"? the output is completely worthless as a debugging aid). Still, I am uneasy about the dogmatic anti-shellscript religion, I worry about the project's overall approach to security when simply accidentally running systemctl as a non-root user causes it to segfault, and it doesn't seem right that a change in pid1 should even remotely impact userland applications at all, let alone as deeply as systemd has.

At the end of the day, choice of default init system isn't going to make me switch from my favourite distro of the last 14 years (apart from a 2 year excursion to Ubuntu), but I think some of my own hostility toward systemd has been a result of the instantly dismissive remarks whenever I've tried to explain a problem I've had with it - by now I'm realizing that perhaps everybody is just too tired to tell the difference between a valid systemd complaint and yet another "get off my lawn" argument. In any case it's made me realize I should really diversify my tastes a little, currently playing with FreeBSD (again) and NixOS - that has to be a good thing.

For me, it's keyscript in crypttab which completely stopped my systems from booting. They're not keen to ever implement becase apparently shell scripts are intrinsically racy (for what it's worth my own keyscripts have a 10s timeout and fall-back to askpass if the crpyt token doesn't become available. I've never had one of my servers reach this time-out, the hardware config rarely ever changes). I wrote about some of the infinite permutations possible to support the use-case of just having a 4-line shell script, but it just seems that systemd is religiously opposed to shell scripts. Eventually, someone pointed out that I could pass keyscript args in the kernel boot parameters which seems to be a partially satisfactory solution for now.

For what it's worth, I do like the declarative nature of systemd for starting processes, socket activation etc. and I have migrated most of my stuff to systemd. It bothers me that debugging dependency issues is still so hard (ever tried "systemd-analyze dot"? the output is completely worthless as a debugging aid). Still, I am uneasy about the dogmatic anti-shellscript religion, I worry about the project's overall approach to security when simply accidentally running systemctl as a non-root user causes it to segfault, and it doesn't seem right that a change in pid1 should even remotely impact userland applications at all, let alone as deeply as systemd has.

At the end of the day, choice of default init system isn't going to make me switch from my favourite distro of the last 14 years (apart from a 2 year excursion to Ubuntu), but I think some of my own hostility toward systemd has been a result of the instantly dismissive remarks whenever I've tried to explain a problem I've had with it - by now I'm realizing that perhaps everybody is just too tired to tell the difference between a valid systemd complaint and yet another "get off my lawn" argument. In any case it's made me realize I should really diversify my tastes a little, currently playing with FreeBSD (again) and NixOS - that has to be a good thing.