Red Hat's Linux Changes Raise New Questions

itwbennett writes "Last month two Red Hat developers proposed to replace the 30-year-old syslog system with a new Journal daemon. Initial reaction was mostly negative and 'focused on the Journal's use of a binary key-value form of data to log system events,' says blogger Brian Proffit. But now, says Proffitt, it seems that the proposal to replace syslog has less to do with the fixing syslog's problems than with Red Hat's desire to go its own way with Linux infrastructure."

Comments

One of the advantages of Linux

[Todd Knarr]

That's one of the advantages of Linux: RedHat can go their own way without needing the rest of us to buy in, and without really messing things up for us. If they provide a reasonable API, it'll either be compatible with syslog with a simple library substitution or we'll quickly see a wrapper library that allows programs to use either syslog or Journal without needing code changes.

I think going to binary's a bad idea, myself. The fewer tools you need working to find out what the error is, the easier it is to debug and fix the problem. But let RedHat try this and see how it works, and then we can decide once we've got some real-world data to compare.

Re:One of the advantages of Linux

[LordLimecat]

There are advantages to not having everything in ascii text, or else we would never see relational databases used for anything. You are right that we will see. I like plain text logs because I am still learning the ins and outs of the major Linux breeds, and not having to learn a special tool for every config file and log makes things easier; but I wont say that there couldnt be benefits to a more robust system.

Re:One of the advantages of Linux

[errandum]

Because if they go their own way the next time you're trying to compile something a bit more complex it won't screw everything. I like red hat based distros because they tend to stay more or less the same over the years, leading to easily configurable systems. The moment they deviate it'll be their death.

Re:One of the advantages of Linux

[skids]

That's one of the advantages of Linux: RedHat can go their own way without needing the rest of us to buy in, and without really messing things up for us.

Not quite true. If PHB insists on RHEL, you're stuck coping with whatever poor choices they make.

Why do I get the sense that all the chafing at the "restrictions" of the LSB/linux-instinct/unix-way/common-sense is just the bellyaching that happens when you realize you're short the talent/energy/whatever to progress and start looking for ways to re-arrange the deck chairs?

Re:One of the advantages of Linux

[esocid]

Maybe entrenchment has some bearing on it. The article mentions the arguments for switching from syslog, and that you can do some of them already in syslog, but nobody does.

However, I agree with you. Let RH try it out. It's not like they can't revert if they find that it was a bad choice. If we wanted all distros to be the same, we'd run windows instead.

Re:One of the advantages of Linux

[GameboyRMH]

If PHB insists on RHEL, you're stuck coping with whatever poor choices they make.

And that's RHEL's bread and butter, corporate support & name recognition, always a requirement at businesses with a NEGFFBIBM attitude. Anyone who doesn't need it will just go with CentOS or some other distro. Those who do need it...well I guess when the troubles become to great they'll have to make a hard decision.

Re:One of the advantages of Linux

[MightyMartian]

No matter your experience, plain-text logs make more sense, especially in *nix operating systems. You have a vast array of tools to search log files with; my favorites being tail and grep. The minute you go to binary logging your options shrink or you end up having to use additional tools to reconvert it to text (ie. the Windows event log).

Re:One of the advantages of Linux

[CAIMLAS]

Looks like they're pulling the same shit Ubuntu pulled with upstart (init replacement). "Let's replace something simple and elegant with something complex, incomplete, and very difficult to fix when it goes wrong".

Sorry, but no thanks. I can see the need for something else, in a limited/special purpose role, but these assholes are aggregately destroying the very basis of what makes Linux a good, robust server choice:

* you can use traditional unix tools from ssh to manipulate and analyze the system
* there are literally thousands of tools for analyzing, manipulating, and storing syslog data
* init is purely linear, whereas upstart is threaded, increasing the possible ways in which it can fail as well as increasing the difficulty of troubleshooting
* KISS means broken things are more obvious.
* KISS means there's less that can go wrong.
* Most Windows guys don't even read the logs, from what I've seen. This could quite possibly be related to the complexity and lack of utility of Event Viewer itself, granted, but even Event Logs can be exported to syslog...

While we're at it, why don't we start using XML or sqlite as a replacement for /etc.

Re:One of the advantages of Linux

[Anonymous Coward]

WTF does NEGFFBIBM mean? Google fails me on this one :(

Re:One of the advantages of Linux

[LordLimecat]

As has been pointed out, there is no reason you couldnt use a new tool to get the output you want out of the database.

You cannot, for example, convince me that noone is able to script MySQL databases, despite their binary nature.

Re:One of the advantages of Linux

[Anonymous Coward]

That's one of the advantages of Linux: RedHat can go their own way without needing the rest of us to buy in, and without really messing things up for us. If they provide a reasonable API, it'll either be compatible with syslog with a simple library substitution or we'll quickly see a wrapper library that allows programs to use either syslog or Journal without needing code changes.

I disagree. In fact, I'd call this a real disadvantage of the FOSS world (bazaar-style development in particular): change comes so slowly because no one else will commit to it. I think it's hard to really take advantage of changes if others just slap an abstraction layer on top of them and otherwise ignore them. I haven't studied this specific example in detail, but off the top of my head:

• software written by people with that attitude won't take advantage of the key/value pair format because syslog doesn't have that
• likewise nothing will switch from a proprietary format to the new common infrastructure
• log analysis software probably won't do anything too interesting with the output files because god forbid someone be left behind

and so this won't be as useful as it would be otherwise.

To take another example I have paid a little more attention to, init was around for far too long, and yet I'm amazed init->upstart happened at all. Even when something is clearly superior, if it takes buy-in from people who want their software to work on the greatest possible range of systems, it's hard to make that happen. init->upstart probably worked because the startup is a relatively unobtrusive piece of the software, relatively easily replaced by distributors.

In contrast, proprietary or cathedral-style development of a whole system does much better at this. They have the sort of unity of purpose where they can agree that something is valuable and exploit it through the whole system. I've seen this at work. Maybe the best example is something that hasn't happened in the Linux world, though: a widely used high-level language. The GNOME people in particular won't commit to anything but plain C, with their painful object model on top of it.

Re:One of the advantages of Linux

[el_tedward]

North-East-Goat-Fried-Farmer-Baked-Interesting-Babble-Manti?

Re:One of the advantages of Linux

[LordLimecat]

Not quite true. If PHB insists on RHEL, you're stuck coping with whatever poor choices they make.

Package management: use it. I would be very surprised if RedHat prevented you from installing whatever logging facility you wanted on your server.

Re:One of the advantages of Linux

[Anonymous Coward]

Nobody Ever Got Fired For Buying IBM

- GameboyRMH (bloody post limiter!)

Re:One of the advantages of Linux

[Compaqt]

I agree in general with "if it's not broken, don't fix it". Witness /. opinion regarding Unity/Gnome changes.

About Upstart, my lowly sysadmin opinion is this: It seems different from the other stuff Ubuntu's been doing in that, AFAIK, it's not alone in this. I think Fedora's going that way too.

Also, with Upstart I know if the webserver crashes for some reason, it'll restart without intervention. Yeah, I know, you're not getting to the root of the problem, but it beats being stuck to a top display looking if something burned.

Re:One of the advantages of Linux

[Iphtashu Fitz]

RedHat can go their own way without needing the rest of us to buy in

The only problem with your argument is that Red Hat has a huge base of paying customers, and money talks.

I manage a small research cluster at a university. It's running Red Hat linux on over 100 nodes. The university has a site license for Red Hat so licensing for the cluster isn't an issue. The decision to go with Red Hat had to do mainly with what distros are directly supported by commercial products like Matlab, Mathematica, Abaqus, Maple, Comsol, Ansys, etc. All these vendors sell lots of software & services to universities, research labs, etc. and they all support Red Hat linux.

I've personally dealt with support departments when trying to run commercial software on non-RH distros, and in some cases they pretty much tell you you're on your own if you're not using RH or one of the other top two or three distros. Most commercial vendors will only state that they support RedHat, SUSE, and maybe Ubuntu and/or Debian.

If/when Red Hat comes out with a new way of doing things then customers like us will start pushing on the vendors to support those new ways. After all, we're tied into using Red Hat, and we need their products to run on it. So the commercial software vendors will start supporting the Red Hat way of doing things to appease their customers. And once the commercial vendors start supporting it then it will slowly but surely make its way into other distributions as well so that these apps can run on distros that other people want to use.

Re:One of the advantages of Linux

[mlts]

I'd disagree. RedHat is only one of two Linux distros that is FIPS and Common Criteria certified. Of course, to people who work with Linux, this doesn't mean much. However when it comes audit time, the auditors either want to see certifications of the OS, or one better has to have a damn good reason (and not a technical one -- one that will appeal to a bean counter who is looking for any excuse to shut your operation down) why the OS isn't certified. Usually one has to explain in great detail why the OS in use is not Windows.

With this being the case, what RH does affects a lot of production items. Take RHEL 6 -- it now uses a new network startup daemon instead of the old network binary. One can switch to the old one, but that one is depricated. Same with commands like ifconfig -- in RHEL6, one uses "ip addr" instead.

Re:One of the advantages of Linux

[LordLimecat]

Looks like they're pulling the same shit Ubuntu pulled with upstart (init replacement). "Let's replace something simple and elegant with something complex, incomplete, and very difficult to fix when it goes wrong".

One could make that argument about solid-state electronics, the move away from punch-cards, the move from paper-based filing, the move to journaled filesystems, etc.

Sometimes progress means letting go of the past, and sometimes it takes a while to fully bake; thats why RedHat doing the QA, testing, and development for the rest of us is a good thing. If it sucks, it will die, and noone really has to acknowledge that it ever existed.

Re:One of the advantages of Linux

[heinousjay]

If noone can't do it, no one can.

Re:One of the advantages of Linux

[rsilvergun]

Yeah, but you add a bunch of overhead to get back to text.

Re:One of the advantages of Linux

[epiphani]

Agreed. I submitted this post yesterday, by the lead developer for rsyslogd (the most common syslog daemon in linux these days). He makes the point that most of the complaints made are actually wrong if they'd bothered to look at the last 10 years of development and IETF work around syslog.

Re:One of the advantages of Linux

[lucm]

No matter your experience, plain-text logs make more sense, especially in *nix operating systems. You have a vast array of tools to search log files with; my favorites being tail and grep. The minute you go to binary logging your options shrink or you end up having to use additional tools to reconvert it to text (ie. the Windows event log).

The more a system becomes complex, the more one needs to see events as part of a whole and do some kind of analysis and correlation. This type of work is done more easily with databases. I like grep like everyone, but if I want to have a nice rollup of events based on time and source, I will get the info much more easily with a SQL query than with a regex piped into a reporting utility piped into a paging utility.

Also I think one has to adapt to a technology, not try to make it work like what was there before (unless he is a one-trick pony). Why would you want to "reconvert" the Windows event log to text? On Windows there is a lot of built-in capabilities for log exploring in Powershell or even in VBS/WMI. A toolbox contains many tools, not just grep.

Re:One of the advantages of Linux

[Crudely_Indecent]

What I don't understand is why you can't achieve both log security and log usefulness with the existing tools.

In a previous job (seems like a different life) - I set up all of the servers to utilize remote syslog. The syslog server then offered the log directory as a read-only NFS exports to each of the servers.

It was quick, it was easy, and it was secure. You could view the local logs on individual servers, but you couldn't alter them in any way except by generating log output.

Re:One of the advantages of Linux

[DiegoBravo]

Many times with a (semi)broken operating system, you don't have all the usual tools.... sometimes your only clue is a syslog driven console text message.

Re:One of the advantages of Linux

[Hatta]

If it sucks, it will die.

On what do you base this assumption? History is littered with sucky technologies that became standard because someone important was pushing it.

Re:One of the advantages of Linux

[pdxer]

If it sucks, it will die, and noone really has to acknowledge that it ever existed.

Yeah, that's what happened with Windows. It...oh wait...

Re:One of the advantages of Linux

[DrXym]

No matter your experience, plain-text logs make more sense, especially in *nix operating systems. You have a vast array of tools to search log files with; my favorites being tail and grep. The minute you go to binary logging your options shrink or you end up having to use additional tools to reconvert it to text (ie. the Windows event log).

Except you already see binary log files on Unix. Log files are frequently compressed with gzip. I don't see a big difference between someone typing (for example) "zcat file.gz | grep somestring" and "redhat-log-cat | grep somestring" assuming that was the name of the tool they used to crap out a logfile from the binary db.

I'd also note that tools like git are happy to store things in binary objects yet still present stuff in textual format to the user.

Re:One of the advantages of Linux

[supremebob]

Funny that you mentioned CentOS, considering that it's basically RHEL in drag...

Re:One of the advantages of Linux

[someSnarkyBastard]

You do realize that this is going into Fedora, not Red Hat proper, right? Fedora is rather infamous for introducing bleeding-edge new stuff and breaking things between updates, Red Hat Enterprise Linux is much more focused on stability and compatibility. Syslog will remain the default on RHEL for a long time yet.

Re:One of the advantages of Linux

[someSnarkyBastard]

Actually Red Hat is championing systemd, which is beginning to spread to other distros as well; OpenSuSE added systemd to their latest release.

Re:One of the advantages of Linux

[LWATCDR]

but what if it is actually better?

Re:One of the advantages of Linux

[Tomato42]

Just yesterday I was debugging why my syspreped Win 7 images weren't installing properly. So I go to sysprep directory and start reading the text log. It didn't have any useful info. So I grabbed the binary log and tried to import it on the same machine in pre-exec environment. It couldn't do it. So I copied it to different computer and tried to open it there, the system claimed that the file was damaged. After 4 hours of struggle to read, copy or convert the bloody thing I went the "Microsoft recommended way" (seriously, that's the solution they suggest in MSKB) and bisected which program caused the install failure. In "only" 6 reinstalls I finally found the culprit.

If it was a Linux distro, a simple cat or tail would have sufficed and it would have been a 15 minute job, not 2 days. If they want my text logs they will have to pry it from my cold dead hands.

Re:One of the advantages of Linux

[vakuona]

So maybe it doesn't suck as much as you think. And maybe *nix sucks more than you think. Just saying.

Re:One of the advantages of Linux

[Tomato42]

If your system needs special tools to see the correlations in errors then your systems is way past its usefulness and should be re-designed with greater modularity in mind.

As for windows logging, sorry to burst your bubble, but it fails when it's most needed, read my previous post: http://linux.slashdot.org/comments.pl?sid=2553896&cid=38231092

Re:One of the advantages of Linux

[armanox]

I'm glad you told me that ifconfig was depricated, I told someone that "ip addr" was not how you get the IP information last week....

Re:One of the advantages of Linux

[Anonymous Coward]

Yeah, but you add a bunch of overhead to get back to text.

Yes, you get the overhead when you as a human try to read the log. In plain text you have the overhead whenever it is used by the system.

There are plenty of arguments for using ascii, efficiency is not one of them.

Re:One of the advantages of Linux

[jgrahn]

Agreed. I submitted this post yesterday, by the lead developer for rsyslogd (the most common syslog daemon in linux these days). He makes the point that most of the complaints made are actually wrong if they'd bothered to look at the last 10 years of development and IETF work around syslog.

But about this part of what he wrote:

"Ages ago (2006?) I implemented high-precision timestamps (including TZ info) in rsyslog, and RFC5424 has brought them to the on-the-wire protocol. As far as I know, syslog-ng supports them for quite a while as well (but I am not a syslog-ng expert ;)). However, all distributions turn high precision timestamps off and set the dumb old format as this is a requirement to keep old tools working."

I enabled high-precision timestamps on my Debian system to get a feel for them. But I had to turn them off again: not readable enough, and took too much screen space making more log lines wrap. The tools weren't the problem; I just couldn't eyeball the damned things!

Re:One of the advantages of Linux

[jmitchel!jmitchel.co]

You can script mysql databases, assuming the data files are intact, mysql is intact and runs on the server, and you have functioning tools and the skill to use them. ASCII logfiles will be readable in spite of spot corruption and can be processed with grep / head / tail -- if these primitives don't work it probably doesn't matter what's in the log.

Re:One of the advantages of Linux

[Spazmania]

tail -f /var/log/messages

In mysql? How?

Re:One of the advantages of Linux

[marcosdumay]

Except that logs are rarely used by the system. And on the few times where performance matters, the system is just interested on the last few lines, in real time. So, you just use them before (or while) writting.

Re:One of the advantages of Linux

[PPH]

Lots of old tools (many built or modded for custom systems) depend on the text file formats. So the solution might be to replace the text log files with structured data and then provide a 'dummy' file system (perhaps as an option), similar to the procfs, that would read the underlying structured data and provide a flat file image to support legacy stuff.

Re:One of the advantages of Linux

[Lunix Nutcase]

Maybe the best example is something that hasn't happened in the Linux world, though: a widely used high-level language. The GNOME people in particular won't commit to anything but plain C, with their painful object model on top of it.

And there's a reason for that. Far better compatibility with the vast majority of.languages.

Re:One of the advantages of Linux

[slack_justyb]

Looks like they're pulling the same shit Ubuntu pulled with upstart (init replacement). "Let's replace something simple and elegant with something complex, incomplete, and very difficult to fix when it goes wrong".

There is a fine line to walk when one starts talking in this way. You have to ask yourself, "Are we doing it this way because there is no better way, ever from this point forward this is the best way to do things? Or are we doing it this way because that's how we've always done it and to do it differently would just be stupid because we've always done it this way."

There once was a point where the goal was for Linux to be as good as UNIX. Now that we are there, some might argue this point, what's next? Was an OS developed many decades ago the apex of servers? Shall we come this far and no further?

I understand a lot of people don't like how distros are bending and breaking the traditional ways servers were handled or how desktops should work or whatever. At some point we should give it a try and see if it works. If it doesn't well, that's why Slackware doesn't have PAM.

We are at a point where we can either stop trying to do new stuff and just refine how cron and syslog and init work and make them slightly better, or we can try something new and see if it makes it better than slightly better. We won't know until we try it out and if it's a dud we can always go back, the code is there.

Let's face it, init's linear method of doing things means that simple services starting up could prevent a login screen until that service is done loading, upstart tries to address that with threading. Maybe this isn't idea for a server and I'd agree with you there. There are plenty of distros that still use the good ol' System-V init.

We all need to realize that Linux cannot just continue to be a UNIX clone or Linux will just fade in popularity like UNIX itself. Does that mean we need to go all crazy and try not to be UNIX? No, that's ridiculous. But it is time to start tossing things at the wall and seeing what sticks. So yeah it is a period of a lot of flux and I don't expect a lot of people to be happy in the interim, but we just cannot continue to kick the same can ten, fifteen years from now, and at some point the change has got to start.

So in the end, I scream when I see GNOME 3 and Unity. I really dislike how complicated the /etc folder has become. I'm tried of /sysfs, /srv, and other suddenly needed root level directories. However, I understand that change cannot be escaped and I need to learn to adapt to that. Some of the changes I really like PAM, iptables, devfs, and so on. It's just a matter of trying stuff out and seeing what happens. If you don't like it, you are free to not like it and continue using whatever meets your needs.

Most importantly let's not confuse KISS with that's just the way we've always done it. I would dare say that's a major hold back currently with IPv6. People just don't want to not use what they've been using for the last ump-teen years.

Re:One of the advantages of Linux

[marcosdumay]

Way to get vendor lockin using free software!

Re:One of the advantages of Linux

[CAIMLAS]

One could make that argument about solid-state electronics, the move away from punch-cards, the move from paper-based filing, the move to journaled filesystems, etc.

Except in this case, syslog format has been thoroughly vetted. It's gone up against some heavy hitters and still proves itself to be superior for what it does. Event Log is the biggest functional competitor, and you've got Eventlog to Syslog mechanisms, but not the other way around. Syslog is trivial to adhere to, so it's easy to log effectively.

Re:One of the advantages of Linux

[The Moof]

In a severely abbreviated, off the top of my head, fill in the blanks on your own fashion:

CREATE TRIGGER `database`.`TailToFile` BEFORE INSERT on `database`.`table`
FOR EACH ROW BEGIN
/* manipulate data and print to file or pipe it elsewhere */
END

You could have config options, piped to /dev/null when not in use, etc.

I've actually used PL/Perl in postgresql to do some pretty interesting stuff with the filesystem via triggers.

Re:One of the advantages of Linux

[mcgrew]

I'm a little offtopic here, but you guys' remarks reminded me to ask: is there a Linux equivalent to dBase or, better yet, NOMAD? I would think NOMAD could be easily ported, since it was (is?) a mainframe DBMS that likely ran on UNIX.

Re:One of the advantages of Linux

[ThePhilips]

The more a system becomes complex, the more one needs to see events as part of a whole and do some kind of analysis and correlation. This type of work is done more easily with databases. I like grep like everyone, but if I want to have a nice rollup of events based on time and source, I will get the info much more easily with a SQL query than with a regex piped into a reporting utility piped into a paging utility.

Typing 'grep <whatever>' is much much faster than: connecting to DB, typing query and realigning rows/columns on screen for readability.

I have to dig quite often through audit-log-like tables in DB created by our software and let me tell you that SQL doesn't make any correlations easy. Especially if we are talking about some production system were you end up self-joining a table with few dozen million rows (what you need to display for example the trivial thing as the time to the next/prev interesting event).

Neither the usual SQL tools are any good at displaying the data - as compared to displaying the SQL itself e.g. syntax highlighting. On text side of things, it takes minutes to create custom syntax for VIM for the problem at hand.

Why would you want to "reconvert" the Windows event log to text?

How many 3rd party applications actually use the Windows Event Log? I have seen probably one or two.

You know why? Because using it is a PITA - I have tried that twice as SW devel already in times of NT4 and W2K. (I was hoping to simplify critical error reporting of the Windows applications (including one GUI-less) and thought myself "WEL is just like syslog!" Oh gosh, Windows API proved me wrong.)

On Windows there is a lot of built-in capabilities for log exploring in Powershell or even in VBS/WMI. A toolbox contains many tools, not just grep.

Oh, so you like all that stuff over something as fool-proof, robust and simple as the grep? OK.

Re:One of the advantages of Linux

[Culture20]

Ugh. Don't remind me of wevtutil.

Re:One of the advantages of Linux

[vakuona]

Windows does what it needs to do. I am not saying it is the best. I am saying it does some things well enough, and that is what matters. I am also implying that *nix and other OSes also suck in other ways. I am not denying that there are reasons, other than technical and non-suckage, that have left Windows dominant, such as abuse of monopoly and network effects (Microsoft would have become dominant anyway, so it is rather unfortunate for them that they saw the need to abuse their position because the judgment hamstrung them for more than a decade).

But if Windows didn't do at least what people need it to do, and some of what people want it to do, it would have died.

Re:One of the advantages of Linux

[hawguy]

No matter your experience, plain-text logs make more sense, especially in *nix operating systems. You have a vast array of tools to search log files with; my favorites being tail and grep. The minute you go to binary logging your options shrink or you end up having to use additional tools to reconvert it to text (ie. the Windows event log).

Except you already see binary log files on Unix. Log files are frequently compressed with gzip. I don't see a big difference between someone typing (for example) "zcat file.gz | grep somestring" and "redhat-log-cat | grep somestring" assuming that was the name of the tool they used to crap out a logfile from the binary db.

The difference is that one zcat tool can handle any gzipped file, it doesn't need to know a thing about the data in the file.

But if when developers start using binary data, yourredhat-log-cat parser will need to understand how each developer writes their data.

Some developers may just use ascii key-value pairs. Some may write out numbers as binary ints and floats. Some may serialize java objects and write those to the logs.

With ASCII readable files you can usually (but not always) decipher the data and write a parser for it. Once you've got a stream of binary data, this becomes much harder.

Which would you rather try to decode:

        Perfdata: 0 100 123123123 20.1

or

        Perfdata=00000000000000640756b5b35468a000

Of course, with proper discipline, this can be prevented from becoming an indecipherable mess, but the same could be said for the current syslog.

Re:One of the advantages of Linux

[mcgrew]

Looks like they're pulling the same shit Ubuntu pulled with upstart (init replacement). "Let's replace something simple and elegant with something complex, incomplete, and very difficult to fix when it goes wrong".

One could make that argument about solid-state electronics, the move away from punch-cards, the move from paper-based filing, the move to journaled filesystems, etc.

No, one couldn't. The difference between a solid state circuit and an old fashioned "simple" one is you replace the tubes with transistors. It is no more complex (although when I was a kid with a soldering iron, it vexed me because I understood how tubes worked but was at a loss to understand the workings of transistors). Punch cards were far more complex (perhaps the correct word I should use is "complicated"), and manual filing was no simpler but far mor elabor intensive. Journaled filesystems make it simpler for the end user.

Sometimes progress means letting go of the past

And they do stupid shit like replacing a car radio's volume knob with buttons. What moron decided that? Glad they went back. Another was nylon shoelaces, wtf? Glad they finally changed again to nylon+cotton.

And I really miss the old gravity furnace in the house I used to live in when an ice storm drags the cables down and the power goes out. The old gravity furnaces didn't use external power, they generated it internally for the thermostat with a little device called a "power pile".

I'll tell you what, though, I far prefer this sci-fi 21st century life to the primitive times I grew up in. But sometimes "progress" isn't very progressive.

Re:One of the advantages of Linux

[tlhIngan]

Looks like they're pulling the same shit Ubuntu pulled with upstart (init replacement). "Let's replace something simple and elegant with something complex, incomplete, and very difficult to fix when it goes wrong".

One could make that argument about solid-state electronics, the move away from punch-cards, the move from paper-based filing, the move to journaled filesystems, etc.

Sometimes progress means letting go of the past, and sometimes it takes a while to fully bake; thats why RedHat doing the QA, testing, and development for the rest of us is a good thing. If it sucks, it will die, and noone really has to acknowledge that it ever existed.

Doesn't have to be that way. After all, take a look at OS X - it replaced init with launchd to manage system startup daemons. It later on (around 10.4) replaced syslog with a different version that doesn't quite use text files anymore - it's stored in a database. Console then loads that database and presents it in plain-text style, letting you see it as a regular log or manipulate it.

I suppose we can blame Apple for influencing LInux?

Re:One of the advantages of Linux

[LordLimecat]

No, Windows suck as much as people think (if not more)

People's perception of its suckiness tends to center around security, despite the fact that every single Pwn2Own has featured a Unix OS (OSX) getting owned simultaneous or before the windows machine. And despite the appearance of features like granular filesystem security, kernel patch protection, ASLR, DEP, etc often before they appeared in Linux OSes.

I would be curious to hear in what ways you think Windows is fundamentally inferior to Linux. Each have design advantages, and are good at certain things, but almost every claim ive seen about how "this OS or that OS (speaking of the modern, still-in-development ones) is inferior" tend to be based on ignorance, not reality.

Re:One of the advantages of Linux

[fahrbot-bot]

Looks like they're pulling the same shit Ubuntu pulled with upstart (init replacement).

To be fair though, Sun/Oracle did the same thing with the Solaris Service Manager. While a minor pain in the butt, the SM actually does a lot of "good things" like providing parallel startup of services, which results in faster booting, and restarting of failed services, etc... And both Ubuntu and Solaris still support legacy {init,rc}.d scripts.

Re:One of the advantages of Linux

[garaged]

Nothing beats plain text for real time monitoring, just try any DB based IDS system

Re:One of the advantages of Linux

[LordLimecat]

Because the beauty of RPM and the linux CLI is that you are 2 commands away from having the sucky technology replaced with the syslog you love. Its really hard to have an entrenched, hard-to-get-rid-of software based on Linux that can be managed by a package manager.

Re:One of the advantages of Linux

[LordLimecat]

I dont think dmesg is going to start printing binary garbage all of a sudden just because the backend storage format is now binary.

And if the system is totally hosed, you can always load the disks into a functional system or boot a LiveCD distro and view the logs from there. Having verifiable integrity on all of the log entries can hardly be considered a bad thing in that scenario.

Re:One of the advantages of Linux

[LordLimecat]

I think we begin to see the problem, if people didnt realize you could do this.

Re:One of the advantages of Linux

[0123456]

I would be curious to hear in what ways you think Windows is fundamentally inferior to Linux.

The big problem with Windows is that it's tied to horrible design decisions made twenty years ago because if they change anything they'll break the proprietary, closed-source WhizzyWriter word processor from 1993 and customers will start whining.

Look at the security hole a while back with Windows loading DLLs from remote file servers when you started a program, for example. For decades Windows has loaded DLLs from the current directory when you started an application, and so various old programs rely on that behaviour and because they're closed source they can't be fixed and because Windows only sells through backward compatibility, users can't be told 'it's broken, tough'.

With Linux the vast majority of software is open source and will be fixed to work with any new changes, and old closed-source software 'it's broken, tough'.

Re:One of the advantages of Linux

[MightyMartian]

MySQL requires the daemon to be running, or at least access to some utility with the MySQL library. If a system has crashed or has reduced functionality due to system problems, a text log that can be scanned with the basic *nix stdio tools is a helluva lot more useful than a binary log.

I hate the Windows eventlog and binary logs in general precisely because they become rapidly less accessible the more issues a system has, which is quite often why you need to delve into syslog anyways. What exactly is the point to reinventing the wheel?

Re:One of the advantages of Linux

[MightyMartian]

So now on top of a crippled system, you've got to move the logs over to a system so you can read them? This is exactly what you're faced with when a Windows system takes a dive, and it sucks.

Re:One of the advantages of Linux

[Billly Gates]

"As has been pointed out, there is no reason you couldnt use a new tool to get the output you want out of the database." ... and end up with the wonderful Windows Registry? 9 out of 10 times non virus related PC problems are ALWAYS related to the registry due to database/index corruption. It is autrocious and prefer Linux because it is text based.

MySQL corrupts and is why real DBAs from other platforms scratch their heads, and insist on PostgreSQL if you want free. Text is just easier not to mention fsck can repair a corrupted file. Not a database or an index however.

Re:One of the advantages of Linux

[Hatta]

Ask Ubuntu users who prefer Gnome 2 about that.

Re:One of the advantages of Linux

[MightyMartian]

gzip is a pretty small utility, just like all the other text-processing utilities. Beyond that, no system I've worked with gzips the active log file, just the archived ones. The active log remains uncompressed because if there is, for instance a file system corruption/crash, a compressed file is often rendered useless, whereas you can still have some hope of analyzing even a corrupted text file for clues.

Re:One of the advantages of Linux

[Billly Gates]

Wow Unix users are so conservative as your post shows.

Windows changes stuff and formats all the time and no one has an uproar. Unless it has a cell phone GUI :)

Sun got a bad rap for replacing inet several years ago with a more event driven initialization/startup system. Apple thought it was a good idea as many MacBooks go to sleep in one network and wake up in another with a different policy and a different set of rules and so on and figured they needed a new netinfo and inet system that was event driven more than anyone. Then it became cool after Apple and people realized inet is quite obsolete and silly.

RHES I am sure are not going to do anything retarded hopefully and keep it in a text format and probably going to do something event driven as well which will be a blessing on a complex server where you wont have hundres of scripts doing something based on what another script does etc. Or they will have more options and reporting with deep internals from the kernel like Windows and Solaris has.

Re:One of the advantages of Linux

[bobaferret]

He still didn't address hack-proof check-summed log entries. I'd love to stop paying annual maintenance for Splunk. I do get t-shirts from them though.... But seriously being able to securely archive in an audit-able format 7 years of log files is needed in the compliance world. But I'm of the opnion that this should be done with a separate tool or something. Anyway just pointing out that it wasn't addressed.

Re:One of the advantages of Linux

[Billly Gates]

SQL or any indexing or Database is BAD.

They get corrupted and no way to fix corruption from fsck unlike regular files with a journaling file system. ...ok some smartass will say even with a journaled file system incomplete transactions get deleted, but mostly the file system will stay healthy.

If the log system gets corrupted from say a system crash or kernel panic you will have no record of it as the database will more than likely get corrupted unless it is a heavy enterprise one like Oracle or Postgres. In that case you will have performance issues.

XML. I hate it as it breaks grep and sed for people who want to do things like at 3:15AM after every backup go grep /etc/var/syslog with a sed -i "crapware backup sucess" > sendmail etc. True I bet you could use a a whole programming language like Perl and call an XML api, but that kills the point for a sys admin. A simple bash script would be more appropriate. Powershell in Windows has that problem as you need to be a .NET developer to use because of XML: files :-)

Nothing wrong with text. Hopefully JSON will work, but I would hope they could replace syslog but keep the format as a text file.

Maybe like Microsoft with the registry, is that RH is concerned over piracy and ORacle and CENTOS stealing their marketshare. It maybe opensource but storing proprietary and cryptic keys might be to prevent the Redhat Kernel from booting.

Re:One of the advantages of Linux

[aesiamun]

Yeah that's efficient... :-/

Re:One of the advantages of Linux

[FoolishOwl]

Hey, who would expect a Linux sysadmin to be familiar with MySQL? You may as well expect them to know Apache and at least one of Perl or PHP or Python.

Re:One of the advantages of Linux

[Billly Gates]

There is a difference.

I was thinking inet fiasco on how Sun and Apple get nailed before it was considered cool when Cannonical decided to do so. It was still text based and backward compatible.

This is database driven with the potential to turn inot the next Windows registry. You know the one where 9 out of 10 PC problems are related to it when it gets corrupted or bad values get thrown in etc? It can get corrupted too if it is in an index. So if the system crashes when it is writting to your disks the system logs get corrupted and you have no idea what happened. Brilliant!

Fsck (or whatever the raid equilivent is. I haven't used Linux on a server in years) can fix text files in a journaled file system. But not corrupted indexes or binary key pairs. You're screwed.

So yes this is a steup backwards. Now if it still stored things in text files but offered event driven and enchanced kernel level logging then it would be cool.

Re:One of the advantages of Linux

[cas2000]

yeah, me too. while it's probably better to have the high-precision timestamps, for me it's more useful to have them readable.

I have the same problem with squid logs - they use unix time_t with milliseconds for the timestamp. more precise but less readable. I filter the lines through a small perl script to reformat the dates when i need to tail or process them:

#! /usr/bin/perl -p
use Date::Format ;
s/^\d+\.\d+/time2str("%Y-%m-%d\/%H:%M:%S", $&)/e;

this is similar to what is mentioned in http://wiki.squid-cache.org/Features/LogFormat but with the improvement (IMO) that the timestamp still only takes one column (compared to localtime() making it take 5 columns), so it doesn't mess up other processing scripts that depend on the detail being in specific columns)

from this;
1322779413.527
to this:
2011-12-02/09:43:33

It would be annoying to have to do that for syslog logs too. I don't really need millisecond precision for my system logs anyway, near enough is good enough. All i need is accuracy and consistency across multiple systems - and ntp gives me that.

Re:One of the advantages of Linux

[alcourt]

Actually, if you are at that point, you need to centralize your logs or use a proper alerting system beyond traditional logging because you are at the point you need to correlate events across systems. "Alert if this event occurs on system A unless you see this other event on system B or C"

Putting a database to manage logs on each system causes grief. We tried that once with wtmp. It hasn't worked out well, and we still don't have a clean replacement for wtmp that records logout times. So instead, I'm faced with custom tools to parse the wtmp file and eliminate events where the user has logged out, leaving events in the file where the user has logged in but has not logged out. (it's painful)

Re:One of the advantages of Linux

[Billly Gates]

The reason people put up with Windows is because they didn't know any better and were new users in the mid to late 1990s when most people first turned on a machine. Us geeks and rich kids did so in the 1980s at home, or perhaps earlier at work with mainframes. We saw how much Windows sucked, but we were a minority in 2001 with IE 6 and WindowsME. Bashing the mac at the time made Windows look better to noobs back then too which was a popular past time that bit us in the ass.

Today, that is changing as witnessed by Vista. Andriod and a resurgent Apple is now changing the minds of people who realize that Windows really does suck. That is the appeal of an IPAD if you just read books, browse the net, and post on facebook.

In the case of RHES the users are experts and wont upgrade. Plain and simple.

Re:One of the advantages of Linux

[symbolset]

XBase for dBase compatibility. The sourceforge project was last updated in 2006, but I imagine not a lot has been done to dBase since then. The format is simple - I reverse engineered it and wrote my own libraries for it once. You'll find NOMAD on Softpedia.

Re:One of the advantages of Linux

[bill_mcgonigle]

tail -f /var/log/messages

In mysql? How?

You missed a requirement: in a form that's still usable when the machine keeps going down hard in the middle of a boot. 'tail messages' still works, nothing to get corrupted or worry about a write-ahead log that can't get consistent.

Not that I spent the day today troubleshooting one of those or anything...

Re:One of the advantages of Linux

[Billly Gates]

Disregard.

I RTFA and realized it was not a txt based replacement and bitching is indeed deserved if that is the case.

Re:One of the advantages of Linux

[alcourt]

For a moment, I thought you were one of the people on our log analysis team that I work with.

There are places where a database is good. I recommend them heavily, but the original log output shouldn't be database. If you want a database of logs (appropriate in many cases), send a copy of the logs off host (which you have to do anyway for PCI and many other audits) and store that copy in a database. That makes it easier to comply with other requirements.

Best of both worlds. Plain text local to box in case Things Go Wrong, and database for cross-system forensic analysis.

Re:One of the advantages of Linux

[alcourt]

RHEL 6 broke that. They stopped supporting syslog and went to rsyslog which is incompatible and is breaking things left and right. Yes, we entered a show-stopper bug on certification with RH.

No, those of you who are rsyslog fans, it is not syslog. It is an abomination wearing a poorly fitting skin of syslog, but the rotting flesh is visible underneath, the miasma of decay of what was once a useful tool has left a shambling terror that cannot be abided.

Re:One of the advantages of Linux

[symbolset]

It's the API legacy issue. Old tools can't cope with changes in the API - in this case, a text stream. New features require these changes.

The right thing to do is a dual implementation, writing to both, until the legacy tools have been updated. Unfortunately some of these legacy tools have fingers and so are very difficult to deprecate. I would say do it both ways if that's what suits you, or let the system administrator choose whether he wants the legacy logging suite or the next generation one, based on his need.

It shouldn't be that hard to write an app to convert the database log to traditional format in an app like "logtail -f /var/log/messages" that maps that final argument's canonical location to the appropriate table and provides output in the traditional format. Add a conditional like "if source stream is a traditional log location and file not found, failback to the equivalent log database converter's output as the stream" to the traditional Unix stream tools, and that should about do it.

I'm a big fan of ASCII text for many things, and for logfiles especially so. But let's not forget that ASCII isn't stored in the computer as glyphs in a row. ASCII data is binary data too. Not that I have a dog in this fight.

Re:One of the advantages of Linux

[alcourt]

Actually, they already have. Red Hat made a real brain dead decision already, and not just in Fedora, on syslog.

We didn't complain about inetadm or svcs. We actually liked those as soon as we saw them. But the lack of syslog on RHEL 6? That's a blocking bug on RHEL 6 certification. No, rsyslog is not syslog. It cannot be monitored to ensure that regardless of the rest of the configuration, it is sending security data appropriately. It tries to send an incompatible format that my other three major platforms (Solaris, HP-UX and AIX) don't use, which of course breaks all my centralized logging infrastructure.

Re:One of the advantages of Linux

[nine-times]

Also if it's broken enough that you can't actually boot, but you can still get access to the files on the disk, then plaintext will be readable by any other OS. If it's a weird binary format, then you have to worry about whether you have the correct viewer installed, if there's even one available.

Re:One of the advantages of Linux

[martin-boundary]

The advantages of plain text vs binary are quite well understood. One of the things they depend on is the purpose of the file format. For archival purposes, plain text is superior. For short term data that must be processed quickly, binary is superior. Logs fall clearly in the archival camp, unless you intend to keep them only for a short time (eg a week or a month).

Re:One of the advantages of Linux

[Todd Knarr]

The problem here though is that the whole reason for the logs being plain text is that the time you most need to be able to read the logs is exactly when things are broken, most services won't start because of the breakage, and your special tools may not be working because most of the system just isn't there. With plain text files, if you can boot into the single-user maintenance shell (not even single-user mode, literally running the shell as PID 1) and get the filesystem the logs are on mounted, you can read the logs and see what happened. With a more complicated system you end up in a catch-22 where you need to fix the breakage to get the tools working to find out what you need to fix to get the tools working.

This is, BTW, why /sbin exists separate from /bin. You couldn't always guarantee that libc was OK, so /sbin had statically-linked copies of critical tools that you could use to fix the system after something had trashed the critical system libraries.

Re:One of the advantages of Linux

[Score Whore]

Change that makes my vendor's life easier (more profitable) at the expense of making my business' life harder (less profitable/incurs expense) is bad -- from my point of view.

Redhat's customers are not in the business of selling linux and linux services. That is redhat's business. The customers' business is web hosting, or ecommerce, or accounting, or HR, or communications. Change that is specifically about making it easier for redhat but doesn't give customers value is bad. This seems obvious to me.

Re:One of the advantages of Linux

[hedwards]

But, those things ultimately were broken, or at least had a compelling downside that needed to be addressed. It's one thing to experiment with how to better solve a problem and quite another to create one that has already been tried elsewhere and found to suck.

The Windows tools work well, until you want to do something that MS hasn't specifically foreseen or doesn't want you to do. It's been decades and MS still fails to provide a convenient way of installing user profiles to a different partition. There are 3rd party tools that will do it and one can use a custom install disc, but MS still doesn't allow it.

I've also found myself in situations where I need another Windows install in order to run the 3rd party tools that are required to figure out why the install went tits up. Something I don't typically have to do with Linux or *BSD as I can just grab the logs with a livecd typically and inspect them on virtually any machine.

Re:One of the advantages of Linux

[shipofgold]

It depends on your skill set.

I personally will get much more information/analysis/correlation with an awk/sed script then I will get with any SQL query, with the added benefit that I can import data from ANY text file and correlate it. I won't be dependent on having all data I am analyzing be in a common format.
TEXT is a common denominator....most any binary file can be converted to text and then the resulting text files can be analyzed. It doesn't work the other way around. You example of windows event logs is perfect....When I want to correlate them to some other text file that isn't in that format, I have to convert them.

It is pretty arrogant on the part of those RedHat devs to think that any system they will devise will have the entire ecosystem of Linux running to use it. You can lead a horse to water, but you can't make him drink. That will always be the case.

Re:One of the advantages of Linux

[LordLimecat]

The big problem with Windows is that it's tied to horrible design decisions made twenty years ago because if they change anything they'll break the proprietary, closed-source WhizzyWriter word processor from 1993 and customers will start whining.

Someone wasnt paying attention when they transitioned to 32-bit, and from thence to 64 bit. Someone also wasnt paying attention when they basically rewrote most everything in NT, and then completed the process in Vista where they DID break everything (which you said they wouldnt do), and provided compatibility in the form of virtualization.

Someone also wasnt paying attention to the fact that MS is big enough that they CAN mandate "update your shit, windows 7 comes out in 1 year and will break legacy cruft", because I saw vendors do it in medical offices, petroleum distributor's modem-based monitoring apps (updated to ethernet and Win7 compatible), and various other areas.

Your beliefs about what they can and cannot do are just wrong; 5 years ago users would say "they cant fix the administrator-by-default thing, because it would break everything. Guess what, thats why everyone hated vista, because they fixed it.

Your example about the DLLs is possibly one of the very few legacy hold-overs that they will stick to, and I had thought they were fixing that. Windows DOESNT sell just through backwards compatibility, people buy Windows because
1) Exchange has historically slaughtered the competition. That (so I hear) is starting to change, but Im not holding my breath; Exchange supports BES and ActiveSync, and through them basically every major phone out there. Besides ActiveSync, what does your android support? IMAP? What about your Palm Pre? POP? Yea, thats real competitive.
2) Office remains the most popular productivity suite out there. Office runs on Windows, with a crappy half-way port to Mac (I understand it is less crappy than it used to be).
3) Most commercial software runs on windows, and I dont mean legacy crap designed for Windows 95.

With Linux the vast majority of software is open source and will be fixed to work with any new changes, and old closed-source software 'it's broken, tough'.

That is a GROSS oversimplification. In many instances, the response is "fuck you, we're the devs and its Working as Intended" or "fix it yourself" or "I no longer maintain this software, feel free to fork it tho". Arguing the merits of FOSS vs commercial / proprietary is nowhere near as simple as you make it; look at chrome vs Chromium and see where all the progress comes from, or compare to Firefox and tell me which is more nimble and innovative, and who gets patches installed quicker.

Re:One of the advantages of Linux

[LordLimecat]

Theres a huge difference between trying to swap out a Desktop Environment in 2 commands (which replaces about a zillion packages) and pulling in a secondary logging resource. Absolutely you can pull in Gnome 2, but its going to break all the GUI defaults-- which im sure is exactly the sort of thing RHEL users care about, right?

Re:One of the advantages of Linux

[LordLimecat]

Um, no, im assuming your bin folder hasnt been corrupted, right? Is there any reason to assume that 'cat', 'tail', and 'grep' would function and 'readJournalLog' would not? Are the cat binaries more robust or something?

Or are you just making assumptions about what would be required to read the new journal?

Re:One of the advantages of Linux

[nabsltd]

You cannot, for example, convince me that noone is able to script MySQL databases, despite their binary nature.

I don't believe that any SQL database has the equivalent of "tail".

Although it probably wouldn't be hard to create a stored procedure that does something like this, I think it would require a trigger on table inserts to not use a lot of system resources. Unless this was planned for by RedHat, you might end breaking their logging system with your attempt to customize.

That said, the current system logger can already output to a database if you want, so I don't see what a new system would offer.

Re:One of the advantages of Linux

[renegadesx]

We got around the rsyslog issues, had to change our build scripts from scratch but we got there.

Im just glad upstart's init support isn't broken considering its a one hit wonder and already has been replaced by systemd in Fedora

Re:One of the advantages of Linux

[LordLimecat]

We saw how much Windows sucked, but we were a minority in 2001 with IE 6 and WindowsME.

It DID suck, I dont think anyone is arguing that.

Today, that is changing

Well thats true.

Re:One of the advantages of Linux

[makomk]

It's the API legacy issue. Old tools can't cope with changes in the API - in this case, a text stream. New features require these changes.

Which Red Hat are planning to change by developing a new Red Hat-specific API that's deeply tied into their init replacement - meaning that other distros won't be able to use it even if they want to!

Re:One of the advantages of Linux

[LordLimecat]

You know the one where 9 out of 10 PC problems are related to it when it gets corrupted or bad values get thrown in etc

Baloney. Just because you bought into the BS spouted by people shilling for registry cleaners, doesnt make it accurate. And a bad setting always has the potential to cause problems, whether its called named.conf or boot.ini or grub.conf or the SYSTEM hive.

Re:One of the advantages of Linux

[makomk]

At least rsyslog is relatively common - Debian and Ubuntu use it too. It looks like The Journal will be basically Red Hat only because it's deeply integrated into systemd and there's no sensible upgrade path to there from the init systems other distros use even if they wanted to change to it.

Re:One of the advantages of Linux

[LordLimecat]

It's been decades and MS still fails to provide a convenient way of installing user profiles to a different partition.

Its not exposed through the GUI, but it takes 1 registry setting to move the location of a user profile. I believe the key name is "profiles", and its in HKLM, but I dont recall the path. To say they dont allow it is just not accurate when they have an article on how to do so-- go ahead and google it.

I've also found myself in situations where I need another Windows install in order to run the 3rd party tools that are required to figure out why the install went tits up

If you mean during the windows install, Ctrl-f10 brings up a console. You can also pull over missing files to a broken windows install.

Regardless, thats not a failure of non-text-designs, but a result of the philosophy of windows. Its not meant to have a zillion CLI tools on the desktop editions.

Honestly, are people also outraged that we moved from the simple FAT filesystem to the journaled and more complex NTFS?

Re:One of the advantages of Linux

[LordLimecat]

If youre dealing with a huge amount of data from multiple sources, I would hazard that SQL is more efficient than a straight up text-based flat-file database. If it werent, there wouldnt be jobs for DBAs.

Re:One of the advantages of Linux

[makomk]

And once the commercial vendors start supporting it then it will slowly but surely make its way into other distributions as well so that these apps can run on distros that other people want to use.

Except that the creator of The Journal has carefully made sure to make this as hard as possible by integrating it deeply into their init replacement which no-one else uses, systemd. Indeed, I don't think other distros could even move over to systemd; upgrading init is hairy even when it's designed to support the upgrade pathway, and somehow I have a feeling that systemd isn't going to support upgrading from a different init because Red Hat just get their customers to reinstall every major release anyway.

Re:One of the advantages of Linux

[LordLimecat]

I could write in about a day a logging program that would write binary data in a key-value format with hashing-- as they describe here-- which would meet all of your requirements. And neither am I a programmer, nor would I even have to use a full-fledged programming language.

It seems like people are assuming a ton of things about how said system will and will not function, just because of experience with totally unrelated database systems.

Re:One of the advantages of Linux

[LordLimecat]

I hate the Windows eventlog and binary logs in general precisely because they become rapidly less accessible the more issues a system has

Not really, the requirements for reading event logs boil down to one or two DLLs, and MMC.exe. Ive been on systems where the whole windows folder disappeared (due to a wigging out drive and filesystem), and was still able to read data out of the event log by pulling those one or two files over.

And whoever said they were using mysql?

Re:One of the advantages of Linux

[LordLimecat]

up with the wonderful Windows Registry? 9 out of 10 times non virus related PC problems are ALWAYS related to the registry due to database/index corruption.

Bull. Ive NEVER seen that occur. I HAVE seen issues with bad permissions causing issues (programs cant read a registry key), but that can occur in any system with access controls. Ive also seen issues with incorrect settings, but that can occur in any system where you can configure settings. And filesystem corruption of vital system files can happen whether you are using a binary format or a text based one.

Im glad to see so many windows "experts" have bought into the "fix your registry now" scam, though.

Re:One of the advantages of Linux

[mabhatter654]

Wow.. I forget there are OSes that don't have serial panel output for error codes! Or Operator "key" codes to force diagnostic modes?

Coming from an AS400 background this article shows how silly "normal" system management is. To answer the parent, the system can be issued boot time commands in HARDWARE (which are also available for virtual machines) that will bring the system to a minimal "restricted" console state. That's like a cornerstone of the system and IBM doesn't mess with that.

Next, the proposal Red Hat has is a very AS400 concept. The History Log (QHST) and the Security Audit Journal (QAUDJRN) are both binary structures that have hard-coded readers built into the kernel. The system maintains internal integrity of the files with extra fields you never see. Of course the AS400 native file system is "DATABASE"-based. So any command that outputs from these displays to a screen or to something that can be instantly searched with SQL.

I'm certain that is what they are trying to make here. Ultimately in security you care about the integrity of the logs more than even recovering the system... Especially when to don't have to restart for a year at a time or more. Frankly, they should add an output option for SQLite file types and everybody can be happy.

Realize that when places like banks use AS400's they mirror the raw audit journals off to another system, often hundreds a day. Because the are binary journals, they are difficult to tamper with because they interlink with eachother. Yet at the same time because they are a data type programs can monitor them for specific events automatically and it's trivial to set up actions to take.

Re:One of the advantages of Linux

[PlusFiveTroll]

In theory, you'd just use the Red Hat recovery cd/dvd disk and have all the tools you need.

Also this is a viewer available, it's already on the Red Hat system. If someone was to make a statically compiled version of the tool, you'd be able to use it on most future RH systems.

Re:One of the advantages of Linux

[sjames]

Fedora is just their unstable distro which eventually turns into RHEL. Kinda like Debian's Sid.

Re:One of the advantages of Linux

[alcourt]

Build wasn't the killer.

How do I, three months later after people have legitimately added custom log events, validate every location that gets security events? How can I prove that the logging is correct because I can't just check in a single configuration? (Don't get me started on how this file should be checked in on every system. Just ... don't.)

My other suggestion to RH was to provide a parser like logger that I can provide an arbitrary log message and it will tell me the destination. I doubt it will be created though.

Re:One of the advantages of Linux

[dudpixel]

the way open source normally works, is that they will do their own thing for a while and if the rest of the community decide its a good idea, they'll adopt it later. eventually, things will merge again and everyone will follow a similar course.

Interesting to watch.

Re:One of the advantages of Linux

[colinrichardday]

Is there any reason to believe that Knoppix (or other Liveon CD distros) has a readJournalLog function?

Re:One of the advantages of Linux

[colinrichardday]

But then you are tied to Red Hat.

Re:One of the advantages of Linux

[sjames]

While punch cards do have a few unique attributes not shared by a text file (mostly bad ones, but a few good), the journaled filesystem looks just like a non-journaled filesystem to the rest of the world. Solid state devices looked to the user just like the vacuum tube devices they replaced except they came on quicker and didn't get as hot.

Re:One of the advantages of Linux

[LordLimecat]

Theres certainly no reason you cant use a package manager to install it to RAM-- you DO know you can install packages on a live-booted os, right?

Presumably, if it becomes a big deal, they all will have it, and youre not going to convince me that a corner case of a corrupted bin folder requiring you to install software to ramdisk on an old LiveBoot distro is somehow a serious argument against what RedHat is doing.

Re:One of the advantages of Linux

[BitZtream]

and end up with the wonderful Windows Registry? 9 out of 10 times non virus related PC problems are ALWAYS related to the registry due to database/index corruption.

You're flat out full of shit :) I'd bet $100 you've never in your life experienced real registry corruption, more likely you at best have had something mangle the actual data not the data base.

Just because the wrong keys get deleted doesn't mean your registry is corrupt any more than rm -rf /etc makes your filesystem corrupt, those both make it unlikely you're system will boot.

No one said they'll use MySQL. A properly designed database won't get corrupted beyond readability any more so than a text file.

Re:One of the advantages of Linux

[Kjella]

Looks like they're pulling the same shit Ubuntu pulled with upstart (init replacement). "Let's replace something simple and elegant with something complex, incomplete, and very difficult to fix when it goes wrong".

And that actually improves boot time quite considerably. I suspect a XKCD link with "Does it run flash full-screen?" "No, but who needs that?!" would be relevant here. Rebooting a server is going to boot a bunch of users and is a big no-no outside scheduled maintenance windows, which are rather generous compared to the boot time. On a desktop there's just one user and cold boot -> working desktop or restart -> working desktop are two important measures. Throw up all the "yes, but you shouldn't reboot you should do $whatever" rhetoric that you like but rebooting is not a big deal to the user. He doesn't care that maybe with a bunch of commands you can unbork what you just borked if a reboot will do it.

How hard can it possibly be to offer some kind of ordered/staggered/synchronized/single threaded toggle anyway? Yes, a multi-threaded boot is complex because you have all the fun of multi-threaded software. If you don't want to deal with it, you shouldn't have to. But at least realize it does provide value to some people even if it doesn't provide any value to you. I'm not going to blame slow boot time on Linux desktop share being <1%, but there are many, many such things that all together say "Great in the server room, but let's leave it there..."

Re:One of the advantages of Linux

[Gizzmonic]

Baloney. Just because you bought into the BS spouted by people shilling for registry cleaners, doesnt make it accurate.

Hah. Guess you've never actually worked on Windows computers.

And a bad setting always has the potential to cause problems, whether its called named.conf or boot.ini or grub.conf or the SYSTEM hive.

So...what's easier to diagnose and fix? Hundreds of thousands of obscure key/value pairs in the SYSTEM hive or changing a few lines of a .conf file?

Re:One of the advantages of Linux

[cstacy]

ASCII logfiles will be readable in spite of spot corruption and can be processed with grep / head / tail

How are those logs being corrupted, and what is the nature of this corruption? And why would binary logs not be similarly "readable"? In fact, isn't more likely that a transaction-oriented database would be MORE recoverable than your (somehow?) garbaged ASCII file?

Re:One of the advantages of Linux

[BitZtream]

PostgreSQL is happy to notify clients of new data view PUSH methods. Perhaps you just need to learn how to use it.

Re:One of the advantages of Linux

[cstacy]

The problem here though is that the whole reason for the logs being plain text is that the time you most need to be able to read the logs is exactly when things are broken, most services won't start because of the breakage, and your special tools may not be working[...] With plain text files, if you can boot into the single-user maintenance shell [...].

This is, BTW, why /sbin exists [...]

Rather than /sbin/grep there should be tools to to some very minimal examination of the syslog database, and support for the most appropriate manner of remote access by the forensics-debugging machine. I know you can do things like this, since I used systems that did this kind of thing more than 25 years ago. (Yes with binary databases, remote connections over serial cables and/or over the network, etc, etc. Ugh. Don't get me started...)

Re:One of the advantages of Linux

[smellotron]

Is there any reason to assume that 'cat', 'tail', and 'grep' would function and 'readJournalLog' would not? Are the cat binaries more robust or something?

If I had to guess, I would say it has something to do with the camel-casing.

Re:One of the advantages of Linux

[swalve]

Linux can feed errors to serial. My problem with switching to database models for configs and logs is that there is no point to it. It seems like it might be one of those deals where it saves the programmers some time, but costs the users tons of time.

Re:One of the advantages of Linux

[swalve]

Wouldn't it be something like select * from errorlog orderby date limit 10? That's WAY easier than tail 10 errorlog.

Re:One of the advantages of Linux

[tqk]

Is there any reason to assume that 'cat', 'tail', and 'grep' would function and 'readJournalLog' would not?

Yes. Your system is really hosed, so you boot it with a LiveCD to read the logs. Does your LiveCD include readJournalLog? Do you really want to play with that possibility?

syslog doesn't need the performance hit that writing binary records would give it. Recovery DOES need the transparency that text offers.

readJournalLog ~= Windows' Registry. Dumb. Seriously dumb.

Re:One of the advantages of Linux

[AragornSonOfArathorn]

But then you are tied to Red Hat.

If you don't want to be tied to Red Hat, you won't buy Red Hat in the first place.

Re:One of the advantages of Linux

[adri]

.. except that relies on your database indexes, and woe be if you have a 1 million line file.

Re:One of the advantages of Linux

[Score Whore]

That's quite the little attack rant. It's so cute. Do you feel better now that you've got it out of your system?

No idea if you're the original AC that posted the content free "change is bad argument" bullshit. But that comment was total bullshit. Unless one provides additional context the post had zero meaning. Zero. It meant nothing. Why? Because change is bad. Change is also good. And it's also neutral. Throwing comments like that out there without the requisite rational are like whipping your penis out and wiggling it in the face of the gal you're trying to pick up. You might think it's convincing, witty and awesome, but it's just stupid and it's not going to get you laid or even a phone number.

If you'd even bothered to read and understand my comment you would not have written the tirade you did. Because I didn't say change was bad. I said changes with particular motivations are bad. I'm not the vendor's beta tester, I'm not running their software for the glory of being in the presence of their product. I'm using it because I determined that it fit my needs at a cost that made sense. The vendor making the wrong changes may well cost themselves right out of my data center.

If you're a software developer, consultant or even a sysadmin, you really, really need to understand exactly why your customer/business has an IT department. It's to support the business, full stop. If you have an idea that seems awesome but doesn't align with that, it's a bad idea.

So in the future, before you grab up your cardboard sword and shield and rush at the adults and squeaking a war cry at the top of your little lungs, you should actually comprehend what they are saying.

Re:One of the advantages of Linux

[DrXym]

I don't see that as a big deal for 2 reasons. First if you are using Red Hat's logging tools, you are almost certainly going to have the client tools for inspecting and if Red Hat are on the ball there will be server and client packages as well as developer libs. The second reason is exactly the same as could be applied to any other package. I can't inspect what's in git's database without git installed to do it.

Re:One of the advantages of Linux

[bill_mcgonigle]

You missed the parents point about security of the logs. If a hacker gets on a system with a text log it is trivial for them to change that log to cover their tracks.

No, he said he feeds the logs to serial - that is, they get recorded on a separate system. You'd have to compromise two systems to cover your tracks on the first, oh, but then a third, etc.

I'm not sure I believe in 'tamper proof' logs on a system where you have root.

Re:One of the advantages of Linux

[justforgetme]

yes, but I think the key point here is that you will add more tools to an already massive toolkit.
When I got started with linux I found it nice that I could just go around and grep, sed and diff
everything and have intelligible output right away.

Even though in general I'm inclined to try out new practices and paradigms I feel skeptical
about this proposal. On one hand you have data manipulation and retrieval RDB style on
the other hand you loose ease of access and intuition because you diverge from Linux text file
paradigm.

Re:One of the advantages of Linux

[LordLimecat]

Hah. Guess you've never actually worked on Windows computers.

I deal with them daily. And I think I have resorted to a reinstall once or twice in my many years on them, believing most problems can be solved with enough research and work.

So...what's easier to diagnose and fix? Hundreds of thousands of obscure key/value pairs in the SYSTEM hive or changing a few lines of a .conf file?

Disingenuous much? Whats easier to fix, an incorrect entry in an ini file, or a deleted /etc folder? What the hell does that have to do with anything, youre comparing a single app's configuration to what is basically a database of settings for 90% of the programs on your system. When things go wrong, they dont just randomly break in a thousand places at once simply because theyre stored in the same file, any more than named.conf is likely to get screwed up simply because your exports file got screwed up.

Re:One of the advantages of Linux

[EvilAlphonso]

You actually don't need to use cat (a command designed to concatenate files) or grep, you can get around just with the shell internals. So with a text based log, you can still go around checking the logs if /bin went the way of the dodo.

Also text is very disk corruption resistant compared to binary... what happens to your nice binary format when a single block of the disk it resides on is gone?

Don't get me wrong, I can sort of see the appeal of their proposed solution as a complement to the existing syslog... but as a complete replacement?

Re:One of the advantages of Linux

[LordLimecat]

Yes. Your system is really hosed, so you boot it with a LiveCD to read the logs. Does your LiveCD include readJournalLog? Do you really want to play with that possibility?

I assume your live distro includes package management, right? Have you never installed wireshark, or vmfs support, or HFS+ support on a live cd?

Re:One of the advantages of Linux

[Anonymous Coward]

It's a new piece of software. It'll link against a hundred crazy libraries for the sake of it.

Re:One of the advantages of Linux

[LordLimecat]

Also text is very disk corruption resistant compared to binary

Thats a gross oversimplification; text is a subset of binary. Binary allows for things like journals and integrety checking.

Re:One of the advantages of Linux

[hawkinspeter]

Where's my mod points when I need them?

Someone mod parent up please.

Re:One of the advantages of Linux

[Aighearach]

Is there any reason to assume that 'cat', 'tail', and 'grep' would function and 'readJournalLog' would not? Are the cat binaries more robust or something?

If I had to guess, I would say it has something to do with the camel-casing.

Yeah, the camels would be more robust except the bin cats all have tophats.

Re:One of the advantages of Linux

[Aighearach]

ASCII logfiles will be readable in spite of spot corruption and can be processed with grep / head / tail -- if these primitives don't work it probably doesn't matter what's in the log.

No. If things like grep are broken, it is that much more critical to find out what went wrong. Some response is needed. Bad hardware, bad software, security breach... you need to know which it is, and what in particular.

Re:One of the advantages of Linux

[garatheus]

I thought that was the point...

Re:One of the advantages of Linux

[pedestrian crossing]

As has been pointed out, there is no reason you couldnt use a new tool to get the output you want out of the database.

And when you get the infamous "Event Log Corrupt" message, you're screwed (happened on an Exchange system that I -really- need the application log for just the other day). If plain text gets corrupted, you can hopefully at least see something. If a binary format gets corrupted and you are depending on the bin->text converter, your mileage will vary considerably, generally in a bad way at the worst possible time.

Re:One of the advantages of Linux

[Aighearach]

I used to see that all the time.
Presumably it breaks less now than it used to.
But it did take them over a decade to shake the bugs out of it.

Re:One of the advantages of Linux

[Aighearach]

Comparing your user ids, I'd say he probably is in fact old enough that, like me, he's seen numerous corrupt registries.

You apparently have no idea how sucky it really used to be. I'll bet if you did a search in MS's own docs on registry corruption, you'd find numerous posts where they talk about fixing it in different scenarios.

I'm glad windoze used to suck so bad, if commercial software had worked right I might never have given it up!

Re:One of the advantages of Linux

[xenobyte]

If it sucks, it will die.

On what do you base this assumption? History is littered with sucky technologies that became standard because someone important was pushing it.

You are so right. Most 'standards' are not the best possible solution to the matter at hand. The reasons for the bad choices are many but they certainly don't get any better from becoming defacto 'standards'.

Do I need to mention examples?

If yes, just think consumer electronics and similar technology for the general public. Why VHS when betamax was far better? Why MP3 when Ogg Vorbis is both better and completely free? Why internal combustion engines and not steam when steam was both cheaper, safer and more efficient? Why 110-120 volt when 220-240 volt can deliver the same effect with half the amps, requiring much thinner wires and lower waste heat? etc. etc.

Yes, I know the answer to most of these... competitors have always battled for market domination and as a contrast to nature it rarely is survival of the fittest which my above examples should illustrate.

Re:One of the advantages of Linux

[Anonymous Coward]

You're comparing a machine where the OS is tied to the hardware is tied to a single company to a machine where doing so is heresy. Linux does support some hardware magic but I don't know if anyone uses it anymore. Anyone use serial consoles? sysrq keys?

The iSeries is built around working with DB tables. All the tools are built around that. The argument here is that moving from a logging system that uses a known set of tools to a system that is going to need a new set for unknown benefits is unlikely to take off. The equivalent is if IBM decided to start storing all logs as flat files or PC files or whatever they're called.

I was going to say that log security has nothing to do with the file format but you beat me to it ("Realize that..."), making your last three paragraphs pointless.

Re:One of the advantages of Linux

[philgp]

*I* didn't know that! Wow, live CDs are even more useful than I thought they were.

Re:One of the advantages of Linux

[gazbo]

If you're using a sensible DB (these are all Postgres examples), I may be able to make your life a little easier.

If you want to compare rows, don't bother with a slow and clumsy self-join, use a windowing function and the lead/lag functions. e.g. to list the interesting events and time between them:

select date, message, lead(date) over (order by date) - date as time_to_next from log where type='interesting';

That only shows 'interesting' events, of course. If you want to show all the other events between the interesting ones as well, add a partition by clause to the windowing function to highlight interesting events, and lose the where clause.

And yeah, vim is a good quick and easy way to explore data. So in postgres, add 'export PAGER="vim -"' to your ~/.psqlrc file, and your query results automatically pop up in vim (may want to switch to unaligned output in that case using the \a toggle).

Re:One of the advantages of Linux

[floop]

The high precision timestamps in Rsyslog, specified in RFC5424 are not epoch seconds. They look like 2011-12-02T04:31:39.171878-08:00 and yes that's microseconds. High precision is essential for multi-source, high volume collection and analysis. It allows you to see the actual order of events when they're coming in from multiple hosts like a large pool of webservers. But hey, if near enough is good enough for you...

Re:One of the advantages of Linux

[lucm]

On Windows there is a lot of built-in capabilities for log exploring in Powershell or even in VBS/WMI. A toolbox contains many tools, not just grep.

Oh, so you like all that stuff over something as fool-proof, robust and simple as the grep? OK.

I am curious: are you using a fool-proof, robust and simple approach such as print-fax-scan-ocr instead of that crazy thing called email? And when you assemble IKEA furniture, are you sandblasting the screws so they are easier to hammer down?

Re:One of the advantages of Linux

[garaged]

And you need to learn to differentiate possible with efficient and practical

Copy the postgres files to another box and grep it, for instance :)

Re:One of the advantages of Linux

[lucm]

For a moment, I thought you were one of the people on our log analysis team that I work with.

There are places where a database is good. I recommend them heavily, but the original log output shouldn't be database. If you want a database of logs (appropriate in many cases), send a copy of the logs off host (which you have to do anyway for PCI and many other audits) and store that copy in a database. That makes it easier to comply with other requirements.

Best of both worlds. Plain text local to box in case Things Go Wrong, and database for cross-system forensic analysis.

Basically that process is called "chinese whispers" - you take the output of a context-aware source (the application) to an intermediary dumb format (a text file) and then you interpret the dumb format to put data in a database designed to store exactly what the initial source offered. From a business perspective, this process adds exactly zero value but it does add a lot of risk (ever heard of "lost in translation"?).

So the exact opposite of what you suggest would probably be the best of both world - store the actual log in a database, and offer a quick way to export that in text format for the people who need to process text with grep and sed. Less risk, more value.

Re:One of the advantages of Linux

[mcgrew]

Hey, thanks! I grinned at XBase, I wrote a dBase clone in Clipper about fifteen years ago and gave it the same name. I'll have to try it out (hope it's better than the one I wrote). NOMAD was a great dbms, I'll certainly have to install that.

Re:One of the advantages of Linux

[lucm]

Same problem with reporting technologies. Some (such as Microsoft RDL) are xml-based, so it is always possible (at least) to search the source or do a quick fix in a text editor. However with other technologies (such as SAP BO) the reports are stored in a binary format, so you need the whole toolkit even for the most mundane task.

Re:One of the advantages of Linux

[jmitchel!jmitchel.co]

What could cause the corruption:
Cosmic rays, Bad sectors, Weird bugs

Why is this a problem:
"Unless it is very carefully done, this kind of format could lose a lot of logging data as the result of a relatively small corruption"
  (http://lwn.net/Articles/467892/)

Re:One of the advantages of Linux

[tqk]

Does your LiveCD include readJournalLog?

I assume your live distro includes package management, right? Have you never installed wireshark, or vmfs support, or HFS+ support on a live cd?

Actually, no. :-) I've never needed them. I have had times when cat5 access to a router was necessary to complete an install, because wifi didn't work right off the bat.

No network access ~= $missing_some_special_app_just_to_read_syslog

WTF is the point of going that way? Is writing records out to syslog THAT CPU or I/O intensive? If so, WHY? I've worked on borked systems that were flooding /var/log with > 100,000 entries per day. THAT's what needed to be fixed.

I don't get it. readJournalLog ~= Windows Registry! Why go there?!?

Re:One of the advantages of Linux

[MightyMartian]

Are you being intentionally dense. Yes, it's all 1s and 0s, but human beings can read text files, even pretty mangled ones, whereas binary files, depending on the format, becomes far more difficult.

Re:One of the advantages of Linux

[LordLimecat]

The point is that everyone is being ridiculous about this. They are positing situations where they are responsible for recovery of vital data on a RHEL box, but then pretending that they are unable to fulfill basic tasks like "Install the prerequisites and binaries necessary to read the RHEL-supported log format" are beyond them.

If this takes off do you REALLY think that a Fedora live cd wont have the binaries on it? Or that it wont be in the debian repositories? Are you REALLY suggesting that a this vital server is running on wifi, and you wont have the broadcom drivers (what, will you need a GUI to install it too?)

Good grief, I begin to wonder if im underrating my Linux skills-- apparently noone knows how to script, or use package management.

Re:One of the advantages of Linux

[LordLimecat]

Then we should move to text-based filesystems, rather than the complex btrfs, ext4, and ntfs, right?

Re:One of the advantages of Linux

[LordLimecat]

Malware doesnt just randomly hit 10,000 places at once; clearly YOUVE never done a by-hand disinfection. Malware targets specific things necessary to cloak its infection, most often the following:
1) exe file handlers-- register another binary to be launched whenever you double click an exe
2) auto-run locations-- use live.sysinternals.com/autoruns.exe to fix basically all of these
3) restrictive policies to block task managers and registry editors
4) currentcontrolset services-- will screw with permissions on registered malware services to prevent them from showing up in a scan
5) sometimes, safeboot gets tampered with too

I can in about 30 minutes and one or two tools detect most of that with a casual scan, and it takes not terribly long to fix if you are familiar with NTFS permissions.

I would note that Linux ALREADY has its "registry formats", like gconf; but then this was never about being rational, it was about ignorant people spouting off about things they dont understand because "theyre binary" and "binary is bad".

Re:One of the advantages of Linux

[LordLimecat]

You can also roll your own tech disk. If you want to see a customized version with all the Ubuntu crap ripped out and tons of networking tools dropped in, take a look at this:
http://98.175.96.149/files/TechToolkitV1.4.iso
Its based off of ubuntu, and I remastered it myself about 1 1/2 years ago.
Note that it has a remote control app (bomgar) baked in under ~/.bomgar, which autostarts on login. You can remove it and remaster it by installing it to disk and then running remastersys.

Re:One of the advantages of Linux

[LordLimecat]

You might need to register some dlls to get it to work.

Re:One of the advantages of Linux

[LordLimecat]

So the answer is "yes, everyone is making wild, worst case assumptions without a shred of evidence". Well, good to know those great IT person analytical skills are in good working order.

Re:One of the advantages of Linux

[kyrsjo]

Yes. Your system is really hosed, so you boot it with a LiveCD to read the logs. Does your LiveCD include readJournalLog? Do you really want to play with that possibility?

I assume your live distro includes package management, right? Have you never installed wireshark, or vmfs support, or HFS+ support on a live cd?

So now you have to get WLAN working before you can read the log, read the log, and then maybe reboot to do something - and then rinse-repeat... Great.

Re:One of the advantages of Linux

[MightyMartian]

Is there some particular reason you're being an ass?

Re:One of the advantages of Linux

[tqk]

The point is that everyone is being ridiculous about this.

Possibly, but that includes *both sides*. I've yet to see anyone say *why* this's even being proposed. What is the problem that this approach solves? What's wrong with leaving the local logs text, and sending $whatever_format to the *LOGSERVER* to do as it pleases with them?

I certainly agree that a server is seldom going to find itself in a place lacking easy access to a network. Servers aren't the only things that FLOSS installs on, you know?

Besides, there's lots of live CDs. This'll have to propagate to SystemRescueCD, GPartedLiveCD, Freesbie, PCBSD, ...

WTF for?!? Just 'cause it *can* be done?!? That's foolish "make work" BS.

Re:One of the advantages of Linux

[FoolishOwl]

I'd expect that, if readJournalLog is critical to a Red Hat distribution, that Red Hat would put readJournalLog on the recovery CD. If it's a small enough executable, given Red Hat's influence on the entire Linux ecosystem, before long most current LiveCDs would include it.

Re:One of the advantages of Linux

[LordLimecat]

Are you SERIOUSLY speculating on a situation where your vital server is running off of WLAN?

Do you even do IT work for a living?

Are you even aware that it is trivial and the work of an afternoon to re-roll a live distro with support for whatever you want?

Re:One of the advantages of Linux

[colinrichardday]

For Red Hat, yes.

Re:One of the advantages of Linux

[Score Whore]

I'm not so much arrogant as not stupid. The replies I've responded to have either been zero content or completely unresponsive to my posts. I wrote the post the way I wrote it because it was evident that the commenter I was replying to has reading comprehension problems and I felt an analogy in a completely different context may help. And given the apparent childishness of the poster the available contexts that might suffice to convey the requisite message were limited to those that resonate with the childish, teen-aged mind. Which means a penis reference or something scatalogical. I went with penis, sorry if you would have preferred something involving giardia or flung poo and perhaps next time I can cater to your peccadilloes.

As far as demonstrating my superiority, how could I not? On the subject of whether certain types of changes are bad for one's customers it is self-evident that I understand the question better. If you don't like the tone of my post, then perhaps you should read this comment sub-thread and see which poster it was that began the ad hominem attacks instead of challenging the point being made? I wasn't the one that used "you" or "your" nine times in three sentences. I didn't start making judgements about the commenter I was replying to. Ultimately when a fly is buzzing around annoying one, one swats it.

Re:One of the advantages of Linux

[Todd Knarr]

And that's one of the reasons things like sash exist. There's a long-standing tradition of trying to pare down /sbin (and /usr/sbin) to the absolute minimum number of items. The fewer things you need to depend on working, the easier it is to deal with the world blowing up.

Most people get to gloss all this over, because when the world blows up they just call the sysadmin and tell them to fix it. The sysadmin doesn't get that luxury.

Re:One of the advantages of Linux

[ckaminski]

You're missing the point. You're doing more work, for less benefit. What are the benefits of changing a 30 year old function that works fucking awesomely? Especially when combined with logrotate.

Re:One of the advantages of Linux

[Grishnakh]

Event logs are not like relational databases. As someone else said here, ascii text isn't the most efficient thing around, but logs aren't meant to be efficient, they're meant to be simple and easy to use when things go bad, and when that happens, ascii text is the easiest thing to deal with. Event logs also aren't multiple gigabytes in size normally. Finally, event logs have totally different requirements from databases. To the OS, they're very simple: they're just a text file, and when something happens the OS just appends some extra data to the end. That's it. Databases are totally different, because they have to guarantee atomicity and reliability in the face of concurrent reads and writes. Event logs don't have that; you don't have to worry about one process reading the log while another process is writing something in the middle of it. They're append-only; two processes aren't going to write to the same piece of data simultaneously like a database has to worry about.

For what databases have to do, they achieve robustness and performance through the way they do things (which is not ascii files). Event logs don't have the same requirements, so they don't need to do the same things to achieve robustness; they already have robustness just by being so simple.

Re:One of the advantages of Linux

[Grishnakh]

One of the reasons many people abandoned Windows and moved to Linux is so they wouldn't be tied to a single vendor with lock-in strategies, weird binary-only formats that you need special vendor-supplied tools to read, total lack of interoperability, etc. It looks like Red Hat is trying to become more like Microsoft, and I don't think that's a winning strategy. If I don't mind being locked in and having no interoperability, why wouldn't I just buy Microsoft instead?

One of the big advantages with Linux is that if I get mad at one OS vendor, it's relatively simple to move to a competing vendor, since they all use the same standards and are interoperable. Switching from Red Hat (before now) to SUSE or Ubuntu server or Debian really isn't that hard. If RH is going to start trying to employ lock-in strategies and eliminating interoperability, then there's little reason to bother with RH any more.

Re:One of the advantages of Linux

[Grishnakh]

What exactly is the point to reinventing the wheel?

Sadly, this seems to be the trend in Linux-land these days: Gnome3, Unity, Mono, etc. Not just "reinventing the wheel", but chasing after some other group/company/trend by making a poor copy with questionable usefulness and then trying to force it on everyone.

Re:One of the advantages of Linux

[tqk]

Are you SERIOUSLY speculating on a situation where your vital server is running off of WLAN?

Redhat has a few very popular downstream distros (Centos ring any bells?). If Redhat goes this way, it's likely they will too. However, they're not specifically distributing "Enterprise" class software as Redhat is. There's no telling where they might get installed.

Do you even do IT work for a living?

Ad hominem; slick. Have you completed your first year on HellDesk yet? :-)

Are you even aware that it is trivial and the work of an afternoon to re-roll a live distro with support for whatever you want?

Are you even recognizing why you'd need to do that? Because you can't read system logs because you, for whatever bizarre, so far unexplained reason, chose to write critical system logfiles that native *nix apps can't read.

To quote dailywtf, "Brillant!"

Again, WHY?!? For what problem does this provide a solution?!?

Re:One of the advantages of Linux

[swalve]

Data is data. If an attacker is in the system, all the data is suspect. If the method to create the binary data is known, then there will be a tool to edit that data in no time. If they were concerned about data integrity, the logs would be encrypted (as close to) end to end (as possible). The decode key would reside on some other system, if necessary. But really, making it harder for logs to be tampered with is kind of like spitting at the Nazis marching down your street. You've already lost. This is in complete violation of the Unix way- keep it simple, stupid.

Re:One of the advantages of Linux

[swalve]

If my DBMS doesn't know where the end is, I've got more trouble than slow reports. (And I was being sarcastic about it being easier.)

Re:One of the advantages of Linux

[ThePhilips]

"Fool-proof" and "regular expression" in the same sentence do not compute [...]

No prob, just assume in my original comment 'fgrep' instead of 'grep'.

Re:One of the advantages of Linux

[lucm]

So you are not only M$ tool, but also now advertise for IKEA???

P.S. And you think that e-mail is that simple? Ignorance is bliss, they say.

Thank you for sharing these valuable thoughts. I will now go for a long walk on the beach, letting your wisdom and insight slowly sink in my soul until they are anchored forever. (Ok I don't live near the beach, but if I did, I would definitely go)

Re:One of the advantages of Linux

[DrXym]

What difference does it make if zcat is generic or not? The point is that if the file is in a binary format, then a tool can be written that turns it into a textual format. I don't see people complaining they need a special tool to make sense of a git repository, so why should it be a big deal to need a special tool of a log file? If the log files are sitting there on the disk then the chances are the tools are there too. I credit Red Hat with not doing something stupid which is of no use to administrators.

Re:One of the advantages of Linux

[hawguy]

What difference does it make if zcat is generic or not? The point is that if the file is in a binary format, then a tool can be written that turns it into a textual format. I don't see people complaining they need a special tool to make sense of a git repository, so why should it be a big deal to need a special tool of a log file? If the log files are sitting there on the disk then the chances are the tools are there too.

The difference is that there will not be one tool that can be written to turn a binary syslog into a textual format. Dozens of different applications write to syslog, each one could potentially have its own binary format. Apache might write data as ASCII name-value pairs, ssh might use a binary data structure, while Tomcat might write out its data as serialized java objects. Maybe Ruby on Rails will use serialized Ruby objects for its log data.

Just like there is no one universal tool to parse syslog today, there will be no one universal tool to convert binary log files to text.

I credit Red Hat with not doing something stupid which is of no use to administrators.

There are lots of reasons to have a binary log file - much useful data could be encoded into a binary stream. No one is saying that it will be useless to administrators, but I'm saying that it could be a huge headache when admninistrators can't even look at the log file until they find the right tool. Even if I don't understand everything my mail server writes to syslog, I can still read it and guess what most of it means. If it wrote in a binary format, I can't even look at it until I find the tool that decodes it, and if I can't find that tool (or the developer never got around to writing it, or if access to the tool requires purchasing the "enterprise" version of the product), I'm out of luck. But it's not just one tool I'll need - I'll need one tool for each software application that writes to the logfile.

Re:One of the advantages of Linux

[dougmc]

Package management: use it. I would be very surprised if RedHat prevented you from installing whatever logging facility you wanted on your server.

True, but in practice that's likely to be ugly.

There's probably quite a few hooks to the logging facility in other packages that would need to be fixed, Selinux would need some updating, etc.

Hopefully Redhat will just let you choose your syslog daemon and have already made things work with any choice you make. They did that with their mail transport agents, so hopefully ...

First post

[turbidostato]

WTF!? First post and the linked article is already slashdotted?

Re:First post

[errandum]

I would mod you up as funny if I hadn't already replied in this conversation :P

Re:First post

[pscottdv]

It's almost like there are people reading the article before they post! That way lies madness!!

Re:First post

[LordLimecat]

Mod parent troll, Slashdot doesnt have articles, only comment threads. At least _IVE_ never seen any articles.

Re:First post

[RoknrolZombie]

There was an article?

Re:First post

[rubycodez]

it's that little bit of comment-fodder under the ads and above the comments. ad-block doesn't seem to work on it

Re:First post

[Stax]

then what's this : http://books.slashdot.org/story/11/11/30/2216218/book-review-the-cert-oracle-secure-coding-standard-for-java

Reads like an article to me.

Avoid binary please!!

[Anonymous Coward]

When everything else is failing ... you still need to be able to dig into the the syslogs reliably no matter what! One little hiccup and you can easily lose everything in most binary type implementations, while at worst you see a little garbage in the syslogs!

Re:Avoid binary please!!

[errandum]

The problem is, they can be easily cleaned. I see where they are coming from with this, to be honest.

Instead of going binary they could just use a database system, with queries and whatnot. Each new application would bring a new table.

This would deal with two things: With proper access configuration it'd be safe from tampering, and it'd be easy to dig through. The amount of garbage that ends up in the logs is mind numbing. (yeah yeah, I know grep works and I still think this would be a better solution)

Re:Avoid binary please!!

[GameboyRMH]

Or just use a network log server, which is both better from a security standpoint and lets you keep your plaintext logs.

Re:Avoid binary please!!

[errandum]

That's a viable option, like many others, but I still think that a very big and very long text log is something that could be optimized by a database.

Re:Avoid binary please!!

[qbast]

And this has been long solved - both rsyslog and syslog-ng can log to database.

Re:Avoid binary please!!

[RedHat Rocky]

syslog is one of those things that needs to work when things break, so one can figure out what to fix.

Making it more complicated with more things to go wrong goes against this purpose.

Example:
Hmm, database server is acting weird, wonder what's wrong? I'll check syslog. Hmm, syslog is toast. Ah.....

Re:Avoid binary please!!

[Jonner]

Whether you "lose everything" is not necessarily function of whether a format is textual or not. There are plenty of robust binary formats such as file systems and MPEG streams. However, being able to read logs from a different system is essential, so the lack of commitment to a set format is troubling.

Re:Avoid binary please!!

[Tomato42]

If there's one thing file systems have no problems dealing with, it's appending data to file. C has even special flags for this mode of access FFS!

Re:Avoid binary please!!

[jgrahn]

That's a viable option, like many others, but I still think that a very big and very long text log is something that could be optimized by a database.

Why? On a modern system you can grep (or perl -ne, or whatever) hundreds of megabytes (far more than any syslog I've seen) in a few seconds. Less if you gzip it first so disk I/O gets minimized.

Re:Avoid binary please!!

[0123456]

Bingo. When your system is failing horribly, which is more likely to work: writing text to a file, or writing data to an SQL database?

99.9% of the time syslog data isn't of much value. The 0.1% when it is is the 0.1% when things aren't working right.

Re:Avoid binary please!!

[marcosdumay]

Databases don't magicaly add tamper resistence. Neither does a checksum algorithm.

For a tamper resistent log you'll need to send the logs to another device, and make that device as tamper resistent as you can. A printer is great for that, but I don't think you'd want all that paper around, so most people just use another computer.

Re:Avoid binary please!!

[bobaferret]

My rather small installation produces just under 10 GB per day of syslog data, I have to keep it for 7 years, in a searchable and auditable format.

Re:Avoid binary please!!

[errandum]

There are ways to make your application (in this case, the linux kernel) aware of the tampering. And it's still way harder to crack into a database in order to delete a few entries than simply opening a text file and removing a few lines.

And as you just said, the database doesn't even have to be in the same computer...

Re:Avoid binary please!!

[errandum]

As I said in the first post, I know you can tail and grep your database, but that's not comprehensive or efficient. Just because you can, doesn't mean you should, and with the amount of junk that gets produced each day I'd say it isn't really the right choice.

The one argument that I consider valid is that a text file is really hard to corrupt, even if a chunk of it goes missing, it'll still open anywhere. But databases aren't exactly known for being unreliable, so I don't think it is really that much of a problem.

Re:Avoid binary please!!

[bobaferret]

I keep a days worth of log files around in normal syslog text format. While maintaining a separate archival system. I'd go nuts if w/o the text version and grep. I find the archive stuff esp irritating when I need entries from different services in context with each other. I'm with you on the avoid binary, but I also understand that you can't be compliant without it. I think we're going to just have to have both. I'm more worried about them taking THAT option away from us.

Re:Avoid binary please!!

[alcourt]

Look at what log events you are generating. That's a bit on the high side unless your "small installation" is closer to a thousand systems. (We're an order of magnitude higher than you, but we are not a small shop).

A lot of log analysis is looking at those events and determining "this is not a security log, able to cut." or other log analysis to winnow down events that are not required to be retained for a long period of time.

Of course "searchable" has been subject to very interesting interpretations. Most locations and audit rules define it as searchable if you can produce results of the past few months in a day or so and the rest within a few days. (Backup archives count if you can recover reasonably quickly).

Re:Avoid binary please!!

[marcosdumay]

"There are ways to make your application (in this case, the linux kernel) aware of the tampering."

Whithout the help of some piece of hardware, no there aren't. Every mechanims you can invent on software can be broken by some piece of software.

Re:Avoid binary please!!

[errandum]

Not saying it can't be done, but that would involve reverse engineering the process in order to find a key and/or algorithm (or whatever would be used to sign) and I'd say that doing so remotely would be quite the challenge. In theory, you're right, it is possible, but I don't think it is probable.

I was thinking about a cumulative scheme, where the next signature would depend on the one before, that way if something goes missing in the middle, you could be sure you were tampered with.

Re:Avoid binary please!!

[rev0lt]

There are syslog compatible alternatives that already can log to a database.

Re:Avoid binary please!!

[bobaferret]

We're not a large shop, just deal with a lot of data, and multiple compliance regimes. Sort of a gateway between some government agencies, and the general public and other government agencies. We do alright, but it just IS a lot of data.

Re:Avoid binary please!!

[marcosdumay]

Normaly, in security, when one says "In theory ... it is possible, but I don't think it is probable" that one is shortly rewarded with data showing that he is wrong.

On this specific case, you are forgetting to take into account that the algorithm (and software) is open. Also, you are forgetting to take automation into account (there is a computer available for the attacker, after all).

Re:Avoid binary please!!

[errandum]

Ok, I'll say it this way:

If you are extremely lucky, you can break RSA or 256bit AES. is it probable? No. The algorithms used in both are well known, and any security algorithm can be broken with enough time and horsepower, but banks rely on these, so I'd guess there is a reasonable amount of trust in both of them.

Do you even understand what I'm talking about? The algorithm used doesn't really matter, the security key does. The whole system can be opensource and the key can be unique. Can it be broken? Yes, it can, but is it feasible? Is it doable in a reasonable amount of time? Do you have 1 month to spare and a server farm just so you can go back and edit the logs (that by then have already generated thousands of new lines of code that need to be refactored).

No alternative would be perfect, but I'd say most of these would be more dependable that simple text files.

Re:Avoid binary please!!

[marcosdumay]

In theory you can not expect to break 1024 bits RSA or 256 bits AES in a lifetime.

In theory you can forge checksums to defeat any security mechanism that depends just on software running on a computer where you are root.

Do you see the diference there? You are not calling it improbable based on a formal statistics analysis.

Re:Avoid binary please!!

[errandum]

The comment about RSA and AES was just to prove to you that because something is opensource, it doesn't mean you can simply break it by knowing the code.

Never said it was impossible, just improbable that someone from a remote shell would be able to do it in a reasonable amount of time. What you're saying is reverse engineering whatever part hides the key, then whatever part defines the algorithm that is used and then the remake very single hash that was done from the point you deleted a log upwards so that it checks out in the end. Then you have to deal with the database security and integrity. And all that without setting off any kind of bells monitoring the system.

When I said you'd be using a server farm I never though you were thinking about decompiling the kernel. Again, I highly doubt you'd be doing it remotely in feasible amount of time, but be my guest.

And all that can be avoided if a hybrid system (like the PS3's hypervisor) is used - and that's something I can see happening if you own a system sensitive enough.

And in the end, yes, it can be done, but you're comparing it to plain text log securiy where the password you already have (root) is enough to change anything you want? Really?

Just more things to break ...

[tomhudson]

Keep on fragmenting each distro ... at a certain point, people will just get tired of distro-hopping and dump the whole mess.

And people ask when the Year f the Linux Desktop will be. It's things likie this, and the constant breakage because of change for the sake of change or to "be different", rather than focusing on stability, that drive people to non-free vendors.

Re:Just more things to break ...

[Synerg1y]

Not to mention everybody who is running Red Hat in production, who will probably be forced to accept the new features if they like security updates.

Re:Just more things to break ...

[imemyself]

Have you ever used RHEL / CentOS? I hightly doubt they will push out a completely different logging system as a normal update in an existing version.

Re:Just more things to break ...

[LordLimecat]

Keep on fragmenting each distro

The whole point of a distro is that it is DIFFERENT from the others around it, not that it is similar. They all have their strengths and weaknesses, and the various things they try can be pulled into other projects.

For instance, Canonical has been talking about rolling Wayland in as a replacement for X in Ubuntu. It might be a phenomenal failure, or it might be incredibly successful. If it works well, Im sure RedHat, CentOS, Debian, etc will all pull it in as well, and some bit of progress will have been made. If it sucks and dies, well, that too is progress.

Re:Just more things to break ...

[0racle]

So the answer is you haven't used RHEL.

Re:Just more things to break ...

[imemyself]

My point is that RH isn't going to make that kind of change in an existing version of RHEL.

Now upgrading from 5 -> 6, or 6 -> 7 whenever that's released. Yeah...stuff's going to change. Maybe they'll put this in RHEL 7. But I don't this significant of a change would be pushed down as a normal update within v5 / v6 even in a 6.x or 5.x update.

Re:Just more things to break ...

[guruevi]

You can also simply substitute your own binaries on the same platform. It's not because RH gives Syslog by default that you can't install Journal and vice versa. You can have them even running concurrently (where necessary). Most people and even sysadmins don't care what is running the logs as long as it's readable unless they're running a dedicated syslog server where any such changes will be monitored.

The standard daemon for mail used to be Sendmail later Postfix. Yet people still run either out of necessity or preference QMail, Sendmail and I use personally a very simple forwarding daemon that sends it to my GMail account.

THAT is the beauty of Open Source (and by extension GNU/Linux). If Windows slips in a new logger (or task bar, security center, .NET framework etc.) you can't replace it, in Linux you can take your bog-standard distro (Ubuntu, Red Hat) and replace piece by piece very simply.

Re:Just more things to break ...

[GameboyRMH]

Hey before Ubuntu went off the deep end it really seemed like the Year of the Linux Desktop was this close!

Re:Just more things to break ...

[garyebickford]

In my case, I stopped using RHEL when it got to be too much of a PITA, and one too many changes that broke stuff that I was using. It's been a while now so I don't recall the details. IMHO a good OS maintains security and otherwise mostly stays the hell out of the way.

Re:Just more things to break ...

[garyebickford]

I dumped RHEL and started using Ubuntu because it was easier to set up a simple desktop and also run various server apps for development, while RHEL seemed to be trying to take over more and more of the system with its own flavors. Now Ubuntu seems to have decided to go the same way. I'm still running 10.04 and trying to decide what to do.

Each of these companies, IMHO, seems to think that what they are doing is Important, Useful and even Visionary. I think it's more about justifying their own existence and expanding their brand. But (to stretch an analogy way too far), "He governs best who governs least". As these companies try to build their own little fortresses, they must inevitably give up more of the countryside, where trees and flowers and deer and bears can get by all by themselves, thank you very much.

I just want to run a reasonably secure, flexible operating system, and run a good 3D-oriented GUI (I'm a visual kind of guy). WRT this particular 'innovation', I regularly depend on 'tail -f' to watch log files on multiple remote systems (that happen to be running FreeBSD), sometimes filtering through several other programs to narrow down and transform the output. I fail to see what is wrong with the present system, and I don't see the point. I do confess I haven't done the research to really answer myself, but there are many aspects of 'the true *nix' that are that way for good reasons - call it the zen of OS. Sometimes the simplest is best.

If they want to do this, then it should be on the back end of the syslog output - tail the syslog into their fancy-schmancy binary system. This avoids disrupting all sorts of system tools, while providing everything that RHEL wants to accomplish.

Re:Just more things to break ...

[RedHat Rocky]

Pfffpt. It's either bitch because things change or bitch because things DON'T change fast enough.

There will always be someone who is unhappy, change or not.

Linux will continue to succeed BECAUSE there are choices.

Re:Just more things to break ...

[future assassin]

>And people ask when the Year f the Linux Desktop will be.
For me that was 4 years ago when OpenSuse just worked out of the box and then it got replaced by Mint.

Re:Just more things to break ...

[mlts]

Believe it or not, this is one selling point for AIX. IBM touts that if it runs in an earlier version of AIX, it almost certainly will run on AIX 7. In fact, IBM actually has a binary compatibility guarantee that anything running on AIX 5.x or 6.x will run on the latest version.

Of course, change is important, because an OS that ends up stagnant will eventually fall to the wayside, but in a production environment, people want to have to change as little as possible during an upgrade cycle.

Re:Just more things to break ...

[devent]

What have desktop Linux with syslogs to do?
You do know what we are talking about? About enterprise systems and the logs, like the apache log, daemon log, kernel log, etc.
We are not talking about anything that have to do with the desktop.
I'm using Linux desktop now for 3 years and besides a dmesg I never looked in the syslogs. And I couldn't care less if dmesg is a database or plain text.

Re:Just more things to break ...

[Jonner]

Keep on fragmenting each distro ... at a certain point, people will just get tired of distro-hopping and dump the whole mess.

And people ask when the Year f the Linux Desktop will be. It's things likie this, and the constant breakage because of change for the sake of change or to "be different", rather than focusing on stability, that drive people to non-free vendors.

Yeah, Microsoft and Apple never break things just to be (or think) different. Differences between RedHat and other distributions are only a problem to users when they switch, so this will likely discourage distro-hopping. That could even be part of RedHat's motive. They are making their systems more proprietary (in the general sense, not related to copyright licenses) just as Canonical is by using their own desktop environment. This is part of a trend of Linux vendors doing things more like non-free vendors. If it works for Microsoft and Apple, why not for RedHat?

Re:Just more things to break ...

[someSnarkyBastard]

Try Arch or Slackware, both are all about the KISS philosophy and are very configurable as a result

Re:Just more things to break ...

[Mike Buddha]

Evolution won't work. Way too much fragmentation.

Re:Just more things to break ...

[drooling-dog]

It's things likie this, and the constant breakage because of change for the sake of change or to "be different", rather than focusing on stability, that drive people to non-free vendors.

Because as we well know, no proprietary operating system would ever make a change without giving you the choice of adopting it or not...

Re:Just more things to break ...

[marcosdumay]

You mean "what problems this proposed thing proposes to solve". Currently, it solves none of them. In fact, it just creates a new problem because the loger now is so complex it should log its actions.

But I mostly agree with you. Distros should experiment new things, and Fedora is the right place for RedHat do that. It is just that this one experiment is dumb, people aready tried that more times than we care to count, and everybody already knows how it ends. And everybody is being that vocal because we fear that we are forced to take part on the experiment, and that it will last for decades, like already happened on other times.

Re:Just more things to break ...

[marcosdumay]

You are already on Ubuntu... Change to Debian. You can get it without all the opinions.

Re:Just more things to break ...

[reub2000]

I doubt they'd even roll out a change like this to the updates repo for a fedora release. More than likely the next release of fedora will use this program by default, and then whenever RHEL 7 is branched, it will use this program by default too.

Re:Just more things to break ...

[alcourt]

kernel upgrades in RHEL our one of the banes of the production SA team I work with. (I'm on the security team). Eventually, they are forced to do so, and it breaks a ton of things that RH provides, not to mention our apps.

Re:Just more things to break ...

[tomhudson]

Yesterdays' in-place update to the latest version of opensuse left my laptop unusable. So, I did a full install ... after booting into Windows to download and burn a dvd, because I could no longer get on the net (wireless had stopped working many updates ago). All of a sudden, the DVD isn't even visible, and every single DE is a mess. My email failed to migrate (good thing for backups). Printer/scanner (supposedly supported) still isn't. Dual screens? Gone - the hardware is no longer detected. And everything is slow as molasses in February. Wireless still doesn't work.

And to top it off? Dual-booting was broken.

After 15 years, I'm throwing in the towel. If I want a server that doesn't break on updates, I'll use FreeBSD. If I want a desktop that works, I'll use anything except linux.

I'm not happy about it, but the linux desktop is a bad joke.

Re:Just more things to break ...

[tomhudson]

And I couldn't care less if dmesg is a database or plain text.

I used tail -f /var/log/whatever_log often ... you'll certainly care that a log file is plain text when your database fails and you can just look through the plain-text files.

Re:Just more things to break ...

[Fished]

Ha. Childs play. Oracle are guaranteeing binary compatibility all the way back to SunOS 5.6, which IIRC was released in 1997 or so?

http://www.oracle.com/us/products/servers-storage/solaris/binary-app-guarantee-080255.pdf

As a practical matter, I <strong>think</strong> you can still run SunOS 4 binaries, but I'm not sure, haven't tried it in many years.

Re:Just more things to break ...

[RedHat Rocky]

I'm sorry for all your troubles.

Hate to add on top, but opensuse is junk in my opinion.

Re:Just more things to break ...

[tomhudson]

Thanks. I'll be installing FreeBSD to see if the desktop "Customization Contagion" has spread. I'm hoping not ... but if it has, there might be a business opportunity, considering the BSD license, if anyone wants to pick up the ball.

Error prevention?

[esocid]

Not that it bothers me, but in forums people are quick to point out that they think Fedora's choice of kernel numbering is stupid. I mention I'm on 2.6.41.1-1.fc15.x86_64, and the first response is, "that kernel doesn't exist." (And yes, Fedora will move to the standard numbering scheme with 17 if I'm not mistaken)
I've found most of RH's decisions to do something their way is to prevent problems down the road. Same for kernel numbering, it was supposedly to prevent repo errors. I don't know for certain, but I'd expect this to also be the case here.

Re:Error prevention?

[broken_chaos]

I can understand the kernel numbering issue -- the 3.0+ kernels are functionally unchanged from the late 2.6 series, but the version number change did break some userspace tools/scripts that (in a poor, but understandable, decision) relied upon the kernel version, often as being in the 2.6 series. Of course, this was a holdover from the old huge swap from 2.4 to 2.6, where almost everything changed.

With Linus suddenly deciding, "Hey, this version is now 3.0, even though there are no compatibility-breaking changes from the last 2.6 kernel.", some of those userspace tools/scripts broke in unexpected ways with version checks, but didn't actually break in effect. So re-numbering the kernel to stave off needing to immediately fix them without much warning was a fairly reasonable thing to do, for the short term.

Overall, I agree with the decision to move to the 3.0 version numbering, though a bit more warning may have helped. Considering just how much the kernel development cycle changed from the 2.4/2.6 transition (namely in there never being a 'full' development branch), making a very clear, albeit arbitrarily timed, version number swap seems sensible to me.

Re:Error prevention?

[LordLimecat]

Overall, I agree with the decision to move to the 3.0 version numbering, though a bit more warning may have helped.

"A bit more warning" is why we're still on IPv4 (though im grossly over-simplifying).

Re:Error prevention?

[parf]

they fixed user tools in fc16 $ rpm -qa | grep kernel kernel-3.1.2-1.fc16.x86_64

Re:Error prevention?

[armanox]

Fedora just happens to be maintained by Red Hat...

Re:Error prevention?

[marcosdumay]

We are still on IPv4 because the floor didn't explode yet, and the world still didn't end.

When the world actualy ends, we'll all switch. In a fast and unordered way.

Whining by some guy with a log analyzer

[Animats]

This is just whining by some guy who wrote a log analyzer that will no longer be necessary.

QNX has had a simple structured log daemon for years. Reading their log never tails off into junk; you always get a clean, current last record. Their solution even works on diskless systems. In many real-time applications, logs are transmitted to some remote location, rather than being kept on each local machine.

Re:Whining by some guy with a log analyzer

[errandum]

Excuse me, it's true their logging facility might be good, but I was under the impression that the junk you see are the junk applications send, so I have no clue how QNX deals with that.

And sending them to a remote location is possible in linux already.

Never tried to use linux in a diskless system though, so I have no idea how syslog would react.

Re:Whining by some guy with a log analyzer

[CAIMLAS]

I don't see you describing anything which isn't possible (and common) with the syslog format. Most environments have remote syslog servers, you can filter events by event type, etc. and you can get a "clean, current last record" in a number of ways quite trivially (tac or last -f, pick your poison).

Changing from syslog format will cause all sorts of problems. There are literally thousands of syslog analyzers in use, custom scripts, and people who use common utilities (such as the above, plus grep/sed/awk/etc.) to monitor and manipulate the files into something else. I personally use half a dozen different tools for this, such as Fail2Ban and loganalyzer. Many of these ship on the default install on systems because they're useful (targetting specific functionality on top of the very specific and minimal functional role of syslog).

Re:Whining by some guy with a log analyzer

[0123456]

Never tried to use linux in a diskless system though, so I have no idea how syslog would react.

I use linux on diskless systems; syslog writes to a RAM disk or over the LAN to a server which does have a disk.

Is he not aware?

[C_Kode]

Is he not aware how terrible syslog is? syslog is ancient and has several series flaws from security to just stupid limitations. It should have been replaced ages ago.

Re:Is he not aware?

[Nos.]

syslog the application or syslog the protocol? syslog the application? Yes, its past due, and things like rsyslog are much better.

syslog the protocol is fine.

The problem with this proposed replacement is that it does not fix anything. The only advantage it gives is to be able to tell if the logs were altered. That's it. You're far better off with a secondary/centralized logging system. Store your logs in text, compressed, encrypted, in a database, it doesn't matter. Just get them to a different location and then not only can you tell that the originals were altered, you can tell what was removed. All while using existing tools.

Re:Is he not aware?

[jedidiah]

Yes. Yes, we get it.

[Hetfield] "New stuff good, Old stuff Bad" [/Hetfield].

Re:Is he not aware?

[jgrahn]

He may be away of it. But I think he is like a lot of other Unix/Linux people, who has some degree of autism. Where they have a hard time dealing with change.

Personally I have a hard time dealing with change for change's sake.

Re:like linux needs more fragmentation

[jedidiah]

> people would rather pay a couple hundred dollars for windows or the premium involved with buying a mac than use linux for free. what does that tell you?

What does it tell me? You are a liar.

Very few people in fact pay for the premium for a Mac.

On the other hand, most people use Windows because "most people use Windows". It has been that way since Macs were an MC68k based platform. This leads to little things like software not being available for Macs or AIO printer devices not working on Macs.

How any of them handle any particular technical detail is largely irrelevant.

Besides, we're talking about an enterprise server vendor we are talking about here. RHEL is not some MacOS wannabe.

Hasn't this somewhat happened

[Murdoch5]

Didn't Ubuntu already change the original implementation of syslog as specified in the RFC? Can anyone name me a current popular and wide spread distribution which uses the original syslog? All red hat is doing is upgrade a dead standard to something modern.

Re:like linux needs more fragmentation

[gmhowell]

> people would rather pay a couple hundred dollars for windows or the premium involved with buying a mac than use linux for free. what does that tell you?

What does it tell me? You are a liar.

Very few people in fact pay for the premium for a Mac.

OMG, you finally admit that the Mac 'premium' is illusory. No, that can't be it. It would make too much sense.

That works both ways

[Anonymous Coward]

You will also be stuck with all the good choices they make.

Reading what they are proposing it seems that is actually a very good idea. When you get out of hobbyist and small environments and into environments with more demanding requirements about security auditing the traditional syslog has not cut it for years anymore. The first step in many environments is usually to rip it mostly off and replace with some more or less proprietary environment.

The new ideas such as improving the reliability of log shipping, reducing possibilities towards tampering, and improving chances for more advanced log analysis are really awesome things - especially for people who are serious about their logging. Syslog and its text format are legacy poison and it will be good to see them die and vanish. Hopefully that happens fast.

Also, keep in mind that that RedHat is still open sourcing that stuff. They will provide tools and APIs - as they require those also themselves.

Re:That works both ways

[mlts]

Even though the syslog is in a binary format, it would be nice to have it also stored in text as well. For example, on some sensitive machines, I would have the syslog redirect to an IBM3151 serial terminal for real time monitoring. This way, I could immediately tell if a job started at its appropriate time, finished, or caused issues.

IMHO, the best way RedHat should implement this is similar to how AIX does logging. It has its own format for logs that are read using the errpt command. However, one can turn on plain old syslog logging and have that able to be stored in a file, forwarded to a log server, or shipped via a serial connection to a secure log drop that has no network access. It would be nice to have a signed, secure format for logs, but also nice to have plain text to watch in realtime and search from without requiring specialized commands.

Re:That works both ways

[gmack]

Rsyslogd already has plenty of ways to move data to a remote machine reliably, mechanisms to prevent local apps from pretending to be each other as well as the option to log to a database. There is no need to rip out the current system.

Re:That works both ways

[LWATCDR]

Well yes you can.
If you read the post from the developer
"My application needs traditional text log files on disk, can I configure journald to generate those?
No, you can’t. If you need this, just run the journal side-by-side with a traditional syslog implementation like rsyslog which can generate this file for you."
Just run journal and rsyslog and you have both systems in place.
Untile journal does everything odds are that it will be run with rsyslog in parallel. A boots and suspender type of solution. if you want to have it also generate a traditional syslog I would think that adding that functionality wouldn't be all that hard. If needed someone will add that feature.

Re:That works both ways

[Anonymous Coward]

In addition to Linux, I work with AIX a lot.

As said, errdemon is the facility that logs all native AIX messages into a database. This database can then be analysed using errpt. Depending on your display options, it's a syslog-style one-liner per error, or a full page (50 lines or more) per error. The latter output format includes sense data and even (if available) the hardware location code and the FRU number of the failing component. MTTR anyone?

AIX also has syslog in case you have applications that expect it. But AIX syslog is modified so that you can pipe its data to errdemon, or you can setup errdemon to pipe all its data to syslog.

The choice is yours but if you look at the *default* filtering that's already possible with the errpt tool, who needs grep anymore? Let alone logwatch, swatch and similar tools.

Re:That works both ways

[Cyberax]

"My application needs traditional text log files on disk, can I configure journald to generate those?"

Well, I'm going to write a FUSE filesystem that allows to read Journal files as if they were simple text log files. With features like customizable representation.

Re:That works both ways

[JabrTheHut]

I've worked with AIX on and off. They have a system called 'errpt' (error report) which is in binary format and sits alongside syslog. It mostly logs hardware events and certain IBM and other vendors' tools and applications.

It's a pain in the ass to use. It's a good idea. It's listable in a couple of ways, and searchable. But sitting alongside syslog it's just harder to use. The tools to search it just don't have the flexibility to search the way you need to search a log file and the ways you can search a text file. Most of the time you just dump it to text and trawl through it that way.

So why the intermediate step? Just keep it in text format.

Re:That works both ways

[LWATCDR]

I would think that just writing a patch to Journal or forking Journal and adding that functionality would be a better solution but if that is what you want to do more power too you.

Re:That works both ways

[Cyberax]

Not really. I have not yet seen the code for Journal, but likely it'll going to be tightly integrated and small as it's going to be started during the initial phases of system init.

Besides, it's good to have a separate utility to investigate journal files in details - not everyone would enable text-based logs.

Re:That works both ways

[LWATCDR]

I had not thought of that. You are correct that it might make a great tool after all.

Re:That works both ways

[alcourt]

Actually, I work in security logging in a very large environment. The last thing we permit is ripping out syslog on generic systems. We do send a copy of the logs to a central system, but we don't allow the client systems to be touched.

The central copy cannot be tampered with. The local copy is not for security, but stability and immediate usage.

There is no such thing as a secure local log, and pretending otherwise is shameful. As to reliability, you get to pick between two evils. The possibility of logs not being delivered, or the possibility of logs not being delivered. The more complex the protocol, the more likely it is that a message gets devoured by the system. Simple protocols may not have guaranteed delivery, but their simplicity has actually helped ensure things just Don't Go Wrong.

Re:That works both ways

[alcourt]

And in our shop, we shut down error report for all security logs (the ones we really care about in my team) and force it to be syslog only.

Re:That works both ways

[NeoMorphy]

I prefer errpt. I can select based on date and time, or I can ask for permanent hardware errors only, or look for errors on one particular device. I can get the summary of recent events first, and if anything pops up, I can then get details for that particular error. You can't get summaries with plain text, you have to read several screens worth of text, which is a pain.

I've never had issues with errpt not working. I think people, myself included, get nervous because of bad experiences with Windows corrupting binary data files. Amazingly enough, it is actually possible to do it correctly.

Re:That works both ways

[Anonymous Coward]

SMIT happens...

Re:That works both ways

[sjames]

The most secure logging of all is to send it out to a line printer behind a vault door. Try editing that after you root the machine!

Re:That works both ways

[ebvwfbw]

There is no such thing as a secure local log, and pretending otherwise is shameful.

Why do you think that? Hook up a hard copy character at a time terminal and log to that. It's as secure as where it is as long as it has paper. You should have good physical security. There are also ways to have write once devices. The point is, if you are looking at it locally and everything is signed and depends on a chain, you can be sure that what you are looking at is secure even if it is local. You don't have to deal with the big mother central log server. If you have missing timeline that's another thing. However if I see something happened yesterday and I have logs back a week, I'll find you with their new system. If you delete something I'll know that too. As it is right now I'd have no clue. To mitigate that we put it into a database in realtime and to multiple machines. Still not impossible but a whole lot harder to hide what you did. I've caught even very experienced people.

The point is the old syslog is dated and something better should replace it. If you have a better idea, let's hear it.

Are Linux Fans Really About Innovation?

[assertation]

It seems like every time a distro tries to innovate they get a lot of screaming from the linux community.

There is this change, the screaming about Ubuntu going with Unity, screaming with every change GNOME makes.

Is the FOSS really about innovation or just mouthing the words?

Re:Are Linux Fans Really About Innovation?

[jd]

Innovation is fine. Invention is better, but if you can't have that then innovation makes a decent replacement. However, Unity isn't really inventive or innovative, and attempting to force someone to use one DWM is definitely a regression.

You are confusing change/novelty with creativity. They're not the same.

And, yes, there SHOULD be push-back. Once it goes past the early adopters, it will make its way to the Real World(tm) where the REAL critics hold multi-million dollar contracts in one hand and a fine sherry in the other. Those critics know nothing about the value of technology, but they know the price of everything, especially that of technology. You WANT the flaws ironed-out before then. You WANT to have put the software not just through the reliability and quality tests but also through the user acceptability tests and the PR tests. You WANT well-tempered systems, honed to damn-well near perfection.

Because, in the end, without those multi-million dollar contracts, the Ubuntus and the Red Hats of the world simply aren't going to bother. There won't be any development at all if we lose the big players at this stage. Linux isn't a garage development project any more, or hadn't you seen the kernel contribution stats on LWN? We NEED the corporations to want to invest not just the time and money they're spending now but more of it. And we won't get that without the PHBs.

Do the PHBs care about Unity or loggers? Directly, no. They care about image and if the unwashed masses turn away from Linux, that's bad image. If there's a security flaw, that's major bad image. If it costs more for the developers to do the same amount of work because of added inefficiencies, especially when the shareholders are baying at the door, that's lethal image. Doesn't matter if Windows would be worse, PHBs won't think like that. Linux is a gamble and it HAS to pay and pay big.

Re:Are Linux Fans Really About Innovation?

[jmitchel!jmitchel.co]

The wise professional is intensely mistrustful of innovation. Innovation introduces new failure modes and deprecates tested methods and experience. Innovation is always born half baked, unreliable and unsupported. By the time all the problems are solved with a given innovation, it's design will typically prove to be just as compromised and unsexy as its predecessor - and then the cycle of "innovation" repeats. Some innovations have sufficient value to overcome their costs - many don't.

And who said that OSS was about innovation?

Re:Are Linux Fans Really About Innovation?

[assertation]

In other words, the answer is "no" :-)

Re:Are Linux Fans Really About Innovation?

[poet]

Yes,

Linux fans are for Innovation. That is why the most innovating distros have the mind share of the community. What you are dealing with here are people who are not the majority. Remember the 2% always whine the loudest.

Re:Are Linux Fans Really About Innovation?

[0123456]

If this was Linus' idea I bet the masses would follow.

If Linus was developing a replacement for syslog, it wouldn't suck.

The problem is that distros want to throw out something that works and replace it with some that sucks, mostly because 'Windows does it that way' or 'MacOS does it that way', or increasingly because 'iPad does it that way'.

We don't care, when their way sucks. Windows logging, for example, is a joke compared to syslog.

Re:Are Linux Fans Really About Innovation?

[sjames]

Innovation is great. A bunch of fiddling with things, not so much.

Re:Are Linux Fans Really About Innovation?

[visualight]

This isn't about innovation. This is about Lennart wanting to control a few key pieces so that he can unilaterally remove obstacles to his own ideas.

IMO, to date, he has not solved one real problem, and I don't understand why anyone listens to him. His ideas are terrible.

Re:Are Linux Fans Really About Innovation?

[bky1701]

Change for it's own sake isn't innovation. People who defend half of the stupid moves in the Linux sphere seem to miss this fact.

Re:RedHat Considered Harmful

[Scutter]

I think the world would be better off if RedHat went off and annoyed some other planet. First dbus, and now this. Why in the name of all that's holy are they making simple things complicated?

It really sucks that RedHat is forcing this change down your throat. If only there were other options. Alas.

Good

[magamiako1]

It's a good move. Parsing syslog sucks. And I don't care how awesome you think you are as a developer--you need to use the system logging facilities to make it easier on those of us who adminster systems.

At the very least a unified format similar to Microsoft's format would be nice.

ID / DATE-Time / Severity / BLOB OF TEXT

Re:Good

[jandrese]

That is probably the only time I've ever heard Microsoft's system logging compared favorably with anything. In my many years of administering systems, I have yet to ever get a useful piece of information out of any of those logs. It's like there's a requirement somewhere that only useless messages are allowed to be logged, and anything that might help an administrator (like an error message when something crashes for instance!) must never appear. Even if the error is something stupid like a permissions issue, you don't get a Linux like "Permission denied on c:/blah/blah/blah", at most you'll get a "An error occurred" or other worthless message.

Re:Good

[lakeland]

Oh, I thought it was pretty good.

I was writing a database and we already had a remote log monitoring tool. From the DB I could log anything I wanted and have the log monitoring tool pick it up. I got the ability to set severity, provide reference codes for easy filtering and provide free-text descriptions.

Re:Good

[0123456]

Clearly you are clueless at reading Windows logs. Which is godawfully surprising since how easy it is.

It's easy because there's nothing of value in them.

'Service LoadsaBollocks crashed and restarted'.

Yep. That's real helpful.

When I replaced the disk on my laptop I was having a hard time figuring out why Windows didn't work after I reinstalled it, and the only useful message was something about 'Cryptographic service failed to start'. Eventually, a few hours later, I managed to figure out that what it really meant was 'Your recovery disk installed some kind of third-party 'disk accelerator' crap which doesn't work so I can't read the cryptograpy keys from the disk. Please uninstall that garbage'.

A little message like 'Cryptographic service couldn't read the key files' would have been a heck of a lot more useful.

Re:Good

[alcourt]

My experience parsing syslog is very different than yours. We had to think a bit about our filters, but one of our best advantages has been the flexibility of syslog.

Re:Good

[El_Oscuro]

At the very least a unified format similar to Microsoft's format would be nice.

Event Viewer - The Event log file is corrupt

The description for Event ID ( 50 ) in Source ( SomeService ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer.

Nice indeed.

Re:Good

[jandrese]

You were putting useful information in the system log? I'm pretty sure that's against the rules.

Re:Good

[FoolishOwl]

'Service LoadsaBollocks crashed and restarted'.

Yep. That's real helpful.

It's much more helpful than any Windows event message I've ever seen.

Windows event logs seem nearly useless to me, but not because they happen to be in XML instead of plain text. The problem is a general one with uninformative error messages from Microsoft products. The most irritating thing is that nearly everything has an associated specific error code, yet there's almost never any way to find out what those codes mean -- as far as I can ever find out, not even Microsoft knows.

I tend to think of this as the most obvious problem with proprietary software: it's exceedingly difficult to find out what the code is actually doing when something goes wrong.

I don't think the shortcomings of Windows events are likely to be shortcomings of a Linux binary journal.

Control

[mpol]

I'm not sure, but I get the feeling that different groups in the opensource community are struggling to get control of their platform. Gnome peeps are doing their own thing, Ubuntu heads off in another direction. Red Hat does their own things.
The last 8 years were somewhat mixed in this regard. There was cooperation, like on freedesktop.org, but olso fragmentation and diversification. Now it all seems to fall apart somewhat. I don't see the different groups come together.

I'm really not fond of some things that are happening, like Systemd and all the other incompatible SysVinit systems. Also the mess that are the main desktops now. Then this new syslog proposal. I doubt other distro's will take this, I expect they will stick with syslog or syslog-ng.

For myself I think I'm going with Debian (testing that is) soon. Once old-school just meant old stuff, but nowadays it almost sounds like the best thing there is. All the new software with less bugs, but not the crummy new inventions which you'd rather let pass by.

Double standards

[Anonymous Coward]

Just so we get your story straight, Mr. Blogger - when media darling Ubuntu trashes 30 years of platform compatibility and portability by moving away from X11, technology pundits like yourself praise them for being forward-looking and innovative. When Red Hat proposes a better mechanism for system logging that is less susceptible to spoofing log entries, for example, you crucify them on your blog for demonstrating the same qualities?

Hypocrites. At least be consistent, if not objective.

Re:like linux needs more fragmentation

[LordLimecat]

It tells you that people want a Mac, it says nothing about the technical merits of the system; youre simply assuming that their reasons for choosing Mac OSX are technical in nature, or that they have heard of Linux, or understand the difference between a GUI and an OS.

Re:like linux needs more fragmentation

[Zerth]

blockquote fail

Re:RedHat Considered Harmful

[rjmx]

Uh-huh. Have you always specialised in cheap shots, or is this a new development?

You're ignoring the size of the redhat customer base and its extensive use in enterprise systems (my own included). If this crap catches on, it's likely to spread to the other distributions; it's best to stop this exercise in change-for-change's-sake before it catches on.

To explain the point about dbus: originally, as a desktop IPC bus, it probably wasn't such a bad idea. It seems, however, to have spread beyond that point: it starts early in the boot process, and seems to be used by more and more processes every time I look. This might not be so bad if it was well-designed, but it's not: chief among my objections to it is the requirement for a reboot every time the thing gets upgraded (or, presumably, crashes). This is one of the things we all bashed Windows over for years and years -- and now that Microsoft seems to be improving in that respect, Linux starts to require it. This is progress????

Re:RedHat Considered Harmful

[LWATCDR]

You are ignoring that you can install rsyslog and run it and journal at the same time if need be.
You can have both styles of logs....

we didnt get to be

[nimbius]

linux by just doing what we've always done. if we did, Linus would still be grading papers, Stallman would probably use use BSD, and the phone in my pocket would probably never exist.
part of what makes me love linux is the undying urge to try something new. granted, thats not everyones opinion. its a bunch of nerds and hackers and really cool people coming together and having the courage to say, "i just made this new thing."

To Red Hat: thanks for trying something new. i really hope it works out and im eager to try it too. just remember, haters are always going to hate. and because its the community that makes up linux, theres a linux for them too.

"ip" instead of "ifconfig" is old news

[Chirs]

I've been using "ip" for at least 8 years now...it actually allows you to assign multiple IP addresses to a single network device. I don't know how anyone lives with ifconfig anymore.

Re:"ip" instead of "ifconfig" is old news

[MROD]

I've been using "ip" for at least 8 years now...it actually allows you to assign multiple IP addresses to a single network device. I don't know how anyone lives with ifconfig anymore.

You mean like?: ifconfig eth0:1 10.0.0.1 netmask 255.0.0.0 broadcast + Of course, changing this in the Linux world merely moves Linux further and further away from the mainstream UNIXes, making managing a heterogeneous environment even more of a pain than it once was. Better to stay as close to a POSIX/XPG4.2 tool set for interoperability reasons.

Re:"ip" instead of "ifconfig" is old news

[Culture20]

I've been using "ip" for at least 8 years now...it actually allows you to assign multiple IP addresses to a single network device. I don't know how anyone lives with ifconfig anymore.

I live with ifconfig because ifconfig can do the exact same thing. ?

it doesn't even have that advantage

[Chirs]

If you have write access to the log file you can delete/change the entry you care about and then re-hash all the entries after that.

Re:it doesn't even have that advantage

[0123456]

If you have write access to the log, you're root, so that's a pointless complaint.

If you want secure logging you have to send the logs to another machine.

Re:it doesn't even have that advantage

[sjames]

If you have write access to the database...

Reproduce ASCII Output

[TheNinjaroach]

I think the move to binary storage for syslog files could be great for efficiency all the way around. A very simple CLI tool that dumps the ASCII syslog equivalent would make for a very nice transition piece.

You could continue using your existing syslog-based tools to monitor / alert / debug / whatever without having to change much at all. As an added bonus, the tool could accept optional search & filter parameters that are applied to the binary form before dumping ASCII output. That would save the CPU a bit of time to grep through thousands of lines of unrelated logs just to report on the one or two system services that you want to monitor.

Re:like linux needs more fragmentation

[Dog-Cow]

The difference is that the Macbook will last 5+ years and the HP will be replaced in one.

Isn't That What "alternatives" Is For

[theManInTheYellowHat]

So you can choose which mail server or java vm you use but not which syslog daemon you want to use?

Oh, you haven't met yet...

[warrax_666]

- False Dichotomy, this is metamatic.
- metamatic, this is False Dichotomy.

Re:like linux needs more fragmentation

[LordLimecat]

Baloney, which part are you implying is inferior? The intel manufactured processor, the seagate manufactured hard drive, the kingston RAM, or the FoxConn motherboard?

As for 1 year, in all my time helping family and friends with computers, the least ive seen a laptop last is 3 years. So 3 years down the road the PC guy gets a brand new laptop with brand new processor and battery technology, while the Mac is still chugging on outdated hardware, and both have spent the same-- how is that a good deal for the HP dude?

Just to be clear, my home-built PC cost about $700 5.5 years ago and is still chugging along. An equivalent Mac (running the then-new Core2 series) would have probably cost around $1400. You telling me that mac would have another 5 years in it and that my desktop is set to die in a year or so? Care to make any bets on that?

Re:like linux needs more fragmentation

[LordLimecat]

how is that a good deal for the HP dude?

Should be "how is that NOT a good deal"

message UUIDs. Ugh

[Anonymous Coward]

128-bit UUIDs are an idea that came out of the Distributed Computing Environment (DCE) project that Microsoft seized and ran with as the mechanism for generating unique identifiers for their COM/DCOM objects. Have a look at the Windows registry (using regedit.exe) to see what the result has been. Huge swaths of the registry are now completely unintelligible because of the reliance on cross-referenced UUIDs, which are impossible for humans to remember. It reminds me of Steve Jobs' famous comment about Microsoft: "They have no taste."

Not saying that the new daemon is a bad idea, but the inclusion of UUIDs in the proposal makes me think that the whole thing needs community review.

Re:like linux needs more fragmentation

[Un pobre guey]

Who the fuck is this "Noone" guy? Sounds like that dude drinks way too much coffee.

so what

[drinkypoo]

there's no reason you can't have a substitute that logs both to their format and to a classic syslog... or a daemon that creates named pipes to let you view the logs from the database

Re:like linux needs more fragmentation

[jedidiah]

> because its all tied together with a brilliant OS that people want to use

Constantly kidding yourself will not make that any more true.

People buy Macs because they are clueless idiots with no taste.

Although even those people are a very small minority.

Re:like linux needs more fragmentation

[jedidiah]

Nope.

Parts are parts and both machines have the same spare parts.

If anything, the HP will last longer due to better heating and cooling design and having those features more accessable to the end user. The desire to "be pretty" or "avoid geekiness" is not an advantage.

Although I could see were certain types of people could be confused.

Re:like linux needs more fragmentation

[jedidiah]

Don't ask any question you aren't willing to hear an honest response to.

If you do get an honest response but one that you don't want, then you really have no one to complain to but yourself.

If the truth is too painful, then perhaps you should try to avoid it rather than courting it.

Re:like linux needs more fragmentation

[jedidiah]

Of all of my "Mac Like" devices, my Apple ones have been the least reliable. I have had one Mini completely fail on me in contrast to several similar ION machines that are still chugging along nicely. Out of 2 other Minis of the older design, one of them has an internal component failure.

So out of 3 Macs, only 1 is still completely in one piece after just a few years.

I have a old slow Compaq that is still useful because I can easily upgrade it including the GPU. My older Minis are doorstops by comparison.

You need to stop kidding yourself. PCs aren't nearly that unreliable.

Better Database + Log Files

[rathaven]

The issue is based on what you need in different scenarios and to meet that I can't see anything wrong in doing both writing to syslog and a database.

Why do both? In larger systems the amount of data is difficult to cross reference and analyse as files due to the amount of sources, size of data, tools to visualise it all, etc. Writing syslog data to centralised syslog services that do use database backends to centralise logs and query/report against them are a key tool in these scenarios. Its one of a number of interfaces you have to analyse what is taking place on your systems.

However, I'd rather use the simplest method of getting log information out of a system if I'm going to use it for debugging an odd situation. There are situations where the overhead of writing to a database or a write remote data might fail and cause no debug information to be written. I'd rather a simple logging system locally.

Re:Better Database + Log Files

[monkeyhybrid]

I think this is the best idea. I can see the beneifts of a binary, hashed log but I also think it would be a step backwards to lose the plain text log most of us probably access frequently and fluently using standard GNU tools and the like.

The sylog daemon should always output plain text logs, no matter what. It's dead simple to access and manipulate, especially when time is of the essence. A config option (probably on by default) would also enable writing to a hashed, binary format. For many uses we can carry on reading the plain text event log as we always have. For the times we need to ensure integrity of the log events, we can access the hashed version with some additional tools, or do something like a 'chklog' to validate the plain text log or snippet we're reading against the hashed version.

Downside is some increased IO but other than that, seems a win-win solution?

why not cool timestamps with syslog-ng?

[allo]

when its not supported by the tools, lets have a /var/log/syslog and a /var/log/syslog.fine-timestamps. New tools use the new file.

howto log the logger

[reasterling]

How do you log errors with the new binary syslog?

You can pry my text logs...

[thatkid_2002]

from my cold dead hands.

Though, being fair - I could see a way that it could be implemented while causing the least amount of distress. Also, you may be able to just turn it off, or have it pipe everything out into a nice text based log with FUCKING USEFUL ERROR MESSAGES. I don't run Windows because I can't fix it when it breaks (constantly) mostly due to the fact that you do not get useful error messages like you do in a Unix-like OS.

I could see the proposed system being useful in a large server setup where you care about logging I/O.

Re:Solution does not stop hackers from modifying l

[reasterling]

Ever had the experience of typing too quick and ending up typing your password in the Login field?

This has been a concern of mine. I have this happen frequently. I take solace in the fact that on the systems I use /var/log can only be read by root.

Just Remove It!

[colsandurz45]

What's to stop you from removing the system logger you don't like and installing whatever you want?? See: http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=1&chap=9

internationalization

[bored]

I didn't see anyone else mention this, but on windows and AIX, one of the reasons for using a binary log format is internationalization. Log messages are little more than application/facility id, log id, and parameters. The when the user displays the message the ids are looked up in a localization table and formatted according to the attached parameters.

Re:internationalization

[alukin]

Unicode and UTF8 is quite simple and elegant solution for log i18n. WTF i need binary log format for?

What this is really all about

[cstacy]

A lot of people are complaining about changes that will destabilize their system - such as log analysis programs that would need to be changed to use the new system. A lot of people are complaining about how they would have to do something different - can't just /sbin/grep. A lot of people are making assumptions about deficiencies in a hypothetical system. There's a lot that is unknown about doing things in this different way, and its outside the experience and skills of many of the posters. And then, a lot of the messages are just bitching about how they hate one Linux distro or another (for inflicting a new GNOME on them, for example). Some good questions, also a lot of ignorant FUD. Seems to me that those are all separate issues, which need to be addressed by more concrete, information. Above all this is the big question of how experimental the distro ought to be, for the customers it is intended to serve. UNIX and Linux are already technologically about 40 years out of date, compared to previous (but not well known/popular) systems. Progress on operating systems is so slow it makes me want to kill myself when I think about.

Re:like linux needs more fragmentation

[colinrichardday]

I've had an Averatec last 5 years. Ok the battery was shot at the end, but the rest was OK. Besides, what are the specs on a 5 year old Macbook?

SysV init "simple and elegant"?

[chris-chittleborough]

The current init system is "simple and elegant"? The one with lots of shell scripts in /etc/init.d, and symbolic links in /etc/rc[0-7S].d to those links? Symbolic links with names like "K09dm" or "S51cups", where the first 3 characters are highly significant?

I disagree.

(Moreover, both upstart and systemd are significantly faster than the current system.)

New genius thing from Lennart!

[alukin]

Well, we have enough examples of wonderful pieces of programming art and creativity of Mr. Poettering. I wander if all of RedHat's top-programmers are genius?
Lennart's systemd is broken by design and pain in the ass, his PulseAudio is pain in the ass for each man who works with sound on Linux, his lidaemon is constantly unfinished and semi-working masterpiece...

This guy is just a soul of destruction. He breaks everything he touches.
Hey, RadHat management guys!!! Do you ever read what people think about your programmers?

PS. Yes, I remember that BSD is irrelevant, according to Mr. Poeterring. That's why I'm going to throw away all RedHat's stuff from my computers at home and at work until it's too late and replace it by FreeBSD and OpenBSD. Hope, BSD community is sane enough to keep guys like this one as far as possible.

 

Re:If you have to ask "if"

[LWATCDR]

umm. You do know that you can also run rsyslog at the same time. AKA not bridges need to be burned....
And if it worse people will not use it. They will use YUM to install rsyslog.

ASR Suckiness Unit of Measure

[greenbird]

No, Windows suck as much as people think (if not more).

There's a unit of measure for suckiness:

3.3) Just HOW MUCH does this system suck?

The ASR standard unit of suckiness is the Lovelace (Ll).
This is defined as: One Lovelace is the amount of force (measured in dynes)
it takes to draw a round ball weighing e Troy Ounces down a tube it fits
exactly (in air) at a speed of pi attoparsecs/microfortnight.

Like Farads, this is a rather large measurement. Thus, Plan 9 sucks a few
mLl, for instance, while your average Microsoft product achieves many Ll.

The Blobs

[qualityassurancedept]

Like how Red Hat started using indecipherable blobs to obscure their contributions to the kernel, I feel like they are just trying to do more and more to differentiate their operating system from linux distros in general while still staying within the framework of freely distributable software. It's not like this move changes much but it does add another layer of uniqueness/expertise that they can sell to their Enterprise clients. An Ubuntu admin is not necessarily going to be able to even read the logs in RHEL and that provides the opportunity for a sale or service charge at some point.