Slashdot Mirror
Secure Syslog Replacement Proposed
LinuxScribe writes with this bit from IT World: "In an effort to foil crackers' attempts to cover their tracks by altering text-based syslogs, and improve the syslog process as a whole, developers Lennart Poettering and Kay Sievers are proposing a new tool called The Journal. Using key/value pairs in a binary format, The Journal is already stirring up a lot of objections."
Log entries are "cryptographically hashed along with the hash of the previous entry in the file" resulting in a verifiable chain of entries. This is being done as an extension to systemd (git branch). The design doesn't just make logging more secure, but introduces a number of overdue improvements to the logging process. It's even compatible with the standard syslog interface allowing it to either coexist with or replace the usual syslog daemon with minimal disruption.
Text is damn convenient to use. How are you gonna grep a binary file?
The binary format part of this is unnecessary, at least as far as I (with limited low level programming experience) can tell. Other people have been suggesting methods which would mean you just need a cryptographic hash in each otherwise plain text line, in a standard manner. Still at least it has got a discussion started.
Set your machine to also log over a secure channel to another machine. Perhaps one that only accepts the syslog entries and no other connections. Problem solved.
The real "Libtards" are the Libertarians!
Back in the late 90's when I first started connecting my home Linux systems to the Internet 24/7, I logged everything imaginable. To prevent tampering/falsification of the logs, I simply printed the log on a continuous-sheet dot matrix printer. Good luck tampering with the printout in my office.
After a while I got to be able to recognize certain types of activity, such as a web user browsing to /index.html, based on the sounds the printer made.
Log entries are "cryptographically hashed along with the hash of the previous entry in the file" resulting in a verifiable chain of entries.
So this means that in order for someone malicious to modify a log entry, all they really need to do is then re-hash all subsequent entries?
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
There is no real problem this solves. You are far better off logging remotely. This does not stop an attacker from hiding his tracks, you'll just know the logs were altered, but you won't know what was removed, or likely if/when you can start trusting them again. Log remotely, use encryption, and use TCP. You're central/remote logger is your trusted source for logs. You close everything except incoming logs. Parse and alert on the logs from there. Its simple to do, its real time, and solves a lot more issues than this type of solution ever will.
The output hashes the last message with the current message:
No binaries, still grepable, single host and most importantly, there is now a trail that can be verified.
Now, without getting into how much i dislike Pulseaudio (maybe because i'm an old UNIX fart, thank you very much), I think there are really serious issues with "The Journal", which I can summarize as such:
1. the problem it's trying to fix is already fixed
2. the problem isn't fixed by the solution
2. it makes everything more opaque
3. it makes the problem worse
The first issue is that it is trying to fix a problem that is already easily solved with existing tools: just send your darn logs to an external machine already. Syslog has supported networked logging forever.
Second, if you log on a machine and that machine gets compromised, I don't see how having checksums and a chained log will keep anyone from just running trashing the whole 'journal'.
/var/log
rm -rf
What am i missing here?
Third, this implements yet another obscure and opaque system that keeps the users away from how their system works, making everything available only through a special tool (the journal), which depends on another special tool (systemd), both of which are already controversial. I like grepping my logs. I understand http://logcheck.org and similar tools are not working very well, but that's because there isn't a common format for logging, which makes parsing hard and application dependent. From what I understand, this is not something The Journal is trying to address either. To take an example from their document:
MESSAGE=User harald logged in
MESSAGE_ID=422bc3d271414bc8bc9570f222f24a9
_EXE=/lib/systemd/systemd-logind
[... 14 lines of more stuff snipped]
(Nevermind for a second the fact that to carry the same amount of information, syslog only needs one line (not 14), which makes things actually readable by humans.)
The actual important bit here is "User harald logged in". But the thing we want to know is: is that a good thing or a bad thing? If it was "User harald login failed", would it be flagged as such? It's not in the current objectives, it seems, to improve the system in that direction. I would rather see a common agreement on syntax and keywords to use, and respect for the syslog levels (e.g. EMERG, ALERT, ..., INFO, DEBUG), than reinventing the wheel like this.
Fourth, what happens when our happy cracker destroys those tools? This is a big problem for what they are actually trying to solve, especially since they do not intend to make the format standard, according to the design document (published on you-know-who, unfortunately). So you could end up in a situation where you can't parse those logs because the machine that generated them is gone, and you would need to track down exactly which version of the software generated it. Good luck with that.
I'll pass. Again.
Semantics is the gravity of abstraction
Your answer is right in the summary. I can use standard syslog in conjunction with it, and then have a process running in the background that notifies me if the integrity of the text file is violated, thereby getting the best of both worlds.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
From the FAQ:
we have no intention to standardize the format and we take the liberty to alter it as we see fit. We might document the on-disk format eventually, but at this point we don’t want any other software to read, write or manipulate our journal files directly.
Not only does it generate logfiles that are not human-readable, they're also in a format that in two years not even their own tool will be able to read. If it is still around in two years, which I doubt.
You can. If you rooted the machine, you have all the data that this daemon has. You can calculate anything it does, including enough hashs to falsify the logs.
Rethinking email
Oh you mean the terribly slow to open, slow to run, tree view application that must be loaded through the GUI? The one where you have to click on each event to view the details? What happens if you can't run this application? How do you access these logs from the Windows recovery console?
Your hashes don't have to match anything. This does not apply. You can just recreate the entire syslog database.
This is on the same crack as the rest of GNOME 3. They've invented the Windows event log, well done! Now I hand you a trashed system, but you can read the disk. You look into /var/log/syslog ... no, you don't. "We might document the on-disk format eventually, but at this point we don’t want any other software to read, write or manipulate our journal files directly. The access is granted by a shared library and a command line tool."
Speaking as a sysadmin, I shudder at this incredibly stupid idea. Are they even thinking of how to get something actually readable in disaster?
http://rocknerd.co.uk
Is this a joke? Or is it someone just trying to push their ideology of what they think should be done to the rest of the world to make their idea a standard?
Doing something like this would be a sure way for Linux to shoot itself in the foot. For evidence, one only needs to look as far as Microsoft who insists on doing it their special way and expecting everyone else to do what they deem as "good". The concept of syslog messages are that they are meant to be 'open' so disparate systems can read the data. How to you propose to integrate with large syslog reporting/analysis tools like LogZilla (http://www.logzilla.pro)?
The authors are correct that a format needs to be written so that parsing is easier. But how is their solution any "easier"? Instead, there is a much more effective solution available known as CEE (http://cee.mitre.org/) that proposes to include fields in the text.
> Syslog data is not authenticated.
If you need that, then use TLS/certificates. when logging to a centralized host.
>Syslog is only one of many logging systems on a Linux machine.
Surely you're aware of syslog-ng and rsyslog.
Access control to the syslogs is non-existent.
> To locally stored logs? Maybe (if you don't chown them to root?)
> But, if you are using syslog-ng or rsyslog and sending to a centralized host., then what is "local" to the system becomes irrelevant.
Disk usage limits are only applied at fixed intervals, leaving systems vulnerable to DDoS attacks.
> Again, a moot point if admins are doing it correctly by centralizing with tools like syslog-ng, rsyslog and LogZilla.
>"For example, the recent, much discussed kernel.org intrusion involved log file manipulation which was only detected by chance."
Oh, you mean they weren't managing their syslog properly so they got screwed and blamed their lack of management on the protocol itself. Ok, yeah, that makes sense.
They also noted in their paper that " In a later version we plan to extend the journal minimally to support live remote logging, in both PUSH and PULL modes always using a local journal as buffer for a store-and-forward logic"
I can't understand how this would be an afterthought. They are clearly thinking "locally" rather than globally. Plus, if it is to eventually be able to send, what format will it use? Text? Ok, now they are back to their original complaint.
All of this really just makes me cringe. If RH/Fedora do this, there is no way for people that manage large system infrastructures to include those systems in their management. I am responsible for managing over 8,000 Cisco devices on top of several hundred linux systems. Am I supposed to log on to each linux server to get log information?
Attitudes like yours cost the industry jobs. It is best for if we store data away into increasingly inappropriate places so that lusers have to pay us to get their own data.
Hell, going back to standard data formats and reusable tools would be the death of a thousand increasingly bizarre specialty languages alone.
As a penance, you should rewrite diff in python to work on sqlite databases. That should set the industry back another few years.
If only his had been done before... Oh, it has. It's called "asl".
http://opensource.apple.com/source/syslog/syslog-132/
-- Terry
Expected rate of log messages from systems I monitor is low enough that ten seconds downtime of syslog wouldn't be noticed except for a few misbehaved systems. Temporary network disconnects, nothing interesting going on, etc. all makes logging faster than that cause an unacceptable volume of traffic. Yes, I could log all those hashes to a table with a mere two week retention instead of the full year required by PCI, but I don't think it would fly the way you intend.
"I may disagree with what you say, but I will defend unto the death your right to say it." -- Voltaire
If we were to accept a binary format, then at least it shouldn't be from a group that says up front:
At this point we have no intention to standardize the format and we take the liberty to alter it as we see fit. We might document the on-disk format eventually, but at this point we don’t want any other software to read, write or manipulate our journal files directly. ... we don’t want any other software to read, write or manipulate our journal files directly
This is absolutely unacceptable for projects in *nix land intending to serve such a central role as logging.
Reading the actual original document, I don't think it focuses so much on security. But to the extent it does, it's pretty pointless. They make noise about an authenticated chain of entries so you can't just modify the middle, *but* that provides no benefit as the attacker can then just rebuild the chain from that point forward. Their answer is to send it to some place that cannot be modified once transmitted. This is exactly the same as remote syslog policies, no additional security, but added complexity for no gain.
Additionally, they *could* have a system with plaintext and a binary format in place and I recommend they change their minds to do so. The binary blob can contain offsets into a corresponding text file. Thus the good old unix way (which the systemd people seem intent on destroying) is preserved while at the same time get their enhancements.
They *do* have some valid points. Syslog can't cope with binary data, it doesn't provide a good per-user logging facility, large text files are hard to search, and syslog has insufficient service/event type facilities making complex analysis a requirement in some scenarios. Even in a simplistic case, I have been left at a loss for 'what string *should* I grep for?' Many services ignore syslog because of it's limitations as pointed out in the artcile, making things that much more complicated.
But at the exact same time they bemoan so many services doing different logging, they propose making yet another facility and recommend keeping rsyslog running because they aren't going to handle syslog messages. They tell people 'tough you have to use systemd' and 'tough you must use our logging'.
They dismiss java-style namespace management due to variable width, which I think is just going *too* far to acheive theoretical performance gains. They get *very* defensive about UUIDs, and I accept when managed correctly they are unique, *but* it adds a layer of obfuscation unless you have a central coordinating master map of UUID to actual usable names. Uniqueness is an insufficient criteria. Have both worlds. An application submits a message with both a human-readable namespace *and* a UUID. If your logging facility already has the UUID, ignore the namespace. If your hash table does not have that UUID, store a mapping between the UUID and namespace. Then your tool has the added bonus of having a way to dump a quick list of currently observed message types to search by.
XML is like violence. If it doesn't solve the problem, use more.