SDSC Secure Syslog

Wee writes "I saw this morning that the San Diego Supercomputer Center has released Secure Syslog, a replacement for the standard Linux/UNIX syslog daemon they've been working on for some time. It adds security and performance features (modular design, highly scalable), while retaining backwards compatibility. According to their announcement, it is the first syslog implementation to target "syslog-reliable" (RFC 3195) functionality and it is the first syslog targeted at very high performance and forensically-sound auditing. It's currently under the UC's "free for non-commercial use" license, but they are looking at moving to a completely open license (BSD-style licensing was mentioned). If you have high-traffic systems and you need reliable syslogging, this might be a worth a look. Those needing syslogging over TCP/BEEP, sockets, etc as well as UDP might also want to check it out."

Comments

TCP syslogging already available

[The Blue Meanie]

If you need syslogging over TCP and want a *way* more configurable system for filtering syslog destinations, including regex filtering and per-host routing, you might want to look at syslog-ng. It works great for me, and is already GPL'ed, so no waiting for a license change.

Re: replacement

[Anonymous Coward]

as a software developer at UCSD/SDSC I know it can be very difficult to get the folks in the UCSD offices to agree to GPL. Difficult, but not impossible. I would expect that it will happen eventually. :) (way to go team!)

Brazil

[SWroclawski]

Wow did anyone else notice the Brazil references in RFC 3195?

It's good to know that geek culture still persists and can perimate into even the most serious subjects.

- Serge Wroclawski

Several problems with syslogd.

[defile]

Standard syslog has several problems which I think are quite serious.

• Remote logging is a joke. There is no authentication, and no notification whatsoever that the event was received by the remote syslog daemon. An attacker can fill the remote syslog with garbage data if they so choose.

• The records are entirely unstructured and not validated. The timestamp, hostname, and process id are all volunteered by the application, not something that's noted by syslogd.

• There is no guarantee whatsoever that an application that has called syslog() will have its messages safely recorded when the function returns. I'm not talking safely tucked away on disk if the system crashes, but even written into the buffer cache by syslogd when syslog(3) returns. See end of post for details.

• If syslog's receive buffer is full, syslog(3) will block. This means that if syslogd cannot keep up with the rate of messages, which is a really easy condition to find yourself in given that by default syslogd calls fsync() after every log file update, your system will slow to a crawl. You cannot even login(1) since most systems record this activity to security logs.

P.S. syslog() returns as quickly as possible. Try an experiment. Generate a random number, call syslog() with this number as a string, and then open() /var/log/messages, seek to the end minus 4096 bytes. Try to find the random number. I have tried it 20 times and never has the number been there by the time read() was called. It takes longer than an application doing syslog()/open()/lseek()/read() for syslogd to record it into to the buffer-cache, let alone fsync() it to disk.

Re:Several problems with syslogd.

[jim3e8]

Valid points, all. Keep in mind, though, that the syslog protocol runs on all sorts of systems, not just servers. It is still true that a simple, UDP-based protocol is easier on embedded hardware than a complex, TCP-based one.

Also, the function of syslog is not only to log routine diagnostic messages but also "distress calls", where the ailing system may only be able to squirt a few bytes onto the network before dying. Formatting, handshakes and authentication are all potential obstacles.

I think that a reliable syslogd is very important, for the reasons you mentioned and more. Sometimes the data recorded by standard syslogd is useless, making its design a liability. But its simple design helped ensure its acceptance. See RFC 3164 for historical notes and a review of the syslog protocol--section 1 is of great relevance.

Re:Hmm...forensically-sound auditing

[Chazmyrr]

Actually many top corporations are getting very serious about destroying documents that are no longer needed/required to be kept. Still, the log would be useful for showing that the file existed, was kept for the required time period, and purged afterward.

try out metalog

[Corporate Gadfly]

Metalog works pretty good for me, YMMV, but I have been using it for more than 6 months. AFAIK, the current metalog does not support remote logging, however, it is planned to be added in the future. A quote from their site regarding its capabilities:

Metalog is a modern replacement for syslogd and klogd. The logged messages can be dispatched according to their facility, urgency, program name and/or Perl-compatible regular expressions.
Log files can be automatically rotated when they exceed a certain size or age.
External shell scripts (ex: mail) can be launched when specific patterns are found.

Metalog is easier to configure than syslogd and syslog-ng, accepts unlimited number of rules and has (switchable) memory bufferisation for maximal performance.

works great for you, but that's all

[halfelven]

syslog-ng is nice, it has the advantages of TCP and a smart filtering scheme, but that's all. For example, logging to SQL is very weak, it's more like a hack.
For example, a syslog version that does everything syslog-ng does, and more, and has decent SQL logging, and a sound security mechanism (PEO) is msyslog
However, even msyslog is quite simplistic when compared to sdscsyslog. IMO, sdscsyslog has an extremely intelligent architecture, that allows for easy expansion, and easy implementation of features that neither syslog-ng nor msyslog can dream of.

An interesting extension of a lousy idea.

[Doktor Memory]

Although it's nice to see people attempting to improve one of the more regularly broken unix core services, I still think that the whole syslog design of a monolithic central multiplexed logging daemon is the Wrong Design.

Dan Bernstein's multilog, from his daemontools package, is, in my opinion, a good example of the correct way to do logging: a single process logs the stdout and stderr of each daemon, goes to great lengths to ensure that no data is lost, and handles logfile rollover automatically. With this tcpserver/tcpclient utilities from his ucspi-tcp package, it can be simply done over a network as well.

Of course, DJB's code, while source-available, is not in any way Free Software, and daemontools in particular currently is inextricably linked into his extraordinarily dubious "slashpackage" system, making it a potentially annoying choice for anyone reselling or distributing systems based on it. Cronolog, which was originally created to manage apache logs, looks like a promising GPLed replacement for multilog, but despite lots of grumbling along those lines, nobody has written a gpl/bsd replacement for daemontools...yet.

Re:It's the obligatory DJB software thread

[Doktor Memory]

the absence of development on some of the packages (djbdns comes to mind)

If you're going to troll, it would help to even pretend to know what you're talking about. Go away.

Re:Define "commercial use" please.

[Anonymous Coward]

Very many software products that have "free for non-commercial use" licenses deem use by a state or local govt to be "commercial use" in that the govt agency uses it for internal purposes exactly the same way a commercial business would be using it for their own internal I.T. needs, hence they want you to pay for it. HOWEVER, you could always claim that you're use in a city government I.T. department constitutes your researching the software for acceptability in applications of government comuter systems security.... and we all know how slow government operates, hence you could be researching it for a long, long time :-)

A time for ipsec?

[Anonymous Coward]

do we really need specialized encryption built into each application? why not just use something at the stack level?