EFF's Logfinder

clonebarkins writes "EFF has just released a new software tool called "logfinder" to help server admins find (and delete) unnecessary log files on their boxen. "By finding unwanted log files, logfinder informs system administrators when their servers are collecting personal data and gives them the opportunity to turn logging off if it isn't gathering information necessary for administering the system.""

Comments

Is a new tool really necessary?

[Lord Kano]

A competent admin will know that his/her boxen are collecting personal data. An ethical admin will get rid of any unneeded data.

LK

Can't subpeona what doesn't exist?

[PornMaster]

Is the point of this that law enforcement can't subpoena records that don't exist?

Re:Can't subpeona what doesn't exist?

[sporktoast]

If an admin is just using this tool to destroy potentially incriminating logs, then they are using it poorly. Like trying to pound a screw in with a hammer.

The use this has for an admin is to survey (or for the less experienced admin, to discover) what logs the system is currently, so that the admin can decide as a policy which logs should be active or not, and with what level of detail. The itch this tool scratches is that many systems as a default keep more logs than perhaps are necessary. A good admin will shut off whatever is deemed unnecessary, based on multiple criteria (security, system load, user/company privacy).

Forbidding the use of log destruction tools (rm?) is moot. Destroying evidence is illegal. Now, laws (or court orders) mandating a level of logging are a completely different matter.

I appreciate the effort but...

[garcia]

I would seriously hope that:

a) the sysadmins are competent enough to handle this themselves. I would think that a sysadmin would know how to use some sort of local file search.

b) the EFF understands that it's not always up to the sysadmins to determine the amount of time to keep logs that might be used against an individual.

Re:I appreciate the effort but...

[Darth_Burrito]

the sysadmins are competent enough to handle this themselves.

I didn't bother to read a description of the tool, but there's nothing to say that a competent admin might not want something like this if it eased his burden. Also, there's the matter of incompetent admins. Many of us wear multiple hats. I do development, support, and administration on linux and windows for a small office, mostly by myself. Suffice to say, nobody can be perfect at everything. I'm always looking for tools that help me automate common tasks and, generally speaking, manage more with less.

Re:Thanks EFF!

[Anonymous Coward]

Pair it with TOR and hacking becomes untraceable! 2005 is going to be a great year.

is this stupid?

[digitalgimpus]

I'm sorry, but this might just be the dumbest move yet they have made...

lots are crucial for many reasons:

1. Hacking attacks (how else do you track them, and prevent them)?
2. Abuse problems (spammers, credit card fraud)
3. aggregate statistics (what percentage of my customers are based in Europe?)

I can't see why someone would shoot themselves in the foot and use this.

Like log files are really intrusive anyway.

Re:is this stupid?

[Dasein]

Don't use it. Seriously. If you are comfortable with the level of logging you have, okay. There are others, who may be in the positions of not wanting log files that identify users because of the expense associated with discovery or complying with a subpeona.

Note, this is why large companies have email retention policies -- because having to do discovery or comply with a subpeona on email records going back years is expensive. So doing this type of thing isn't anything new or sinister.

Re:is this stupid?

[ducomputergeek]

I have to agree. I don't do much system administration work outside of our data-storage network these days, but even then I want to see what employees are putting on the 6TB system. Why? We've found porn before, against company policy, warezed games, against company policy, personal files, etc. before. For most random logs, they are archived every month and backup to optical media every quarter. But the storage system is supposed to be used to store completed video projects for the past year. Nothing else. The front office and bean counters have their own servers just for accounting information etc.

Back when I was doing more web-based server admistration logs were my friends. We could tell people trying to find an open mail port, hacking attempts, DDOS attempts, etc..

The EFF is becomming more like the ACLU in many regaurds. Had a very good reason for formation, but at some point went off the deepend. Privacy on the net is non-existant. Never was there, but somewhere along the way someone thought that the Internet was going to be this great tool of anaminity and I want to know who thought up that bloody idea. Because it was and always will be flase.

Just as an example...

[PartialInfinity]

This is just EFF's way of reminding sysadmins to be vigilent about their log files, it's not meant to replace good administration habits.

Re:WTF is Boxen?

[MojoSF]

I also think of it as a nod to the old days when a room full of DEC VAX computers would be referred to as vaxen.

Re:Excellent

[ab384]

Just two observations: (the second of which is actually relevant to this parent)

1. It took me around 3 minutes to find out that this thread applies to POSIX-like systems only (ie. won't work on this winXP). The fact should really have been mentionned in the summary. I only say this because recently, some summaries seem to have been "hastily" written.

2. I am myself wary of huge, hidden log files that either winXP itself or other programs create. As the only user and sysadmin on this system and keen to minimise disk wastage, I would want to prune all logs regularly. Trouble is, they aren't all *.log files. So, how do I find them, short of going through every single program and investigating any logs it might or might not create?

Re:Thanks EFF!

[innerweb]

As is always, that which helps to protect the innocent can be used to protect the *evil*. The problem is the innocent do not know what is being done, and the *evil* are studying and learning to use and abuse. Nothing new there.

InnerWeb

Log Retention Policy

[dRn-1]

I can see how useful logfinder could be/is. And how it along with relevant administration, can eleviate the possibility that your systems are harbouring evidence of criminal activity... Could they seize any systems they deem necessary? I certainly wouldn't want any of my systems seized because I don't have a log retention policy, and hence when they ask do you have logs for such and such @ such and such a date, a reply of I'm not sure wouldn't go down too well! In short and IMHO having a log retention policy is a good idea... think I might recommend one myself.

Re:Oh, yeah

[stephenbooth]

In an ideal world every system would be administered by a well trained and experienced system admin, or a trainee admin being mentored by one, who had plenty of time to investigate and maintain the machine. In practice most system admins are people in other roles (developers, DBAs, desktop support or even receptionists) who have been handed the task of managing half a dozen white box Wintel servers (with maybe a SCO or Linux box or even an aging Sun box in the mix) and probably a Netware server doing file and print, most were built and installed by someone one of the manager's knows or have been inherited third hand from another company. If they're lucky they get a training course where they'll learn a few of the GUI screens, more likely they'll be given a few dozen pages of handwritten notes (aka 'the manual') and told to go to the nearest Waterstones/Borders/Whatever and buy a book if they need more.

That was pretty much my first job. I had trained as a C programmer; then I found myself managing 70 desktops running various versions of Windows, a dozen or so White Box Intel based servers running Windows NT 3.51 and 4.0, a SCO OpenServer box, an Alpha running VMS, a 3 member VAX cluster running VMS and an RS6000 running AIX. All with no usable documentation or training. A little later they added in DBAing the Oracle databases and managing the network (a variety of devices from 3Com, Cisco and Bay), at the time I only knew a bit of SQL and wasn't really sure of the difference between a router and a switch. After spending a lot of money on books then a lot of time reading them (I didn't have web access at the time, when I did I started reading websites as well) I eventually learned what I needed to know.

This script is a separate issue. Inpractice I don't expect those sorts of admins to run it, they probably wouldn't know what to do with the information if they did. Where I think it would be useful is for the professional admin who suddenly inherits a bunch of machines (maybe they've moved companies or their company has merged with another). Put this script on them and run it for a few days then see what it turns up. No matter how wonderful and professional you are unless you built and installed a machine yourself and can guarantee that no-one else has ever had the root/admin password to a box you can't be 100% sure that there's not some process running somewhere that is quietly logging something somewhere. No-one who manages a non-trivial number of machines has time to check every machine to make sure that there are no new or unexpected services that have snuck in (and remember it's not something you could do once and then not again, you'd have to keep on doing it). That's why you need scripts that look for anything that could point to unexpected activity. Not just looking for anything that looks like a log on a box but also ports that shouldn't be open (I've lost count of the number of times I've found a box with port 25 open when I know I've disabled SMTP, only to find that someone has re-enabled it without telling me) or unexpected activity on a switch or firewall port. Not only do we have too many machines to manage but also users who delete files they shouldn't which then must be restored from backup, managers who constantly demand reports on system availabity stats and projects that we have to keep an eye on to make sure they don't run wild and break every standard we have.

Stephen

Re:Good work, but...

[Secrity]

I am not a lawyer, the following is a general discussion, your mileage and your laws may vary. It is possible that some jurisdictions may have laws that require the retention of data, I know of no such requirement in the US. Did I mention that I am not a lawyer?

There is a difference between evidence tampering (illegal) and system administration (legal). If you remove data because it may be incriminating, you are tampering with evidence. It would also be illegal to delete data after you receive a subpoena or other legal demand. If you don't collect data or you have a policy to remove data after a certain period of time, you are administering a system. Another valid system administration policy is to remove log data when you fill a certain amount of disk space. You could also have a policy that says that you do not back-up certain logs. If you maintain logs or other data, a documented data retention policy is a Very Good Idea.

Re:Interesting Motive

[Anonymous Coward]

Given that the EFF is motivated to oppose Internet regulation, this release should not be particularly surprising. Providing a tool of this sort to sysadmins grants those administrators a measure of "plausible deniability" when investigators request activity logs. If the administrator can say, "We wipe secondary logs four times a year and the more important ones yearly; it's our [written?] policy and it's coded into our software," the investigators have a weaker case for suggesting in-house conspiracy or negligence than if the log audits were carried out on an impromtu/"whenever-they're-getting-too-big" basis.

Automatic log auditing is definitely a double-edged sword. It can provide cover for many types of misbehavior, but also gives the average sysadmin more legitimacy, especially in the eyes of the technically semi-literate.

If nothing else, this release might inspire some of the more skilled admins out there to write their own code to automate their auditing, especially given some of the comments below on the *cough* functionality of the EFF package...