Known-Good MD5 Database

bgp4 writes "Have you ever examined a system you thought was broken into but you weren't sure? If only you had run an integrity verification program like osiris or Tripwire first you could have figured out what programs had been changed. In an effort to help out in the instances when you can't answer the question "what was this like before?" we've constructed a searchable database of MD5 and SHA-1 hashes for files in many standard operating systems. You can search using the filename or the checksum and see if you have a trojaned binary or an overactive imagination. Currently at knowngoods.org we have many FreeBSD, OS X, Linux, and Solaris installations checksummed and cataloged. If you have other programs or distributions you would like to see in the database, please let us know."

Comments

What about source builds?

[Anonymous Coward]

Wouldn't this be useless to anybody that builds from source?

Re:What about source builds?

[Cerlyn]

Indeed; the capability of such a system is a bit limited with operating systems like FreeBSD, which actively *encourage* their users to build/rebuild from sources. IIRC, FreeBSD actually only gives intermediate security updates in source code format so you have to compile them (not too hard: cd /usr/src ; make buildworld).

So, recording the checksum to /bin/ls, etc. is a bit flawed in that when I do a "make buildworld", my custom configuration parameters from /etc/make.conf get used, overriding CPU type, if Xfree86 is installed, etc. Since my system's parameters likely will not match FreeBSD's master build system, there is a high chance that the checksums after I do a rebuild are significantly different.

But for non-source distributions (Redhat, etc.) this concept is excellent, assuming that no one compromises the database or the OS kernel. Unfortunately, no database checksummer will ever counteract the case when the OS kernel itself is compromised, potentially returning one file when scanned and another when executed.

Still, it wouldn't hurt for them to record source file checksums as well; after all, having an independant checksumming group would require them to be compromised as well as the FTP network, making an attacker's life harder.

Re:What about source builds?

[pVoid]

Indeed.

In fact, this system would be best suited for systems which aren't OSS... such as windows =)

crowd boos... stones and rotten tomatoes fly as author runs for cover

:)

ooooo nifty

[netwiz]

I've been wondering when something along these lines would be available.

[devil's advocate] However, how do we know that the pregenerated checksums are correct? Who watches the watchers? [/devil's advocate]

Yah, yah, I know, the easiest way is to inspect the source for the minicompiler, the main compiler, and the program by hand, then build all of them step-by-step until you're done, then use the final binary to generate your hash. I wonder, tho, how much drift might there be in using a pre-built compiler (say I D/Led the binaries for GCC and the libraries to go with it). One tiny change in machine state (or any other number of things I would suppose) could result in the final binary being a single byte off, and the whole thing's a wash.

Granted, I may be talking out of my ass here, could someone w/ some hard-core coding knowledge or CS experience expound on the above?

What about Windows OS?

[scubacuda]

I didn't see the ability to search for Windows MD5 hashes.

Considering its history of vulnerabilities, I'd think that this would be pretty important...

Re:What about Windows OS?

[kubrick]

What about viruses that change the structure of the files they infect? Especially ones that haven't been spotted by the anti-virus firms yet (rare, I know, because they probably develop and release most of them).

Also, can't people still use disassemblers to 'crack' files, and maybe add backdoors at the same time?

Both of these activities would be reflected by checksum changes.

Re:So what about the obvious scenario...

[BitHive]

Then I imagine that as soon as someone changes a hash, many secure systems will indicate they've been comprimised, and the whole thing will be quite obvious to sort out.

Something's wrong here

[phr2]

If we need an external database of md5's to authenticate so many different files, that means that md5's weren't really the right authentication method to begin with. It's better to use digital signatures.

The fancy way to do that is with an Authenticode-like system for signing files. Distro maintainers would sign the files in their distros, and users could also sign their own files. A simpler way would be to just have a big, signed list of md5's in some file that tripwire checks against. Tripwire would check the signature on the file before believing the md5's in it. Or the list could contain individual signatures per file instead of just hashes.

A centralized md5 database doesn't feel so right with the free software spirit, which says (legitimate) users could modify the files at any time, or just recompile them with a slightly different compiler, etc.

"False" senses of security

[Hizonner]

Spoken like a true second-year student.

The reality of the matter is that, while it certainly would be possible for somebody to gag a machine to evade all your wascally checksumming tricks, they frequently don't do so. And when they do it, there's the usual arms-race lag between the time when a new method of checking comes out and when they update their tools to evade it. And there's a cost to them for each defense they evade; if you want to avoid every defense you ever hear of, you basically have to roll your own rootkits, which is a huge time investment.

And a kiddie who's out there collecting hundreds of boxes has no particular incentive to be anal about holding onto yours.

... and everybody makes mistakes. Yes, you're right, looking at checksums gives you absolutely no security against omniscient adversaries with infinite resources. Luckily, real adversaries are not omniscient and have limited resources. Yes, you'll even miss some of the real adversaries. You'll also catch some. Probably a lot. Nothing is perfect. Deal with it.

Fucking pompous amateurs.

Re:But what happens...

[kasperd]

when they trojan your MD5 checksummer?

Then your compromised system might apear to be clean. I have actually seen a system where that has happened. But the intruder forgot to trojan the rpm executable, "rpm -Va" revealed everything. But had the intruder trojaned the rpm executable too, that wouldn't have worked. The only secure way to use the verification tool is to boot from a readonly media and run the tool from there.

Versions?

[tconnors]

OK - debian seemed to have one version there - r5, whatever that is. How does it handle apt-get upgrades? If r5 is reffering to something like stable, then even stable changes over time (contrary to what some poeple think ;-). So do they take the checksums from a machine that was just apt-get upgraded last night, or what? If they mean an actual yearly or half yearly release, who on this world does not apt-get upgrade when there is a security fix released? So your system sure as hell aint going to match theirs.

Then I can't imagine how you would be able to automate this, so it checks all the binaries in /bin /sbin, /usr/sbin etc - do they have some alternative to HTTP for their database?

Doesn't seem overly useful to me....