Forensics Tool Finds Headerless Encrypted Files

gurps_npc writes "Forensics Innovations claims to have for sale a product that detects headerless encrypted files, such as TrueCrypt Dynamic files. It does not decrypt the file, just tells you that it is in fact an encrypted file. It works by detecting hidden patterns that don't exist in a random file. It does not mention steganography, but if their claim is true, it seems that it should be capable of detecting stenographic information as well."

Comments

Don't worry

[sakdoctor]

The company has "innovations" in it's name, so their product probably won't work.
If it did work against true crypt, which is a yard stick of well implemented encryption, I'm sure they'll come up with a counter measure by the next minor release.

Also: In before XKCD strip.

Re:Don't worry

[Kjella]

Since encryption implements a lossless conversion then the data is not random. BECAUSE random data is just that random.

Encryption in ECB mode leaves a very clear pattern, because identical input blocks leads to identical output blocks. Pretty much every other block chaining mode doesn't though because they mix it the preceding blocks, so i'm guessing an implementation flaw because the cryptographic primitives are pseudorandom, they have no distinguishable non-randomness unless you know the exact key.

Re:Don't worry

[Anonymous Coward]

You realize that you aren't saying anything at all, right? Your argument is that since encrypted data is different than random data (an assumption you make without stating), encrypted data will look different than random data.

In reality, one of the standards for encryption algorithms (and block chaining methods) is that they produce a pseudorandom output. In fact, block ciphers are often called upon to operate as PRNGs when given random input data. The idea is that they will produce a significantly larger amount of pseudorandom output data than the random seed data.

BTW I do mathematical cryptanalysis at a university...

Re:Don't worry

[Stray7Xi]

BECAUSE random data is just that random.

Any kind of analysis that answers the question of whether a piece of data is random or deterministic can't do so with certainty. You can't prove a string of a million 1's wasn't randomly generated. Every piece of random data long enough will have substrings that appear to be a pattern.

Give a voice recognition program a low enough certainty threshold and it'll pick out words from below the noise floor. But the lower you go, it'll make more and more mistakes and eventually it'll pick out words from plain white noise.

Re:Don't worry

[PitaBred]

That's why I name my TrueCrypt volumes stuff like "moo.zip"

"Awww, jeez... the damn thing's gotten corrupted! My boss told me to keep my sensitive company files in an encrypted zip file, and it keeps screwing up"

Just because security through obscurity isn't good as the only defense doesn't mean that it's not quite handy in addition to others.

Re:Don't worry

[MoxFulder]

I wish I had mod-points for you.

Finally we hear from someone who knows WTF he/she is talking about.

Just to expand a bit: encryption algorithms (except for one-time-pad) don't produce truly random output. But all good, modern ones seek to produce output that's as indistinguishable as possible from truly random output, as a necessary but not sufficient component of their security. There are a variety of techniques to produce pseudorandom data based on a variety of sophisticated mathematics.

It seems like the height of hubris to claim that one software program can reliably detect all these different kinds of extreme slight deviation from perfect randomness.

A more plausible approach (as others have pointed out), is to look for files that do appear to be totally random. Such files are likely to be either (a) the output of a random number generator, or (b) encrypted. All files that have some useful content in their present form have some structure or non-randomness.

Re:Patterns?

[Firethorn]

The fact that there's order in the encrypted information doesn't change the fact that, to an outside observer that doesn't know the original information or the key can't tell the difference between the encrypted information and true random noise. That's part of the point.

If they can tell that it's not random, that's a start on cracking the encryption and gaining the original information.

Re:Plausible Denial?

[Animaether]

"That's cute, sir - now give us the other password"
- "what other password?"
"for the hidden truecrypt volume"
- "what hidden truecrypt volume??"
"the one that's being referred to by half a dozen applications' most recently used files lists"
- "oh err.. that's uh.. another drive entirely"
"very well, then hand us that other drive"
- "err uhm.. my dog ate it?"

If you're really, really serious about these things, maybe you could work super-diligently to prevent leaving any clues as to that hidden volume's existence.. odds are something's going to bite you in the behind somewhere though.

Re:Patterns?

[Jah-Wren Ryel]

Dear mods, that's meant to be facetious. Some of you seem to be a little trigger-happy so you won't understand why I shouldn't have to explain that.

Make your joke and take the moderations like a man.
If you are going to explain that it is a joke, you might as well not bother in the first place since explaining takes away all the fun.

Re:I'm calling BS

[zindorsky]

OK, I checked it out. Here's how they "do" it:

1. No File Header.

2. (File size % 512) = 0

3. Successful X2 and Arithmetic Mean tests on certain bytes.

4. File size greater than 15 MB.

Step 2 == entropy tests.

In other words, they detect random looking files (which implicitly implies "no header") whose size is 0 mod 512 and is greater than 15MB.

Big fucking deal. It might be true that on your system, the only files that meet these characteristics are TrueCrypt volumes, but again it's trivial to create non-TrueCrypt files that meet these criteria. Simply, any true random file (whose size meets the above requirements) will be detected as a TrueCrypt file.

I stand by my assessment: BS.

Re:Is unreadable data really encrypted data?

[e4m]

There is a reason that high-quality encryption was once classified as a "munition" by the US government. You cannot accidentally create it. You need a very good PRNG or an algo such as AES. Don't worry, your formats will (and cannot) be confused with encrypted data.