Forensics Tool Finds Headerless Encrypted Files

gurps_npc writes "Forensics Innovations claims to have for sale a product that detects headerless encrypted files, such as TrueCrypt Dynamic files. It does not decrypt the file, just tells you that it is in fact an encrypted file. It works by detecting hidden patterns that don't exist in a random file. It does not mention steganography, but if their claim is true, it seems that it should be capable of detecting stenographic information as well."

Comments

Patterns?

[causality]

It works by detecting hidden patterns that don't exist in a random file.

I should first say that I'm rather ignorant about encryption but I hope someone will be able to explain this. I was under the impression that any sort of good-quality encrypted data is indistinguishable from completely random data. That seems to directly contradict the ability to determine whether a volume contains encrypted data by means of locating patterns. Is this really a contradiction?

Re:Patterns?

[causality]

And how many completely-random files do you have on your computer?

One, and a second file that's pretty close. /dev/random and /dev/urandom.

Dear mods, that's meant to be facetious. Some of you seem to be a little trigger-happy so you won't understand why I shouldn't have to explain that.

Re:Patterns?

[Andy Dodd]

Actually, if you use the wrong block cipher mode, it's easy to distinguish between an encrypted file and random noise. AES-256 encrypts 128 bits of data at a time (with a 256-bit key). If you use the same key and the same block of data (ECB mode), you get the same output and can determine that there's something there.

If you modify each block with some known quantity that is different from block to block, then the output becomes much less patterned. For example, Counter (CTR) mode XORs or adds an increasing count to each block of cleartext, so that if you have two identical blocks of cleartext, the output is very different. Cipher Block Chaining (CBC) takes the encrypted output of block N and XORs it with the cleartext of block N+1 before encrypting that block.

Re:Patterns?

[geekboy642]

Another thing would be Truecrypt's refusal to overwrite certain parts of that "random" data inside the not-hidden container. Gives it away that it's protecting the integrity of a hidden container.
Why do people constantly make this mistake?
TrueCrypt cannot know a hidden partition exists, *unless* you enter the inner volume password. It will cheerfully let you write right over the inner volume without so much as a by-your-leave, if you only give it the first password. It is true deniability, assuming this tool can't distinguish "encrypted blank space" and "encrypted data".

Re:Patterns?

[Kjella]

That's called a known cleartext attack. If they already have the original file then the point of encryption is moot.

1. It's usually called a "known plaintext" attack.
2. Detecting patterns in ECB mode encrypted data is not a known plaintext attack.
3. Known plaintext attacks are most definately not moot.

A known plaintext attack means that you can derive the key or some intermediate to decrypt other data encrypted with the same material, and is highly useful. For example, you could send someone a mail, an instant message, upload a file to a server or whatever and if stored on an encrypted disk you have a known plaintext. If that'll let you figure out the key, big uh-oh. I actually used this on some encrypted (standard password protected) zip files, they have a known plaintext attack. Basicly I had one zip file with contents I already had, and other zip files with contents that I didn't have. But from having both plaintext and ciphertext from one file, I could decrypt all the other files too.

Umm...

[drakaan]

s-t-e-g-a-n-o-g-r-a-p-h-y...not stenography.

Re:Don't worry

[SerpentMage]

What I am guessing is that they are doing Gaussian analysis. It is actually quite simple, and not too hard to implement. If a data set is truly random then the statistics will have some basic indications that it is random.

Since encryption implements a lossless conversion then the data is not random. BECAUSE random data is just that random.

Though it would not be that hard to get around this because the statistics can be fooled. Actually would not be that hard to do that. Thinking about it, rather interesting problem...

BTW I do statistical and probabilistic analysis in a hedge fund...

Benford's law

[tyrr]

This is probably another application of the Benford's law.

Who Cares?

[DomNF15]

The Wikipedia page on TrueCrypt already indicates that the volumes can pretty much be detected since they are always divisible by 512, it's just impossible to PROVE they are TrueCrypt volumes...

Be enlightened: http://en.wikipedia.org/wiki/TrueCrypt

Re:Plausible Denial?

[jroysdon]

I thought one feature of TrueCrypt was the ability to have two passwords. One password unlocks your "non-secret" data. The other password unlocks your "secret" data in a hidden volume.

http://www.truecrypt.org/docs/plausible-deniability

The point is both sets of data are stored in one big binary blob. It'll all look like one big fat encrypted mess. In fact, if you are not careful, your non-secret data can overrun your secret data.

To get around this "randomness" problem, after creating your non-secret partition, fill the partition completely with something (copy a few public domain books over and over until the partition is full). All the "randomness" will be gone with encrypted data. Then delete everything and put back in just the smallest amount of non-secret data you need to store in order to appear legit. The "randomness" is still there, as only the FAT entries are deleted, but all the encrypted data is still filling up that whole binary blob.

Now, create your secret partition and use it. Be sure to use it just short of the non-secret data's amount (as they fill from the opposite end), otherwise your non-secret partition will be corrupted.

This link helps with the graphics:
http://www.truecrypt.org/docs/hidden-volume

The one downside is that the non-secret side, if it fills up with too much data, will override your secret side. That's why your have backups and this is just for transport anyway, right?

TCHunt Does this very well

[e4m]

TCHunt found all of my TrueCrypt volumes. It's free too. http://16systems.com/TCHunt/index.php

Re:Don't worry

[1729]

I seem to remember that being a scene from The Wire.

It first appeared in the David Simon's Homicide: A Year on the Killing Streets. The anecdote had been passed down within the Baltimore Police homicide squad, and was presented as a true story in the book. Simon later adapted this and other events from his true crime books for use in The Wire.

Re:Don't worry

[Atzanteol]

Sounds like the do something like the free ent utility. It calculates a "randomness" of files. It can be quite useful to tell "data" from "encryption."

Re:Plausible Denial?

[shutdown -p now]

If you are actually seriously using TrueCrypt so that the NSA (or law enforcement in general) won't get ya, you'd be an idiot to do so from Windows, or even your typical desktop Linux. I'd probably make a separate Linux (or BSD) install just for that, with home directory mounted in ramfs by default. Then make an image of its clean untainted state, and then everytime I need to access the encrypted drive, dd the image to a USB flash stick, boot from that, and only then mount the TrueCrypt volume and work with it. Once done, `shred` the stick.

Re:Don't worry

[Jane Q. Public]

If that is the case, then the solution is trivial: initially fill up the encrypted file space with pseudorandom data.

Re:Don't worry

[Anonymous Coward]

"...and eventually it'll pick out words from plain white noise."

Oddly, there's a wonderful lady(Deutsch is her last name, I forget the first, Diane maybe) who proved that the human brain will do the same thing with utter gibberish. From the same repeated nonsensical sounds, I heard "copy", "on me" "meow" and a ton of others. So I agree with you 100% in that it couldn't be 100% certain there was anything encrypted at all.

Re:Sure they do! :)

[Randle_Revar]

# ignores commands preceded by a space
HISTCONTROL=ignorespace

of course then you have to remember to put a space in front of any commands you don't want recorded

This is complete BS, and is easy to test

[anom]

This is complete sensationalist crap. Truecrypt isn't broken, (probably) nor are any of the other programs they possibly claim to have broken.

This is easy to test for yourselves folks, I just did it in 5 minutes.

dd if=/dev/urandom of=/home/me/somefile.jpg bs=512 count=10000

Performing this command and then scanning the resulting file with "File Investigator" results in the file being detected as a headerless encrypted data file.

Whoever pointed out that they simply identify any randomly filled binary file of a size of a multiple of 512bytes is correct.

TrueCrypt doesn't use ECB mode, hasn't for some time, etc etc etc. Stop freaking out every time someone claims to have broken it.

Re:Plausible Denial?

[FilterMapReduce]

What we really could use is a distro meant specifically to prevent this this, with (among other security features) default configurations that don't save any data about what your applications have been doing. Perhaps Paranoid Linux, if it matures.

Re:Don't worry

[JamesP]

And that's why it's recommended to compress things before encryption.

Re:Don't worry

[MSG]

I don't think so... It's recommended that you compress things before you encrypt them if you plan to do both (usually for network transmission). If you encrypt and then compress, your compression will not be very effective. Good encryption produces very few patterns, and patterns are what compression applications need in order to function.

Re:Don't worry

[Jane Q. Public]

If you have 2 files, each of 1M bytes, and one is filled with all 1s, but the other is filled with 500 copies of "10001101" followed by all 1s, the entropy is NOT identical. Please show your calculations.

I assert, once again, that 500 patterns followed by 1s is not of the same entropy. If it were, Fourier analysis would not work.

Re:Don't worry

[hoggoth]

I am a computer forensic investigator, and I know what the structure of a zip file looks like internally. It's NOT a blob of random bits. Even a corrupted zip file has a well defined header, indexes, etc.

It's extremely difficult, if not impossible, to hide data from a good investigator who has the time and motivation to investigate thoroughly. If I find a large file containing only random bytes, it is NOT a normal thing and I will look into it further, especially if the file size is an even multiple of 512 bytes. If I can find traces of TrueCrypt ever having been used on that drive I will have a pretty good idea what I'm looking at. I can try to decrypt the file using every possible string found on the hard drive, including bits of memory saved to the paging file and hibernation file. If I manage to decrypt and open the file and find it is formatted with the FAT32 filesystem instead of NTFS I will be very suspicious that this was done because there is a hidden "plausibly deniable" inner volume. I will then work on cracking that open like I did the outer volume. I will also report to the authorities I am working for that there is a significant possibility of a hidden volume. They will use their social skills to get the key from the owner.

The real limitation is that cases usually DON'T give me enough time or resources to investigate that deeply, or the lawyers manage to bury the issue of an encrypted file and it doesn't get addressed. The best bet for a person with something to hide is to make it very difficult and time consuming for an investigator to get to the bad stuff, and hope his case isn't that important to warrant the time to dig deeply. In practice that means if you cheated your partner in a small business and hide it very very well I probably won't find it. If you killed someone I will find it.