File System Forensic Analysis
nazarijo writes "The field of investigative forensics has seen a huge surge in interest
lately, with many looking to study it because of shows like CSI or the
increasing coverage of computer-related crimes. Some people see a
career opportunity there, and are moving toward computer forensics, marrying
both law enforcement and investigations with their interest in things digital. Central to this field is the study of data storage and recovery, which requires a deep knowledge of how filesystems work. Brian Carrier's new book File System Forensic Analysis covers this topic
with clarity and an uncommon skill." Read on for the rest of Nazario's review.
File System Forensic Analysis
author
Brian Carrier
pages
600
publisher
Addison Wesley Professional
rating
9
reviewer
Jose Nazario
ISBN
0321268172
summary
The standard for digital filesystem forensics
It's easy to think that computer filesystems are relatively simple things. After all, if 'dir' or 'ls' don't show what you're looking for, maybe an undelete program will work. Or will it? To be a decent, trustworthy expert in forensics (a requirement if you plan to participate in any criminal investigations), you'll have to learn how filesystems really operate, how tools like undelete and lazarus work, and how they can be defeated.
Carrier's book isn't a legal book at all, and it doesn't pretend to offer much insight into the law surrounding forensics. Instead it focuses on technical matters, and is sure to be the gold standard in its field. This is important, because it comes at you expecting you to have some knowledge, even if only informal, of what a filesystem contains. With a basic understanding of data structures, you'll get a wealth of information out of this book, and it will be a good reference long after you've first studied it.
File System Forensic Analysis is divided into three sections. These are arranged in the order that you'll want to study them to maximize the benefit you can hope to achieve, namely an understanding of how to examine filesystems for hidden or previously stored data. The first three chapters cover a fundamental series of topics: Digital Investigation Foundations, Computer Foundations, and an introduction to Hard Disk Data Acquisition. While they start at a basic level (e.g. what hexadecimal is), they quickly progress to more developed topics, such as the types of interfaces (SATA, SCSI, IDE), the relationship of the disk to the computer system as a whole, and how data is stored in a file and filesystem at a basic level. A lot of examples given use Linux, due to the raw, accessible nature of UNIX and UNIX-like systems, and the availability of tools like 'dd' to gather data.
Part 2 covers "Volume Analysis," or the organization of files into a storage system. This introduces the basics of things like partition tables (including how to read one). The next few chapters cover PC-based partitions (DOS and Apple), server-based partitions (BSD, Solaris and GPT partitions), and then multiple disk volumes like RAID and logical volumes. With this introduction, the final chapter of the section covers how to use these filesystem descriptions in practice to look for data during analysis. Filesystem layouts, organization, and things like journals and consistency checks are covered with a clarity and exactness that's refreshing for such a detailed topic.
Having covered the basics of filesystems, Part 3 covers the bulk of the book and material. Several chapters follow that specifically show you how to analyze particular filesystems by using their data structures to direct your reads. A range of filesystems are covered, including FAT, NTFS, EXT2 and EXT3, and the BSD types UFS1 and UFS2. Each filesystem has two chapters, one devoted to concepts and analysis, another entirely about data structures. Dividing each filesystem type like this lets Carrier focus first on the theory of each filesystem and its design, and then the practical use of its design to actually understand how to pull data off of it.
The real strength of File System Forensic Analysis lies in Carrier's direct and clear descriptions of the concepts, the completeness of his coverage, and the detail he provides. For example, a number of clear, well-ordered and simple diagrams are peppered throughout the book, explaining everything from allocation algorithms to NTFS alternative data streams. This use of simple diagrams makes the topics more easily understood, so the book's full value can be appreciated. This is the kind of thing that sets a book apart from its peers and makes it a valuable resource for a long time.
Finally, Carrier brings it all together and shows us how many aspects of filesystems can be examined using his "sleuth kit" tools, freely available and easy to use. Without appearing to hawk this tool at the expense of other valuable resources, you get to see how simple and direct filesystem manipulations can be done using a direct approach. This kind of presentation is what makes File System Forensic Analysis a great foundation.
Overall I'm pleased with File System Forensic Analysis, I think that Carrier has achieved what few technical authors do, namely a clear explanation of highly technical topics which retains a level of detail that makes it valuable for the long term. For anyone looking seriously at electronic forensics, this is a must have. I suspect people who are working on filesystem implementations will also want to study it for its practical information about NTFS. Overall, a great technical resource.
You can purchase File System Forensic Analysis from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
It's easy to think that computer filesystems are relatively simple things. After all, if 'dir' or 'ls' don't show what you're looking for, maybe an undelete program will work. Or will it? To be a decent, trustworthy expert in forensics (a requirement if you plan to participate in any criminal investigations), you'll have to learn how filesystems really operate, how tools like undelete and lazarus work, and how they can be defeated.
Carrier's book isn't a legal book at all, and it doesn't pretend to offer much insight into the law surrounding forensics. Instead it focuses on technical matters, and is sure to be the gold standard in its field. This is important, because it comes at you expecting you to have some knowledge, even if only informal, of what a filesystem contains. With a basic understanding of data structures, you'll get a wealth of information out of this book, and it will be a good reference long after you've first studied it.
File System Forensic Analysis is divided into three sections. These are arranged in the order that you'll want to study them to maximize the benefit you can hope to achieve, namely an understanding of how to examine filesystems for hidden or previously stored data. The first three chapters cover a fundamental series of topics: Digital Investigation Foundations, Computer Foundations, and an introduction to Hard Disk Data Acquisition. While they start at a basic level (e.g. what hexadecimal is), they quickly progress to more developed topics, such as the types of interfaces (SATA, SCSI, IDE), the relationship of the disk to the computer system as a whole, and how data is stored in a file and filesystem at a basic level. A lot of examples given use Linux, due to the raw, accessible nature of UNIX and UNIX-like systems, and the availability of tools like 'dd' to gather data.
Part 2 covers "Volume Analysis," or the organization of files into a storage system. This introduces the basics of things like partition tables (including how to read one). The next few chapters cover PC-based partitions (DOS and Apple), server-based partitions (BSD, Solaris and GPT partitions), and then multiple disk volumes like RAID and logical volumes. With this introduction, the final chapter of the section covers how to use these filesystem descriptions in practice to look for data during analysis. Filesystem layouts, organization, and things like journals and consistency checks are covered with a clarity and exactness that's refreshing for such a detailed topic.
Having covered the basics of filesystems, Part 3 covers the bulk of the book and material. Several chapters follow that specifically show you how to analyze particular filesystems by using their data structures to direct your reads. A range of filesystems are covered, including FAT, NTFS, EXT2 and EXT3, and the BSD types UFS1 and UFS2. Each filesystem has two chapters, one devoted to concepts and analysis, another entirely about data structures. Dividing each filesystem type like this lets Carrier focus first on the theory of each filesystem and its design, and then the practical use of its design to actually understand how to pull data off of it.
The real strength of File System Forensic Analysis lies in Carrier's direct and clear descriptions of the concepts, the completeness of his coverage, and the detail he provides. For example, a number of clear, well-ordered and simple diagrams are peppered throughout the book, explaining everything from allocation algorithms to NTFS alternative data streams. This use of simple diagrams makes the topics more easily understood, so the book's full value can be appreciated. This is the kind of thing that sets a book apart from its peers and makes it a valuable resource for a long time.
Finally, Carrier brings it all together and shows us how many aspects of filesystems can be examined using his "sleuth kit" tools, freely available and easy to use. Without appearing to hawk this tool at the expense of other valuable resources, you get to see how simple and direct filesystem manipulations can be done using a direct approach. This kind of presentation is what makes File System Forensic Analysis a great foundation.
Overall I'm pleased with File System Forensic Analysis, I think that Carrier has achieved what few technical authors do, namely a clear explanation of highly technical topics which retains a level of detail that makes it valuable for the long term. For anyone looking seriously at electronic forensics, this is a must have. I suspect people who are working on filesystem implementations will also want to study it for its practical information about NTFS. Overall, a great technical resource.
You can purchase File System Forensic Analysis from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Don't forget to mount the drive as read only!
___ www.lingo24.com Language and translation solutions - online
Why in the hell would you choose a dull career like forensic investigation based on a TV show? That would be like becoming a cop because you want to be like Dirty Harry. How many of these gits go into college for this kind of career, because they think it's going to be exciting and they're going to discover the case-cracking evidence in a few hours, grab their gun and go make an arrest?
I would say a book on how to snoop on people hard drives and see what they deleted is pretty privacy invasive? Most legal investigations are invasive by their very nature.
How long will it be before there are a million "IT Forensics" certification mills out there advertising on the radio to knuckle-dragging GEDs to come get certified and make $$$ in this "HOT, NEW, EXCITING INDUSTRY!!!"
This sounds really interesting. I've been fascinated for a while with how the file / folder metaphor has become so entrenched that people have a difficult time imagining any other way of thinking about it.
As the OS has become more sophisticated, most computer users now never see things like a disk defrag. They really think that there is a file, all in one spot in their computer, that sits literally next to other files in the same folder. The idea that you can recover a file that has been "deleted" seems like deep wizardry, with no thought to the more impressive wizardry that makes "files" out of pieces of metal with a magnet.
Best Windows Freeware
For alternate opinions on the book see this review by Rob Slade in RISKS Digest, and this short rebuttal of Slade's review by Simson Garfinkle.
And no, I don't work for these people. I just think they make some nifty geek toys.
No, that's not why I have SCSI drives on my home server. Honest; it's for the RAID performance....
//Information does not want to be free; it wants to breed.
Install an old version of windows, unpached with no firewall protection.
I suggest getting: Incident Response (Kevin Mandia and Chris Prosize) and also Computer Forensics (Warren G. Kruse and Jay G. Heiser). Both are an excellent read, and the Mandia book has some wonderful documents to use for real-life situations.
In all, a good review of the book. However, the focus on forensics is left out of the review -- just wanted to point out that the book is more than a text on file system management, search, and data recovery.
Although, of course, the book does a very good job of being that as well.
"Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
Custom built 5.25" bay metal box, front side key locked switch controlling 12v powered spark igniter for magnesium primer charge; remainder of the box filled with thermite. Install in the computer's top bay. You can generally get all the way through at least eight drives that way, but if you have vertical mount drives, you'll want a second kaboom bay in the lowest 5.25 bay. Have a good UPS, and have a metal-bottomed water tank below the computer (camoflage as an overclock device), because that much thermite does NOT stop quickly.
They can pry my PGP key from my computer's cold dead... um, slag. =)
//Information does not want to be free; it wants to breed.
a series of how-tos and standards docs
At the behest of the DOJ, NIST has been grinding out standards on how to forensically analyze a hard drive an other arcana for several years now.
NIST even provides tools: http://www.cftt.nist.gov/
SLASHDOT: news for people who can't concentrate on work or have no life at all and got tired of yelling back at the TV.
I will look forward to watching SCSI-Miami.
I know that encryption is a topic unto itself but it is becoming more and more common for people to create PGP Disks or DriveCrypt disks.
How do those things fit into this topic? I mean, the filesystem stuff is great and interesting but it doesn't seem to do any good if all you can recover is a PGP Disk file*.
Can someone much smarter than me tell me how data forensics deals with that????
* PGP Disk: a pgp encrypted file that can be mounted as a drive letter. It is, literally, a file just sitting there on your harddrive. You mount the file (after providing the secret passphrase) and voila! - you now have an encrypted drive to copy files in and out of.
I do 'forensics' sometimes. I was freelance fixing computers for a while when one of my clients asked me to find out what her husband was doing online. For a princely sum I began doing 'stealth' missions for many distressed spouses. I uncovered a lot of dirt and presented it with the understanding that I never be named or asked to testify.
Morally, it's a dark-grey zone, but it payed well and I provided the hard evidence needed to end a few broken marriages. All my former clients are better off after they found the truth.
It was odd explaining to the ladies that the VAST majority of men on the web look at porn, and that it's not anything to worry about. I was looking for personal ads, dating sites, child or extreme porn, and S&M personals sites.
It's exciting to get the call at 8am to come and clone a drive on-site. I then take it home and get what I can from it however I can, from mounting and browsing to hexdumping and grepping.
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails
Related links:
Digital Forensic Tool Testing Images
Brian's Tools - Includes links to SleuthKit and Autopsy
Forensic Tool Kit free trial
FTK is a nice tool to play around with for Windows users, especially with the testing images. The free trial does have a limit of 5,000 files per image so if you create or work on testing images you may have to get rid of extraneous junk and leave the good stuff. SleuthKit and Autopsy are great for the *nix environment. After you get those tools working you might give Scan of the Month challenges 24 and 26 from The Honeynet Projecta shot. They're both pretty fun and challenging. Don't worry if you don't know what you're doing. Both of the challenges have writeups done on how to accomplish the tasks and what tools were used if you need guidance.
> Mmmmm young girls...
You'd better hope nobody does a forensic analysis of YOUR filesystems.
- For the complete works of Shakespeare: cat
Crooks who are "smart" are going to encrypted systems and making darn sure there's no unencrypted writable storage lying around. This, plus tamper-evident computer including tamper-evident keyboard and keyboard-connectors and a faraday cage makes it very hard on the police.
/dev/hda"? I knew you could.
:).
Can you say "boot with Suse Live CD and encrypt
This only works in jurisdictions that can't force you to reveal your passphrase. In those jurisdictions, smart crooks outsource thier IT to North Korea
That still leaves plenty of forensics work for criminals using other people's computers such as white-collar crooks and the 99% of crooks who aren't smart.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
You'll want more than a water tank below the computer since water doesn't stop a thermite reaction. Try a couple of layers of firebrick or some other ceramic that won't shatter due to exteme heat.
Honestly, this job is probably the coolest I've done. We get the run of any joint we enter. We get to crack people's passwords, read their stuff, and pry into the details that they're trying to hide.
Outside of the unreal timeframe, it is a bit like television. I've been on location at 1 AM acquiring hard drives so that the debtor principles didn't know what we were doing. Walking through the data center with my mag light at that hour of the morning comes pretty close to that feeling you get when you watch CSI on TV. Most of the time, we tell the people on location we're making "backups" of the data so that we can preserve the data in the event of a crash. There's definitely a social element to forensic work (at least in bankruptcy cases).
A typical acquisition may go something like this:
You set up, pull your forms, start noting observations, pull the drives, hook them up to the little black box connected to your laptop's firewire port (a write-blocker), and start having a look at the data. If you've got what you're looking for, you acquire the drive and put everything back together. Boot it all up and be on your way.
You may be doing this in the CEO's office, or in the data center looking for a mail server. The top officers are usually the most important, since they have the most important correspondence and data.
It's a fun job. It's every bit as exciting as what you see on television (for once).
If you don't want anyone to find out what you have been doing on your computer, then a hammer is the best choice.
I found that too... I got Hammer to defend my computer, and any time someone tries to take the drive away for forensic examination Hammer stops them by saying "You can't touch this!"
"Slashdot - News and Chat Sites Deviant". (Click "homepage" link above for details).
defenetly a little extreme, but as the other replier stated that water wont stop thermite very quickly. In reality you dont need that much distructive power to distroy a harddrive.
If I had my way, I'd just put a small shapped charge ontop of the harddrive. Small enough to distroy the harddrive (and probably some other stuff in the machine w/ fragmentation) but not big enough to blow up the entire machine. Cases are preety well built now adays, and with some re-enforcement they could take a small shapped explosion (that was not pointed at them). But this is all under the guise that you can get your hands on all this stuff.
What can the real person do to protect themselves is a better question. What quick/distructive meathods are there for the real person.
snowulf.com
On a more fundamental level, privacy is a conditional right. A person has to behave in order to enjoy it. It is not a shield for wrongdoing. Moreover, in a marriage it is patently obvious that both are willingly giving up privacy. I have fewer qualms with spousal snooping than that on kids or employees.
But beware, the discoveries hurt!
It's really profitable... I was charging $200 an hour. Spent a ton of time digging around on a bunch of CDs, a hard drive and thru a couple of email inboxes. Plus my client had a key logger.
cool stuff.
Rather than being so worried about what is there or not, the deeper and far more difficult question is: why is it there?
With the existence of zero-day exploits, spyware-zombies-for-sale, broadband, etc., how can anyone convince a jury beyond a reasonable doubt that someone put the bits there THEMSELF without a confession or video of them actually putting the content there?
People are going to jail because of this shit. Digital evidence is an oxymoron.
For a law firm, I investigated a drive that had been stolen by a former employee. The drive had been recovered, and my task was to determine what he had done with it and whether he had taken or tampered with any of the intellectual property on the drive. It paid very handsomely for the amount of work involved, and it was an intellectual challenge. That said, this book may have made it easier (I didn't read the review in-depth or the book itself, but I assume it wouldn't make the task more difficult).
/, /usr, /home, and so forth; he had mounted it on /mnt1, /mnt2, /mnt3, and such).
In this case, I determined that the employee had mounted each partition on the drive to a separate mount point, not in the original structure (such as
It's not as glamorous as extreme porn or personal ads, but it was still interesting.
You DO NOT want a water tray at the bottom. What makes you think a little bit of water will stop thermite? You need a tray full of sand. The thermite is hot enough to seperate the hydrogen out of water, not a great move.
I bet thermite is probably kinda hard to get your hands onto
Do you really think that aluminum and iron oxide are that hard to get a hold of? Anyone who has passed high school chemistry could make it.
In my experience it is harder finding a way to light the thermite then it is to acutally make the stuff.
> why don't you use Linux and simply create a drive image straight from the raw device without mounting at all
Because once you start blathering on and on under cross-examination about raw devices, MD5 hash integrity, etc., the jury, which will probably consist of morons, will slowly doze off into la la land and blow off evrything you are saying.
Much better to spend $500 and tell the jury, "Jethto, Earlene, I got this here special dee-vice that physically prevents tampering."
To quote (fairly accurately IIRC) a juror in the Vioxx trial that just ended, "They started talkin' all that science talk and it was like - wah wah wah wah wah wah" (sound of the Teacher talking from the Charlie Brown videos).
Give a man a fish and you have fed him for today. Teach a man to fish, and he'll say "WHERE'S MY FISH, YOU IDIOT?"