Slashdot Mirror
Windows Forensics and Incident Recovery
The intended audience, according to the author, is "anyone with an interest in Windows security, which includes Windows system and security administrators, consultants, incident response team members, students and even home users." The author assumes the reader is familiar with basic networking (including TCP/IP) and has some Windows administration skills. Some programming ability, though not actually required, will help out greatly with reading and understanding the many examples provided, and will let you make your own modifications (this is encouraged by the author throughout the book).
The chapter on data hiding was a real eye-opener -- it's amazing the things Microsoft has implemented as part of the operating system (and included applications) that can be used to hide things. Discovering the hidden information is talked about, as well how it is hidden. Sample topics include file attributes, alternate data streams, OLE and stenography. This is an excellent chapter with many examples; I found myself stopping after each subject to try out each of the discussed techniques.
The next chapter delves into incident preparation. Carvey addresses some of the things that administrators can do to harden their systems. He goes over the application of security policies in general, as well as intelligent assignment of file permissions. He then covers Windows File Protection and how it is implemented, and includes a perl script to implement your own file watcher. He touches briefly on patch management and anti-virus programs, then moves into monitoring. He provides quite a few scripts, and discusses other means by which you can monitor your system.
The next chapter describes tools that can be used in incident response. This chapter has quite a lot of information and took me the longest to get through, because of all the tools mentioned that I had to download and check while I was reading the book. Carvey uses a mixture of his own perl scripts and programs that can be downloaded from places like Sysinternals, Foundstone, DiamondCS and others. All of the tools used are open source (or are at least freely available). That equips the reader with a low-cost toolkit, especially important to the home user or small business owner who cannot afford to buy the commercial equivalent. Carvey does acknowledge, though, that there are quite a few commercial tools with great functionality out there.
The first part of the incident-response tools chapter deals with the collection of volatile information (processes, services, etc.); this is a vital part of live analysis. The second part deals with the collection of non-volatile information (the content of the Windows registry, file MAC times and hashes, etc.) and tools for analyzing files. Carvey also shows how some of the tools complement each other, and that there is not one almighty tool that will find all the data you need. (This is also proven by example in a later chapter when he talks about rootkits.)
The next chapter deals with developing a security methodology, and it's handled differently than in most books: the author presents the material as a series of dreams that a Windows system administrator has, showing how an individual can come up with and fine tune a methodology as incidents happen. Carvey has used this approach before in a series of articles entitled "No Stone Unturned" for SecurityFocus.com, and the creative approach appeals to me. As he moves from dream to dream, you can relate to the admin's circumstances (and mistakes), and how be and becomes better at responding to different incidents.
The next chapter talks about what to usefully look for with the tools the book has introduced. It discusses infection vectors, types of malware and rootkits, and demonstrates tools and techniques for detecting them. This is where the author makes a clear point of why you would need to run several different tools, even if some overlap. His example uses an installed rootkit; running a particular program from a previous chapter, he shows that it fails to find that anything untoward is running -- it takes another program from the same chapter to actually reveal the rootkit's presence. By cross referencing the output for both programs, you can see why you should run more then one type of analysis tool for certain areas to make sure you are not missing anything.
Finally, the author dedicates an entire chapter to his own Forensic Server Project, a two-pronged approach to live forensic analysis which uses two machines simultaneously. The first piece, the Forensic Server Module, is the listener software; this runs on a clean PC where the data will be sent from the compromised system. The other piece, called the First Responder Utility, runs several of the programs and scripts from the incident tools chapter on the compromised system . After installing everything needed for both parts of this system, I followed the author's instructions on how to run it. What a slick tool! I ran it from a couple of PCs on my home network and was able to get a lot of the information that was described in the book as well as hash values for each log file that was produced, and a general log of everything the First Responder Unit did. The whole principle of this is that when you have an incident there will be very little interaction with the compromised system, since everything is scripted to begin with.
The framework that this software constitutes is very flexible. I was able to add two new features to the Forensic Server Module and the First Responder Utility with very little code. The first addition I made was to mark all the logs as read-only on the file system after they were written from the Forensic Server module. The next addition I made was to add a perl script to scan the c:\ drive of the PC that the First Responder Utility was running on. After I made both additions, I tested everything out, and it worked great. I had my extra log files and they were all read-only. My hat goes off to the author for coming up with and including this in the book, a really nice piece of software.
You can purchase Windows Forensics and Incident Recovery from bn.com. Slashdot welcomes readers' book reviews. To see your own review here, carefully read the book review guidelines, then visit the submission page.
You have an older server that has a relatively small hard drive. You get a complaint from somebody saying they can't do so and so on the server. You look on it and find that the hard drive is full. You think, oh great, somebody printed a 200mb print job again and filled up the hard drive. Well, time to reboot the server. You have just lost an enormous amount of evidence and you may never know where the pirated games, movies, and music came from.
I am a windows/linux sysadmin and when something goes isn't working right, my first thought is not *I'VE BEEN HACKED*, no its "stupid {some software}, {some company} can't program to save their life. time to restart the service or reboot". I will agree that discovering a problem at this point is too late - you're already 0wN3d. Instead of focusing on forensics, we need to focus on proactive measures - use group policies to enforce better security policies; use ntop, nmap, snort, gfi languard, and ms baseline security analyzer to check your systems; dump your linux and windows boxes to a syslog server that notifies on any irregularities; use SUS, SMS or something similar to patch all systems quickly and efficiently. If we are more proactive, then forensics will be less of an issue.
And that kind of an IT response (problem? reboot!) is fine, because you've got work to do after all.
However, at the point that you discover that there's been an incident, you should note what's running (ps), what's open (netstat), who's logged on, and what drives are mounted. Document everything that you do (date/time, action, reason for action), pull the plug, and call your lawyer.
Incident response is difficult as the scale of the "response" can vary so widely. Sometimes it's enough to run a virus scanner and reboot. Other times you may find you've been 0wn3d by international hackers conducting IP theft (I've seen it) and you need to proceed very carefully. And then there are the HR investigations...
My main point is simply that when geeks start to tread into the area of incident response/forensics, it's important for them to know what they're doing. When you're operating in the context of traditional IT, an IT response is okay. When you're operating in the context of an investigation, a traditional IT response is typically the worst thing you can do.
You are also very correct, though, in noting that pro-active security is where the typical sysadmin should have his/her focus. You don't want to be the sysadmin who let in the international hackers stealing all your company's intellectual property.
Jon
"Forensics" on a live system is a misnomer. For incident response, collecting live data on open ports, running processes, logged on users, and mounted devices is useful and sometimes necessary. Investigators should be sure to check -- gingerly -- whether any encrypted volumes are mounted.
/tmp directory (or any other mount point, potentially) that allows seamless file-and-directory semantics but that never writes anything to disk.
/tmp directories in the process? That's liable to get you in the doghouse, that's for sure!
Why a "misnomer"? Forensics, in the usage of the phrase "computer forensics", is an extension of our usage of forensics to refer to the presentation of evidence in court. The word initially means "methods pertaining to proving a proposition by logical argument". It doesn't matter whether the collection of data is live or not, it's still "forensics".
Your statements, despite your credentials, suffers from the failing of being Windows-centric (come on "registry keys"?), and (worse, still) Encase-Centric. And your advice about pulling the plug is horrifically oversimplified.
What happens when someone's walked into your LAMPS servers through an unpatched OpenSSH vulnerability, but hasn't installed a rootkit? If you just "pull the plug", the real evidence of the intrusion goes bye-bye because none of it was written to the hard drive. You'll probably want to look at netstat, maybe take a core dump to a remote machine, and generally examine the state of the running machine. THEN, depending on the type of filesystem, you either A) pull the plug, B) issue a 'flush' and then pull the plug, or C) go through the machine's normal shutdown procedure.
Think about it: If you just pull the plug without thinking the matter through, you stand a good change of fucking things up. Yes, many sysadmins cause problems by trying to do too much work on their own, or taking steps that actually destroy existing evidence, but that's no reason to give blanket advice that can be as shitty as it is good!
The correct solution: train the sysadmins to be first responders, or at least train them enough to be capable of keeping their hands off the systems until qualified help arrives. Whether you have in-house incident response people or you hire an outside firm, make sure that your admins consult with them before moving on anything that could lead to a legal issue.
Your biases are reflective of desktop experience on Windows platforms, which is a large part of forensic work, I'll grant. But it's far from all of it, and it's actually counter-productive on many non-Windows machines. For instance: are you familiar with a thing called tmpfs? It's a RAM-based filesystem for the
Guess what? JUST ABOUT EVERY LINUX DISTRO USES TMPFS, NOWADAYS! Can you imagine how badly you would screw up investigations if you went around pulling plugs on a bunch of Linux machines, and losing the entire contents of their
There's no excuse for looking like an expert but giving poor advice, even if this IS Slashdot.
I run patches when I do hardware maintenance. If that means I run a few months behind the latest and 'greatest' Microsoft patches, so be it. I run a router with a stateful packet filter, an application-level software firewall, an IDS, and multiple anti-virus/anti-trojan scanners. If someone manages to get into my system and stay there, they deserve it.
-- "Government is the great fiction through which everybody endeavors to live at the expense of everybody else."
Commenting on the pulling the plug...
this is generally suggested for law enforcement who respond to a crime scene for which there is a standalone running machine and a qualified computer forensic examiner is not readily available. In the "bag and tag" process, the responding cops are then told to pull the plug (literally...not from the wall, but from the machine itself) and then bag and tag the machine. This might seem odd at first, and yes things can be lost, but the key is obtaining "forensically sound" evidence which is admissible in court.
Just as you wouldn't want the computer guy touching the dead body on the floor, you don't want the average police guy trying to do an analysis of the machine. Even if he does open a few folders and find undeniable evidence of the crime, a good crossexaminer would rip that up in court. The idea is that you allow the least amount of "tampering" as possible before the guy who knows what he is doing gets ahold of the machine.
Now, that said, if you walk in and there is an unsaved but important-looking document sitting on the desktop, you're not going to want to pull the plug. At that point, you take pictures and then have several witnesses present to try to preserve the document by saving it to an external drive. In that case, you gotta do what you gotta do.
However, the plug-pulling strategy, I can second, is a standard practice for first responders when a live, on-site replication of the machine is not possible. In that case, you really do have to pull the plug, bag, tag, and examine later.
*-*-*-*-*-*-*-*
"We are Linux. Resistance is measured in Ohms."
Windows-centric? The title of the book is *Windows* Forensics and Incident Response. I guess this is Slashdot so I should expect to be flamed for being on-topic...
/tmpfs. And then the plug would be pulled.
My top post isn't about EnCase.
As others note here, standard forensic practice -- especially when you don't know what you're doing -- is to pull the plug. That's what the DOJ says, at least.
In the example that you give, of the OpenSSH vulnerability, it's very likely that the intruder will leave filesystem artifacts. sync() runs every few seconds. Syslog keeps track of all kinds of events (not to mention shell history files!). Sure, if someone's got root they can cover their tracks completely, but we're rarely dealing with Moriarty -- usually it's Moe and his friends, Larry and Curly.
Additionally, the distinction needs to be made between forensics and incident response. That distinction is definitely murky, but it exists. When you suspect an "incident", which is usually a code word for "exploited vulnerability", yeah, gathering live data totally makes sense, and I'd do it. Once I'd gotten my data, I'd then pull the plug. For forensic software, mildly corrupt filesystems resulting from powerloss is not a big deal. But if you think about the potential for losing artifacts that is entailed from performing a clean shutdown, you can see why abrupt power cut-off is the way to go.
For example, tmpfs. Sure, tmpfs is a "RAM"-disk, but that memory is backed by swap. If I'm the kernel, I bet I'm going to see that a lot of the files in this 'tmpfs' are relatively unused -- swap them away! Now, consider what happens when you perform a shutdown. Processes are killed, users are logged out, daemons are signalled, runlevels are changed, etc., etc., etc. That sounds to me like a pretty invasive procedure, one that could very well involve lots of swapping. So then you lose tmpfs. As noted above, if I really cared about the live data, I'd capture it, as simply as possible, and then I'd pull the plug. So I'd run ps, netstat, maybe lsof, and ls -ls
As far as getting in the doghouse, my experience has been that it's far better to be in everyone else's doghouse but have your General Counsel love you than to have everyone else love you and be in your General Counsel's doghouse. That's the key point, really: When legal gets involved, IT doesn't matter.
cheers,
Jon