The Honeynet Project Has A Winner

AltGrendel writes with a welcome followup: "The Honeynet Project has announced a winner. OK, actually they announced the three winners and have posted the results here. Details as to how all this was accomplished will be posted though the next few weeks. Congratulations to all finalists!" This project has been mentioned a few times before; if you thought running Linux made you obscure enough to largely escape the attention of random and non-random malice, this is a thorough (if depressing) reason to think otherwise. Hats off to the Honeynet Project and participants for putting this labor-intensive analysis together.

Comments

Re:Inflated damage numbers

[dair]

...that shoplifter caused tens of thousands of dollars worth of damage?

You missed out the bit where he says:

losses should only be allowed if such losses can actually be proven, unlike for example the Steve Jackson Games case where a 911 document which could be purchased for some US$30 was valued at US$79,449 for purposes of estimating damages.

Stealing some M&Ms is a self-contained and easily recognised (there's one less bag on the shelf) act, so the damage would be limited accordingly. A compromised system could have suffered any amount of changes - you need to spend the time to understand exactly what did change before you can be sure you've covered it all.

-dair

Re:Damn

[volsung]

Yes, to be truly secure, you should copy your RPM database to CD or Zip, and boot the removable media (with rpm on it) to check your installation.

However, you'll be able to catch the vast majority of rootkits even with RPM on a comprimised system. So far I haven't seen one smart enough to cover its tracks that extensively. (Not to say that it doesn't exist, it just isn't very common. Patching the RPM database is not needed because the average Linux workstation user doesn't even know about rpm -V)

Further in the same article...

[gavinhall]

Posted by srvivn21:

That is not to suggest that every intrusion warrants a complete forensic investigation, but in some circumstances it is entirely appropriate and needs to be done quickly (and correctly).
A stolen bag of candy is easy enough to quantify. An intruded system is another matter.

There is NO security and won't be until we use bio

[crovira]

The problem is that our systems are designed around a fundamental flaw, security by what you know (anything you know can be known by others.)

We have to get to systems designed around security by what your are.

We have no security and we won't until we start using biometric characteristics for security (even the primitive A.F.I.S. retrieval key for fingerprint identification, 13 points of reference with self-relative x,y coordinates, ie direction, shape, depth characteristics & 26 double precision floating point numbers, in a single string) is a lot longer than some lousy digit PINs.

There's much secure identification available.

Basically what this means is that secure systems will be extemely secure with unforgable keys (imagine using your you DNA sequence for encrypting your e-mail, in hardware.)

Re:There is NO security and won't be until we use

[Quikah]

While I would agree that such information is unforgable now I really doubt it will stay that way. DNA is just information after all, and remember if we can see it we can copy it.

Besdies I could always just cut off your hand and get through your fingerprint/dna security. :)

I envision a computer of the future knowing who I am by the way I act. So I sit down and have a nice short little chat with my computer and it logs me in because it knows who I am by the way I talk to it. A bit of a crazy idea, but I think it would be neat.

Re:Here's another winner

[Felinoid]

A short lesson in stocks by Jeffery "Felinoid" McLean.

The economy dumps. All stocks dump.
Microsoft bad mouths Linux.. Linux stocks dump.
Microsoft is sued into oblivion.. All tech stock dump "including Linux stock".

Linux makes money.. Stock dumps.

VA Linux loses billions and on brink of death....
Stock shoots the roof.

when will people get it?

[kennedy]

If you are really so worried about blackhat hackers why not just use a hardware firewall and NAT and give every machine an unroutable ip. use port forwarding to direct traffic towards your mail and web servers, and just make sure your all your daemons aren't grossly insecure.

I'm really getting sick of this "which os is the most secure?" crap. if you want out of the box anally secure go with OpenBSD. period.

Re:when will people get it?

[strags]

Those are both good suggestions, however, once again, there's the trade-off between security and useability. For instance, I can't admin remotely with your scheme.

As for "Make sure all your daemons aren't grossly insecure"... well, that's rather the point, isn't it?! A bit like saying "If you're worried about security, then just make your box secure" - strictly speaking, correct, but not terribly useful.

Strags

Re:Inflated damage numbers

[Mike Schiraldi]

I respectfully disagree.

Note, however, that i'm not defending the criminal in any of these examples. If someone walks into my house through an open door, they're still guilty of trespassing. However, it's not as bad as if they rooted through my drawers or set the place on fire.

But let's say they do break into my house and set fire to my filing cabinet. It costs $50 to buy a new filing cabinet, $5000 worth of my time to get my files back in order, $1000 to hire a detective to figure out how they broke in, and $30,000/yr to hire a security guard. I would blame the criminal for the $50 and the $5000, but not the $1000 or the $30,000/yr.

--

Re:Inflated damage numbers

[Mike Schiraldi]

A better one is that someone broke into the supposedly locked & secured office, opened the safe & stole a money wrapper.

Excellent point! Here's a new analogy: Let's say they just picked a lock without damaging it, opened a safe without damaging it, took nothing, damaged nothing, and left a note that said, "You need better security. I broke into your safe."

I'd say they're guilty of the criminal charges of trespassing, burglary, and breaking and entering. However, i don't think they're guilty of any civil charges, and i don't think they should pay one penny to the store in question. Even if the store in question has to conduct an extremely expensive review or purchase a new security system.

--

Re:Inflated damage numbers

[Mike Schiraldi]

[it's a huge pain to recover from a breakin]

Absolutely. It sucks. It's an astronomical expense. The intruder is an asshole. And guilty of criminal charges. But that doesn't mean you should be able to send them a bill for your new security measures. Security measures are YOUR job, and YOUR expense. Damages are their expense, but should only include damages, not investigative costs or new security.

If the police or the store spend ten million dollars to figure out that you were the one who stole a pack of M&Ms, that's their expense.

Why would you be so careless with your servers then?

Who said i advocated being careless with servers?

--

Re:Inflated damage numbers

[Mike Schiraldi]

What if they then post on IRC the exact steps needed for someone else to break back in?

If that's as far as it goes, then no new crimes have been committed. If someone else uses that information to break in and break a bunch of stuff, then the first person, IMO, is guilty of accessory to [whatever]. In this case, the first person should be expected to pay for damages. (The second person, too)

--

Re:Inflated damage numbers

[Mike Schiraldi]

I don't accept that crackers are performing a social service

Me neither. I hate that argument. People who break into other people's systems are assholes and criminals, and should go to jail, for about the same amount of time as people who break into other people's houses.

If someone breaks into someone else's site and gets caught then they deserve what they get. Nobody forces people to go cracking. If you can't do the time don't do the crime.

Absolutely.

At $2K per day consulting rates that is a non-trivial amount of cash.

Absolutely. Their expense, not the intruder's.

--

Inflated damage numbers

[Mike Schiraldi]

"But all it takes to re-install Red Hat is 30 minutes. How do you come up with US$2000 damage?" ... When a system is compromised, and the data on it and its network are compromised, it is not simple to determine the extent of the damage without a lot of work. We do not know if the blackhat stold peoples passwords, hacked other systems, has implemented sniffers, etc. This argues for strong prevention, defense in depth (including monitoring in depth), and trained responders. If all the administrator does is re-install the OS, they are doing a wholly inadequate job of responding to a security incident, as the extent of damage may be far greater then a single system.

So if someone steals a packet of M&Ms from the local grocery store, and the grocery store conducts a full review and decides to hire a $20/hour security guard, spend $1500 installing cameras and a closet-circuit TV system, and install a checkpoint at the candy aisle, that shoplifter caused tens of thousands of dollars worth of damage?

--

Re:Inflated damage numbers

[Mike Schiraldi]

How much do you think it would cost Boeing for this incident? How much would you, as a potential passenger on this aircraft, expect them to spend on this one incident?

An incredible amount of money. But it's their own fault, not the intruder's fault. The intruder is guilty of a lot of criminal charges and should go to jail for about the same amount of time as they would if they broke a window, climbed into the office, and rooted through everyone's desks. But no longer than that. And they certainly shouldn't be blamed for the expense of investigating the breakin or beefing up security.

--

Re:Inflated damage numbers

[spudnic]

Now assume that a clerk at a large grocery store noticed someone hanging around the candy isle looking suspicious. The clerk walks over just in time to see the person inject something into a pack of M&M's and throw it back onto the shelf. Knowing that they have been seen, they run out of the store.

The clerk picks up the packet of M&M's and can hardly see the whole created by the needle. It is so small that it would almost certainly go unnoticed. How many other packets had the person messed with before they where noticed? None? Half of them? Which half?

Just assuming that you had caught them in the act on the first and only packet of candy they messed with would be foolish.

It could take a REALLY long time to go through each piece of candy to see if there was some indication of tampering.

Now in this case it might be cheaper to just throw away all potentially tainted candy, but say there is a world candy shortage and each of those packets cost $1000.00US...

You either pay the price of lost candy, or hire VERY competent people that you absolutely trust the ability of to search each piece with a microscope looking for problems.

Could get very expensive.

Re:Inflated damage numbers

[Amokscience]

How about if someone breaks into Boeing's network and the new airline design is accessible on that network. Furthermore assume that the breakin isn't detected for a few weeks and that lots of work has been done. Now, suppose the person didn't steal anything.

How much do you think it would cost Boeing for this incident? How much would you, as a potential passenger on this aircraft, expect them to spend on this one incident?

This actually occurred several years ago and the estimated cost to Boeing was (IIRC) over $200,000. They had to verify that their data integrity was ok, that work hadn't been tampered with, and so on into the dull sysadmin stuff...

So, yeah, a simple act can cost many times as much as its face value. I don't believe someone would let it stretch as far as your example (and I hope not if it's just M&Ms) but the principle is in place.

Re:Inflated damage numbers

[kevinank]

A better analogy than the M&M's would be the airplane passenger who jokingly mentions that he plans to hijack an airplane.

Hundreds of people are delayed while the airplane is searched for explosives, plus there is the cost of security guards to hold the person in custody, and the staff to complete the search.

That is easily $20k worth of damage, especially when multiplied by the number of of people who visit a busy web site (as opposed to the ~200 that might be delayed at the airport.)

Re:Inflated damage numbers

[gfxguy]

I agree...it sounds like MPAA and RIAA inflated damages from pirating, they can make up any numbers they want.

Re:Inflated damage numbers

[jred]

>So if someone steals a packet of M&Ms from the local grocery store, and the grocery store conducts a full review and decides to hire a $20/hour security guard, spend $1500 installing cameras and a closet-circuit TV system, and install a checkpoint at the candy aisle, that shoplifter caused tens of thousands of dollars worth of damage?

No, but that's not a good analogy. A better one is that someone broke into the supposedly locked & secured office, opened the safe & stole a money wrapper.



jred
www.cautioninc.com

Re:Inflated damage numbers

[Ian Wolf]

The two situations are completely different! I would argue that $2000 is a low estimate. That amount only reflects the cost of the Admin(s) time spent investigating a security incident and fails to even cover the costs of data loss, downtime, or even possible legal action.

What if the company is a health care provider and confidential medical records are compromised? The legal firestorm and damages would make $2000 look like a penny on the sidewalk.

Now if that bag of M&M's had been an instant winner for a new car, then you might have had something there.

Re:Inflated damage numbers

[Ian Wolf]

The problem with that analogy is that you can clearly determine whether or not they took or changed something. In the case of a system storing all kinds of data, this cannot be so easily determined.

Re:Inflated damage numbers

[rabtech]

I'm glad you aren't in charge of security for any major websites, since you obviously have no clue.

Assuming you wiped the machine and reinstalled, you have to spend hours restoring from tape (assuming you kept good backups), then replacing any material missing since that backup, and finally reset any permissions or settings that may have been lost. This all assumes that you KNOW when the hacks were initially conducted.... for all you know, your last 6 months of backups might also contain virii or backdoors!

If you don't do a reinstall, then you've got to carefully examine every folder. Look at the configurations for all deamons and running processes. Check over log files. Leave no stone unturned. Leave no file unchecked. If you don't examine everything, you risk missing a trojan or backdoor installed somewhere. You need to change passwords on that machine and other machines that might have been accessed from it (like database servers, etc.)

All these things, and many more. To do any less would be like saying "PLEASE, COME VIOLATE ME AGAIN!"

A more correct analogy would be if a bank were broken into without anyone detecting it until a week later..... would you still entrust your safety deposit box and money to them if they decided not to make any changes to their security protocols or go over all the video tapes frame by frame looking for any problems, as well as hiring new guards and possibly bringing in consultants to review their procedures? Absolutely not.

Why would you be so careless with your servers then? I'll repeat my earlier lament: THANK GOD you aren't in charge of any important servers.
-------
-- russ

"You want people to think logically? ACK! Turn in your UID, you traitor!"

Re:Inflated damage numbers

[Zeinfeld]

So if someone steals a packet of M&Ms from the local grocery store, and the grocery store conducts a full review and decides to hire a $20/hour security guard, spend $1500 installing cameras and a closet-circuit TV system, and install a checkpoint at the candy aisle, that shoplifter caused tens of thousands of dollars worth of damage?

If someone breaks into a candy store with a large jar of cyanide and a syringe then it is pretty logical to infer that he may have poisoned one of the packets of candy and that therefore the entire stock has to be destroyed.

I don't accept that crackers are performing a social service. If someone wants their security analysed they can ask someone.

If someone breaks into someone else's site and gets caught then they deserve what they get. Nobody forces people to go cracking. If you can't do the time don't do the crime.

In any case it certainly takes more than 30 minutes to do a complete installation of any operating system from scratch on a main server and restore all the facilities to full working order. It may be possible to return a dumb client machine quickly but rebuilding a server takes time. At $2K per day consulting rates that is a non-trivial amount of cash.

Re:Inflated damage numbers

[Zeinfeld]

Absolutely. Their expense, not the intruder's.

Given the number of people who go to jail for cracking I can't think of anyone who got a raw deal. It is pretty rare for someone to go to jail for a first offense.

There is certainly a problem with political prosecutors looking to goose their poll numbers for re-election by inflating the damage caused.

The only multi-year sentence that I can recall offhand was for Mitnick and he get several warnings. His sentence did not stike me as steep compared to what people who are convicted of house breaking a third time.

Given the number of people on sentences of 20 years to life for minor drugs charges under the 'war on drugs' I don't think that crackers get a raw deal.

I don't think that a 10 year sentence would be unfair for someone writing a highly destructive virus.

Re:Inflated damage numbers

[Zeinfeld]

Oh wait, you weren't advocating that? I would hate to create a Straw Man Argument.

It is not an uncommon argument in the cracker sub-culture.

Until a couple of years ago it would appear regularly in print as a journalist wrote up a story on the basis of an interview with a cracker group.

These pieces were notable for their complete lack of balance, the journalists would never contact a security professional such as myself to give the other side and describe the effect that the attacks have.

Another cracker conceit is the pieces were noted for is the idea that security consultants are all former crackers - most of us are not. In fact a security consultancy firm will almost certainly do background checks and refuse to hire anyone with a cracking conviction.

It has taken several years and a lot of work with the media to demolish the false image that crackers have been allowed to project.

Re:Inflated damage numbers

[Kujako]

What if they then post on IRC the exact steps needed for someone else to break back in?

"What is this talk of 'release'? I do not make software 'releases'. My software escapes, leaving a bloody trail of designers and quality assurance people in its wake!"

Link not quite right

[Brento]

It was missing a slash. Here's the real link:

http://project.honeynet.org/challenge/results/

Re:Link not quite right

[interiot]

Damn. I was hoping someone had just cracked the site... ;)
--

Re:Link not quite right

[spezz]

See, I always just assume the sites are /.'d

Re:Linux a target? I'm not surprised

[Kartoffel]

err, s/VMX/VMS/. See how obscure that stuff is? I can't even remember the name properly.
--

Re:Damn

[Kartoffel]

The use of RPM to compare checksums of original files vs. current files is particularly ingenious.

When doing forensics, never trust the tools that are already installed on the compromised system. Download or bring a floppy with a known good copy of cksum(1) or something.
--

Linux a target? I'm not surprised

[Kartoffel]

Of course kiddies are going after Linux systems. Suppose you wanted to 0wn a few boxes, for whatever reason. Would you rather 0wn....

• a commercial *nix box where most of the common linux-centric 'sploits won't work?
• some (l)user's windows machine that gets rebooted and/or crashes all the time?
• a mac?
• some obscure legacy OS running in a factory or big business (w00t, be a VMX h4x0r)
• Jonh Doe's RedHat box running with an out-of-the-box config?

If I were a kiddie (hypthetically speaking, of course), I'd be going after a target that's easy, plentiful, and stable enough to stay online, allowing me to use it for scanning, bouncing, running b0ts, etc. Big Linux distros have new exploits uncovered and published every day. Linux is gaining popularity, too. All the new users installing it don't know jack about locking down their computers and it makes Linux a very ripe arena for the kiddies.

Does this make Linux a bad operating system? No, of course not. It just means that in order to really be secure, you still have to know what you're doing.
--

Re:All this seems very costly and complicated.

[jidar]

I realize that was a big tongue in cheek, but that is what we do. We have a very small network, but it's made up of commodity hardware, so when there is an intrusion, the machines on that network are replace with fresh installs. This has happened twice in 3 years. Of coarse that isn't going to work for everybody, but anyway...

Insurance Claim Damage Numbers

[Speare]

"But all it takes to re-install Red Hat is 30 minutes. How do you come up with US$2000 damage?"

More typically, a company takes out insurance against such "disasters." When the company is attacked, they have to make a claim to the insurance company. They're knee-jerk worst-case numbers. It is often these numbers that are quoted to media.

asdf

[rbreve]

uh?

Apparently...

[soulsteal]

The people who got in left a small notice with bad grammar... something about bases belonging to "us."

All this seems very costly and complicated.

[SpanishInquisition]

Why not simply throw away the computers and buy new ones.
--

Re:All this seems very costly and complicated.

[Ian Wolf]

Because you still have to find out what they did. At least if you want to protect yourself in the future. Its also prudent to find out if there has been a theft of data.

Security professionals, programmers, psychologists

[Glowing Fish]

The previous Slashdot article mentioned that psychologists were working on the project. Is it just me, or would being a spychologist trying to put together a psychological profile of 5cr1p7 k1dd13z be a very amusing job? I can imagine a psychologist sitting around, trying to guess what fonts and color schemes would be the most amusing to cr4ck3rz.

On a serious note, I wonder if any psychological research has been done that debates the notion of kiddies\crackers as being teenage males with bad social skills. It would be interesting to see if more cracker types wern't (for example) socialy outgoing and healthy college students who like the thrill of cracking. Or maybe their is a substantial amount of middle aged computer programmers that have access to high tech computers, and use them for breaking into other systems out of boredom.

despite what the previous Slashdot post said, the documents available don't seem to have any kind of abstract of the psychological characeteistics of the people that broke into the system.

Re:Lessons learned

[Glowing Fish]

You are right, but for cultural reasons, Americans will never listen to your logic. I don't know about people in other parts of the world.

In America, everytime a plane crashes, or someone overdoses on OTC medications, or someone shoots themselves in the foot or head with a hand gun, the media, activist groups and politicians swarm all over it. Sure these things are bad, but to a certain extant, they are unavoidable. American culture seems to be averse to the fact that things don't work or work in unforeseen ways, and that sometimes people get hurt or killed from these things.

Computer security is no exception to this, people are not going to accept the fact that running a computer implies the risk of having it broken into, and that there is not much to be done about it and that no one is to blame when a security hole in your chosen OS causes loss of thousands of dollars worth of sensitive data. Many people will continue to view security holes as the result of near criminal negligence.

And the script kiddies wins

[xercist]

the needless use of cat award!

bash# cat /etc/inetd.conf | grep tel
cat /etc/inetd.conf | grep tel
telnetstream tcp nowait root /usr/sbin/tcpdin.telnetd


--

Re:Security professionals, programmers, psychologi

[Ian Wolf]

This just in...
Psychologists say k1dd13z are 50cc33r m0mz.
According leading psychologists studying a particular kind of hacker, called "script kiddies", have reason to believe that they aren't really kids at all. According to the latest profile, the average script kiddie has an affinity for SUV's, cocker spaniels, and anything from the Pottery Barn. This startling discovery has led them to believe that the average "script kiddie" is actually a "soccer mom"...

Linux vs. NT security

[strags]

There is a great deal of argument about whose OS is more "secure".

It is true that both systems can be well secured if the sysadmin knows what he/she is doing, keeps up with the latest patches, and keeps a wary eye open for attacks.

It is also true that an inexperienced sysadmin can use either OS to create a box that is dangerously insecure.

So, why do I believe that Linux will always have a security advantage over NT? Because the source code is available for anyone to look at. Exploits are found, and sometimes ... err ... exploited! But - the most important thing is that they're located, and fixed!. Linux is equipped to deal with potential problems very quickly, due to the sheer number of people combing through the code. Microsoft (and any closed-source developer) simply do not have the required number of people to check their code for vulnerabilities. This means that weaknesses can only be located and fixed by a small number of people. Yes, it also means that it's harder for crackers to find exploits, but, as the last couple of years have shown, they've still managed admirably.

Security is not a quantity - it's a process, and Linux moves faster.

This doesn't apply to just security problems, but also to bugs in general. We notified MS about a bug in their TCP implementation some time ago. All we can do is twiddle our thumbs while we wait for a fix (which may never arrive). If their stack were open-sourced, we could identify the offending line of code, and come up with a fix ourselves, which we could then submit for approval, thus speeding things up no end.

Strags

Damn

[graveyhead]

I wish I had access to this when that darn RedHat "Ramen" worm hit us. There's some invaluable tricks here for finding out exactly what an intruder has done. The use of RPM to compare checksums of original files vs. current files is particularly ingenious. I tried to do something similar with "find", but apparently some root kits actually mess with files date stamps, making things more difficult. All in all, good stuff to know.

really quite good for security

[ant_morgan]

things like this are really positive for all our high security OS's out there. Hacking/Cracking is an extremely positive activity when put to actally helping me secure my box.

Conclusions about operating systems generaly wrong

[Zeinfeld]

One of the depressing end results of these projects is that they tend to come down to people making staements like 'we proved system x to be better than y' as if this was a soap powder comparison test.

Unfortunately in the real world security is much trickier. Simply installing system X does not necessarily mean you get the better security. Configuration is everything.

Quite often it comes down to did the guy who installed the O/S know what they were doing. More often it comes down to did the person comming after him screw it up?

Windows NT can be reliably and securely configured, however you really have to watch out for keeping up with the latest Microsoft patches.

Unix can be reliably secured, particularly if you don't install sendmail which was the root of 30% of CERT reports a few years back.

Unfortunately no mainstream O/S ships designed to be secure out of the box, and those that do tend to be military O/S which are practically unusable.

Here comes the catch with UNIX security, to secure a UNIX system I take off every package and every service that I don't absolutely need. I'm not talking about removing finger from the inetd, I am talking about removing the binary for finger, ftp, rlogin, telnet and every other executable file that is not critical for the system to run - if possible including X-Windows and emacs.

Now the result is secure but by the time I am finished the 'UNIX' I have left has no resemblance to a machine most folk would want to use. If you put back the executables I have taken out then you are back to roughly the same degree of exposure as Windows NT.

Another problem is that 'security' standards for operating systems are all pre-net. Even the common criteria which were meant to be the latest and greatest appear to be written by someone who thinks that the problem is preventing access conflicts on multi-user machines. Unfortunately while that is an interesting problem it has nothing to do with todays problems of securing networks. Is a server in a client/server configuration a single or a multi user machine?

More interesting than the statistics for which machines got hacked first would be the description of the attack strategies employed.

What I would like to see is a return to the type of security we used to have in operating systems like VMS where processes could be given specific privillege levels. I would like to prohibit the process displaying my email from doing anything other than drawing to the display visual - no taking a look at my address book, no sending off emails to anyone else.

Lessons learned

[CyberDawg]

Is the real lesson here how to increase security on your Linux box, or how to perform forensic analysis after a crack attack, or why you should/shouldn't pick Linux? No. None of the above.

I have long contended that the applicable formula is
convenience = 1 / security

The safer you want your system to be, the less convenient it will be to use the system.

If you have a computer for fun and entertainment, you don't want to make keeping it secure a full-time job (unless, of course, that's your idea of fun). Take some reasonable precautions, keep good backups, don't tempt fate, and get on with life. If you get hacked, deal with it.

If you have a mission-critical system in a business environment, then hire a professional sysadmin to keep it secure. This is not a do-it-yourself job, whether you're using M$, Linux, Solaris, MacOS, OS/2, or BSD.

Crime or a Job?

[Regolith]

What you describe would indeed be considered a criminal act, however it still amuses me that thousands of businesses pay security consultants through the nose for that same service when they could just wait for a burglar to break in and leave them a note detailing how they did it for free.

God bless our current model of capitalism (or is it consumerism?) which allows people to either make tons of money or be sent to jail for a number of years for commiting exactly the same act.

Tricky stuff

[nate1138]

Forensics on a compromised machine is really tricky. So many things to look for. I suppose this just proves the best defense for your machines is to be ever-vigilant, bordering on paranoid :-)