Slashdot Mirror


Forensics On a Cracked Linux Server

This blog entry is the step-by-step process that one administrator followed to figure out what was going on with a cracked Linux server. It's quite interesting to me, since I have had the exact same problem (a misbehaving ls -h command) on a development server quite a while back. As it turns out, my server was cracked, maybe with the same tool, and this analysis is much more thorough than the one I was able to do at the time. If you've ever wondered how to diagnose a Linux server that has been hijacked, this short article is a good starting point.

219 comments

  1. Story is FUD from a M$ shill by Anonymous Coward · · Score: 2, Funny

    A Cracked Linux Server? Ha! He should live so long!

    1. Re:Story is FUD from a M$ shill by Anonymous Coward · · Score: 0, Funny

      Cracked Linux server? Oh Noes, that's unpossible! Teh Lunix is UNBREAKABLE!

    2. Re:Story is FUD from a M$ shill by FST777 · · Score: 0, Offtopic

      Larry, is that you?

      --
      Free beer is never free as in speech. Free speech is always free as in beer.
    3. Re:Story is FUD from a M$ shill by Anonymous Coward · · Score: 0

      They are 40 days that they had consumed CPU cycles "doing nothing"!!!

      $ time { cat /dev/zero > /dev/null ; }
      Ctrl-C

      real 40d21h28m40.726s
      user 4m5.648s
      sys 40d21h24m21.325s
      $

  2. Yeah obvious FUD article by Anonymous Coward · · Score: 5, Funny

    Why Slashdot would such obvious anti-Linux FUD is beyond me. Maybe the M$ advertising dollars are turning their heads.

    The bottom line is that a LINUX SERVER CAN'T BE CRACKED.

    Maybe this admin got his login info phished by Nigerian scammers, I don't know. The guy probably is wondering why his Ebay account has a bunch of negative feedback and his MySpace is all jacked up and hasn't put 2 and 2 together with that time he responsed to that clever email asking for the triple whammy of MySpace/Ebay/root on your servers so that you could clear the money transfer.

    That or he didn't have his updates turned on and had an outdated BIND. And its not like BIND means Linux is unsecure.

    Even not that the idea that Linux is crackable is laughable and not worht front page at digg let alone slashdot. You don;t see Technorait or Bruce Perens' site posting garbage like this ever so why slashdot editors can't see thru it i dont kno.

    1. Re:Yeah obvious FUD article by Inakizombie · · Score: 0, Redundant

      Break out the BBQ! Its flame bait!

    2. Re:Yeah obvious FUD article by PPH · · Score: 1, Insightful

      The bottom line is that a LINUX SERVER CAN'T BE CRACKED.
      Its not impossible. There are admins dim enough to configure a system so as to be crackable. Its not like a Windows system. It takes work, but idiocy knows no bounds.

      Replace 'LINUX' with another version of Unix (the name of which will be withheld to protect the innocent). Some years ago, I ventured out onto the shop floor where I worked and encountered a terminal logged on to a critical production server. Nobody responsible seemed to be around. Typing 'whoami' returned 'root'. I promptly called the IT department's computing security group, informed them of the problem and hung around to see who showed up. After about 15 minutes with neither the original user or IT security appearing, I just logged the system off and left.

      Who knows what damage could have been done to that system before I arrived?

      --
      Have gnu, will travel.
    3. Re:Yeah obvious FUD article by JeremyGNJ · · Score: 3, Insightful

      Great attitude to have. It's like saying "no one can pick my front door lock". Vulnerabilities are found all the time, and just because they are found and patched, doesn;t mean that someone couldn't have exploited them before that point.

      Don't be blinded by your religion.

    4. Re:Yeah obvious FUD article by ATMD · · Score: 5, Funny

      *whoosh*

      --
      Nobody else has this sig.
    5. Re:Yeah obvious FUD article by Inakizombie · · Score: 1

      Don't be blinded by your religion. <3

    6. Re:Yeah obvious FUD article by DrSkwid · · Score: 2, Interesting

      I had a co-lo rental from Pipex. Linux 2.2. They noticed it was broken in to, cut us off, charged us to re-image the box on which they had left a tar of the drive. OK sounds fair enough, but they re-imaged it with EXACTLY the same Linux 2.2 install and it was infiltrated again by the time I got the email telling me it was back on. I fixed it by hand and never told them lest they charge the company again. Happily I quite soon after.

      --
      There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
    7. Re:Yeah obvious FUD article by Anonymous Coward · · Score: 5, Funny

      . o <- Joke

      ..O <- You
      ./|\
      ./ \

    8. Re:Yeah obvious FUD article by lawpoop · · Score: 1
      --
      Computers are useless. They can only give you answers.
      -- Pablo Picasso
    9. Re:Yeah obvious FUD article by Anonymous Coward · · Score: 0

      Why Slashdot tolerates ignorant responses like this is beyond me.

      Any software, as long as it's connected to a network or otherwise accessible by a malicious third party, is open to attacks. It's quite difficult to make something 100% bug free, uncrackable piece of software unless it's the "Hello World" program that you wrote in some C for Beginners class. Actually, even with something so simple there can be errors that can be exploited. The cost of reaching that 100% bug free state becomes prohibitively high with any piece of software that's relatively complex.

      The bottom line is that any piece of software with any degree of complexity *CAN* and probably does have bugs that can be exploited. If Linux were truly bug free, then by definition things like patches would not exist for Linux.

      What's really laughable is that you truly believe that turning on updates makes you impervious to exploits.

    10. Re:Yeah obvious FUD article by suggsjc · · Score: 0, Redundant

      ASCII art is lame If you really want to blast them Then try a haiku

      --
      When I have a kid, I want to put him in one of those strollers for twins and then run around the mall looking frantic.
    11. Re:Yeah obvious FUD article by Anonymous Coward · · Score: 0

      Sorry, I don't get it...

    12. Re:Yeah obvious FUD article by suggsjc · · Score: 5, Funny

      Dang HTML Formatted default, forgot the <br>'s

      ASCII art is lame
      If you really want to blast them
      Then try a haiku

      So in my rage, I wrote this (and used the code layout):
      Today I posted
      Today I looked like an ass
      It is Friday, beer

      --
      When I have a kid, I want to put him in one of those strollers for twins and then run around the mall looking frantic.
    13. Re:Yeah obvious FUD article by 32771 · · Score: 1

      I actually had to look up 'ls -h'. It supposedly prints out file sizes in a human readable format (man ls says "1K 234M 2G"). I thought that mere humans aren't granted access to UNIX machines.

      I wonder what the crack did to ls. It probably printed smilies instead of 'K', 'M' or 'G', i.e. big smilies for G and a scowl for no extra letter because of the low efficiency or what not.

      Another option could have been ls output in rosy warm colors - eek ...

      --
      Je me souviens.
    14. Re:Yeah obvious FUD article by Anonymous Coward · · Score: 0

      Do you think if you took that stick out of your ass then you might be able to smell flowers and fresh cut grass, hear laughing happy children, pet a kitten, and get the joke that everyone else is enjoying?

    15. Re:Yeah obvious FUD article by stephanruby · · Score: 1

      Forgetting to log out happens all the time. In College, anyone who forgot to log out from our unix student lab would find the next day that they had sent obscene emails and love poems to a few select faculty members.

    16. Re:Yeah obvious FUD article by rattis · · Score: 1

      You don;t see Technorait or Bruce Perens' site posting garbage like this ever so why slashdot editors can't see thru it i dont kno.
      Maybe not, but I did see this on Bruce Schneier's Blog last week or so.

    17. Re:Yeah obvious FUD article by Antique+Geekmeister · · Score: 1

      I can't access the article right now: please tell me it was cracked because a script kiddie read is Subversion passwords from an NFS home directory or an passphrase-free SSH key. Please?

    18. Re:Yeah obvious FUD article by pionzypher · · Score: 1

      It replaced ls with a backdoored version. One in which -h did not function or was not present.

      --
      I'll believe in corporations having personhood when Texas executes one... - advocate_one
    19. Re:Yeah obvious FUD article by UncleTogie · · Score: 1

      Well said. It's like two teams of kids at Easter, one team wearing black hats, the other white. Let the kids loose in a field full of Easter eggs. Most people like the white-hat team better, as they don't pick up their eggs and throw them at other people. However, that doesn't mean the less-liked team will find fewer eggs, nor refrain from throwing them. I'll have to quote Scotty again:

      "The more they overthink the plumbing, the easier it is to stop up the drain..."

      As for the poster's belief that any physical/electronic security is idiot-proof, I'll remind 'em of the next maxim:

      "Build an idiot-proof machine, and the universe will build a better idiot."

      --
      Don't tell me to get a life. I'm a gamer; I have LOTS of lives!
    20. Re:Yeah obvious FUD article by It'sYerMam · · Score: 1

      I read this comment.
      I am clearly no poet;
      it seemed rather odd.

      --
      im in ur .sig, writin ur memes.
    21. Re:Yeah obvious FUD article by mackyrae · · Score: 1

      No logs, so no way to tell, but during the break-in they ssh'd into another box as the ftp user, so if this server had an ftp user set up, it's possible they did the same thing, trolling for open ftps.

      --
      look! it's a bird, it's a plane, it's....a girl? yes, a girl browsing Slashdot on Linux
    22. Re:Yeah obvious FUD article by doktorjayd · · Score: 1

      yep,

      and it looks like the same set of backdoors that i (unhappily) came across about 8 yrs ago now.

      the backdoor'ed ls / ps / top wouldnt report the presence of the nefarious files/programs running, so you had to dig a little deeper to work out what was going on.

      i think i worked out ls was borked, then copied a 'good' ls from another box to ~/bin/ls , then used that to work out what files had been changed/moved/backdoored.

      the .bash_history in the OP seems pretty familiar, getting in, grabbing files from another (compromised) box, then setting up & not _quite_ covering up their tracks, at least giving me an in to work out what was goin on.

      iirc, they used '...' as the directory name from /, so even with the good ls, i think i missed it the first couple of times!

      this was a long time ago ( 2.2 kernel ? ), and was on my first real job. in the end i think the root cause was either a brute force ssh attack, or one of the various buffer overflows in the older versions of bind.

      either way, it was format, reinstall time!

    23. Re:Yeah obvious FUD article by sigaar · · Score: 1

      Because Bruce Perens:

      1. is a good enough sysadmin to secure his box
      2. doesn't need "his friend" to check out his box if it doesn't do what it's supposed to.

      I work for a hosting company with several thousand servers (client's hire a real or virtual server and have root access) and based on what I see on a dialy basis, I can tell you the following:

      1. Linux servers do get compromised.
      2. The vast majority of compromises on Linux servers are application level.
      3. Root compromises tend to be cause by:
                a) disgruntled former employees (we can track your IP, don't you know?)
                b) inexperienced sys admins doing stupid things like enabling root login via ssh, not restricting access to their own IPs, and dumb passwords like 'password' or 'penis' or '123456' or blank (who would think of just pressing enter?).
      4. The majority of application compromises are merely spammers figuring out ways to send spam via your box - usually by brute forcing smtp passwords or backscatter (thanks to qmail's dumb defaults)
      5. What's left of application level compromises (I'd guestimate about 20%) are website code exploits (yes, people still insist on turning register_globals on and safe_mode off and then wonder why they're hosting a PayPal phishing site)
      6. Windows servers get compromised far more often.
      7. Windows servers more often than not get root compromised. Not sure of the detailed reasons since I don't work with them.

      --
      sigaar
    24. Re:Yeah obvious FUD article by bekenone · · Score: 0

      Its unfortunate that Non-Linux Users associate Linux as A complete system (its not).
      If you mean KERNEL yes your right...(who want to do that anyways?).
      but Distro wise?
      yes THEY CAN be cracked, anything can be cracked.
      UNIX wise, it depends on whats flawed on a "STOCK" install.
      I repeat... "STOCK" install.
      As well as wrong permissions on things that talk to things, that talk to things, that talk to things. LOL! :)
      and using what already exists on a system to help you do things (long live | !).
      flaws are the key.

  3. Forensics by DrDevil · · Score: 5, Insightful

    Where did the word forensics come from? This is the completely wrong approach if working forensically. Can slashdot please use not use sensational titles! "Analysis of a cracked box" maybe more appropriate.

    1. Re:Forensics by Leftist+Troll · · Score: 1

      Troll? I hardly see how looking at your bash history qualifies as "forensics".

      Don't get me wrong, the article is somewhat interesting. It's just not an accurate headline.

    2. Re:Forensics by extrasupermario · · Score: 2, Insightful

      For those that do not knowingly experience 'cracked' linux boxes (re: not knowing everything to look for), articles like this are a great way to learn from others. Kudos to 'lars' for sharing his findings with the world and reminding us all that security is an evolving process.

    3. Re:Forensics by eln · · Score: 4, Insightful

      This article is somewhat helpful as it does show one way to catch crackers, although he goes about it somewhat clumsily (an "ls" command that doesn't accept a flag you know to be valid, especially when that flag has been aliased on your own shell for months, should instantly tell you you have a cracked box) and the method by which he finds out where the rootkit is is due to a mistake that most non-moron crackers would not make (neglecting to remove the .bash_history file).

      It's unfortunate that this cracker made such an elementary mistake, it would have been interesting to see more advanced techniques in detecting rootkits. However, his analysis of the rootkit itself does provide some good information as to what a typical rootkit will generally do (replace core binaries, hide itself, use innocuous-looking names, etc).

    4. Re:Forensics by Anonymous Coward · · Score: 5, Funny

      On the one server I have backdoor access to .bash_history is symbolically linked to /dev/random

      It makes for an interesting read :)

      Anonymous in case the admin actually reads slashdot.

    5. Re:Forensics by SIIHP · · Score: 0, Redundant

      "Can slashdot please use not use sensational titles!"

      ??? ...

      BWAAAAHAHAHAHHAHAHAHAHAHAAHAHAHAHHAHAHAHAHAHAHAHAH AHAHHAHHAHAHAHA (cough cough).

      Thank You. Really, that was awesome.

      --
      I only go to buffets for the unlimited soft serve.
    6. Re:Forensics by Anonymous Coward · · Score: 1, Interesting

      [~]:apache$
      lrwxr-xr-x 1 root wheel 11 Dec 20 2006 .bash_history -> /dev/random

      I'm pretty sure it is. I didn't use any crazy exploits or anything. It's an old computer that I once had access too when I was in school. It's just a lesser used machine and all I use it for is bit torrent (on a .edu).

      I created a few users such as "apache" and "sendmail". I'm not claiming to be a haxor by any means, and I just use it, like I said, for bit torrent.

      'apache's root directory is actually a mounted DMG file that I have mounted to /tmp.

      With OSX it's pretty easy.
      Create DMG: /usr/bin/hdiutil create -size 1t -fs HFS+ -type SPARSE -encryption -stdinpass -volname objc_sharing_ppc_23 data

      Attach DMG: /usr/bin/hdiutil attach -readwrite -private -mountroot /tmp -nobrowse -stdinpass "/Library/Application Support/LiveType/LiveFonts/Pro Series/Script.ltlf/data.sparseimage"

      Detach DMG: /usr/bin/hdiutil detach /tmp/objc_sharing_ppc_23 >> /dev/null

      128 bit encryption on that home directory. No one really questions large files in /Library/.

    7. Re:Forensics by SIIHP · · Score: 2, Funny

      What redundant? Did someone else tell him his post was hilariously funny?

      Are you too stupid to know what redundant means? I guess you are.

      Hey mod you're an dumbass.

      Wait, "dumbass moderator" see, THAT is redundant.

      --
      I only go to buffets for the unlimited soft serve.
    8. Re:Forensics by Antique+Geekmeister · · Score: 4, Funny

      Ohh. I thought you had accidentally copied a newbie-written Perl file to to .bash_history. That explains why it looked so coherent!

    9. Re:Forensics by jnelson4765 · · Score: 2, Interesting

      I've seen root's .bash_history symlinked to /dev/null used on a couple of incidents - at least the date of the symlink creation can be used to tell you exactly how long they've been there...

      --
      Why can't I mod "-1 Idiot"?
    10. Re:Forensics by maxwell+demon · · Score: 1

      lrwxr-xr-x 1 root wheel 11 Dec 20 2006 .bash_history -> /dev/random

      What happens to the entropy pool when bash writes its history into /dev/random?
      --
      The Tao of math: The numbers you can count are not the real numbers.
    11. Re:Forensics by Anonymous Coward · · Score: 0

      It gets XOR'd with what bash writes.

    12. Re:Forensics by Anonymous Coward · · Score: 0

      You don't have to symlink. A simple "export BASH_HISTORY=/dev/null" will do the trick.
      It won't save you from kernel exec logger though.

  4. How did he get access and On tools by morgan_greywolf · · Score: 5, Insightful

    And the most important question is, how did he get access in the first time? The server was running Ubuntu 6.06 LTS (i386) and was fairly updated. The compromised could be caused by:

            * An exploit unknown to the public.
            * A user accessing this server from an already compromised host. The attacker could then sniff the the password. It's a very good question, because if the guy was keeping his server up-to-date, then these two are the most likely scenarios.

    On tools...it's important to note that in forensics on a Linux box, your friends are ethereal (for watching packets on open connections), netstat (to see what's listening), and strace (shows you what UNIX API calls a running process makes, which gives you very good idea about what's going on.)

    Other tools: nmap may be useful for seeing what's going on with 62.101.251.166 and 83.18.74.235. The service detection options, in particular. Always do this on a sandboxed host. Something running in a VM might be useful in this regard.

    Anyway, nice article. This is almost exactly how I proceeded when one of my own servers was hacked a few years ago.
  5. cat .bash_history by Anonymous Coward · · Score: 0

    I though even script kiddies knew unset HISTFILE... hmm...

    1. Re:cat .bash_history by Anonymous Coward · · Score: 0

      If I bust your server, you'll see me upload, compile, install (to /sbin/sh) and exec my own shell.
      If you're lucky, you might find I the source behind.
      The shell is a working Bourne shell, so nothing interesting there. However, being a Bourne shell, there's no history file generated from there.

    2. Re:cat .bash_history by Anonymous Coward · · Score: 0

      You really need "ttyrpld". (Just hope your logging server does not get hacked too.)

  6. Looks as if there was another way... by sphealey · · Score: 3, Funny

    Looks as if there was another way to crash his server...

    sPh

  7. This is not forensics by Gandalf_the_Beardy · · Score: 5, Informative

    Forensics has to be useful in court. This is not - it's tainted evidence. Now if they took the original disk out, copied it with DD or similar to a file and mounted it as loopback and worked on that, then that's a first start to a forensic analysis.

    1. Re:This is not forensics by andreMA · · Score: 1, Insightful

      Uh, just because the term "forensics" is sometimes used in a limited sense in the legal sphere doesn't mean it can't be used in a more casual sense elsewhere. If he'd called it a "postmortem" would you be complaining that it wasn't performed by a licensed medical examiner?

    2. Re:This is not forensics by Gandalf_the_Beardy · · Score: 1

      No but postmortem is better - it's clearly not being dissected by scalpels. Just that computer forensics is a strict discipline that has a chain of custody and doesn't tamper evidence, and this would give the misleading impression that the actions carried out are OK. As a learning step by step article for finding out what happened and doing a post-mortem, then I think it is actually a pretty darn good article and one that I'll be saving to show to people who want to learn. Just not one that you would be able to use in criminal court.

    3. Re:This is not forensics by eln · · Score: 1

      How is postmortem better? Postmortem means "occurring after death," but this box is not dead by any means.

    4. Re:This is not forensics by Quarters · · Score: 2, Insightful

      "...sometimes used in a limited sense in the legal sphere..."

      The definition of the word forensics is, "The use of science and technology to investigate and establish facts in criminal or civil courts of law." The original poster's argument is correct. This was not forensics. It was an analysis.

    5. Re:This is not forensics by Gandalf_the_Beardy · · Score: 1

      From the article. "....most notably the web-server apache refused to start...." OK so it probably did other things besides serve web pages, but as a web server it's as dead as a dodo since the logfiles were hosed.

    6. Re:This is not forensics by Quill_28 · · Score: 1

      If they were using RAM as a partition wouldn't you lose all this data?

      Isn't there other data that could be lost if you shut the machine down?

    7. Re:This is not forensics by Xtifr · · Score: 1

      What dictionary are you using? M-W online gives me:

      1. The art or study of formal debate; argumentation.

      The 1913 PD Websters gives:

      "Belonging to courts of judicature or to public discussion and debate; used in legal proceedings, or in public discussions; argumentative; rhetorical; as, forensic eloquence or disputes."

      One of the paper dictionaries I have here says: "1. belonging to, used in, or suitable to courts of judicature or to public discussion and debate."

      (Emphasis mine.)

      By that, I think this became forensic the moment it was decided to post it to slashdot. :)

      (And yes, I found the source of your quote, but you cleverly failed to mention that what you quoted was the second definition, not the complete nor even the primary definition.)

  8. Not enough information by downix · · Score: 0, Redundant

    What was his setup? How did they access? And who had access?

    --
    Karma Whoring for Fun and Profit.
    1. Re:Not enough information by Stormx2 · · Score: 1
      From TFA:

      And the most important question is, how did he get access in the first time? The server was running Ubuntu 6.06 LTS (i386) and was fairly updated. The compromised could be caused by: * An exploit unknown to the public. * A user accessing this server from an already compromised host. The attacker could then sniff the the password.
  9. rkhunter anyone? by jshriverWVU · · Score: 3, Informative

    I have rkhunter on all of my machines, sends a nice email letting me know of any changes in system files.

    1. Re:rkhunter anyone? by kwalker · · Score: 0

      So does mine, along with an alert e-mail because it doesn't work with SELinux running most of the time, it doesn't like /dev/.udev, and it isn't updated for my newest distros. it's to the point that I'm about to uninstall it because of all the false positives.

      --
      ... And so it comes to this.
    2. Re:rkhunter anyone? by Mr2cents · · Score: 1

      But if your system is compromized, do you still trust rkhunter?

      --
      "It's too bad that stupidity isn't painful." - Anton LaVey
    3. Re:rkhunter anyone? by Anonymous Coward · · Score: 0

      "I have rkhunter on all of my machines, sends a nice email letting me know of any changes in system files."

      Can't you just set permissions so that nobody can change those files?

      After all, if someone has sufficient access to change the file without the file-owner's permission, then presumably they also have sufficient access to modify your email script?

    4. Re:rkhunter anyone? by Obsi · · Score: 0

      Root bypasses permissions. Otherwise, how'd you delete a file some noob chmod 000?

    5. Re:rkhunter anyone? by bhtooefr · · Score: 1

      Hmm, I forsee a tiered approach being useful, much like what Windows does.

      First, there's the administrator permission level, which is supposed to be the Windows equivalent to wheel (and Administrator being the Windows equivalent to root.)

      Then, there's LocalSystem, which has higher permissions than Administrator. (And, Administrator is necessarily slightly lower than *nix root.)

      Of course, there's still exploits against the LocalSystem account, but it's not as easy to get to as Administrator or an equivalent account.

    6. Re:rkhunter anyone? by Xabraxas · · Score: 1

      You're talking about Role Based Access Control. Linux has this too.

      --
      Time makes more converts than reason
  10. They got the webserver too by EvilRyry · · Score: 1

    Looks like the server is down for some forensic analysis following a break-in as well. Too bad. Wonder how they are going to do the analysis on the server without TFA?

    1. Re:They got the webserver too by Gandalf_the_Beardy · · Score: 1

      Let's hope mine doesn't need some analysis shortly. http://www.chris-street.demon.co.uk/article.php.ht m

  11. Taking Bets... by sanimalp · · Score: 1

    Ill bet his root password was "password"... oh, wait, "password1" is the new norm now..

    1. Re:Taking Bets... by nschubach · · Score: 1

      Actually, this month I believe it's "password08"

      --
      Every time I start to have faith in humanity, I ruin it by driving to work between 7 and 8 am.
    2. Re:Taking Bets... by Anonymous Coward · · Score: 0

      Always use a capital letter, its more secure. Password1

  12. Can't read TFA by PPH · · Score: 1
    I guess the rootkit on my system prevents me from reading any articles on how to detect and clean up rookits....


    Time to put my tinfoil hat back on.

    --
    Have gnu, will travel.
  13. Hey by Joseph1337 · · Score: 0

    You`re SURE that he wasn`t running Windows?

  14. I had to do this once. by Anonymous Coward · · Score: 4, Funny

    We had a cracked linux server at work one time and I took it upon myself to find out who did it. Long story short: some server monkey decided it would be a fun idea to ride his bike around inside the data center and smashed into one of the racks.

    1. Re:I had to do this once. by CompMD · · Score: 2, Funny

      im in ur datacenter breakin ur racks

    2. Re:I had to do this once. by manifoldronin · · Score: 1

      That means some human being had some cracked _ribs_ there, you insensitive clod!

      --
      Tyranny isn't the worst enemy of a democracy. Cynicism is.
    3. Re:I had to do this once. by PayPaI · · Score: 1

      Do you work for 365 Main by any chance?

  15. Mirror by W2k · · Score: 3, Informative
    --
    Quality, performance, value; you get only two, and you don't always get to pick.
  16. Further discussion... by meringuoid · · Score: 5, Informative

    Bruce Schneier posted this a few days back. Consensus is that it's not that good an analysis, but that the attacker was even worse. Some discussion also of whether it is better to take the machine offline immediately (and risk alerting the attacker that he has been rumbled) or to begin your analysis with the machine still live and operational. I for one side with the 'shut that thing down NOW' faction.

    --
    Real Daleks don't climb stairs - they level the building.
    1. Re:Further discussion... by morgan_greywolf · · Score: 1

      It's not a great analysis, no. In my case, I was actually able to find the hole (an unpatched BIND with a known exploit -- ouch! That'll teach me to keep my patches up to date!), the attackers IP address, (which was not easy to find. I had to sleuth around a bit a contact a few sysadmins before I traced him down to a cybercafe in China) but it's a good start.

      Taking tha machine offline immediately -- bleh. It depends on the box. In my case, my box was nothing more than a old machine being used as a firewall. He was never successful in getting through to my boxes behind the firewall, he tried...but something he saw must've spooked him or made him disintrested, because he stopped looking and just left the box open as a zombie. My guess is that's all he was after was a zombie anyway. So shutting down the box would have saved me exactly what?

    2. Re:Further discussion... by Anonymous Coward · · Score: 0

      . So shutting down the box would have saved me exactly what?

      His zombie spending the time spamming anyone and everyone while your network gets added to every spam list, even those that are next to impossible to get off of.

    3. Re:Further discussion... by Andy+Dodd · · Score: 4, Interesting

      On the other hand, shutting down the box ASAP makes it much harder to find the guy.

      For example, one of Vodafone Greece's first reactions to finding that some of their switching systems had been rootkitted was to remove the offending software. This removal was one of the main contributing factors to the authorities having no chance to ever find the group that had compromised the system, that along with a couple of other screwups led to Vodafone getting fined a pretty hefty sum.

      http://en.wikipedia.org/wiki/Greek_telephone_tappi ng_case_2004-2005

      IEEE Spectrum had a recent article that had MUCH better information than Wikipedia though, I don't have it with me at the moment unfortunately.

      --
      retrorocket.o not found, launch anyway?
    4. Re:Further discussion... by morgan_greywolf · · Score: 1

      Bleh. I had already blocked off the mail port with IPtables. It was the fact that it was sending thousands of mails an hour that I noticed it was a problem at all.

    5. Re:Further discussion... by Ecuador · · Score: 1

      Actually there was one lead. They knew which 3 cellphone antennas the wiretappers were mostly using for their phones, so they had a triangle containing their location. That "triangle" (quite unsurprisingly IMHO) didn't contain much of importance other than a certain well-known embassy.
      The fact that the programs AND LOGS were purged, was either clear ineptitude and idiocy (a quite likely scenario), or deliberate (the conspiracy theory scenario) - which can't be ruled out as the breach seemed like an insider job.

      --
      Violence is the last refuge of the incompetent. Polar Scope Align for iOS
    6. Re:Further discussion... by twitter · · Score: 1

      IEEE Spectrum had a recent article that had MUCH better information than Wikipedia

      If you think there's some actual information missing, please add it. Please don't add the Spectrum's BS about how skilled the attackers were and how rare the event.

      --

      Friends don't help friends install M$ junk.

    7. Re:Further discussion... by discord5 · · Score: 3, Informative

      IEEE Spectrum had a recent article that had MUCH better information than Wikipedia though, I don't have it with me at the moment unfortunately.

      http://www.spectrum.ieee.org/jul07/5280 for those interested.

    8. Re:Further discussion... by Bearhouse · · Score: 1

      Oh, the horns of dilemma

      Do I shut down & clean, thereby protecting my clients, data, shareholders and above-all my ass?

      Or do I play amateur cop / responsable citizen (depends on your point of view), and try and sniff and smoke the bastards out?

      Tough call.

      Having said that, some of my clients are massive multinationals, (like Vodafone), and they seem more preoccupied with cutting costs than taking this kind of threat seriously. Whilst a local entity - to take your example, Greece - could not necessarily justify investing in a proper defensive & forensic team, such a team would be trivial in terms of cost on a global level. It would probably be efficient too, since such attacks are - for the moment - relatively rare.

    9. Re:Further discussion... by dbIII · · Score: 1

      whether it is better to take the machine offline immediately (and risk alerting the attacker that he has been rumbled)

      Preaching to the converted here - but who cares what the attacker thinks? There is little or no chance of tracking down the origin in a hurry if they have a clue and little chance to prosecute if you do. From what I've read from a legal perspective it's better to pull the plug, pull out the drive and make sure that anything you do is unquestionally readonly in front of witnesses anyway. It's likely that what you can work out from the compromised machine off the network (and booted from something you can trust) is all the information you are going to get. There is the possiblity that they might go after something else nearby but it would usually be easier for them to do damage or work out information from the compromised host.

      I had an idiot that took a reasonably secure host, gave all the email users shell access, put a compiler on the thing, screwed up permissions (I think /usr, /lib and /var were world writable top to bottom) then opened up ssh to accept connections from any ip address. All it took was one user with a common name and the password of "coffee" and a dictionary attack got a script kiddie in. In that case "grep" had been replaced with something that segfaulted with common options so a spam monitoring script set off the alarm. The script kiddie changed the file attributes on a lot of their stuff to make it difficult for newbies to remove them and easy for anyone else to find them. Obviously there may have been other stuff there too - so it's a case of putting the disk in a bag with a warning label to look at if I'm really bored, a good excuse to get everyone to change their email passwords, to set up a decent system with options that were unavailable previously and get management to put limits on the changes unskilled staff could do to it.

  17. Meta-cracking by CopaceticOpus · · Score: 5, Funny

    Oh, I see, it's a clever DOS attack:

    1. Infect Linux server of some guy with a blog.
    2. Guy blogs about how he dealt with said infection.
    3. Blog posting gets linked to on Slashdot.
    4. Millions of computers attempt to access the blog, hence bringing down the server.

    Don't you see? We've a socially engineered botnet!

    (And please, for the love of all that is sacred and funny, don't reply to this and add steps for "???" and "Profit". It's just tired and completely not funny. And the clever little variation on that theme you're thinking about posting right now isn't funny either.)

    1. Re:Meta-cracking by Anonymous Coward · · Score: 1, Funny

      5. ???
      6. Profit!

      (oh, come on, you asked for it)

    2. Re:Meta-cracking by Anonymous Coward · · Score: 5, Funny

      1. Find clever little variation that is funny
      2. ????
      3. Profit!

    3. Re:Meta-cracking by slightcrazed · · Score: 0

      4. Prophet? 5. ????? 6. Profit!

    4. Re:Meta-cracking by Anonymous Coward · · Score: 0

      1. Post a meta-joke about "???"/"Profit" jokes on Slashdot.
      2. ???
      3. Karma!

  18. Wish I would have known... by DoctorPepper · · Score: 1

    I got hacked back in February - March 2001 time-frame. I made the mistake of setting up my Linux server as a router, and left my Samba and NFS shares active. This kind of info would have really helped me then.

    --

    No matter where you go... there you are.
    1. Re:Wish I would have known... by Anonymous Coward · · Score: 0

      yes, and maybe you can read the manpage for "netstat -an | grep LISTEN" to see what other services are running

    2. Re:Wish I would have known... by Anonymous Coward · · Score: 0

      I got in a car accident, it was in the junteenth through roctober time frame.

      I was in a bad car crash, but forgot to wear underwear that day.

      This kind of info would have really helped me back then.

  19. Casual approach to forensics by Vario · · Score: 1

    Before everybody complains how he could have done the analysis much better I think it reflects quite well the approach a lot of people would use here. If my friend would ask me about a failing apache server my first reaction would not to dd the whole system.

    Unfortunately the article is a little low on details about the running configuration. Ubuntu 6.06 seems like a solid distribution security wise, so where all current patches installed, was there a weak root password? Was root ssh login enabled?

    It is quite lucky that the attacker was not really experienced and more or less just used the scripts he downloaded somewhere without knowing exactly what they were doing. Otherwise without anything like tripwire this might have gone unnoticed for quite some time.

    1. Re:Casual approach to forensics by mr_mischief · · Score: 1

      Okay, I've not yet RTFA. Did it specifically say, "bog standard Ubuntu 6.06 with absolutely no additional software and only bare necessary configuration changes needed for system differentiation purposes"?

      I ask because everyone seems to be looking very closely at the initial OS distro, and almost any server that's been put into useful production has been tweaked in some way from the official packages. Stuff gets compiled from source. Custom stuff gets coded. Packages get installed out of third-party repositories or straight from vendor sites. Daemon configurations get changed, firewall rules may be tweaked, and additional modules for existing server daemons get added.

      Hell, they could have done something as stupid as allowing root logins through unencrypted telnet, then actually using that "feature".

      Oh, well, off to RTFA to see if it contains any of the answers.

    2. Re:Casual approach to forensics by mr_mischief · · Score: 1

      Okay, I read a cache of the article. It doesn't answer any questions about what might not be stock Ubuntu 6.06, but simply assumes that uname/motd says it all.

      Another option for a perfectly secure box that wasn't mentioned in TFA is that the friend could have run a Trojan that opened the initial hole.

  20. *Bourne* Shell? by Spy+der+Mann · · Score: 4, Funny

    The shell is a working Bourne shell

    I knew it! Jason Bourne was involved in this!

    1. Re:*Bourne* Shell? by nschubach · · Score: 1

      If Jason Bourne were involved in this he'd be standing right next to you...

      --
      Every time I start to have faith in humanity, I ruin it by driving to work between 7 and 8 am.
    2. Re:*Bourne* Shell? by Anonymous Coward · · Score: 0

      Was this a very poor attempt at making a joke, or are you seriously ignorant of the most popular shell ever?

    3. Re:*Bourne* Shell? by el+americano · · Score: 1

        o <-Joke

      . O <-You  O <- Jason Bourne
      ./|\      /|\
      ./ \      / \

      Whoosh! ... bang!

      --
      Those are my principles. If you don't like them I have others. -Groucho Marx
  21. All the log files will have been changed to: by Anonymous Coward · · Score: 0

    j00'v3 b33n PWN3D! I 4M 3r337.

    1. Re:All the log files will have been changed to: by Anonymous Coward · · Score: 0

      "3r337"? So, "hacked by chinese?"

  22. Re:How did he get access and On tools by eln · · Score: 4, Funny

    I think it's probably the fact that the owner of this system had the root password set to "GOD" as all good sysadmins do. The hacker's extensive experience hacking the Gibson made getting into this system a cakewalk.

    Clearly, we as sysadmins should rethink the long-standing policy of setting all root passwords to either love, secret, sex, or god. Perhaps we should at least add another password to the list, like "unhackable" or something truly secure like that.

  23. Re:Heh heh heh heh hehe hehehe eheheheheheehehehe by Anonymous Coward · · Score: 0

    You still don't know I 0wnz0red you teh hole t1m3?

    Hahahaa, stupid kid. Check your log files @ /dev/null

    God I feel smug _now_. But then again, I AM a Lunix user.

    > Go back to play with your nintari. Press A to start and B to stop, you knows...

  24. Re:How did he get access and On tools by jimicus · · Score: 1

    There's a few things which immediately spring to mind:

    1. We already know that it was meant to be running Apache. Perhaps there was some PHP application which wasn't very secure? Even so, if that were the case then the exploit they used must have been fairly convoluted because it probably wouldn't have got them root access immediately.

    2. We don't know what other services were supposed to be running, how/if they were firewalled and secured. SSH, for instance, is only as secure as the weakest password on the box - for best results you probably want to combine it with minimising the number of shell accounts, only allowing root access through private/public keys and using denyhosts (or similar) to automatically block bruteforce dictionary attacks.

    3. We don't know how secure the desktop PC which was used to administer this box is. There is an awful lot of Windows-based malware out there - it wouldn't surprise me if there's more than one piece which looks around for when you start a connection to a host on port 22, enables a keylogger and sends the results back.

  25. Re:How did he get access and On tools by Anonymous Coward · · Score: 0

    On tools...it's important to note that in forensics on a Linux box, your friends are ethereal (for watching packets on open connections), netstat (to see what's listening), and strace (shows you what UNIX API calls a running process makes, which gives you very good idea about what's going on.) Yes, I think if you know what all this means, your friends are definitely ethereal.
  26. Likely dictionary ssh attack on a random user by Anonymous Coward · · Score: 0

    Linux has many local exploits and very little effort to fix them sadly. If you have 'regular' users, you need to keep cracklib hooked in to all password change methods and try to use john the ripper often on password files. OpenBSD has a likely safer userland, but in general local root exploits happen on many services that are necessary on a server. The best thing would be to have an OpenBSD jump server with no extra services/tools for users that can easily be monitored/rebuilt. Even better, expose daemon servers (apache,etc) by NFS only if users must touch their actual filesystems.

    1. Re:Likely dictionary ssh attack on a random user by morgan_greywolf · · Score: 1

      Huh? If you're running a server with 'regular' users, and you're using even remotely dictionary-based passwords, you deserve to get hacked.

    2. Re:Likely dictionary ssh attack on a random user by sumdumass · · Score: 1

      I dunno. what about automatic account lockout after 5 unsuccessful tries and stuff like that?

      There are some things that can be done to prevent dictionary attacks from working. Or at least from working enough that they would succeed. there used to be an email program you could run (monkey business or something like that.) that would just return hits on anything tried other then the actual accounts. It was designed to make harvesting addresses as useless as mailing to the dictionary itself. I don't see why something like that couldn't be implemented. If you crack a password to a random account it could give you a no login allowed or something like that. I mean if you look though your logs and see 5 ip addresses hitting rejected logins 5000 times on one or two days times, it is pretty safe to assume you can bann the IP at the router or put it in a hosts deny list somewhere refusing access to the server for anything.

      I have a server that does exactly that. It looks though the log entries and after so many failed login attempts it emails me and I just block the ip at the router. I have been looking for a way to automatically block the router but I'm not using one with enough features I guess. All my other servers are private so there isn't much of a worry from the outside there.

    3. Re:Likely dictionary ssh attack on a random user by LordSnooty · · Score: 1

      Prevention of brute force ssh attacks? The answer is sshdfilter.

    4. Re:Likely dictionary ssh attack on a random user by sumdumass · · Score: 1

      I have heard of this in the past but couldn't remember any specifics about it. Thanks. This looks very good and promising.

  27. Still a good read. by Seakip18 · · Score: 1

    He was collecting a good bit of data there. If he pulled the drive out before doing that, he would of lost all volatile data, including possible info that hadn't been garbage collected. Granted, a dump of the RAM should have been his first command, since everything before it risks trampling de-referenced addresses.

    If your going for a court case, your better off with the mountain of information than just a sheet of what really matters.....unless your the RIAA, then you make accusations at dead grandmothers.

    How server was accessed in the first place is what I really want to know.

    --
    import system.cool.Sig;
  28. Raise your hand by tie_guy_matt · · Score: 5, Funny

    Raise your hand if you typed "ls -h" on your box just to make sure it still works right.

    1. Re:Raise your hand by Anonymous Coward · · Score: 5, Funny

      C:\>ls -h
      'ls' is not recognized as an internal or external command,
      operable program or batch file.


      Oh noes!

    2. Re:Raise your hand by nschubach · · Score: 1

      (while I get the joke) I wouldn't go anywhere without ls! Even Windows.

      --
      Every time I start to have faith in humanity, I ruin it by driving to work between 7 and 8 am.
    3. Re:Raise your hand by smellsofbikes · · Score: 1

      >Raise your hand if you typed "ls -h" on your box just to make sure it still works right.

      C:\Documents and Settings\user>ls -h
      'ls' is not recognized as an internal or external command, operable program or batch file.
      C:\Documents and Settings\user>

      Crap! I'm screwed! Someone hacked my system!

      --
      Nostalgia's not what it used to be.
    4. Re:Raise your hand by Antique+Geekmeister · · Score: 1

      You're right. You should find and punish the person or group who forced that software to be installed on your machine.

    5. Re:Raise your hand by bky1701 · · Score: 1

      Yep, looks like your Linux has been cracked. Someone sunk in and installed Windows over it. Bastards, why couldn't they just take credit card numbers?

    6. Re:Raise your hand by smellsofbikes · · Score: 1

      I was hoping someone would reply that I was indeed screwed, but it wasn't because of the intruder software -- you're close enough to get the prize.

      --
      Nostalgia's not what it used to be.
    7. Re:Raise your hand by jafac · · Score: 1

      ls.bat:
      dir

      --

      These are my friends, See how they glisten. See this one shine, how he smiles in the light.
    8. Re:Raise your hand by rickb928 · · Score: 1

      i hate you

      --
      deleting the extra space after periods so i can stay relevant, yeah.
    9. Re:Raise your hand by Anonymous Coward · · Score: 0

      @dir /d %1 %2 %3 %4 %5 at a bare minimum..

    10. Re:Raise your hand by dbIII · · Score: 1
      ls -h

      ls: illegal option -- h

      usage: ls -1RaAdCxmnlogrtucpFbqisfL [files]

    11. Re:Raise your hand by cdrom600 · · Score: 1

      I have the exact same file on my Windows box.

  29. Re:Ssshhh.... Secrets Revealed... by FuzzyDaddy · · Score: 1
    AC because I am a MS insider

    Suuuure you are.

    --
    It's not wasting time, I'm educating myself.
  30. Does rtkhunter... by Anonymous Coward · · Score: 4, Insightful

    Does rtkhunter send you a email when the cracker changes /usr/bin/rtkhunter so that it won't email you the attacker's changes?

    If you think that rtkhunter will protect you from a Linux kernel module rootkit your completely delusional. NOTHING will _reliably_ locate a LKM rootkit. That's the point of it.

    Think about it. Rtkhunter relies on the ability of the kernel to accurately indicate files sizes, file names, and running proccesses as well as a bunch of other little detail things that normal rootkit makers tend to get wrong. When that kernel is subverted and controlled by it's new owner to give rtkunter, as well as other processes (such as your bash shell) false information about the system then those things are completely worthless.

    It's the same as virus scanning on Linux (or any other system). Once the attacker gets root access then they have access to the kernel. Once they have access to the kernel they can use the kernel against you to hide what they are doing. Since userspace runs on top of the kernel then any sort of activity can be hidden by making the kernel lie to anything running in userspace.

    This includes logging daemons, rootkit detection software, administrators, virus detection, rpm checksums, or anything else that people use to give themselves a FALSE sense of security.

    There are two ways to reliably detect a rooted machine.

    The first way is to use a network-based Intrusion Detection System (IDS). One of the best ones is commercially supported open source application called Snort. These guys can be hooked up to networks in a passive and completely undetectable way and are used to monitor traffic. They will alert administrators to any unusual network activity.

    Network based IDS can be fooled, but as a administrator your at least operating on the same playing feild since your own software isn't used against you.

    The second, and more reliable way, is to use a checksum-style IDS. MD5deep, AIDE, or Tripwire are 3 very good examples of this.

    However how people use these things are completely worthless. If you keep the checksums and run the checksum software on the same machine as the one your trying to detect, then it's not good. Since they rely on the kernel any kernel-level rootkit can defeat them and the attacker can edit and substitute incorrect checksums.

    In order for stuff like AIDE to be usefull it needs to be ran from read-only media and from a different operating system then the one your checking. (for example booted up in a knoppix cdrom, or a removable disk in a dedicated unconnected-to-any-network 'Tripwire' machine)

    Both forms of IDS are very expensive and difficult to correctly use. Virtual machines make this stuff somewhat easier, but it's still much better to have dedicated machines for these things.

    rtkhunter is nice if it's job is to make you feel good. If it's job is to make sure your machine is secure then it's shit. (no offense to the rtkhunter authors, I am sure they understand it's role and effectiveness.. to bad their users don't tend to) It's only good for kiddies that don't know better and if your being owned by kiddies then you have bigger problems.

    1. Re:Does rtkhunter... by mutterc · · Score: 1

      There's an interesting third approach, used by Sysinternals's (now part of MS) RootkitRevealer for Windows.

      Basically, enumerate all the files on the system using the usual OS APIs. Then, scan the entire raw disk, and enumerate all the files on the system by manually interpreting the directory structures stored on-disk. Any files whose directory entries exist on-disk, but don't show up in the OS's API (with a few standard system exceptions) are being hidden from the OS API layer by a rootkit.

      It's certainly theoretically possible to fool, by having your rootkit hook the APIs used to read the raw disk, and returning innocuous values, but that's a good bit harder to do than the other stuff rootkits usually do. Some rootkits fooled it by not hiding their files if the process trying to look them up was named RootkitRevealer.exe, so the tool took to making a randomly-named copy of itself and executing that.

    2. Re:Does rtkhunter... by Anonymous Coward · · Score: 0

      That's why I use ossec - http://www.ossec.net/

      -It does:

      rootkit detection
      integrity checking
      log analysis

      Giving me a better view of what is going on....

  31. selinux? by burnin1965 · · Score: 4, Insightful

    Does Ubuntu install selinux and a policy in a default installation, or is it necessary to add it later?

    I've only performed one Ubuntu install and most of my experience is with Red Hat and Fedora linux distros. Fedora installs selinux with a targeted policy enforcing by default which I think is a good thing. I had an experimental Fedora web server with PHPbb installed which was comprimised via the PHPbb application but looking through the log files it appeared that selinux had thwarted attempts to root the box or setup a zombie to connect to an irc server.

    Other than the mistake of an outdated PHPbb application I also made the mistake of allowing execution of code in /tmp, lesson learned. But it was interesting to see selinux do its job and I'd be curious if it was utilized in this instance.

    1. Re:selinux? by Neil+Watson · · Score: 1

      I think SElinux is still a mixed bag when it comes to distribution support. My attempts at using SElinux with Debian have been disappointing. Red Hat AS4's SElinux works out of the box but, it is not enabled by default.

    2. Re:selinux? by OmegaBlac · · Score: 2, Informative

      Does Ubuntu install selinux and a policy in a default installation, or is it necessary to add it later?
      No, one must install it manually. Getting SELinux into a default installation for future release is being worked on though: https://wiki.ubuntu.com/SELinux?highlight=%20selin ux%20#2910857737223089520
    3. Re:selinux? by quanticle · · Score: 3, Informative

      Ubuntu, as of the latest version (Feisty Fawn), does not install SELinux. If you want that functionality, you'll have to install it yourself. I think this is because SELinux policies can be difficult for beginning users to navigate. Also, when SELinux thwarts execution of some file, there is often no explicit message stating that the file was blocked by SELinux, please change your configuration. In all too many cases, the user is left on their own to figure out why their file isn't executing.

      --
      We all know what to do, but we don't know how to get re-elected once we have done it
    4. Re:selinux? by TheAverageGuy · · Score: 1

      Is AppArmor similar to SELinux? I saw AppArmor installed by default on the latest "tribe" of Ubuntu's upcoming "gutsy" release.

    5. Re:selinux? by ShaunC · · Score: 1

      I think SElinux is still a mixed bag when it comes to distribution support.

      Just pick up the nearest telephone and state your problem; the issue will be patched in the next release.
      --
      Thanks to the War on Drugs, it's easier to buy meth than it is to buy cold medicine!
    6. Re:selinux? by pembo13 · · Score: 1

      I have multiple installations of CentOS5 with SELinux, and I have never experienced this.

      --
      "Thanks for all the money you paid to us. We've used it to buy off ISO among other things" -Microsoft
  32. Re:How did he get access and On tools by arivanov · · Score: 5, Informative

    All of these will help only if it is cracked by amateur sr1pt k1dd10tz like in this case. If it is cracked properly you will not see anything or spook off the intruder. He will either go underground or destroy the box with all of your data (not that you should try to use it as it may have been altered).

    I have seen a number of rootkits for Linux as far back as 97-98 which were considerably more advanced. It was a bit of an arms race between the admins (including me) and the guys who were breaking in. By the end the best rootkits could:

    1. Load a whole hidden fs with tools into a ramdisk or hidden area on the filesystem not visible using normal tools.
    2. Hide all sockets, processes and files belonging to the rootkit completely. You simply could no longer see them using netstat, ps and other similar tools.
    3. Monitor network driver state for the promisc flag and "scrub" backdoor traffic out of it so it is no longer visible using tcpdump and ethereal.
    4. Adjust memory totals and df so that you do not see them. This was also the only way we found to catch it. Try to allocate 95% of the remaining free memory and see the system oops magestically.
    5. Doctor logs so that you could not notice anything.
    6. The rootkit itself handled all connections via something that looked like ssh. I never managed to figure out how it loaded. One of the executables in the system loaded at startup was backdoored. Probably sendmail or one of the other daemons it could not do without.
    7. The rootkit managed to masq changed files completely. Tripwire and md5sums were reporting all OK while executables were being changed.

    That was a the tech level in 97. I would expect 10 years later a good rootkit to be even better. Looking at the blog post I can only laugh.

    If you suspect a system is cracked:

    1. Take it offline and take the disks out. Analyse the system completely offline looking at the disk from another system mounted as ro (on SCSI discs use the RO jumper). Never ever even try to start it. Nowdays knoppix is a great help. Most importantly - do not fsck systems before mounting as the rootkit may hide in orphaned areas which fsck will fix.

    2. If you are monitoring traffic, monitor it on a switch span port or create yourself a simple multiple interface box which serves as a firewalling bridge (so you can hijack the more interesting bits and alter them). Lex Book PCs are a good choice as they can run either Linux or BSD and are as portable as a laptop. A recent Via with 2 Ethernet ports is also a good choice as it can handle up to 1GB of traffic across as a bridge.

    --
    Baker's Law: Misery no longer loves company. Nowadays it insists on it
    http://www.sigsegv.cx/
  33. Sorry, nice try, no by SIIHP · · Score: 0, Flamebait

    "The definition of the word forensics is..."

    No, that's A definition. Here's another

    1 : an argumentative exercise

    OP was wrong, and so are you.

    --
    I only go to buffets for the unlimited soft serve.
  34. Re:Ssshhh.... Secrets Revealed... by dedazo · · Score: 5, Funny

    I am a MS insider

    The 220,000 or so members of the Slashdot Members Who Post Authoritative Statements On The Inner Workings Of Microsoft To Support Their Arguments warmly welcomes you to the club.

    --
    Web2.0: I love when people Flickr my cuil and digg my boingboing until my google is reddit and I start to yahoo
  35. Re:I had to do this ounce. by Anonymous Coward · · Score: 0

    Hey if it was a "racked" server to begin with, it was only one Letterman-gets-evil-off-his-meds move away from being "cracked" or "hacked".

  36. Ineffective rootkit by whoever57 · · Score: 1
    The "rootkit" does not seem to be very effective at hiding itself and the malware processes:

    These two processes show up using (our backdoored) "ps", so I guess that why the attacker renamed it to "smbd": root@server1:/var/.x/psotnic# ps axuw | grep smb root 3799 0.0 0.4 8592 2156 ? S 11:00 0:00 /usr/sbin/smbd -D
    In fact, the whole crack of the server seems to be pretty amateurish. Still, even if the analysis was not very good, it is interesting article.
    --
    The real "Libtards" are the Libertarians!
  37. That's it, I'm switching to Windows by Maltheus · · Score: 3, Funny

    Security is very important to me, I can't be screwing around with something that can be so easily cracked.

  38. twitter by Anonymous Coward · · Score: 0, Troll

    Oh please twitter, please post in this thread and tell us all how this is Microsoft's fault. I can't wait for your explanation for this one!

  39. Re:How did he get access and On tools by sootman · · Score: 5, Informative

    If you suspect a system is cracked:
    1. Take it offline and take the disks out.


    And I've been told don't use the 'shutodwn' command--instead, pull the power plug out of the wall. A rootkit could include a cleanup routine that gets run at shutdown time.

    --
    Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
  40. Cracked Linux.... by Wikkiwikki25 · · Score: 0

    Screw it I'm switching to windows.

  41. Re:How did he get access and On tools by arivanov · · Score: 2, Informative

    Correct. Always pull the plug out of the wall the moment you suspect that something is wrong. This is what I meant when I said - take it offline (my fault, should have written it better). If it is compromised the data on it is worthless anyway and you need to go back to backups so the loss of data from pulling the plug is trully in the "who cares" area.

    --
    Baker's Law: Misery no longer loves company. Nowadays it insists on it
    http://www.sigsegv.cx/
  42. Cracked spell checker? by LilGuy · · Score: 1

    Do not use if the seal has been broken.

    Questions? Call 1-800-no-spell

    --

    You're nothing; like me.
  43. 9 billion names by Anonymous Coward · · Score: 0

    Don't see any need to add another password quite yet. There's still 8,999,999,999 other names to choose from.

  44. Re:How did he get access and On tools by markxsd · · Score: 1

    >love, secret, sex, or god Perhaps we should at least add another password to the list, >like "unhackable" or something truly secure like that. or what about "password" nobody would guess that?

  45. Re:How did he get access and On tools by jargoone · · Score: 1

    A friend recently forced me to watch this movie. I can't believe I was depriving myself of it for all these years.

    Kernel who?

  46. The title is totally misleading by baggins2001 · · Score: 1

    Linux servers cannot be cracked. They may be borrowed every once in a while but not cracked.

    --
    He who said 1,000,000 monkeys on 1,000,000 typewriters would eventually type the great novel, never saw an AOL chat room
  47. Re:How did he get access and On tools by sootman · · Score: 1

    And disks have gotten very good in the last few years. I haven't seen any (immediate) data loss from hard power cycling/plug pulling in I don't know how long. A former co-worker used to turn her G5 off every day by pressing the front button. I saw her do this once and said (very nicely) "You know it's better to shut down from the menu, right?" and she answered "Yeah, I know you're not supposed to do that, but it's faster." She had been doing that nightly (or maybe just weekly) for a couple years.

    --
    Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
  48. Forensics only needs to be... by Anonymous Coward · · Score: 0

    ...good enough to identify the sonofabitch that cracked your machine so you can go and crack his balls with a baseball bat. Screw the courts.

    --amazing, the captcha to type this is "offenses"

  49. Forensics has an established meaning in security by spun · · Score: 1

    In computer security, 'forensics' has a well established meaning. Any computer security class will teach proper forensic procedures that preserve the trail of evidence for use in a court of law. As this is an article about computer security, I and the other posters naturally assumed the word was used in that context. This analysis is not proper forensics, and the evidence gathered would likely be inadmissible in court.

    That was what was meant. You can argue semantics and definitions all you like, but anyone with even a few course credits in computer security will be unimpressed by your presumption and general lack of security related knowledge. Hell, even someone who merely spent a few days boning up for one of the security related certs will be amused.

    --
    - None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
  50. It's easy to blame M$ for this. by twitter · · Score: 1, Insightful

    If you read the article though, you get to the conclusion:

    The compromised could be caused by: An exploit unknown to the public. [or] A user accessing this server from an already compromised host. The attacker could then sniff the the password.

    I'll give you two guesses at which one of these methods were used by the script kiddie who's mistakes are so well spelled out in the rest of the write up.

    Security is only as strong as your weakest link. If you use an OS with a one in four chance of compromise, you have a one in four chance of giving away all of your passwords.

    --

    Friends don't help friends install M$ junk.

    1. Re:It's easy to blame M$ for this. by dedazo · · Score: 1
      You know you've reached the end of the rope when someone mentions you by name on an outrageously funny comment designed to get people to laugh...

      ... and then you go and make everyone proud.

      --
      Web2.0: I love when people Flickr my cuil and digg my boingboing until my google is reddit and I start to yahoo
    2. Re:It's easy to blame M$ for this. by twitter · · Score: 1

      Did you write this little AC troll thread, dedazo? Your obsession with me is flattering for me but sad for you.

      In any case, I never mind injecting a little sense and truth.

      --

      Friends don't help friends install M$ junk.

    3. Re:It's easy to blame M$ for this. by Anonymous Coward · · Score: 0

      This isn't "sense and truth." This is another tantrum. Did you get stuffed into a locker after study hall again, Twitter?

      Can you provide concrete evidence that the code exploited on this Linux/Apache box was contributed by Microsoft? Because that would be "sense and truth". It would also be evidence of Hell freezing over, flying pigs, and Andy Kaufmann faking his death. Care to try?

    4. Re:It's easy to blame M$ for this. by Anonymous Coward · · Score: 0
      In any case, I never mind injecting a little sense and truth.

      We know what your "sense and truth" look like.

      Why just a few days ago you were trying to desperately pin the Skype fiasco on "M$".

      We know your "truth". Thousands and thousands of times over, lots of "truth" in the name of Free Software.

    5. Re:It's easy to blame M$ for this. by The+Bungi · · Score: 1

      God, you truly are stupid. I mean, not in the sense someone would say "you're stupid" in the comfort of their home while posting online when faced with an opposing argument, but truly stupid in the real-world sense of stupid. I hope to hell you have some redeeming attribute like being able to whistle Beethoven's Fifth through your nostrils.

    6. Re:It's easy to blame M$ for this. by dedazo · · Score: 1

      No, that would be one of your actual fans. You're famous now, a shining bright light in the Free Software community. Everyone looks up to you. They know you by your interwebs handle. Bask in that warmth and enjoy it. The rest of us poor little mere mortals are mostly amused.

      --
      Web2.0: I love when people Flickr my cuil and digg my boingboing until my google is reddit and I start to yahoo
  51. Re:Forensics has an established meaning in securit by SIIHP · · Score: 1

    "That was what was meant."

    No actually, it wasn't.

    "You can argue semantics and definitions all you like"

    And I'd be right, and there's nothing you can do about it. Funny how your kind always tries to make it into "semantics and definitions" when you're wrong, like trivializing what you're wrong about makes you less wrong.

    --
    I only go to buffets for the unlimited soft serve.
  52. I really had no idea. Thank you for posting. by Anonymous Coward · · Score: 0

    I have often wondered how people get away with this. However, your ability to detect where the shananigans were coming from was amazing! I really had no idea just how deeply compromised a machine can become in 2 seconds?! It's scary. I have often wondered how I can check my machine's status. Thanks for giving me a good start. God bless you for posting this. I've learned a mountain from this. Thank you.

  53. right... by Anonymous Coward · · Score: 0

    because the Linux system was so secure that a third-party was able to sniff a plaintext password right off the wire. Judging by the article author's own writeup, and the experiences of others in this thread, I'm going to go with 'unknown remote exploit'. As another poster said it, "Don't be blinded by your religion".
     
    PS, the word you were reaching for was whose. Who's is a contraction for either who is or who has.

  54. Who needs clever hacks? by rastoboy29 · · Score: 3, Interesting

    I work in a large, low-end datacenter.  Almost all the servers there are rented buy non-technical people, who for some reason feel qualified to run web hosting businesses.  There are so many exploits going on there at any given time, we can't really do anything about it--especially as theoretically the customer is responsible.  So when they call in because their server is running slow, we usually find a php hijack happening, tell them their server has been compromised, and suggest that they do something about it.

    It's pretty appalling.  We would need an army of sysadmins--an army which is currently employed already--to really do something about it.  Most of what we see are primitive script kiddie hacks, but guess what--that's good enough, and rarely are the perpetrators hunted down.

    Who knows what the more sophisticated hackers are up to!

    1. Re:Who needs clever hacks? by Antique+Geekmeister · · Score: 1

      Not only are you right, but even when the perpetrators are hunted down, nothing happens to them. Take a look at the Morris Worm and the David Lamachia case for good examples of how perpetrators escape punishment. (Morris's father was the head of the NSA: LaMacchia was an MIT student and MIT's lawyers did a great job of stonewalling the prosecution to avoid a student being convicted, and apparently encouraged the prosecution to file charges trying to set an unlikely precedent, making people who host warez responsible for them even if they don't get paid.)

    2. Re:Who needs clever hacks? by baggins2001 · · Score: 2, Interesting

      I wouldn't be to critical of the techy in this situation.
      It's more about 2 screwed up business models (If you look at it from a technical point of view).
      They want cheap servers with bandwidth, buy cheap servers and buy shitloads of bandwidth. Offer them for really cheap prices ( 10,000 Servers. They may have five or six people on a shift for maintaining these systems. These guys are responsible for patch management and backup/restore, plus they have to physically replace the systems which crash (Usually there is very little forensics done. It's down, yank the box replace it and restore. This usually happens to about 15 boxes a week. Plus you have the hardware update cycle. There's another 100+ getting yanked per week). So these guys are usually pretty busy. There are only a few guys who actually look at the system and try and determine why it is running slow, but they aren't there to fix problems. There in place to tell customers they have a problem and tell them that they need to fix it or let them restore it(very very nicely). They aren't there to go through the intricacies of a hack.

      Comp 2) Some guys heard about this web thingy and heard he can make money doing it. He knows very well that he can't have less than a full server for his 12 orders a week. Of course he originally thought it would be thousands, especially since he went out and had a professional build the whole site for him for $500 (looks good). He occassionally calls this guy up to update his site for $50 (content mind you).

      So now we have 2 business's with interest in a server and neither one gives a shit about security. (Of course the techs working Company 1 do, but they don't have time for that)

      Which brings us to Comp 3. These are the guys Comp2 turns to when their server isn't fixed or keeps crashing due to poor security. They charge 10% more, but this time Comp2 asks them about security. Comp3 answers yes we are vigilant about security "We do patch management and are vigilant about monitoring for hackers". "Ahh, you monitor for hackers" Comp2 says "I'll take it". Never realizing that he is getting no more than what he was getting from Comp1.
      But won't Comp1 go out of business? No Comp1 is getting Comp3's old customers for the same problem.

      Basically if you aren't paying $250/month for computer and bandwidth and paying $300 for management of a system, your getting a Dell Dimension in a barn somewhere. And Odd's are pretty good that a hacker is going to get it or a cow is going to shit on it.

      --
      He who said 1,000,000 monkeys on 1,000,000 typewriters would eventually type the great novel, never saw an AOL chat room
    3. Re:Who needs clever hacks? by Anonymous Coward · · Score: 0

      You ask:
      "Who knows what the more sophisticated hackers are up to!"

      Since this story is off the main page [and I can avoid the flame war] ... the link is to the comments - the story is also worthwhile.
      From what I've gathered a handful of people were hit with this [myself included] - it's reeks of a test run, most everything in the comments is true, even the crazy ass sounding stuff has some merit.
      X-platform, anything, my colleagues and I know it's a hardware based attack.
      Truly, a wonder to see operating - it even mocks you when you think you're making "headway".
      Good luck and check your boot blocks.

      http://www.securityfocus.com/cgi-bin/index.cgi?c=a rticlecomments&op=display_comments&ArticleID=11372 &expand_all=true&mode=threaded

      Forensic Discovery [free book download]
      Dan Farmer and Wietse Venema

      http://www.porcupine.org/forensics/forensic-discov ery/

  55. Re:How did he get access and On tools by stevied · · Score: 1

    It's certainly a long time since disk heads had to parked manually - you shouldn't ever see damage to the disk itself from a power failure. The usual source of data loss is OS and drive controller caching.

  56. Re:How did he get access and On tools by Antique+Geekmeister · · Score: 2, Interesting

    I'm afraid that most software tools are not inherently better than those in 1997: most attackers, and even most successful attacks, are by script kiddies with tools. Even skilled crackers like Mitnick consistently make foolish mistakes. (In Mitnick's case, it was leaving messages mocking his victims and getting the FBI really, really mad at him,, angry enough to actually prosecute.) There are plenty of vaunted crackers who make other amazingly stupid mistakes, both programming and social.

    The IRC-bot creators seem to be among the worst of the script kiddies. Frankly, IRC should go the way of open relays. Too much of the traffic is illegitimate to justify allowing it through any firewalls or any ISP provided system. It should be blocked even before non-ISP-server bound SMTP, simply for damage control.

  57. Re:How did he get access and On tools by Anarke_Incarnate · · Score: 1

    Deception Toolkit. Learn it, love it.
    http://all.net/dtk/download.html

  58. mod_php by OrangeTide · · Score: 0

    Friends don't let friends run PHP.

    --
    “Common sense is not so common.” — Voltaire
  59. Re:How did he get access and On tools by naapo · · Score: 1

    6. The rootkit itself handled all connections via something that looked like ssh. I never managed to figure out how it loaded. One of the executables in the system loaded at startup was backdoored. Probably sendmail or one of the other daemons it could not do without.

    I once had a machine compromised with very similar symptoms. It turned out that it was infected with a loadable kernel module rootkit. Back then I didn't even know that such beasts existed! However, I managed to find and examine it by compiling my kernel without module support, so the rootkit couldn't start. Originally I thought that the kernel was just acting weird and tried to configure it as simple as possible. When booting, it warned it could not load the rootkit module. :-)

  60. 'forensics' also means 'public speaking' by spun · · Score: 1

    When I said 'that was what was meant,' I meant that the posters to whom you were replying were using the word forensics in the proper, computer security related context. You presume too much in assuming you know what others meant. Crow your triumphant pedantry to the world, it won't change the fact that we are all laughing at your utter lack of knowledge.

    The funny thing is, even the definition you tried to apply does not fit. The term 'forensics', when used in the context of 'an argumentative exercise' means public speaking. Perhaps you've heard of the forensics club at your high school? Thats what 'forensics' means in that context. Looking up words you don't know in a dictionary and misapplying them does not make you seem wise.

    --
    - None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
  61. Re:How did he get access and On tools by pionzypher · · Score: 1

    My previous employer was waaaay ahead of you. All of our systems in the field had the root password set to 'cat'. No way in hell was any snot nosed cracker going to try THAT three letter word.

    --
    I'll believe in corporations having personhood when Texas executes one... - advocate_one
  62. If this had been windows... by Anonymous Coward · · Score: 0

    There would be 'haha' and 'insecurebydesign' tags. The hypocrisy just shows how retarded you peons are and why the computing world will not take you serious (less than 7% desktop share for over 10 years straight for mac and linux each, and apache losing ground to IIS). Have fun making hysterical senseless replies, I won't bother checking since I know they're going to be retarded and the same crap that's been rebutted thousands of times. BK.

    1. Re:If this had been windows... by Anonymous Coward · · Score: 0

      Well personally, all the boxes that I care about run OpenBSD, but that's also a laughing matter around these parts (but not so much to the hackers).

  63. Re:How did he get access and On tools by dan+dan+the+dna+man · · Score: 1

    ++

    This article was junk, as you point out the state of play 10 years ago was already way ahead of this.

    I always assumed lack of progress was caused by the kiddies discovering that a large install base of Windows machines was more profitable than the odd *nix machine that took a bit of work to get into.

    feh.

    --
    I don't read your sig, why do you read mine?
  64. Re:How did he get access and On tools by SpaceLifeForm · · Score: 1


    I don't even trust Putty just because it *is* windows.

    --
    You are being MICROattacked, from various angles, in a SOFT manner.
  65. Re:Heh heh by Anonymous Coward · · Score: 0

    Even though I'm not very strong using Linux; I am not smug! I desire to move to the Linux platform instead of Microsoft for many reasons( too many to mention, LOL! ). If Linux is really this insecure then what are you telling people? By posting your comment; are you no less smug than the smugees you claim to hurt by doing the things you do to their systems? I don't get it? I really want to make the transistion but this makes me think perhaps I should use a Mac instead? How can I protect myself from somebody like you and enjoy my computer? I'm just a regular user.

  66. Re:How did he get access and On tools by frinkillo · · Score: 0

    You forgot 'pencil' ;)

  67. basic hijacked server 101 by Anonymous Coward · · Score: 0

    First thing you do is shutdown the server and boot from your CD drive.

    From there you get out your archive copy of your RPM checksums, and you run a checksum test against that CD. The programs that don't match have been modified.

    Thats the benefits of RPMs.

  68. Who modded this Informative?! by Anonymous Coward · · Score: 0

    There's fuck all evidence that Windows had ANYTHING to do with the exploit. Twitter's just karma-whoring to make up for the smackdown he took this week.

    Still think Linus Torvalds is a "M$" shill for not being as irrationally paranoid as you, Twitter?

  69. Some Elemental Precautions by SwedishChef · · Score: 2, Insightful

    We have numerous servers on various subnets and have learned a few elemental precautions to having more than one server cracked:

    1. Change the ssh port to something other than 22;
    2. Use different root passwords on each machine;
    3. Use selinux to block connections from IP addresses you do not control and to ports you don't want the machine connected to (like 6667);
    4. If possible route all packets through a bridged machine which you can then use to monitor activities... be especially wary of IRC connections;
    5. If you have email users set them up as nologin or /bin/false;
    6. If you use ftp do not allow anonymous logins or, if you must allow connections, do not allow anonymous uploads;
    7. Configure syslog so that it logs to several locations; and,
    8. Use access lists on the routers to limit connections both in and out (including the new ssh port);

    Crackers often forget to change lsof (list open files) and that utility can often be used (or reinstalled) to determine if a machine has been cracked and where the nasty bits are hidden.

    --
    No one ever had to evacuate a city because the solar panels broke!
  70. Teh house of cards by Anonymous Coward · · Score: 0

    I think Slashdot needs to stop acknowledging the fact that teh Lunix has no security.

    It only emboldens teh enemy (i.e. teh MiKKKro$$$l0th). If anyone were to realize how teh Lunix's security is provided entirely by teh obscurity, teh entire house of teh cards would fall down.

  71. Re:How did he get access and On tools by drsmithy · · Score: 1

    A former co-worker used to turn her G5 off every day by pressing the front button. I saw her do this once and said (very nicely) "You know it's better to shut down from the menu, right?" and she answered "Yeah, I know you're not supposed to do that, but it's faster." She had been doing that nightly (or maybe just weekly) for a couple years.

    On a G5 (and, indeed, most PCs anf Macs <6-7 years old) pressing the power button should result in a clean shutdown.

  72. Re:How did he get access and On tools by arivanov · · Score: 1

    In my case the attacker did not leave the rootkit on the system. We never managed to find it.

    We found a couple of backdoors now and then none of which was particularly fancy. For example sendmail had an extra command added which executed a shell, etc. So I suspect that he loaded the rootkit straight into memory over the network after accessing the compromised machine through the backdoor. As a result it was never present for forensics.

    The most unpleasant bit was that he nuked the machine at the slightest suspicion of being observed.

    --
    Baker's Law: Misery no longer loves company. Nowadays it insists on it
    http://www.sigsegv.cx/
  73. Re:How did he get access and On tools by arivanov · · Score: 1

    Couldn't agree less. IRC must die. If you need a chat server for work you can always run jabber.

    --
    Baker's Law: Misery no longer loves company. Nowadays it insists on it
    http://www.sigsegv.cx/
  74. Re:How did he get access and On tools by Antique+Geekmeister · · Score: 1

    I have no idea why you say "Couldn't agree less" when eliminating most if not all IRC services is exactly what I meant. Open relay mail servers are relentlessly exposed, hounded, and blocked by most email servers.

    But by the way, do not begin to pretend that most installations of Jabber are any better administered than most installations of IRC. Plain text passwords stored on the server is just an amazingly bad idea: it's almost as stupid as Subversion keeping your user passwords in plain taext in your home directory.

  75. Re:How did he get access and On tools by owlstead · · Score: 1

    Would holding the power button for 5 seconds alert the root kit? Because that's what I normally recommend instead of pulling the plug. It would be a shame to damage the power supply just to shut down the computer.

  76. Re:How did he get access and On tools by jc8088 · · Score: 1

    I think it's probably the fact that the owner of this system had the root password set to "GOD" as all good sysadmins do. The hacker's extensive experience hacking the Gibson made getting into this system a cakewalk Great... Now I have to change all my passwords ;-)
  77. Re:How did he get access and On tools by arivanov · · Score: 1

    Sorry, too high blood level in the caffeine subsystem when posting the GP. I was in absolute agreement. IRC must die.

    As far as Jabber vs IRC vs the rest of the IM I agree they all suck and they can all be used for zombie control. You can write a BOT that logs in on yahoo, AIM or anything else you like. I used to have a Yahoo Messenger BOT that talked to a MON alert system and pinged me when something went apeshit in the network (you could also get network status and such). Writing it was quite trivial, unfortunately Yahoo changed the protocol and the underlying perl modules stopped working. Modifying that code for zombie control would have been trivial as well.

    The difference IRC makes is:

    1. Its tradition. It has been the stomping ground of 1337 wankers since the mid-90es. Gives cred to socially deficient people.

    2. Ability for the user to gain some level of administrative control over a chatroom (aka channel) and exercise it to exclude everyone else from it so it can use it for its own nefarious purposes.

    By the way, as far IM of any form is concerned it is also yesterdays day tech for BOTs. Newer ones build peer-to-peer networks and encrypt them. As a result finding the command and control center becomes practically impossible.

    --
    Baker's Law: Misery no longer loves company. Nowadays it insists on it
    http://www.sigsegv.cx/
  78. Virtual machines rule... by TheLink · · Score: 2, Interesting

    If I suspect something is wrong with my home machines and I didn't care to figure out what happened, I'd just revert the relevant virtual machine to a clean snapshot, disconnect the network connections and patch, restore data etc.

    If I did care, I could either suspend the virtual machine or make a snapshot of it.

    Virtual machines are cool :). Once x86 hardware gets more efficient at running VMs (including IO), I think I'll run everything virtualized. You can't get away with doing that red pill, blue pill thing to my system if I do it first :).

    If you don't run machines in a VM, I believe the proper way to do forensics is to pull the plug (not sure if attackers would tamper with fsync) then make a copy of the drive using hardware that is certified to block writes to the drive - there are few vendors about selling such hardware and software to go with it. Google should show up a few.

    If you do it any other way, any evidence gathered could be considered suspect or tampered with by the defense, or you could accidentally destroy your evidence, or you could be allowing the attacker to destroy the evidence.

    Doing what the chap did in the article is definitely not "forensics", anymore than stomping all over a murder scene while touching everything is forensics or a proper investigation.

    --
  79. Security by DaMattster · · Score: 1

    Security is more than just making certain that your operating system is patched and up to date. Good security practices only start at OS updates. Look at your perimeter defenses. Are there ports that you could specifically block outgoing as well as incoming? Are you using a strong password? This article is not FUD at all. This could happen to any system where competent security audits and mesasures are not taken. In fact, this gives a junior admin some guidelines to begin the detective work of figuring out if a system has been compromised. Bottom line, this attack should not have happened as it likely was not done by a skilled attacker. A competent admin will also be aware of what is running on his or her machine and the open files. As a final note, once a system has been compromised, I could never trust it again without a full blown format and OS reinstall.

    1. Re:Security by DaMattster · · Score: 1

      As a side note, I love the FreeBSD concept of jails. If you must run services that are known problems, run them inside of a FreeBSD jail. If the machine becomes compromised, the attacker cannot break out of jail because the OS is unaware of anything existing outside of the service jail. The OS thinks the jail is the root. Simply wipe out the jail and reinstall the jail with added extra security measures.

  80. Why are you so fucking stupid? by SIIHP · · Score: 1

    "When I said 'that was what was meant,' I meant that the posters"

    Here's what you're not getting cunt.

    I don't give a fuck what YOU meant. YOU are an imbecile, who consistently posts garbage and makes me what to scratch my eyes out after reading the nonsense you think is worth posting.

    And in this case, you're exactly fucking wrong.

    "The funny thing is, even the definition you tried to apply does not fit."

    NO IMBECILE, THE FUNNY THING IS THAT YOU'RE SO STUPID THAT YOU THINK I WAS TRYING TO APPLY A DEFINITION. I WAS RESPONDING TO ANOTHER IDIOT WHO THINKS THERE'S ONLY ONE DEFINITION OF THE WORD, WHICH MY POST PROVES WAS WRONG.

    WHICH WAS MY POINT THE WHOLE FUCKING TIME BUT YOU WERE TOO STUPID TO GET IT. I WAS TELLING HIM THERE'S MORE THAN ONE DEFINITION, NOT TRYING TO APPLY THE DEFINITION I GAVE.

    Get it now you fucking moron? Do you understand why you should have kept your fucking loser mouth shut? Or are you too stupid to grasp that too?

    --
    I only go to buffets for the unlimited soft serve.
    1. Re:Why are you so fucking stupid? by Gandalf_the_Beardy · · Score: 1

      With regard to your impassioned and erudite diatribe above it's clear that your version of forensics, not in the computer related mien admittedly has me wanting spit chips. For your information, I am well aware of the meaning of the word forensics. I am also well aware that English is a contexually sensitive language and that the vast majority of the speakers appreciate this and do not require the context to be explained to them. You may or may not appreciate this, I really don't care. You have proven yourself capable, without justification or reason of ad homeniem attacks (that's Latin, by the way) on others, for the simple justification of inflating what appears to be a small ego. Please have the courtesy as such not to cross my path again, for I fear I find you most distasteful and boorish.

    2. Re:Why are you so fucking stupid? by spun · · Score: 1

      You lose.

      --
      - None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
    3. Re:Why are you so fucking stupid? by SIIHP · · Score: 1

      "You lose."

      In other words you know what I said is true and you realize it, so you have nothing left to say that refutes it.

      But you supported my point about your post quality quite nicely.

      --
      I only go to buffets for the unlimited soft serve.
    4. Re:Why are you so fucking stupid? by spun · · Score: 1

      Here's a clue for you: no one cares what you say. No one here respects you at all. I'm only responding because it amuses me to keep yanking your chain. I bet you'll feel compelled to respond to this post, too. I can make you do things. Froth at the mouth. Good boy! Post an angry response to this. There you go, who's a good boy? Here's your treat: another human being is paying attention to you. Doesn't that make you feel all warm and tingly? After all, bad attention is better than none, right?

      At this point it's not even about wining the argument, everyone knows I won several posts ago. It's about getting you to humiliate yourself in front of everyone. Here's the hilarious part: even knowing that, you will be unable to keep from responding to this post and humiliating yourself further. You can try to justify it all you like, but those justifications only work in your own head, the rest of us are just laughing our asses off.

      --
      - None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
  81. Re:Heh heh by Anonymous Coward · · Score: 0

    I dunno if English is your fourth language or you're 12 or what, but here's a tip: you sound like a moron. Stop saying things like LOL. Any post with 5 questions in a row should probably be rethought, especially if some of them aren't actually questions. Don't respond to arrogance with pathetically sincere and self-deprecating crap. Most importantly, don't even respond to posts like the one you responded to, as you clearly don't understand the mindset of the poster. You'll come to understand how people work as you listen to them more, but right now you clearly don't, based on the tone of your reply.

  82. Re:How did he get access and On tools by eyeye · · Score: 1

    sync
    sync
    *pull*

    --
    Bush and Blair ate my sig!
  83. Re:How did he get access and On tools by Anonymous Coward · · Score: 0

    Yes it can, because on many systems the hardware sends e.g. an ACPI event to the operating system when the power button is pressed. I don't know why pulling the plug would damage the power supply, but maybe that's possible.

  84. gsg by Anonymous Coward · · Score: 0

    examplehttp://www.example.com/example

  85. Not forensics by pbaer · · Score: 1

    The point of forensics is to provide legal evidience, and what he did wouldn't be useful in court. For this to count as forensics he should have at a minimium needed to image the server's disk(s). Then calculate and store the images md5 hashes, so he can prove the images haven't been tampered with. Then mount the images on a trusted computer and use a hardware write blocker. Then and *only* then should he poke around.

    If any slashdotters are interested in actual forensics I highly recommend File System Forensic Analysis by Brian Carrier. Best part is the author only uses FOSS tools so anyone can follow along (in addition to pragmatic reasons).

    --
    There are 11 types of people, those who know unary and those who don't.
  86. I asked why you were so fucking stupid by SIIHP · · Score: 1

    "Please have the courtesy as such not to cross my path again, for I fear I find you most distasteful and boorish."

    YOU responded to ME you stupid fuck. We crossed paths because your reading comprehension sucks, and you thought it would be a good idea to respond when you didn't know what the fuck you were responding to.

    Please do me the favor of choking to death on your lunch today, as the world would easily be a better place without you.

    "You have proven yourself capable, without justification or reason of ad homeniem(sic) attacks (that's misspelled Latin, by the way) on others, for the simple justification of deflating what appears to be a small ego."

    FYP.

    Fuck off now.

    --
    I only go to buffets for the unlimited soft serve.
    1. Re:I asked why you were so fucking stupid by Gandalf_the_Beardy · · Score: 1

      Hehehe.... If you've stooped to correcting grammar and spelling then you've clearly lost whatever argument you thought you may have had. Buck your game up, mediocrity is a vice of the doomed.

  87. Re:How did he get access and On tools by sootman · · Score: 1

    No, she pressed it and held it until the machine went dark.

    --
    Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
  88. Call me master by SIIHP · · Score: 1

    "Here's a clue for you: no one cares what you say."

    You obviously do or else you wouldn't respond.

    "I'm only responding because it amuses me to keep yanking your chain"

    You'd have to start first. Your posts were serious until I S you TFU, then suddenly you're "yanking my chain".

    "I bet you'll feel compelled to respond to this post, too."

    You mean like you did when you HAD to post "you lose"?

    Right, that was YOU yanking my chain.

    "At this point it's not even about wining the argument, everyone knows I won several posts ago."

    Was that the post where you lied, or where you posted a reply to an argument I never made? And how does either of those things "win" an argument that we were never having in the first place?

    "It's about getting you to humiliate yourself in front of everyone."

    You mean like you did when you argued a point no one ever made, or when you got told that you're an imbecile and started responding by "yanking my chain" because I hurt you widdle feewings?

    "Here's the hilarious part: even knowing that, you will be unable to keep from responding to this post and humiliating yourself further."

    You mean like you did when you posted a response to an argument I never made, or when I S you TFU and you got your widdle feewings hurt?

    At any rate, if I were to "humiliate" myself with this post, that wou;d put me three humiliating posts behind you in this thread alone.

    I can take it, can you?

    And really, it's pretty obvious when I told you off, you got your feelings hurt. Go ahead and lie, but the tone of your post only proves it. Did you cry (yes, but you'll lie and say no).

    And I LOVE how you use my "go ahead and reply" gimmick, but you should give me credit for using it, you know very well who you stole it off of and I'm a little sad that you use against me knowing you learned it from me.

    I guess that makes me your master, since you're learning how to post from me.

    I like that, when you respond, call me master.

    --
    I only go to buffets for the unlimited soft serve.
  89. Mackenzie Morgan naked and petrified!!! by Anonymous Coward · · Score: 0

    Dear Beautiful Mackenzie Morgan (an Actual Girl):

    I'd like to sneak up behind you and start fondling you violently and then as you struggle to try to escape I'll take a scientifically-proven magic petrification ray from my bag and zap you with it, and it would first disintegrate all your clothing, leaving you gloriously naked, then it would start the process of transforming your body into marble, inducing in you a massive magically-induced which would be captured eternally as your body is turned into solid stone from the feet up to the head gradually, freezing your final moan of ecstasy as you become a beautiful, cold lifeless statue, but with your mind still alive inside the statue, aware of everything that happens to you. I would put you in display in art museums so that everyone could admire your spectacular naked & petrified teen body, then I would put you on a pedestal in my apartment and admire you constantly, and climb up on the pedestal and make love to your stony form, getting my penis raw & red from the friction, and covering your beautiful hard marble skin with my spooge, my beloved naked-and-petrified queen.

    (NOTE: This is just a fantasy; I would not actually do this.)1

    p.s. I like masturbating to your Blogspot picture.

  90. A lesson for us all by KnezLazar1389 · · Score: 1

    This is a great object lesson to the public regarding the fact that Linux boxes can be hacked just as anything else can. We need more security training for everyone who will be administering a Linux system. Even if there is nothing particularly sensitive on your system, you do NOT want it being used as a bot! That said - for larger organizations, hosting a critical application on swarms of little boxes isn't the answer. I suppose there's a reason why some people might call me a "mainframe bigot." For large-scale applications that need extreme reliability and uptime, along with the least likelihood of penetration by hackers, host them on IBM "big iron." If you have a lot of *nix boxes of any flavor, you're going to save a ton of money by way of electric power, floor space, and support time by switching over.

    --
    http://twitter.com/1389 http://pownce.com/1389 http://www.technorati.com/people/technorati/1389 http://www.gleamd.com