Slashdot Mirror

← Back to Stories (view on slashdot.org)

How To Secure A Cracked Box

Posted by CmdrTaco on from the learning-things-the-hard-way dept.

Noel sent us a collection of stories from rootprompt on how to secure your box. The articles include Denial and truth, Watching and Waiting, Hunting the hunter, The Sniffer, and Rebuilding the system to recover from the crack. It's an interesting discussion on what it's like (and enough to churn the stomachs of anyone who's ever been there).

27 of 210 comments (clear)

  1. Re:Dot Matrix Printers and security? by InfoMonk · · Score: 3

    Even though Clifford Stol used that method in the infamous Cuckoo's Egg, operating systems seemed to have progressed well beyond that type of jerry-rigging. With the potential for autoarchiving log files, automating their conversion into different formats, and not too mention the cheap availability of older PCs that can serve as independent and secure log servers, a dot matrix seems to be a resolution only for the most paranoid sysadmin. (Of course, dot matrix printouts do still retain their age old hacker appeal.)

  2. Re:Dot Matrix Printers and security? by vyesue · · Score: 5

    uhhh...

    auth.* /dev/lp0

    ...might be a way to do this without tail -f sucking half your processor 24/7.

    man syslog.conf, dude.

  3. Can't say it enough: use Kerberos. NOW! by coyote-san · · Score: 3

    It seems that I can't say it enough. Install and use Kerberos. NOW!

    SSH is great for connecting to a shell account, but you may still leak passwords once you've established a secure connection to your "trusted" network. Kerberos, properly installed, ensures that your passwords *NEVER* appear *ANYWHERE* in plaintext, and rarely appear in ciphertext. After all, you never know when someone has compromised one of your local tools, e.g., psql.

    --
    For every complex problem there is an answer that is clear, simple, and wrong. -- H L Mencken
  4. Re:Why can't we all just get along by ajs · · Score: 3

    You say this jokingly, I presume, but what if such a beast did exist? I would run a client of such a system *if* it contained a Slashdot-like moderation system that allowed people to propose code (e.g. post code they wanted to run to a public forum). When a piece of code gets enough votes, it "goes live" and people start executing it. Would this result in problems? Sure. Do I care? No.

    Seems odd, no? Well, I say the Internet was put in place by people who had bigger dreams than a really fat pipe for advertizing. I think the Internet is actually a cool thing, and should be used to its fullest. This would give it that chance, but would also come with risk. Ok, I can do risk....

    Anyone up for writing it?

  5. Re:how to secure a cracked box: by Admiral+Burrito · · Score: 3

    well, if your binaries are all on read-only media, maybe there are subtle backdoors hidden in your rc.files; maybe configurations files for daemons have been subtly altered to provide a way back into your machine even after you think it's resecured.

    Indeed. I've spent many hours thinking about this...

    Suppose you lock down your system really tight. You use Linux capabilities or BSD securelevel to set your binaries and config files (and directories! don't forget the directories and their parents or "mv" followed by "cp" is all it takes to trojan your stuff) read-only and your log files append-only in such a way that not even root can mess with them.

    Being a security-concious person you insist upon changing your passwords regularly. This requires /etc/passwd to be writable by root.

    Your login shell is specified in /etc/passwd.

    Some intruder gains root, discovers he can't trojan the system binaries or wipe his footprints from the logfiles because of all the lock-down you've done. No problem! He changes your login shell in /etc/passwd to point to a little program that chroots you into a special jail directory heirarchy where all of your usual tools and logfiles can be found, in trojan form. Since the intruder hasn't altered the protected stuff in /bin, /var/log, etc. he hasn't done anything your capabilities system can prevent.

    Bingo, you are now the clueless luser in the honeypot.

  6. Why not CDR? (Re:Dot Matrix Printers and security? by yellowstone · · Score: 4
    I have been playing with the idea of hooking up an old dot-matrix printer to print out certain log files, or lines from log files with keywords in them.
    Why not stream them to a CDR? IANASE (I am not a security expert), but it seems to be CDRs are also write-once, but have the additional advantage of being greppable (not to mention cheaper and more environmentally friendly -- you would have to kill a lot of trees to print out the text that fits in 650 megs...
    --
    150 Opening BINARY mode data connection for slashdot.sig (129323052 bytes).
  7. How to secure a cracked box by Hard_Code · · Score: 5

    Items you will need for this procedure:

    1. Superglue
    2. Strip of cloth or large bandage
    3. Tape, twist tie, or rubber band

    First, apply superglue to both sides of crack, and press pieces together. If superglue comes into contact with hands, follow instructions on back of package to remove. Do not attempt to lick off superglue.

    Wait. Until you're tired of waiting.

    Take strip of cloth or bandage and tie it around box, perpendicular to the axis of the crack. Secure cloth tightly by either tying it in a knot, or by using tape, a twist tie, or a rubber band.

    Refrain from dropping or throwing your box out a window to avoid the risk of future cracks.

    (sorry, something makes me do this)

    --

    It's 10 PM. Do you know if you're un-American?
  8. Re:a good reason not to use *nix by stab · · Score: 5

    Well, to be honest, its your fault for using BIND!

    BIND is notoriously insecure, so you should always run it in a chrooted environment if you are going to use it.

    Also, investigate alternative, and far superior servers for services you want to run.

    Instead of BIND, look at Dan Bernstein's DNSCache package, which is lightweight, stable and uncrackable. In fact, he offers a monetary reward to the first person who can find a security hole in it.

    Similarly, replace sendmail with either qmail, exim, or postfix and get a superior, more intuitive feature set, and better peace of mind security wise.

    Also, look at a more secure OS than Linux, for example OpenBSD which has not had a remote security hole in its default installation for over two years now.

  9. Re:a good reason not to use *nix by Hardwyred · · Score: 3

    If the box was being used for the same purpose that a windows box can serve, why run bind anyway? The problem is not the OS, you'll be hard pressed to argue that comparing linux running no services and a windows box running no services, that linux is less secure. Or any un*x for that matter. The key is to know the purpose of your box from the start. Are you building this box just as a gateway? Then you shouldnt need any services running. If you are going to use a linux box as a router, then think of it as a router. If you are going to use it as a firewall, then think of it as a firewall. How many firewalls have you seen, PIX and what not, that have DNS or mail servers running on them? None. The problem is not the OS, the problem is education.

    If you want an all-in-wonder box that will do your masq'ing and firewalling and mail hosting and web hosting and DNS and wash the dog, then you need to at least research the services you are going to use and be prepared for the attacks. BTW, a do-all box is just a bad idea IMHO. Whats the point of having a secure firewall and then running non-secure public services on it? A little forethought would have saved you a lot of time.

    ... and the geek shall inherit the earth...

    --
    www.linux-skunkworks.com
  10. Dot Matrix Printers and security? by billh · · Score: 5

    I like the idea of certain log files that cannot be erased, so...

    Upon installation of SDSL, I will be moving my webhosting services to my home. I have been playing with the idea of hooking up an old dot-matrix printer to print out certain log files, or lines from log files with keywords in them.

    Am I hopelessly out of date with this idea? I have seen some mention of systems like this, and I think it will be a good complement to other security. The idea is that if I get a penetration, I will at least have an idea about *when* the initial intrusion was, and be able to work with that.

    Anyone else with a similar system care to comment?

    1. Re: Dot Matrix Printers and security? by InitZero · · Score: 3

      You should never have security logs the machine those logs secure. While I find the printer idea pretty darn cool, syslogd directed to another box would be fine, too.

      Assuming, of course, that machine was secure.

      I worked at a company where the most secured system in the entire building wasn't the firewall, mainframe or the accouting system. It was the syslog box.

      The only service the box was running was syslogd. It allowed no telnet access and all ports except 514 were closed into and out of that system. In order to physically touch the machine, you had to break a seal on the box (literally, a locked plywood box with a fan in the back) to get in. (Remember, seals are not locks and locks are not seals.)

      Our position was that you couldn't keep people from doing bad things but that if someone did a bad thing, we wanted to be able to hunt said person down and render vengeance not heard of in thousands of years.

      Of course, not even this system was fool-proof given UDP and that the network had to be up. But nothing is perfect.

      Dot matrix printers rock.

      InitZero

    2. Re:Dot Matrix Printers and security? by synaptic-impulse · · Score: 5

      here is the way I do it:

      we have many systems in house and collocated that get scanned and attacked regularly. we use syslog to pipe ALL logs back to a central server. this server then runs LOGCHECK against the logs, and emails and prints all signatures found.

      Logs are reviewed as the come in via email - and daily the printed logs are reviewed by several ppl to ensure that "many eyes" look for anomolies.
      This way - we never miss anything that looks strange.

      We ran this system with no filters for about a month and a half to determine what items would be ok to ignore (standard system events, cipe key syncs etc)

      In addition we run port sentry, and lids. port sentry will permanently block any IP that scans us (we get scanned at least 3 times a week) and lids is setup to make all log files (and others) to be APPEND ONLY - even by root.

      Unless our systems get compromised AND the hacker can unlock lids - he really doesnt even have root access.

      Last - any scan that comes in gets investigated.
      1. permently block that IP
      2. trace the IP (ping,nslookup,whois etc)
      3. contact that site/isp/etc. via email with the log excerpts that show the attack.
      4. archive for eventual turn in to FBI

      here is something that you will really find interesting: this is the response from one scan that came through:

      We sincerely regret any inconvenience/consternation the probing from 216.181.81.11 may have caused you and/or your organization. The machines that have had the name excaliber.barnhard.net have been the subject of a number of attacks which have been investigated by the FBI and in some case may still be under investigation. Based on the prior investigations we agreed to make a reasonable effort to collect data concerning any subsequent attacks, and in particular any attacks which may have some relationship to prior events. Whereas it is possible that three different random hackers have figured out independently that the machines bearing that name are used for testing/training on our network and have used an exploit suitable for whatever variant of Linux happened to be installed at the time. I think as the number of times it gets hit increases it is increasingly unlikely. Regardless, the boys/girls responsible for this are likely unaware that once an attack is confirmed we activate an upstream monitoring process that records all of the incoming packets, which we will forward to the FBI. Our poor abused testing machine then gets backed up to tape, wiped, and then reincarnated when needed again. It is interesting, but it is also getting old fast. As such, we have made the decision that our future test machines will be locked down boxes like our production equipment. If anyone is interested in the construction of suitable blackhole boxes that could assist the FBI in tracing these folks, instead of having to leave hacked machines live I think it would be a good thing. I am sure they would be interested. If we could lay a cracker trap that would only cost a modest amount of bandwidth and CPU cycles that could gather the necessary evidence on the cracker without enabling them to carry out real attacks, I know I would be interested.

      Once again, thanks for letting us know you were scanned, We sorry to have darkened your doorstep in these regards. The machine has been taken down and subsequently replaced.

      If you have any questions related to this matter I can be contacted at the address/email/phone shown below. Our contact with the FBI is Special Agent Kevin M. Walsh who can be reached at kwalsh@leo.gov.

    3. Re:Dot Matrix Printers and security? by ion++ · · Score: 3

      if you want more security... and is really paranoid like me ;-0
      then why not this senario

      ADSL -> hub -> server
      also in the hub is a network cable, that has the SEND lines cut over, so the machine only can receive.
      On this machine you constantly "record" anything on the network, much like the tivo.
      Then you run real time checks on the netpackets, and the most strange you log to your hd. If it is big enough, and the site is small enough you could have a day, a week, perhaps a month's data on the disk of suspisious connections.

      As for the syslog... why not send them over the serial line into the previously described computer, and on this computer dump everything into a text file so no command could ever be executed, simply anything from com1 is saved as /var/log/log_from_server

      and then you run your logcheck program on the log.

      ion++
      ps: i remember someone video recording the console which was writting everything to the console.

    4. Re:Dot Matrix Printers and security? by Nater · · Score: 3

      I tried this for a while, but my printer was a little weird dealing with log files, so I eventually got rid of it. When I did tho, I deleted /dev/lp1 and then remade it with the device numbers for tty9, and left the syslog configuration files alone.

      Some time later, someone did try to get in, but they saw a bunch of stuff logging to "/dev/lp1" and left. So even if you don't have a printer, if you can make it look like you do, you'll scare off a few that way.

      --

      I like to play children's songs in minor keys.
      "We're all sons of bitches now." --J. Robert Oppenheimer

    5. Re:Dot Matrix Printers and security? by Colitis · · Score: 3

      How about logging via serial port?

      A machine that's only saving to disk anything coming in via serial and has no network connection will be *very* hard to crack, and you have the advantage that your logs are still in electronically searchable form.

  11. Why can't we all just get along by Shoeboy · · Score: 5

    A modest proposal for making life easier for DDoS crackers

    I have an idea. I think it's brilliant. When you want to DDoS a big site into the stone age, most of your time is spent infecting hosts to use in the attack. This is annoying and it causes us to behave in antisocial ways. If I wan't to bring down Yahoo, I want to do it NOW!!, not after I finish setting up a subseven network. All the work I have to do makes me pissy. When I get pissy I wipe your hard drive to cover my tracks. Now you're pissy too. Misery loves company.

    What I have in mind is a massive voluntary distributed computing effort along the lines of Seti @ Home. I call it kiddie @ home. Basically, those of you with cpu and bandwith to spare should sign up. When you aren't using your computer, I'll use it to launch SYN attacks and settle grudges. Now I don't have to crack anything, and you don't have to bother reading cert advisories. We're both happy.

    What do you guys think? Can I sign you up?

    --Shoeboy
    (former microserf)

  12. A gram of prevention is worth a Kg of cure.... by fluffhead · · Score: 5

    Try securing your systems BEFORE they get cracked. A good few places to start:

    Insecure.org, especially this top 50 security tools page.
    SecurityFocus the disseminators of the BUGTRAQ list among others.
    Attrition.org, especially their security page.
    And of course 2600, the l0pht, and Phrack for the latest tasty street info....

    #include "disclaim.h"
    "All the best people in life seem to like LINUX." - Steve Wozniak

    --

    #include "disclaim.h"
    "All the best people in life seem to like LINUX." - Steve Wozniak
  13. Re:Interesting... is there such a thing for Win NT by AugstWest · · Score: 4

    No NT Admin should ever be without NT Bugtraq.

    Subscribe to the mailing list and sit back and watch your inbox. Dig through the archives if you're a new user. You'll be amazed at the sheer volume of security issues that floats through on a daily basis.

  14. Where to begin? by Inoshiro · · Score: 4

    As others have said, you should always reinstall after noticing your boxes have been cracked (you'll also want to check on things to see if you can determine the point of entry and person(s) responsible).

    The better solution is to just not be cracked in the first place. The way to do this is to be known-secure. How do you do that? Audited code, such as OpenBSD provides peace of mind. Secure logging (i.e.: logging to another internal machine whose job it is to accept log reports) -- this gives you a nice write-only log target, making it easier to trace intitial probes and attacks.

    Next, you'll want to check existing services, and review any services you want to add. I discussed this in Securing the Border, parts 1, 2, 3, and 4.

    You might also want to read "Auditing Kuro5hin" where I found a root compromise on Kuro5hin.org when reviewing the system with Rusty, the site owner and creator. It has tips on how to recover cleanly.
    ---

    --
    --
    Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
  15. Be sure to catch Noel's next article by ch-chuck · · Score: 4

    titled: "How I delt with over 48,762 simultaneous http connections refered from /."

    Part 1. The onslaught
    Part 2. I've never seen a disk so busy
    Part 3. Out of swap space
    Part 4. Internal Server Error
    Part 5. The crowd finally goes away

    --
    try { do() || do_not(); } catch (JediException err) { yoda(err); }
  16. System Design From The Beginning by I0ta · · Score: 3

    If you've been hacked, reload.. It's that simple. If you designed your system 'right' from the beginning, this isn't a big deal. Here are some basic steps I've used that anyone can use.

    1) Get your *nix (or any os) setup the way you want, with patches, drivers, etc..Don't load application software yet. Create an Image of that machine.(ghost, drive image, etc..)
    2.Load your applications.
    3.Set your syslog to mirror your logs on another server.
    4. If possible, try to move your 'data' directories (from your applications) to another directory for just 'data'. (You'll have to create symbolic links from their original locations.)
    5. Backup your DATA Directory/Drive ONLY!

    Too many times do I see people backup their entire system whether it be Winblows or *nix. If you get a virus, or comprimised binary, that file/binary will be backed up! If you don't catch the attack, all of your backups could be infected.
    A good rule of thumb is too only backup your DATA, not your binaries. After all, you own the software, right ? *grin*
    Then, the obvious solution after a hack is to:
    1) Reapply your OS image (ghost, drive image, etc)
    2) Apply new patches/fixes/close security holes.
    3) Reload your Applications
    4) Reload your data
    5) Point the applications to your data on the other drive.

    Yes, it can be a long, drawn out affair initially, but whether it be a hacker or just plain system crash, the restoration process goes rather smoothly.

    -Iota

    --
    God is Real Unless Declared Integer
  17. how to secure a cracked box: by vyesue · · Score: 5


    reinstall.

    seriously, if your machine has been compromised by anyone other than a completely retarded skriptkiddie, chances are there's going to be lots of "new functionality" in some of the bins on your machine. reinstall from read-only media.

    1. Re:how to secure a cracked box: by vyesue · · Score: 3

      well, if your binaries are all on read-only media, maybe there are subtle backdoors hidden in your rc.files; maybe configurations files for daemons have been subtly altered to provide a way back into your machine even after you think it's resecured.

      if your machine is owned, tripwire can be subverted. it's not trivial to use tripwire correctly, and even if it is used correctly, it can still be tricked.

      as to your last point - once one machine falls, other machines on the netowrk become progressively more prone to falling too. think communists in SE asia, you know? :D

  18. a good reason not to use *nix by jeffstar · · Score: 4
    I have been compromised twice before. Both times through bind. The first time some hax0r group found my box by scanning for computers running bind. they installed all kinds of root kits, and i didnt realized it was jooked for a few months. Its a headless ipmasq, so as long as it works i dont care... Anyway i unplugged it from the net when i found out. I found out when i tried to install something and ps aux was not showing a bunch of things that were supposed to be running. they had messed with it so it didn't show their eggdrops.

    the next time was bind again, but the guy rebooted the box for some reason and then i found him on irc (was using the same nick as the account that he added, and IPs matched), and i asked him how he did it and he said bind.

    i dont run bind anymore ...

    i reinstalled after the first time, but not the second.

    That kind of thing can't really happen with windows, yeah you can get back orifice but norton antivirus takes care of that. For someone moving from windows to linux (say like my dad) if he hears that he has to check some web page and subscribe to mailing lists to keep on top of latest exploits that will root his box, it's a good reason to stay with windows.

  19. Two Letters: PM by Amphigory · · Score: 3
    Namely, Preventative Maintenance. If you're running Linux, with it's fast turnaround of bug fixes, you can prevent most cracks by just installing the OS upgrades on a regular basis.

    C'mon guys... I know you love your uptime. But if you download the Redhat (or Debian, or whatever) updates once a week, install them and reboot, you'll save yourself a world of trouble. Depending on the updates, you don't even need to reboot -- but it's usually the easiest way to make sure all the daemons have been restarted. Plus it cleans up your memory pool.

    I have seen many boxes compromised. But there are two configurations I've never seen hacked:

    1. Redhat w/ latest updates.
    2. OpenBSD.
    Note that closed source OS's seriously suffer in this area. Running Solaris (second only to Linux in the number of exploits), your best bet is to replace /every/ server program you can with the latest Open Source alternative. I've seen sun take 6 months to turnaround security bugs (granted years ago, but still).

    --

    --
    -- Slashdot sucks.
  20. no such thing as recovery... by blaine · · Score: 4

    Only rebuild. The only possible ways of fully recovering a cracked system are:

    1. reload the entire system from a known-good backup

    2. reinstall the entire system

    However, #1 isn't always possible. First of all, if you don't keep backups of your system, you are SOL. Even if you do, if you don't keep backups around for long periods of time, it is possible you don't have a backup from before the initial intrusion.

    If anything, you CANNOT trust ANY data/programs/etc from the cracked system. ANYTHING and EVERYTHING could have been modified by the cracker. Trying to plug the hole after its already been used is pointless, as you have no way of knowing what they've changed. If you just update whatever program was the problem and move along your merry way, you're just asking for a repeat of the initial breakin.

    --

    -[Blaine]- "'Oh dear,' says God, 'I hadn't thought of that,' and promptly vanishes in a puff of logic."
  21. Re:Interesting... is there such a thing for Win NT by stokessd · · Score: 5

    Anyone know of a place for Windows NT Security?

    Betty Ford Clinic.

    Sheldon