Slashdot Mirror


How Should One Respond to a Network Break In?

Jety asks: "I am the sole IT support for a medium sized residential real estate office. It has a network of one main server, 10 office workstations, and another 40 or so agent's personal computers. I discovered via logs that recently someone made about 50 remote login attempts to the server, guessing at passwords, but it would appear that they were not able to gain access. They did, however, leave an IP address in the logs. It turns out to be an Exchange server for another business in the same city. What is an appropriate response to this sort of failed break-in attempt? How seriously should one react? How should it be presented to management, and should you encourage them to over or under react? Should the other business, whose server was used to launch the attack, be informed? Should you try to surveil them first to learn about who is doing their tech work? With what tone should they be approached and/or accused? What would a suitable response from that company entail?"

26 of 96 comments (clear)

  1. First and foremost, cover your ass. by TripMaster+Monkey · · Score: 4, Insightful


    Document everything in writing, discuss the situation with your superiors, and seriously consider initiating some form of legal action. If you are the first to get litigious, you stand a better chance of having the situation resolved in your favor. Unfortunate, but true.

    --
    ____

    ~ |rip/\/\aster /\/\onkey

    1. Re:First and foremost, cover your ass. by Saeed+al-Sahaf · · Score: 2, Insightful
      I discovered via logs that recently someone made about 50 remote login attempts to the server, guessing at passwords, but it would appear that they were not able to gain access.

      If you really want to, try to find out who admins the other server, and make contact. Are they competitors, that would change a lot of things. BUT, this sort of thing happens several times a day to the servers I admin. Generally, there is nothing to be done about it, trying to notify the offending source is usually ignored. More than not (precisely always), it's Windows boxes infected with bots.

      Why waste any time on this?

      --
      "Who are in control, they are not in control of anything - they don't even control themselves!" - Glen Beck
  2. Call 911 by H8X55 · · Score: 3, Funny

    Call 911 and let the Patriot Act take it from there... No one from that company will be trying to pwn you again.

    1. Re:Call 911 by ian+rogers · · Score: 2, Funny

      And make sure to use the word "pwn" while on the phone with 911. :)

  3. It probably isn't even them by Stone+Rhino · · Score: 5, Informative

    You shouldn't start out accusatorily, because it's most likely that they're not the ones attempting the breakin. It's more likely that their box has been hijacked and is being used as a proxy to launch attacks against your computer for someone else.

    After all, who uses an exchange server as their terminal to log in to other computers? If it was one of the desktops, then it would make sense that they were attacking.

    --


    Remember, there were no nuclear weapons before women were allowed to vote.
    1. Re:It probably isn't even them by linzeal · · Score: 4, Interesting
      Unless he lives in a large city I highly doubt your suspicions, if that happened in my town of 30k you can bet all the beans in Boston that some summer intern has gone rogue. I have dealt with similiar things while working for an art gallery in Phoenix. We had our WWW server compromised at a datacenter that we did not control and a trojan was installed in a scratch directory with the name of a popular program for digital art manipulation at the time. One of my friends downloaded and installed it on the main point of sale machine in the front of the shop and it almost immmediately attempted to phone home to a ip address owned by a competing art co-op who had been dissed by us in a play performed at our gallery the month before. It was stopped by Tiny Personal Firewall which was installed on all machines in the gallery.

      We did not call the police, instead we found out the format it was sending information in and what it was reporting. So we took the program and installed it on disconnected machine to play with it. It scanned a hard drive for Jpeg, PDF and PSD files and than sent them in a zipped file to the address every night at 3 am. So we had a meeting to decide on what we should send them. We decided to send someone they did not know to photograph inside their gallery when they were not looking. After we had most of their new installation photoed and scanned, FYI this is before digital cameras were cheap.

      After that we found out where they lived and took pictures of them leaving their houses in the morning for some who lived nearby, their licence plates and inside of their cars, where they worked some with pictures of them working and sent it to them a few days later. About a week after that we took pictures of someone taking pictures of us from across the street in a car we did not recognize and blew up the image to find the culprit who we told the competing gallery about which promptly took his whole installation including 2 computers synchronizing motion to music (just a program downloaded off the net) and left all of it in the back of the building in central Phoenix in broad daylight. Virtually nothing survived, lol. Some people were pissed we took photos of them and their art but I believe it it legal to do so in public. Correct me if I'm wrong.

  4. Simple by rylin · · Score: 5, Insightful

    You try contacting abuse@ the other company.
    If that fails, you call them up and ask for their tech-lead.

    You already have your logfiles, and reasonably secured server.
    What you can gain here is a partnership - or at least an exchange of favors every now and then - between your company and the remote one.

    That said, if the other company isn't responsive, you firewall them to hell and get on with your daily work.

    You'll want to give management a brief notice about what's happening before you do this, obviously.
    After you've talked to abuse@, you tell management what happened.

    Now is the time to see over your authentication schemes. Are your users logging in over SSH? With passwords instead of keys? (Hint: keys are nicer).

    After this is said and done, you paypal me $90 for doing your job.
    Cheers!

    1. Re:Simple by EricV314a · · Score: 2, Funny

      And this is why consultants should demand their fee UP FRONT

  5. Just inform them by dtfinch · · Score: 3, Insightful

    No damage was done to you, except the effort you put into investigating. They, on the other hand, will probably want to catch whoever's actively using their server to launch attacks.

  6. Don't overreact by Nos. · · Score: 4, Informative

    Start off by blocking remote logins (ssh?) from anywhere except where you want to allow people to log in from. Second, I would send a polite, email to their tech contact, or if you can't find that, regular post mail to the company. Don't overreact. Their are a lot of ssh worms out there. I have one machine where I watch for these kinds of things. I see at least 3 or 4 worms hitting my box a day.

    1. Re:Don't overreact by Nos. · · Score: 5, Informative

      Speaking of which, I was just chatting with a buddy who has a Brute Force rule setup in IP tables. Too many connections from a single IP within a set amount of time creates a temporary ban of that IP.

      Here's what he wrote to an IRC channel we were on (this is untested but should be close):

      • iptables -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW -j SSH_Brute_Force
      • iptables -A SSH_Brute_Force -m recent --name SSH --set --rsource
      • iptables -A SSH_Brute_Force -m recent ! --rcheck --seconds 60 --hitcount 4 --name SSH --rsource -j RETURN
      • iptables -A SSH_Brute_Force -m limit --limit 3/min -j LOG --log-prefix "SSH Brute Force Attempt: "
      • iptables -A SSH_Brute_Force -p tcp -j REJECT
      Again, I haven't tried this yet, but generally speaking, 4 ssh connects within 60 seonds on eth0 will result in a 3 minute ban - I think.
    2. Re:Don't overreact by Mercury2k · · Score: 2, Informative

      I would have to advise AGAINST email contact. Remember, if their email server is compromised, chances are the person you engage in converstaion with is infact the person who is trying to break into your machine, and thus will go unreported to the people who can do something about the problem. A better solution is to do a whois on the domain name and try to get a phone number of the company involved. Also, dont phone just one contact number is multiple numbers are givin. If the admin contact is actually trying to break into your machine, they can simply play dumb and just stop attacking your machine since they now know you are on to them. You are better off also talking to the CEO or the billing contact, since this would involve two seperate people and the odds are that two people wouldnt be working togethere because of legal issues. Remember, a crime is better to commit when less people are involved since telling or approaching someone else about criminal activities puts the initial person at risk of being ratted out.

    3. Re:Don't overreact by 4of12 · · Score: 2, Insightful

      Be careful with implementing auto blcoks on connections since systems like that can sometimes be abused to cause a denial of service.

      --
      "Provided by the management for your protection."
  7. It doesn't have to be the exchange server by maddskillz · · Score: 3, Insightful

    There is a good chance the whole business uses the one IP for everything, so it could be anyone at that business (or anyone accessing an unsecured wireless network they have setup, etc) that is attacking your network

  8. Personally... by Anonymous Coward · · Score: 5, Funny

    I always celebrate. Oh wait, you mean as the victim? Hrm..

  9. Diplomacy by Penguin+Programmer · · Score: 2, Informative

    My guess is that it's that script trying to bruteforce random SSH servers, as mentioned on /. a couple weeks ago. My server here at work has been hit too, although the attacking machines were in Europe and Korea in my case. I emailed the owners of the IP blocks the attacks came from and have left it in their courts. My system is secure (I'm the only one who can login via SSH and I have a damn good password), so there was no harm done.

    I think before you jump to any conclusions about it being malicious on the part of the other company, you should call the tech there and let him/her know what's happening. If it is indeed a script then there's no harm done and the other tech can take care of fixing their system. If it was actually a malicious attack, then you can try and figure out who is responsible.

  10. Big Friendly Letters by SDMX · · Score: 3, Funny

    DON'T PANIC.

  11. Depends by linuxwrangler · · Score: 4, Insightful

    Frankly I'm a lot more afraid of a successful breakin that I don't discover than heaps of unsuccessful attempts that I do.

    Essentially everyone who attempts to hit my ftp server with anonymous is trying to break in - the address is only known to a few people who have accounts and I can see from the logs that the other attempts are just scripted tries.

    Similarly, I'm see several attempts every day to log into my machines via ssh (where an attempt may involve from a dozen to hundreds of tries to log in). Don't even get started on what I see in the http or smtp logs.

    I work at a small company, too, and I could pull everyone off their jobs and still not have enough manpower to investigate each attempted breakin, locate and contact the appropriate parties, etc.

    As mentioned elsewhere, most of these machines are compromised so you are really spending your time to provide unpaid antivirus support for the other party's machine. You have to pick your battles.

    Depending on my workload and the probability of a positive result I'll contact someone as a courtesy. Generally my criteria is that I am able to make telephone contact with a person responsible for the machine relatively quickly.

    --

    ~~~~~~~
    "You are not remembered for doing what is expected of you." - Atul Chitnis
    1. Re:Depends by Johnno74 · · Score: 2, Interesting

      A while ago I was setting up a win2k server on my connection at home with an external ip address (yes I patched it before I went online :D).

      One of the last things I did was disable FTP, and then on some whim I checked the ftp logs...

      Someone (no doubt a bot) had connected to my ftp server with anonymous, created a directory, changed into the directory to make sure it really existed, then deleted the directory and logged out.

      No doubt my IP address was now on some list of open ftp servers.

      I was very tempted to leave FTP going for a while and see what turned up there, but then I realised I probably wouldn't like what I found so I left it disabled

  12. Re:Call/email them by Aeiri · · Score: 4, Informative

    It might be not even them due to spoofing, but most likely, it is unauthorized use of their machine.

    How would you go about brute forcing a server using IP spoofing? With IP spoofing, you don't get the packets to return to you, they get returned to the server, then dropped. No complete TCP connection can be made.

    Therefore, SSH would never get the packet to begin with, and even if it did, and got your full packet, it wouldn't send the "success" or "failure" to you.

    That computer is obviously either compromised (most likely), or being used by authorized personnel to launch this attack (very unlikely).

  13. Re:Set a trap by Anonymous Coward · · Score: 2, Interesting

    those scripts that try to brute-force ssh don't even look at your files. they rm -rf to make more space, and then run an IRC server to trade porn with, and also start trying to break into more machines.

  14. Let the readers decide by kmahan · · Score: 2, Funny

    You could always just post the IP on Slashdot.

    Some might consider that overkill though.

    --
    Invalid Checksum. Retrying.
  15. It's called NAT by b00m3rang · · Score: 3, Insightful

    Just because their NAT router has a port forwarded to an Exchange server doesn't mean that the Exchange server was necessarily the machine where the attack originated. It could have been that machine, or any other machine on the network.

  16. From a grizzled old security dude.. by Anonymous Coward · · Score: 2, Informative

    "You must be new here" is what comes to mind.. I get hundreds of these per HOUR on most of my boxes. It could be anything: a curious worker, a hacker, a virus, a script gone bad.

    First thing, check your important file checksums, run tripwire, or whatever. If you don't have a tripwire-like system set up, or a backup set you can compare against, you've got another problem, but let's assume somehow, you are sure your files were not compromised.

    Once you're sure no damage was done, relax, the system did what it's supposed to and rejected the traffic. Do a quick audit to make sure everything is up to date, you're not running any insecure junk, no version numbers are revealed, IDS signatures are up do date, and so on.

    It's likely just a virus or a hacked box.

    My algorithm for dealing with this is:

    if self.friend_of? other_business.admin
    self.contact other_business.admin
    else
    document anomoly
    possibly_firewall other_business.ips
    get_back_to_work!
    end

    In other words, it's NOT YOUR PROBLEM if the other guy is hacked. In fact YOU could be blamed for it (yes, this shit happens, people are idiots). DO NOT portscan or telnet or attempt to learn anything about the other box (which you already did, oops).

    Be sure to DOCUMENT everything. If you visit their web site, document it. If you call them, document who you talked to. Just document everything, even if you just file it away.

    Whether or not you contact management is up to your business culture, position, etc. In my opinion, it's your job to deal with this stuff and if you "escalate" every little port scan, you're just making a lot of useless noise. However if the other business is a competitor of yours, or there's some business impact here, or they've been doing it for months, you should tell your management.

  17. Autobanned? by phorm · · Score: 2, Insightful

    a) They'd have to know the IP's of the allowed machines
    b) The ban would only last 3 minutes.
    c) A 3 minute blockout is much better than an owned server :-)

  18. What A Manager Would Expect by reallocate · · Score: 2, Insightful

    My perspective is that of someone, in a past life, who hired network techs.

    If this happened in my organization, I would expect three things from my network people:

    1) Follow and stay within established policy; I would expect you to do what is needed to protect the security of the network short of attacking the presumed culprit. If it came to that, bring the network down. Attacking the apparent culprit puts my business at legal risk and you do not get to make that call.

    2) Notify me (management) as soon as possible. Give me all the facts and answer my questions. Lay out my technical options objectively. Explain to me why our network was vulnerable and how we can remedy that. Don't try to spin me so I under or over react. It is my network and you work for me; I won't take kindly to attempts to manipulate me.

    3) Then, follow my instructions.

    --
    -- Slashdot: When Public Access TV Says "No"