Handling Viruses in an Uncontrolled Network?

An anonymous reader asks: "Recently I've gotten a (volunteer) job looking after a small (approximately 500 computer) network, located within a large block of student flats. We've been having numerous problems with viruses over a few years. They spread like crazy on our network, with 100megabit connections in every residents room. Every so often they 'go off' and start a flood, which of course takes the entire residence network down. I've tried desperately to educate users on the virus problem, but those that are the problem don't care - they ignore every warning they get and just buy a faster computer to compensate for their systems sluggishness. As we only need two or three ping flooding computers to bring down the network it's hard to keep our network up whenever a worm starts its payload. What solutions have Slashdot readers came up with this and similar problems?" "Keep in mind that I'm doing this on a volunteer basis, and that my own study time and personal life takes first priority. The residence isn't prepared to spend more money bringing help or a replacement in, which I can understand given that I pay them rent that I would prefer not to increase. I also don't have any control over the network infrastructure itself, just over our DHCP server. I can't force users to keep their computers safe, as I don't own the things - all it seems I can do is point them to the *FREE!* virus scanner and local Windows update mirror and urge them to protect their computer, and offer to help out those that need it - (although due to time constraints, personally helping out everyone in a 500 member network isn't a possibility).

I can also email off a request to have certain IPs dropped off at the switch, but those users have to come back online soon enough. Whenever someone is infected I try and sit them down and make them realize that keeping their computer safe is their responsibility, and they always seem very attentive whenever we're discussing when they get reconnected to the network, but soon after they'll be infected again."

Comments

Is this really that hard?

[Scott Lockwood]

Hm... Seems pretty simple to me.

1. Have someone at the school make them sign something that says they will have virus protection and spyware protection on their machines, and that it be kept up to date. Failure to keep the machine clean can result in suspention of service at any time.
   
2. When one of them has an infected machine that starts pinging the shit out of your network, Unplug them!
   
3. Point to document mentioned in step 1 above when they start whinging about it.
   

There see, that wasn't too hard!

Re:Is this really that hard?

[fembots]

Is 1. "Profit!!" ??

Re:Is this really that hard?

[_Sharp'r_]

Or slightly faster:

1. IDS set to trigger on specific patterns and events (if you have been seeing this stuff on your network constantly, you'll know what to look for already.), you can even set some up free using FOSS.
2. the IDS alerts then trigger shutting down their switch port and notify an admin. Depending on your switch port mapping database, you can even email the user.
3. See Scott's post above for signature/cleaning cycle.

Re:Is this really that hard?

[Anonymous Coward]

But theproblem is these are students and they have work to do. by pulling their plug you are not allowing them to get the work done that they are I presume there to do. So thats not an option.

That's like saying you can't take a drunk driver's license away, because he needs it to drive to work.

His "need" is his problem, not yours. He should have thought about that, before he decided to engage in activity that threatened other people.

Fuck this whole "buy more filters" thing. Place the burden on the users, and then users who behave intelligently, won't have any burden. That is the fair thing to do.

Re:Is this really that hard?

[Chyeld]

To put this politely, if they can't be bothered to keep their system clean, they can't have access to the free network.

To say that 'they have work that must be done' is ignoring the fact that the umpteen (insert hyperbolic number) other users ALSO have work to that must be done and in this case the good of the many out weighs the good of the few or one (damn, did I actually find a good excuse to use that line?).

Yes, by all means, research methods to contain and control any outbreaks to reduce the issue when they do occur; but in this case prevention is far, far, more effective than mitigation.

Re:Is this really that hard?

[MrAnnoyanceToYou]

Ah... nah. I'd say,
"1. Write short document stating that in 'reparation for virus damage' computers would occasionally be confiscated when they managed to infect multiple computers connected to the local network
2. Notify them of this agreement and make them sign it
3. When one of them has an infected machine that starts pinging the shit out of your network, give them a 'first warning'
4. Point to document in step 1 kindly, in writing, and create yourself an Ebay account.
5. Repeat
6. Profit and learn to laugh evilly."

Re:Is this really that hard?

[tehcrazybob]

Indeed.

My school has a very effective setup for controlling outbreaks. To start, the network is MAC filtered. Any time you connect to the network with an unlisted MAC address, your browser is redirected to a page containing the university Terms of Service for the network. You read this information, toss in your university ID and password and click I AGREE, and the program adds your MAC to the list.

As outlined in the TOS, there are no warnings. If your computer exibits any viral behavior, your network access is removed. Unless your virus was email-related, you still have access to the mail servers. When you try to use the internet again, you are once again taken to a limited page, which politely tells you that your computer appeared to be infected with a virus. You are given basic cleaning information, as well as the tech department phone number and email address in case you need help. They can also provide you with tools like AdAware, since you won't be able to download these yourself. Then, once you are confident your computer is clean, you call the tech department, and they run a quick check to see that your computer is no longer showing viral activity. At this point, your network access is returned.

There are no warnings. As soon as you cause a problem, the problem (you) is removed. Once you fix the problem, access is restored. I don't know their policy for repeat offenders, but I assume there is something.

Re:Is this really that hard?

[Vengeance_au]

Thats just wrong. Here in Australia, you lose your drivers license for drink driving, and you are not behind the wheel of a car for however long you get pinged (6 months being the minimum). You can plead the case in court, but there are very few exceptions made.

I fully support this policy - you decide to risk MY life on the roads, you pay the penalty. Can't get to work now that you've committed a crime and are doing the "time"? Well, hopefully you will realise how important having a license is to your life, and you won't ever drink/drive again. And also, be thankful you didn't injure or kill another road user, pedestrian or even yourself...

To segue this back onto topic, same rules should apply in this situation. You put others at risk or deny them access to the network due to your inability to load a freely available, well publicised and mandatory on the network you are using tool, then you do the "time". Access cut off and you can't work? Well, perhaps next time you will ensure the virus scanner and firewall software is running, you won't have the issue, and those around you are not impacted.

No more access

[nizo]

Forcing people to have up-to-date virus/firewall software before they can even connect to the network would be a good start. Turning network connectivity off for offending computers/users for progressively longer spans of time after they infect the network seems like a good deterrent as well. I suppose posting the names of people who infect the network and bring it down might work, though the screams from the public beatings might make it hard for you to sleep at night.

DOOOMMMMED

[Anonymous Coward]

You are DOOOOMMMMMED.

Simple.

[Skudd]

Write your own virus to send them massive payloads of anti-virus software. :P

Re:Simple.

[jemenake]

You need more power. Otherwise you will fail in your job

Just about all of the posts thus far have been along the lines of "Pull the plug on the people who don't care until they *do* care.... IF the administration will let you."

Here's an alternative to the "IF the administration will let you" part. Make use of the fact that nobody else really understands what it is the wizard (you) does behind the curtain:

Implement whatever service termination solution you feel necessary (whether by writing/downloading some automated system, or by doing it manually yourself). When the offender calls to complain, *don't* say that they were shut off administratively. Tell them that the massive traffic from their machine "overloaded" the port they were on (tell them it's kinda like a circuit breaker on house wiring).

They'll say that this never happened before. Tell them that they've got a newer, more-aggressive virus.

They'll ask that their port be "reset". Tell them that, due to all of the machines that they helped infect, and to the convoluted process for "resetting" a port, there's a backlog of a couple days before you can get their port reset.

Maybe they'll ask if you can just plug them into a different port. Tell them that they're all maxxed out.

At some point, Administration might ask why this is happening. Tell them the same thing you told the users... new, nastier viruses. They might ask what new equipment they could get to fix the problem. Tell them that the BFS-9000 can do it... but it's very... very expensive. It would be much cheaper for everyone to just use virus protection.

Uncontrolled Viruses

[Anonymous Coward]

Have you considered spankings? At least for the hotter co-eds. After all, they should know better.

3 Strikes policy?

[fahrvergnugen]

It sounds like you've been completely neutered. If at all possible, talk to the administration about instituting a "3 strikes" policy. That is, if someone's computer causes a network-wide issue 3 times, their network drop stops working for the remained of the year.

That'll clean their acts up in a hurry, or at least make your life easy.

Re:3 Strikes policy?

[lakeland]

Remainder of the year probably isn't smart in an environment that previously has seen no enforcement. I'd be using a sliding scale with punsihment at each stage in order to get people used to the idea that you are serious.

Something like: first offence, 24hr ban; second offence, 7 day ban; 3rd offence, 1 month; 4th offence, one year and an email to all 500 with the photograph of the person who has been stuffing up their computers.

Once you've got people used to the idea they will be punished you can swap to something like the 3 strikes policy. But at first you're going to get idiots testing you, and so two warnings is too soft while a year-long ban is hellova hard for a first punishment.

There are alternatives of course. Install an 802.11g network in parallel with strict rules. Disobey them once and you get a stern warning, twice and you're banned for life from it. That way you'll naturally see people migrate to the network which 'works' without the fight with idiots.

Oh, I'm assuming this is targetted at teenagers at or near college level. If you're dealing with mature adults then it is much easier.

Move out?

[Eezy Bordone]

Seriously, volunteering to be THE on-site tech support for 500+ users is insane, especially since you're not even getting a discount on your housing. Quit the job or move out so you can worry about your own network.

Ban them

[nadamsieee]

Isolate the computers that are spreading the virus and shut down their access to the DHCP server based on their MAC address. Then make the reconnect process as painful (yet educational) as possible. >:)

Seems simple enough.

[FyRE666]

If you can't put the bad users on a slow switch, and force them through an even slower proxy to make their life hell, then see if you can't organise a minimum disconnection period. Say 10 days or so to reconnect the idiots who keep getting infected. Since you control the dhcp server, you could filter them out by their mac address so they can't wander over to someone elses room to connect. Yes, they could probably circumvent this with a little knowhow, but let's face it, an idiot who's managing to become a virus writer's bitch every week isn't likely to have too much in the way of technical knowledge...

A recent Poll...

[Shadow Wrought]

Regarding revenge might help you come up with, shall we say, colorful solutions to your problem. Either that or figure out a way to have all of their papers "lost" due to the virus;-) In this regards, I would suggest that you channel your inner BOFH.

Wasting your time

[ibpooks]

It really sounds like you're wasting your time.

You don't have control over the users, the machines, or the routers; so what the hell can you expect to do?

Sounds like the best option is to unplug the offending machines from the patch panel until they can demonstrate they are virus-free. Although that is likely not a viable solution if these are paying customers.

Re:Wasting your time

[Knara]

It can be viable if the students had to sign an AUP from the campus IT department when they moved in (which I get the feeling is fairly common these days on major campuses). Worked at a place where they just turned off the switch port of offending machines, and then if the student wanted to get access back, they had to call in to the help desk and go through the process of setting up a schedule technician visit, which may be pretty far off depending on the time of year.

Was kinda hairy the first couple weeks of fall semester for the techs and the helpdesk (which will happen no matter what), but very few repeat offenders.

Stop volunteering

[lelitsch]

Seriously, it seem like this is an unsolvable problem and neither the users nor the administration seem to want to spend any effort in fixing it. So the sooner you realize that there is nothing you can do, the better. Help out with the IT system at your local Humane Society, womens shelter, or similar instead.

Oh, and get your own DSL or cable modem.

Simple.

[grasshoppa]

You need more power. Otherwise you will fail in your job ( unless you take to violence ).

Students need to be kicked off the network until their computers are clean. If they are kicked off x times, they are off until they come to you and sign a form saying they understand how to keep their computer clean. y more time(s), they are off for the rest of the semester.

Simple, effective. You will need a couple decent switchs capable of shuting down ports ( or you could just yank the wire ).

If you don't have this level of power over the network, get rid of any access you do have. The higher ups only want a scape goat.

DHCP server is all you need.

[strredwolf]

Just reconfigure the guys that keep spewing to ether deny access, or return that the computer's IP address is 127.0.0.1.

When they come in complaining, babysit them at their computer.

Stage virus drills

[bigtallmofo]

Send them emails with executable attachments. If they click on the attachments, ban them from the network for a week.

Send these out frequently. Soon they'll instinctually hit the DEL key when something with an attachment comes in.

Sure it's an option

[CarrionBird]

by clogging the network, they prevent other people from doing thier work. It's standard procedure at some universities to shut off the ports of problem systems.

Re:Sure it's an option

[David Horn]

I work as a "student advisor" at Leeds University and every student is issued with a free license to McAfee Virusscan Enterprise.

When connecting for the first time, they have to enter their university username and password so the IP address can be tied to their MAC address and the computer logged.

If their software detects viral traffic from their PC, they're automatically cut off from the net and a webpage comes up explaining why. They don't get re-connected until myself (or one of my colleagues) verifies they have virus scanning software installed and their PC is clean.

First few weeks of term there were a lot of people cut off, but virus infections now are next to nothing because everyone has the software running.

Apart from this, the internet connection here is extremely good. Fast and reliable, and no port blocking.

Egress filtering

[MoogMan]

The idea is simple: Egress filtering.

Strict policies on outgoing traffic for untrusted networks is essential.

I would suggest a default policy of something like www, ssh, msn/aim im, p2p programs (possibly, depending on the uni's rules and regulations).

Providing you have a mechanism for giving the students access to other ports when necessary, then there should be no problem enforcing a strict egress policy.

NetReg

[DA-MAN]

I also don't have any control over the network infrastructure itself, just over our DHCP server.

With this you have all you need to run a NetReg server within your infrastructure. With this you can allow users to register their machines automatically. Any user with a virus or other such malware gets their dhcp entry deleted, and they are on a private network that goes to where you define. I would allow antivirus sites, antispyware sites, and windowsupdate only (or better yet, a local mirror).

Have them send an e-mail to user@host once this is complete and you can re-activate their lease.

Re:NetReg

[vco123]

1. With DHCP and Netreg, you do control the network. Keep your registered leases short ( 2 hrs ).
2. Be sure to disable external DNS calls at the router ACL, to force people to use Netreg.
3. Run 2 instances of BIND with Netreg and selective DNS forwarding to allow Windows Updates, LiveUpdate, IT Support and Spyware. ( see Netreg-l from last August).
4. Bump infected computers out of registration, so that they can't phone home as easily. Alternatively, use groups with ISC DHCP to force an infected MAC to use the Netreg bogus DNS to "quarantine" them.
5. If you can, ask the network dudes to disable 25,135,445/tcp for your unregistered IP ranges. That'll limit the infected PC a bit.
6. If you start to see a virus frenzy, shut ports off fast. It'll save time later.

I've run a 4000 computer RezNet this way for 4 years.
As to infected computers, I'm working on a Netreg extension that includes a "Your're infected" group. It's like being unregistered, but DNS forwards to a virus notification page.

Re:You are in control!

[courcoul]

Amen to that! Or, it just may be that his post is only the ceremonial position of "official scapegoat" that takes the fall when the poop really hits the propeller blades... Short recipe for the cure (provided he IS the admin):

• Get an extra PC on the backbone of the network, so it can monitor all the traffic. Anything bigger than a x486 is good enough, say with 128MB or more of RAM.
• Install OpenBSD ( http://www.openbsd.org/ ) on it (most hardened free OS around, so the hackers can't take you down so easily).
• Install SNORT ( http://www.snort.org/ ) on it. Configure to work as a network IDS and keep it up to date with the latest vulnerability/virus plugins.
• Once SNORT gets wind of an infected machine, set it to do one of three things:
• • If you have the tech skills to set it up, have SNORT block out the switch port where the offending PC is plugged in AND send you a message. When the owner cleans up their act, reactivate the port and restore connectivity.
  • Else, have SNORT send you a message with all the details and YOU do the port blocking, if you can. The rest proceeds as above.
  • Else, have SNORT send you a message so you can bitch whomever has the capability to block the port. The rest proceeds as above.
• If your authority is so puny that you cannot do any of these things, you could resort to sending out a mail to all the rest of the users of the network, and letting them know who the miscreant screwing up their connectivity is, and let peer pressure do its thing...

Good luck!

Why does the network go down?

[g-san]

Have you figured out exactly why a few infected computers is bringing down your whole network? I could see if they are scanning local subnets, you would have a lot of broadcast ARP packets. If they are scanning remote network IPs, you may be filling up a cache on the outbound router. Are you sure you don't have a few people just playing with NMAP? Is it inbound traffic or outbound? Identify the nature of the traffic when the network implodes, look for a pattern, and see if you can mitigate that. Use ethereal for that.

This is a *switched* network isn't it? Hopefully yes, and with a firewall also. I really can't see why someone would need inbound tcp/135,137,138,139,445,1025 or udp/135,1026-1029 nowadays. That would prevent malware that is not spread by email or Explorer. I won't recommend you dictate the browser or email client people use, but it's a possibility to have a outbound web proxy not forward any requests from IE.

You might also want to look into snort, you could at least have it alert you when the problem starts, or shut down ports, but sounds like you have not had much luck with that. Note rather than drop people off the face of the earth, at least make sure they can get to antivirus sites and microsoft updates. This is tough without access to the infrastructure but would improve things.

Another suggestion is if you do not have alot of room to room traffic, and you do not have a 100mb conenction to the net, configure all ports to 10mb. At least that way it takes more than 10 users to flood your 100mbit backbone. And users accessing the net are always throttled by your outbound connection so they won't know the difference.

I assume you volunteered for this because you like like this stuff. Note that if you *did* spend more time on this problem than your schoolwork, and came up with a solution, you might not even need to finish school.

Ok....

[Audacious]

We've heard from the:

1. "It can't be done" crowd.
2. "Be tough about it" crowd.
3. "Go behind their backs" crowd.

and others....

How about this:

1. Get everyone's e-mail address so you can send all of them e-mail at the same time. How do you do that? Ask them to e-mail you - that's how. Of course, disinfect anything they send you because they probably will have a virus or two.

1a. How do you get all of them to send you the e-mail? Go buy some of those blank business card sheets (Avery I believe makes these), print up your message, get someone to help you break them apart, and then just tape them to each person's door. In this way you: 1)Don't have to talk to them, 2)Don't try to force them to do what they don't want to do, and 3)Can do it on your own time (like on a floor-by-floor basis). Cost: Probably about $10.00.

1b. Your message? It should be something like:

Dormitory SysAdmin needs your help!
We need your e-mail address as we
are trying to remove viruses and want
to be able to keep you informed. Thanks!
myemailaddress@thedorms.edu

1c. Put notices on doors leading into the dorm and/or bulletin boards also asking for e-mail addresses. If you can, have someone hand the things out to people as they come in and out of the dorms.

2. Set up a blog where everyone can meet and talk about problems. Use the e-mail addresses to send your notice out about the blog and how to access it.

3. Set up appointments with people to meet with them to show them how to protect their system from viruses, ads, cookies, and other problems.

Ok, let's say you've gotten some responses and want to start to go to other people's rooms to help them out. You want to:

4. Use the scheduler built in to every operating system currently in use (ie: Mac OS X, Windows98se and up, Linux, BSD, Solaris, etc...). For those OSs which are older (although I can't see anyone currently in college using an Apple ][+ or even Mac OS 9.x or earlier) download and bring with you some sort of a scheduler. (Even the Apple //gs had a CRONTAB program!). Set their machine up so it automatically, every day, trys to download the latest and greatest updates for the OS, SpyBot, AdAware (or whatever you use), your virus protection program, etc.... The MOST IMPORTANT THING THOUGH - is to always explain what it is you are doing to the person's computer. Don't just dump a bunch of things onto their system. Bring a flyer that explains what it is you are doing and why. Set their system up so they can win and so they don't have to rely on you to be there to make everything function correctly. All of the virus/cookie/ad checking software out there can be set up to function on its own. Some of them (like most virus checkers) have their own scheduling software built in.

4a. NOW! Here is the important thing! Set the virus/ad/cookie (or VAC for short) to AUTOMATICALLY e-mail you with the results. This too can be done via the scheduler. Give the automatically generated e-mail a special header (like [VIRUS|AD|COOKIE] REPORT FOR ROOM X). There are e-mailer programs for all operating systems which run from the command line. So just make a little batch program/shell script to create your report and e-mail it to you. Again, write it all down in the flyer you are going to give them so they don't freak when their system suddenly starts doing things (like checking for viruses or sending e-mail).

4b. Most virus software's report will read "VIRUS FOUND" and then tell you where and when the virus was found. Write yourself a short Perl/PHP/C/ script which will read these e-mails and sort out which one have viruses and which ones don't have them. Since you made the title have the room number on it - you automatically know who is having problems. So you can e-mail them back and set up a time to go over to fix any problems they might be having. Further, you can produce statistics on where the greatest problems are and post these fi

You are NOT punishing the wrong person.

[arete]

You should certainly punish the virus writers, if you can catch them. And you should possibly punish M$ for how big of a hole IE still is, even if Windows itself is better than it used to be. But none of that matters.

To use society's resources, you have to follow society's rules. I can go buy any car I want and drive it at 200 mph - on my own track. But if I want to drive on streets I have to follow the rules, as they apply to my actions (hitting things) even when they may not necessarily have a direct negative impact (speeding, driving on the sidewalks) have only a paper impact (licensing, insurance, registration) or only a preventative impact (headlights, brake lights...)

I can also go buy a used car and have the brakes suddenly fail, running over someone's garden. Note that even if I didn't know, I'm still responsible for the cost of that garden, (unless I JUST bought it and can pass the blame to the previous owner) If the brakes were recalled, it's still my fault for not getting them fixed. If they WEREN'T recalled, but should've been, then that's not my fault.

If you're already providing appropriate, simple, free, publicized resources _that they didn't use_ they are being negligent at best. Kicking them off until sometime after they fix it is a MINIMUM penalty for such negligence.

Argueably they should have to pay for the cost of your time to fix their computer (mandatory since they didn't do it the first time) and to repair any problems caused by their problem - and STILL be penalized in terms of being online.

(Personally I believe that a kick-until-fixed first warning is probably a necessary threshold of publicity - but even the second time they aren't listening I think it'd be very reasonable to escalate it.)

To be clear, I don't think it's reasonable in today's world to hold them accountable for anything their computer does. I think it's NECESSARY to hold them accountable for not following your security procedures to defend against it. Which means you're still going to be snuffed by the virus that exploits the OS hole noone has put out a patch for yet - and I wouldn't blame that on the first kid to get it.

I agree with the other posts - you have to get kick/ban/unplug authority, you have to quit, and/or you have to get paid. 1 of those might do...

netsquid software package works well for this

[gabesk]

This is the method used at Texas A&M University, which I attend, for their residence hall network.

We use netsquid, http://netsquid.tamu.edu/, which is essentially some code that ties into snort to provide automatic filtering by mac address and notification.

It works quite well.