Slashdot Mirror
Handling Viruses in an Uncontrolled Network?
An anonymous reader asks: "Recently I've gotten a (volunteer) job looking after a small (approximately 500 computer) network, located within a large block of student flats.
We've been having numerous problems with viruses over a few years. They spread like crazy on our network, with 100megabit connections in every residents room. Every so often they 'go off' and start a flood, which of course takes the entire residence network down. I've tried desperately to educate users on the virus problem, but those that are the problem don't care - they ignore every warning they get and just buy a faster computer to compensate for their systems sluggishness. As we only need two or three ping flooding computers to bring down the network it's hard to keep our network up whenever a worm starts its payload. What solutions have Slashdot readers came up with this and similar problems?"
"Keep in mind that I'm doing this on a volunteer basis, and that my own study time and personal life takes first priority. The residence isn't prepared to spend more money bringing help or a replacement in, which I can understand given that I pay them rent that I would prefer not to increase. I also don't have any control over the network infrastructure itself, just over our DHCP server. I can't force users to keep their computers safe, as I don't own the things - all it seems I can do is point them to the *FREE!* virus scanner and local Windows update mirror and urge them to protect their computer, and offer to help out those that need it - (although due to time constraints, personally helping out everyone in a 500 member network isn't a possibility).
I can also email off a request to have certain IPs dropped off at the switch, but those users have to come back online soon enough. Whenever someone is infected I try and sit them down and make them realize that keeping their computer safe is their responsibility, and they always seem very attentive whenever we're discussing when they get reconnected to the network, but soon after they'll be infected again."
I can also email off a request to have certain IPs dropped off at the switch, but those users have to come back online soon enough. Whenever someone is infected I try and sit them down and make them realize that keeping their computer safe is their responsibility, and they always seem very attentive whenever we're discussing when they get reconnected to the network, but soon after they'll be infected again."
myswitch> (enable) set port disable
-dk
Dream with the feathers of angels stuffed beneath your head.
Just reconfigure the guys that keep spewing to ether deny access, or return that the computer's IP address is 127.0.0.1.
When they come in complaining, babysit them at their computer.
--
# Canmephians for a better Linux Kernel
$Stalag99{"URL"}="http://stalag99.net";
Definitely cut people off when they are infected until they are cleaned up.
Hit them in areas they care about and they'll start being more careful. Figure out where those motivational places are (disconnections, fines, losing IM privileges, etc.)
Post a policy that has escalating punishments for each subsequent time they are infected, particularly if it's obvious it's their fault. This could be a rising fine, or that you don't reconnect them as fast... If they are disconnected 1 day for first offense, 2 for 2nd, etc, they might learn real fast.
You could also consider cutting off certainly protocols at the firewalls or for particular users, either for security or as punishment.
Nah, it's a volunteer position, no real power and no pay. The guy probably isn't even allowed to ban MAC addresses (I'd bet), and any policy he comes up with is probably just going to be taken as a "suggestion" by the unwashed masses. If possible, try to come up with a system for re-establishing connections that conveniently "forgets" chronic offenders. If they can't get their daily dose of SWG (or whatever the kids are playing these days), they'll come into line soon enough. Make network problems their problem. If you've got a machine that's ghost pinging away, throttle them down at the switch, or "accidentally" block their IP at the firewall. If you get really frustrated, try making it fun for yourself. I mean, you've got a pretty good idea of whose machines are vulnerable, right? I'm sure you can find some creative way to relieve your stress with a list of vulnerable IPs and open ports. Not that I'm advocating any illegal or unethical activity or anything. *wink wink*
Set the bar high, then bring a tall ladder.
First off - something that EVERYONE should be doing - make sure spoofed packets dont leave your network. This helps you, and it helps those of us (like me) who run websites who are frequent victims of DDoS attacks - you just may reduce my DDoS from 3Gbit/sec to 2.9Gbit/sec :)
So... you know your internal addresses. You know your external addresses. At the external firewall, block all packets going out that don't have a matching source address in the header. Most all virii nowadays use spoofed headers to hide the actual source - simply block packets that match this criteria.
Second, you can use QoS at the firewall level to prevent one computer from using more than their share of bandwidth. Nearly all firewalls (even open source Linux and BSD solutions) offer quality QoS.
Third, you can identify virii that cause issues, and detect them - usually they are built with backdoors on a certain port - check for that port being open, and block their access.
Fourth, institute a punishment for students who don't fix their issues. One warning, then they lose access for a period of time. This needs to be their responsibility - just make sure that help is available to students who can't protect themselves, perhaps a student IT club can help them or something like that.
Depending on how sophisticated your switching hardware is, you might be able to implement QoS there, to prevent a single system from flooding the network. Additionally, you may be able to simply throttle back each port (if you have a 100Mbit uplink to the internet, set each port to negotiate only at 10Mbit).
Also, choose software packages for different platforms that you can recommend they use to fix any problems that arise - standardization makes management easier.
If you have the budget for it, you could look into locally placed firewall boxes whose focus is to detect and eliminate virii - they're expensive and less common than your standard SonicWall box, but can be found. Might be a last resort unless you have deep pockets.
Good luck!
by clogging the network, they prevent other people from doing thier work. It's standard procedure at some universities to shut off the ports of problem systems.
Free Mac Mini Yeah, it's
The idea is simple: Egress filtering.
Strict policies on outgoing traffic for untrusted networks is essential.
I would suggest a default policy of something like www, ssh, msn/aim im, p2p programs (possibly, depending on the uni's rules and regulations).
Providing you have a mechanism for giving the students access to other ports when necessary, then there should be no problem enforcing a strict egress policy.
I also don't have any control over the network infrastructure itself, just over our DHCP server.
With this you have all you need to run a NetReg server within your infrastructure. With this you can allow users to register their machines automatically. Any user with a virus or other such malware gets their dhcp entry deleted, and they are on a private network that goes to where you define. I would allow antivirus sites, antispyware sites, and windowsupdate only (or better yet, a local mirror).
Have them send an e-mail to user@host once this is complete and you can re-activate their lease.
Can I get an eye poke?
Dog House Forum
Students in our dorms have no need for Microsoft ports, which is the primary reason worms can take down the network. So i block port 137,138,139,445 at the switch port level.
Granted this doesn't solve the virus problem on the computer, but it sure does prevent it from taking down the rest of the network.
That's not an easy fix at all. Who are you kidding? If you had to spend less than 5 minutes a week with each computer that's already over a 40 hour work week right there -- and I doubt any solution is that quick. You're not understanding the numbers involved here -- and that's not including travel time, plus being able to meet then on their schedule. Ain't going to happen with student users on broadband who feel it's their God-given right to abuse.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
- Get an extra PC on the backbone of the network, so it can monitor
all the traffic. Anything bigger than a x486 is good enough, say with
128MB or more of RAM.
- Install OpenBSD ( http://www.openbsd.org/ ) on it (most hardened
free OS around, so the hackers can't take you down so easily).
- Install SNORT ( http://www.snort.org/ ) on it. Configure to work
as a network IDS and keep it up to date with the latest
vulnerability/virus plugins.
- Once SNORT gets wind of an infected machine, set it to do one of
three things:
-
- If you have the tech skills to set it up, have SNORT block out
the switch port where the offending PC is plugged in AND send you a
message. When the owner cleans up their act, reactivate the port and
restore connectivity.
- Else, have SNORT send you a message with all the details and
YOU do the port blocking, if you can. The rest proceeds as above.
- Else, have SNORT send you a message so you can bitch whomever
has the capability to block the port. The rest proceeds as above.
- If your authority is so puny that you cannot do any of these
things, you could resort to sending out a mail to all the rest of the
users of the network, and letting them know who the miscreant screwing
up their connectivity is, and let peer pressure do its thing...
Good luck!Had this in tradeshows for years. If you cannot control both Layers 2 & 3, forget it.
You need to AT LEAST be able to login to the switches/routers to read MAC tables at the instant there's a problem. ARP would be nice too. You need make no changes, but read-only in non-negotiable. Otherwise give up the job.
Once you have that, you can perfect the steps to find out what's happening when it's happening. THEN you may use whatever eloquently violent steps others are suggesting.
A b/w mgmt appliance would also be a smart investment, they can provide unusual evidence that's remarkably useful. (We'd look at the top talkers, when TCP sessions >800/5 min, we'd know we're lookin' at a naughty person.)
If your responsible for an improvement of the situation, and you're not given the tools, then resignation is the only course. Sticking it out with your hands tied is pointless torture: you'll never get a break, and the torturer will get tired.
having worked for a university where they had to manage some 30000 people, just about all of whom had computers, the solution is basically the same thing everyone here is saying. There is very active monitoring several levels deep into the network. If a computer is sending out crap and flooding the network or otherwise being a PITA, it is disconnected. The port is just automagically shutdown and a note is made in a DB so that when the call comes in to the hell...help desk, the poor slave... technician knows what it's about and can direct the user to a place to pick up a CD-R with the "Cleaner of the Week" for whatever virus it is this time. Repeat offenders have a very rough time indeed getting back on the network. In addition, the dorm buildings each have a *psuedo* router. I'm not exactly sure who makes it (I wasn't in that part of the staff, I was on the IT team for some scientests), but the joy of it was internal connections are fine, outbound from the dorm connections get QoS'd to hell and back again.
One side note, when you do start pulling the plugs make sure you've got the following lined up.
1) Management authorized you to have that kind of power. In writting.
2) Buy a weapon. Seriously. I kept a live blade sword in plain view behind my desk just in case. Some of these college students / scientests are friggin nutz.
3) Stock up on some booze. Patience can be easily recharged with the right liquid beverage ^_~
hth
I forgot to mention, we used ettercap to detect attacks.
Ettercap:
http://ettercap.sourceforge.net/
Netreg:
http://www.netreg.org/
Netdisco:
http://netdisco.org/
Have you figured out exactly why a few infected computers is bringing down your whole network? I could see if they are scanning local subnets, you would have a lot of broadcast ARP packets. If they are scanning remote network IPs, you may be filling up a cache on the outbound router. Are you sure you don't have a few people just playing with NMAP? Is it inbound traffic or outbound? Identify the nature of the traffic when the network implodes, look for a pattern, and see if you can mitigate that. Use ethereal for that.
This is a *switched* network isn't it? Hopefully yes, and with a firewall also. I really can't see why someone would need inbound tcp/135,137,138,139,445,1025 or udp/135,1026-1029 nowadays. That would prevent malware that is not spread by email or Explorer. I won't recommend you dictate the browser or email client people use, but it's a possibility to have a outbound web proxy not forward any requests from IE.
You might also want to look into snort, you could at least have it alert you when the problem starts, or shut down ports, but sounds like you have not had much luck with that. Note rather than drop people off the face of the earth, at least make sure they can get to antivirus sites and microsoft updates. This is tough without access to the infrastructure but would improve things.
Another suggestion is if you do not have alot of room to room traffic, and you do not have a 100mb conenction to the net, configure all ports to 10mb. At least that way it takes more than 10 users to flood your 100mbit backbone. And users accessing the net are always throttled by your outbound connection so they won't know the difference.
I assume you volunteered for this because you like like this stuff. Note that if you *did* spend more time on this problem than your schoolwork, and came up with a solution, you might not even need to finish school.
fwiw, Firewalls don't protect against most viruses. They'll stop people from getting directly owned over some RPC port from the internet, but they won't stop people from getting owned by some malicious webpage they visit.
The only way to keep a Windows computer safe is to install patches and virus protection software on the individual computers. Work *must* be done on the individual computers.
At my school, there were paid student techs that fixed stuff like that. These guys need someone who will walk from room to room, fixing computers. Doesn't sound like a volunteer job to me.
There are no trails. There are no trees out here.
The student co-op where I lived had around 150-170 machines on the network at any given time. We required each user to 'register' through a php form on the local administrative box. Until the user had registered a given machine (mac address) we redirected all web traffic to the 'you must register to use the internet' page.
We generated id keys for each house member ahead of time and required that they have this key to register. When the user came to get the key we gave them a quick overview of what they should and shouldn't do and introduced them to the software cache on the local network (free AV software, firefox, ad-aware, etc..).
Once the user had the registration key in hand they could go back to their room & register their machine in their name (or any number of machines), we then cleared that MAC address for access to our dhcp server.
The benefit of forcing registration is that we knew who owned each machine and where the person lived. If any virus or trojan was bad enough to endanger network we could go to the switch for that person's floor and pull the plug on their connection.
Alternately if a machine on the network started spewing virus payloads we could just revoke dhcp access and boot the offender off the network - we didn't have to worry about notifying them of virus infestations, we could wait for them to come to us saying "my internet doesn't work, can you fix it?"
I'm a student at the University of Waterloo (Ontario, Canada), and they have a simple solution.
When you get to residence, you sign a form that says you agree to monitor your computer, keep it clean of viruses, up to date with Windows update, et cetera. The terms are made very clear in it. No agreement, no use of the university network.
On your first offence (banned p2p, virus, anything like that), your network drop is disabled until you pay $25 (Canadian dollars; cue jokes about 2 cents USD) and sign a form acknowledging what you did wrong and that you will take action to avoid it in the future. In addition you have to clean up whatever triggered the disconnect in the first place.
Second offense? Disconnected for the rest of the term. That's the end of that.
Hope it helps!
Join the Empire! http://www.empirereborn.net/
Start your documentation with "Connection to the in house network is a privilidge, not a right." get them to sign a "take reasonable steps" form. Hand them a bunch of URLs pointing them to the freebie stuff.
Now, because you have access to the DHCP server, why not assign IP address based on MAC address, and set the lease time to something low (say 30 minutes)? If there is an offending computer, assign them a "jail" IP address that only allows them contact with the patch server. Once they have patched up their system, and added antivirus software, you take them out of "jail". It isn't perfect, but it will cause you a whole bunch less headaches.
A sig is placed here
To display how futile
English Haiku is
Indeed.
My school has a very effective setup for controlling outbreaks. To start, the network is MAC filtered. Any time you connect to the network with an unlisted MAC address, your browser is redirected to a page containing the university Terms of Service for the network. You read this information, toss in your university ID and password and click I AGREE, and the program adds your MAC to the list.
As outlined in the TOS, there are no warnings. If your computer exibits any viral behavior, your network access is removed. Unless your virus was email-related, you still have access to the mail servers. When you try to use the internet again, you are once again taken to a limited page, which politely tells you that your computer appeared to be infected with a virus. You are given basic cleaning information, as well as the tech department phone number and email address in case you need help. They can also provide you with tools like AdAware, since you won't be able to download these yourself. Then, once you are confident your computer is clean, you call the tech department, and they run a quick check to see that your computer is no longer showing viral activity. At this point, your network access is returned.
There are no warnings. As soon as you cause a problem, the problem (you) is removed. Once you fix the problem, access is restored. I don't know their policy for repeat offenders, but I assume there is something.
Computers need to explode more often.
The most powerfull goal you have here is to segment your network.
You can do this strictly through the DHCP server by using several scopes.
Pass out the following IP's and give your main gateway multiple IP's, or have a machine act as proxy (with multiple gateway ip's for your lan's).
With enough segments, you can isolate problem PC's down to groups of ten or less depending on how you break up your private (or even public) ip's. This will make the majority of others users on your network unroutable to malicous virus's.
Just make sure your gateway (the one with all the .1 IP's for each segment) doesn't route traffic through itself to the other segments.
Gateway = 172.30.1.1, *.2.1, *.3.1, *.4.1, etc....
172.30.1.1 255.255.255.0
172.30.2.1 255.255.255.0
172.30.3.1 255.255.255.0
172.30.4.1 255.255.255.0
etc........
If you have a minimal budget, and your users dont need public IP's, you can buy a bunch of SOHO routers... for about 10-15$ a piece.... 300$ can get you 20 linksys's....
put 25 users on each linksys (with the WAN ports connected to your gateway).... and your users cant directly attack each other (except for the smaller networks behind the linksys's.
If your users have no need at all for direct access to each other... just set out your scope as 255.255.255.255.
192.168.1.1-255 / 255.255.255.255 gateway: 192.168.1.1
now you r users can only reach the gateway and themselves.
As to email virus's, with DHCP you can force traffic to move through any machine you like, and set up a proxy between your "real" router and the network.... that proxy can filter port 25.... looking for viral email.
These solutions arent perfect, but they will greatly slow down propagation across your network, allowing you to respond much faster to problem children without having one bad computer infect everyone else. --VISION
--Idiots, Every single one of YOU, A flaming mass of conglomerated morons, hey wait a second, isnt that how RAID works?
If all you've got is control of the DHCP server, your hands are pretty tied. I would suggest setting up fixed leases and BOFH'ing students into submission. Kill the lease of infected machines, then bring 'em back once the infected system is clean. You don't have to be a dick about it, just bring the system back on at your leasure. Of course, you've got class all day and an exam tomorrow, oh and you're going home for the weekend...
Make it clear in polite, simple terms what the users responsabilities are, what will happen if they don't keep their system clean, and why you have to take the action you do. Maybe put together a standard "so you fucked up your system and got kicked off the network" sheet. Educate as much as possible. Yes it feels like you're talking to a wall. But the users will either evolve (get sick of being off the net) or die (find other ways of getting their computering needs met.)
Some people have suggested Microsoft SUS. You need to be able to apply a group policy, or make registry changes on the remote machine. Since you're not inchage of the domain controller, this is a moot point. Also, SUS only works on XP and 2000, so it may not help all users.
There are some people that if they don't know, you can't tell 'em.
This is the method used at Texas A&M University, which I attend, for their residence hall network.
We use netsquid, http://netsquid.tamu.edu/, which is essentially some code that ties into snort to provide automatic filtering by mac address and notification.
It works quite well.
I have been working on a similar network for some time, and dealt with similar problems. I don't know if these are optimal solutions, but here is how we are doing it:
/ONLY/ connect to the DNS server and the HTTP/HTTPS proxy server. This server provides the user with a message about the computer being infected, links to several sites with patches, free AV and updates. And a note that they will have to contact an administrator to get access renewed. The user can continue browsing freely, but don't do anything else. If they want to get back to the usual network they have to clean up their computer.
First of all, we have build a simple management system based around SNMPv3. You want this. Take a course in enterprise management or read up on it yourself. The day you stop writing scripts and use a management system instead is the day when you begin to come out on top of the problem. OpenWBEM can be a start if you want to know what can be done.
Here is our setup:
Incoming connections are blocked. There has been a discussion about removing this block and allowing "safe" ports. At the moment the issue is rather pointless as we are behind a NAT due to lack of IP space. Outgoing connections to DNS, SMTP and HTTP/HTTPS are filtered to force people to use our servers. Some of the more notorious p2p protocols are capped to keep the bandwidth usage from going insane.
We have a central register of users. To use the network you have to register and pay a symbolic sum each month. Then you get access to the connection in your room. You are responsible for what happens from your connection. This register gives us an easy way to contact users. To be allowed to join the network you have to sign a paper stating what you are allowed to do and not do. Our TOS are pretty restrictive, but without them we wouldn't be able to manage the net.
After some network outages (Code Red...) we have implemented a quarantine VPN. We have several IDS spread out, and if they detect a computer spreading malware they move the computer to the quarantine VPN. On this VPN the computer can
We also have several special checks for "evilness", most important rouge DHCP servers and ARP spoofing. Anybody caught by these simply get their connection pulled until they have explained themselves. Administrators are notoriously slow when it comes to returning connection to people knowingly doing malicious things on the network.
Hi,
:)
Here is how we deal with this issue on our 225,000 user unmanaged MAN (we are a large urban K12):
We use all managed switches, an IDS lets us know when a PC starts acting up (at least if it's a virus that produces traffic, which seems to be the norm these days) we use Nessus to scan the host, which is usually not running any personal firewall, see if we can contact the person directly (name or room number in the netbios table) and if that fails, shut off their port in the switch that serves that part of their building. In extreme cases we have turned off entire rooms, floors, and even a whole 3000 student highschool at one point. This tends to get people (read: the LAN folks and the users) to understand that they are actually on a network with other people.
You might want to play with hogwash http://hogwash.sourceforge.net/oldindex.html (I have not personally used this, we have a similar (commercial) device that does this kind of thing) and see if that will help you drop some of the outbound traffic/identify infected hosts. Of course regular snort can be configured to modify iptables so you can automatically deny infected hosts net access.
We are at present 4 months away from having managed office systems (insha'Allah) and 4 years from seeing them out in the schools. It's going to be a long, tough, fight... Gee, thanks, Mr. Gates.
peace,
jcw
PS: eeye has a bunch of free scanners for windows machines, and there is ample documentation on IDS and scanning solutions "out there". I find that knowing your current level of risk and where your problem users are (i.e. where things are likley to start) makes work a heck of a alot less stressful.
I work with exactly the same situation, helping maintain a halls of residence network where machines are owned by the students. We have a the following setup which seems to work pretty well:
:p)
:-)
1. the switches drop any traffic between machines in the network to stop malicious traffic propagating, (except to the server obviously
2. all students data quantities are monitored so if a student is using a large amount of bandwidth consistently over a number of days an enquiry is made into whether the student is aware that they are sending/recieving a lot of data. If they were only downloading linux distros or something thats fine, however if they were only checking email then they machines connection is blocked until a virus scan is complete and the machine is fixed.
3. Regarding security, a CD and infosheet is handed to users on arrival to the halls with a slip they have to sign saying that if their machine is found to be sending viruses/spam etc then it will be disconnected from the network until it is fixed (by them). The CD contains Spybot/Adaware and AVG antivirus for those who don't have antivirus software.
4. Ports access is heavily restricted, no p2p traffic for example. (I'm from the UK and the laws that were explained to me are that if a company/organisation runs a network which is engaging in illegal activity then the company is just as liable for copywrite theft as the users are, as they are responsible for their network and must take "reasonable" actions to prevent it)
As a warning you will get a lot of flak from students for "restricting the access that they paid for!" even though in the actual halls contract that they sign is states that "internet access is provided for academic use only".
While this seems a little harsh if people really wanted to do LAN gaming for example they can always set up a separate network to do so.
Hope that helps
Sam
Warning, comments may not have been passed by the sanity department of my brain.
Computers on a college campus are different than computers in business. A company owns its computers and therefore has full discretion over them. On a college campus, the students bought and payed for their own personal machines. They have a right to download and install whatever they want onto them. And I agree that they are going to be pretty upset if the internet connection they are paying for is turned off. However, in my experience working at a college, the college was able to say that it owned the network and that allowing students to connect to that network was a priviledge, not a right that it offered to its students. If the student's computer was harmful to the network, they were disconnected from the network for the greater good of the students at the college. They could be reconnected when they could prove their computer was clean.