How Do You Handle Ethernet Port Management?

MTL-Stalker asks: "I am currently investigating the best way to handle Ethernet port management for an organization with over 75,000 Ethernet ports spread out over 700+ sites. I was wondering how members of the Slashdot community are handling this issue in their organizations? Obviously this is as much a business process issue as a technological solution. In today's threat-filled networks, it seems like asking for trouble to rely on a simple switch based 'port enabled/port disabled' methodology. Do you think Cisco-style port security (tying a MAC address to a particular port) or PACLs (port access control lists) are worth the effort? Are products like Cisco Campus Manager or HP OpenView worth the cost and deployment headaches? Do they address your security concerns? How many of you are using homegrown scripting and/or SNMP solutions? How many ports can you effectively manage with these solutions? I would also be interested in knowing what industries these solutions are being implemented in."

Comments

Re:My dad's solution

[Harry Balls]

The OP is talking about physical Ethernet ports, not about TCP or UDP ports.

What about 802.1x security ?

[CineK]

This way you could tie particular users to their VLANs, not the machines to the ports, which can be quite annoying when a user wants to change his/her desk.

802.1x should be combined with some decent endpoint security solution
(see recent Gartner reports on this)

HTH

Marcin

Re:What about 802.1x security ?

[Philip K Dickhead]

VLANs can be a headache too - especially with 802.1x, which requires replacing your existing access layer switches with 802.1x capable ones. You DO get the benefit of integrating your wireless access infrastructure with the copper stuff.

Are yu all/mostly Windows (2000+)?

Look closely at Windows Domain and Server Isolation. It is an IPsec based infrastructure security solution, all managed with existing infrastructure. The IPsec policy agent is on the OS, and policy is easily managed centrally by Active Directory and Group Policy. It really is great - and can interop with other IPsec stacks like Linux and Solaris. The default auth mechanism is Kerberos - but x.509 can be used in parallel for interop. Kerb is dead easy.

If this is even only an 80% solution, it should be explored. There are no hardware costs in most cases, it can be phased in without field visits, and you probably already own it.

http://www.microsoft.com/technet/security/topics/a rchitectureanddesign/ipsec/default.mspx

I wish that one of the big Linux vendors would do something like this with IPsec and OpenLDAP. We have spent years matching the desktop, when developing advanced infrastructure management is where the winning game has moved.

Re:What about 802.1x security ?

[maxhead]

Actually, you do not necessarily have to replace the access layer switches to enjoy dot1x. Placing a dot1x-capable switch upstream that supports mulitple logins on a single port can be an intermediate step and bring most the benefits.

In general, I advise customers to lock down every port in their network with 802.1x and to provision guest VLANs that are GRE-tunneled to a switch in the DMZ. This segregates all the guest traffic from corp traffic at L2 so the only way for a guest to access local corp servers is via the internet and back through the corp firewall rules.

Guest-Intruder VLAN

[chill]

I've always had good luck with not necessarily tying a MAC to a port, but rather a list of approved MACs. MAC not approved gets automatically shunted to an isolated VLAN. If they bring up a browser all they see is a "welcome guest, call IT" screen. Both Cisco and HP switches can do this.

Re:Guest-Intruder VLAN

[Anonymous Coward]

I've always had good luck with not necessarily tying a MAC to a port, but rather a list of approved MACs.

You guys always try to do things the hard way. For true ethernet port management just use this.

Re:Guest-Intruder VLAN

[cybrix]

Is that what they use for broadband over powerline?

Serious business

[voice_of_all_reason]

The internet: Homework Help for both teenagers and network administrations :)

mac security

[v1]

Given how easy it is to change your mac address, (I can do this at will on my ethernet AND wireless) I would hope no serious security system relied entirely on that one factor. We have to assume the serious criminals have all the easy angles covered.

Re:mac security

[Alioth]

A large proportion of break-ins (particularly malware type break-ins) are not due to malice: quite often they are because a contractor/employee brought in their personal malware infested laptop and saw fit to connect it to the corporate network. Nearly all the problems I've seen on company networks are not due to malice but due to people doing silly things like this.

A huge number of corporate network problems can be solved just by keeping the honest people honest with things like MAC address approval.

RADIUS

[Lehk228]

i would suggest using a RADIUS login to manage user access

since RADIUS was originally designed for ISP's managing users it is good dealing with hostile clients and other riffraff as long as you are on a switched network

Too easy...

[__aaclcg7560]

One port at a time! The best part is that you don't need to be an MCSE tech to figure that one out.

Why?

[Dolda2000]

I'm not exactly in charge of any large area networks, so I'm probably just ignorant, but why would you want to limit physical Ethernet access to begin with? All your actual services are properly authenticated, aren't they? Is it for DoS prevention or proactive security or something completely else?

Netdisco

[arnie_apesacrappin]

As far as port management goes, you may want to look at Netdisco. If I recall correctly, UC Santa Cruz was using it to manage about 20K ports. It's open source, so you so should be able to customize it for your environment. I haven't run it personally, but the demo looks impressive.

When considering how to secure the ports, I think you have to find the balance between security and functionality. If you lock down each MAC to a specific port, how much time will you spend managing it? Whenever there is a connectivity problem, will you have to fight with the other groups assuring them that it isn't the network?

As a final thought, you generally get out of a network management system what you put into it. With a network as large as yours, there isn't a silver bullet to fix all of your problems. Whether you customize, roll your own or use vanilla off the shelf software, you need to figure out what makes the most sense for your business. Good luck. It sounds like you need it.

Poorly

[Sycraft-fu]

Well, that's the truth for our orignization. You don't want ot know how we do it. What you should look at for that scale, is probably dynamic VLANs. Cisco has good solutions, I'm sure you can find vendor neutral ones as well, but I'm the kind of guy who will push a Cisco solution in general. At any rate the basic idea is that when soemthing gets connected it's MAC is checked and then a VLAN is assigned to the port based on it. So no matter where a computer is connected, it's in the same area network and security wise. This also means that unauthorized computers can be put in a nothing VLAN with no access.

It's not a magic bullet security wise, but it really makes management easy. You want all your engineers in a given VLAN, just assign their MACs to it. Then if one goes to a new office and nobody tells you, doesn't matter the hardware takes care of it for you.

Good maps and schematics...

[Fallen Kell]

Well, first thing you want to have are good site network layouts in a CAD program, preferably done in scale. Do not worry about every single wire (it is nice though at least for the pulls from the floor to the closet's patch panels) but get the major items, devices, and closet feeds.

As for what connects where, well, that needs to be part of your asset management system to be really effective. Some type of database which contains records for each class of object (like computers, servers, switches, routers, etc., which also has fields for location and network port connectivity. Obviously you would want a relational style database, with one to many relationships for network connectivity since you may have multiple network interfaces on different devices. Now the hard part, actually making this part of your processes. You need to have this updated, and really the best way is to make sure that people have to go through the process in order to get on the network. What this means is that you absolutely must use something like "port security". If regular people can move a system from one location to another and just disconnect one device and connect this one and it works, you will never be able to keep any tracking/management system up-to-date. It will be up-to-date for a whole 5 minutes after you do an inventory of that cube/office/location before someone somewhere decides that they are taking over the room down the hall because it is closer to the window, or is next to the exit...

I can't state that enough, you need to FORCE EVERYONE TO USE THE SYSTEM. If one person doesn't use it, then everything he/she does will be under the radar and not detected which makes having such a system pointless because it doesn't contain valid data, and you might as well have done "/dev/random > my_network_layout".

Re:Turn them All on

[swordgeek]

My choices here were to mod you down, or to reply. I'm chosing the high road, I think.

Your suggestion has merit--turn on the damned ports, let people plug in, and get work done. Lower admin overhead, faster response for the end user, and everyone can get on with their work.

However, you seem to have an attitude problem, and I suspect it takes three days to get you on the network because nobody really gives a shit if they get around to doing your bidding. Doing work for people who believe they know your job better than you do is about as much fun as slicing open veins, and rather less satisfying. MAC address-based port connections may not be the perfect security solution, but they are one powerful layer in a multi-tiered environment, and they're absolutely not a toy. Consider: People bring personal laptops to work, plug in to the LAN, and a virus spreads because the primary virus scanners are at the perimeter firewall. The ENTIRE FUCKING COMPANY is now down for between six and 72 hours. Oh, but that's OK because you didn't have to submit your laptop for scanning, and could start working immediately. Clearly your work is more important than anyone else's in the whole company.

Here's another scenario: A company has a mixed user environment of PCs and Unix workstations. We can declare that every port is enabled, but what ports are enabled on which network? What if the networks are split by division?

Contrary to what your fantasy world might suggest, IT is NOT there to block your progress! They want to get things up and running as fast as possible, and with as little overhead for themselves as feasible. Opening all ports in a moderately large company is neither feasible nor intelligent.

I think that you pretty much defined yourself as a legitimate troll (note: Not your post, but YOU) with this comment:

"I am so tired of the IT group doing huge make work projects in the name of security/scalabilty/Enterprise/CRM/blah blah blah. What a bunch of crap. You know us users out here... We really do have work to get done."

So you have real work to do, but they are a bunch of slackers inventing work because they have nothing better to do.

You, sir (or madam), are an asshole. I predict for you a long and frustrating career of nobody doing what you want, just for the sake of pissing you off. Good riddance.

simple

[Keruo]

Use epoxy. Just mix the two compound and fill in un-used ports.
Great securitywise but kinda limits future expanding.

Migration path: manual-scripted-RADIUS-802.1x

[kalvyn]

I just recently stopped working for a government agency and I was responsible for managing port security on about 6000 ports. Our current end-game solution is to use 802.1x, however due to certain regulations, our agency couldn't operate a CA, so we couldn't feasibly request a new certificate for each host everytime one completes an accreditation process. But we were implementing everything else until we could get there.

Our short term solution is to standup a RADIUS server and use it for port-security. This isn't quite as good as 802.1x, but provides the same level of scalability without going as much in-depth. You bascially have your switches (assuming they have this ability) check the radius server for allowed MACs. This works the same as the MAC ACLs, but is centrally managed. We haven't gotten that far yet either, as we didn't have a RADIUS server. (more stupid regulations that make that a headache)

So, the current process is to manually change the MAC address on each port on each switch. We initially turn on port-security on the switches, and for the newer ones (Cisco 3550/3560/3750) once we determine that all the users are on that need to be on, we drop all other ports into a dead-end VLAN that has no access. The remaining ports we drop into our data vlan (we also have dedicated vlans for voice, wireless, video, and infrastructure management). Once we've established that, we secure the MACs to the ports. All port security violations are logged to a syslog server and the switches are set to restrict access. This prevents useless work of re-opening ports when some user decides to plug-in their home machine to download the latest Linux ISOs or torrents. For further changes (i.e. when a new machine gets put on the network), a call is made to the helpdesk which routes the ticket to the networking team (that's me) and I unlock the port. We then have to notify the security team, which scans the machine for vulnerabilities and applies patches as needed. After that, it is managed by WSUS and SMS.

Now this sounds very tedious, but it isn't that difficult to manage. For the last 2 months, I managed all port security by myself, as well as down network links, some remote office firewalls, and new switch installs. Port security helpdesk tickets were typically closed within 2 hours of the request (assuming the helpdesk tells me about them). As a bonus, and because I'm lazy, I wrote some scripts for WSH that will connect to a switch, get a listing of all port-security information, compare it to DHCP leases on Windows servers, and output a table that shows which host is on which port. I also expanded this for use on WAN links where it will recursively access all switches at a site, stopping when it reaches a router and display the same information on a per-switch basis. A pretty handy report. Useful for telling you which hosts aren't using DHCP (so you can ensure they belong there). The only real requirements for this to work are that the switches use CDP on infrastructure links and they support ssh. You also have to have a CLI ssh client that supports putting the password on the command line (or certificate based auth if you can set that up, I don't think Cisco devices support it, although I think kerberos works :)