Slashdot Mirror
Managing Huge Networks with Open Source Tools?
An anonymous reader asks: "I work for a large multinational firm with a network that spans the globe and am responsible for evaluating the software we use to monitor our network. Our department has a lot of money, and we're usually willing and able to spend it on good commercial software. Recently though, I find myself evaluating and approving more and more open source software. We are actually in the process of replacing some of our commercial tools with software like Nagios, LooperNG and syslog-ng. We are also evaluating MRTG, RRDTool, ntop and a host of other tools. The problem is that there's just too many of them, most of which are not maintained anymore. Here's my question: What other open-source tools do you use to monitor your networks? I not just looking for names, but how long you've been using them for, how easy / hard is it to administer and I guess how well it scales as the network grows. More importantly, are their respective projects still alive and kicking?"
For my large network at home, I have a little device with lights that blink when the network is working. I think the guy at the store called it a switch or something.
He who laughs last is stuck in a time dilation bubble.
...unless you have the resources to maintain it yourself.
You will find gaps in the utility of any set of tools you use, but a maintained project will be likely to have a community of people who can verify any patches you submit and provide tips on using the software in the most effective manner.
Stop the world; I need to get off.
I created and use JFFNMS (Just for Fun Network Management System) to monitor my network at work.
Its also used by a lot of people to monitor big, and medium networks.
Its fully mantained, and customizable.
- Smells Like Open Source Code
We used to use Nagios here to monitor a large number of services on a large number of servers. We eventually agandoned it and replaced our "is the server up" monitoring with simple scripts that call fping. The problem with nagios is that the process model starts to fall apart at several hundred monitored servers/services and we really did not want to dedicate a farm to monitoring.
A slick tool is Gkrellm, which has real-time graphical status for memory/temperatures/net/disk. Can be run in "server mode" (so no need for X on the monitored server). Lots of plugins are also available, from SNMP to ping tools. The project is well alive. Don't know if it floats your boat, though, as you're mentioning huge networks.
Feel ready to own one or many Tux stickers?
rrfw.sf.net is a nice gem of an app. It can automatically discover a lot of high end snmp equipment, and set itself up for monitoring that equipment. It's a mix of XML and Perl, and is really easy to add support for more hardware.
FlowScan/flow-tools/RRDTool/CuFlow.
Well, don't forget it as much as get something better, Cacti. Cacti is a frontend to MRTG & RRDTOOL and offers a lot of awesome improvements, such as a web frontend to add devices, device "profiles" to enable a common monitoring set for things such as Cisco routers, servers, etc and a whole lot more. We used MRTG here at our (Windows only) network, and I'm slowly moving it all over to Cacti for all of the above plus a lot more.
Looking for hardware (Currently need: Large Etch-a-Sketch) Have one? See my journal!
In Soviet Russia the insensitive clod is YOU!
I bought Open Source Network Administration by Kretchmar to answer this question. I was looking for open source tools to be used in a service provider environment and was unpleasantly surprised at what was revealed in this book. However, since it seems you're looking for enterprise-ish stuff, I highly recommend this.
You'll find many of the tools within to be quite useful during both day-to-day operations and troubleshooting as well as long term planning on your network. The author does a fairly decent job of walking you through a basic installation of each tool.
Slashdot reviewed it here.
Here are most of the tools discussed in this book.
"A clear conscience is usually the sign of a bad memory."
An advantage you have is the financial resources to make support arrangements for open source software. These arrangements can be standard, such as purchasing support from a company that expressly supports the OSS project. Or, it could be less traditional, such as finding the authors and making arrangements with them in advance. If you have the financial resources to do so, I would strongly suggest making support a criteria of selection.
From looking at what we've achieved with these I would say that you will need to be careful trying to scale them to large networks. They can start huge numbers of processes each minute, when monitoring many servers.
It depends what you're monitoring, of course - in our case we are monitoring maybe 20-30 operational parameters on each server. If we were only monitoring a single parameter then we could probably look at around 1-2000 machines from a single P4-based monitoring box, without any real problems. Using a 2.6 kernel on the monitoring box would also dramatically increase the scalability of it all.
Scalability issues bite similarly with rrdtool: numbers of parameters monitored per server can ramp the load on the monitoring machine(s) quite quickly. Again that is process load, not CPU load though, and a 2.6 kernel will be significantly better in this area. It can also be resolved by scripting the collection process better - not just running some collect-the-statistics routine from cron every minute.
If you're looking at monitoring 1000's of systems though, maybe you have enough of a budget to be able to plan around these issues.
I'm sure that ultimately all monitoring apps run into issues with how many (parameters * servers) each monitoring system can monitor too.
You say that your department has lots of money and you are happy to waste it on commercial software. You also say you like open source but are worried about it being maintained and supported.
Sounds to me like you could create some jobs for a couple of talented programmers. Everyone wins.
This post contains benzene, nitrosamines, formaldehyde and hydrogen cyanide.
http://www.opensims.org
If you have the budget why not fund a project to make sure it doesn't stop being developed. This would make good use of that money and help everyone.
Not sure how helpful this will be in huge environments, I live in the small to midsize market, but here are some tools that I have found useful in the past:
Not exactly a monitoring tool, but definitely the most versatile all around auditor I have ever found: Nessus.
Ettercap is a good sniffer.
The MRTG tool has been a godsend when I have had managed devices to deal with, and I have heard very good things about the RRD tool and Cacti.
Tripwire is freely available for Linux and the BSDs, though the Win32 version has not been open-sourced.
One tool I have not been able to find in F/OSS is a Windows event log monitor (though believe me I'm still looking).
I've recently taken the position in a large multi-national company to monitor our devices as well. We are only about a year into it. We also use a mixture of OpenSouce and commercial tools. The thing I've noticed mostly with most software today is that they are becoming more and more a mix. For instance our main managment software is currently Netcool. Netcool is commercial but it utilizes Apache, Tomcat and alot of Perl/CGI. On the commercial side it uses Java, and thier own proprietary DB called Omnibus as well as a Sybase Communications protocal. The end result it is extremenly flexible and works across all platforms I've tried with little trouble. I really don't think the future of software will be as black and white as OpenSource vs Commercial. Its really going to be a mix of both. The benifit is lower costs to the customer because of less development effort for the vendor. I like Netcool alot because its very nature is to be as flexable as possible. Its possible to use Netcool for other puposes that the develpers never thought of. Our HP NNM system is good but is nowhere near as flexible simply because it can only go as far as HP wants it to go. Hence it is only used for network discovery and SMNP trap collection. Bottom line is that the best choice in software today is the software the includes some OpenSource code.
Both tools give us a much better view of our network, and what our various devices are doing.
Man watching 6 MSCE's around a sun box, looks alot like the opening scene's of 2001:space odyssey...
What tools are popular for IP address management? Most of what I've seen is pretty basic. Are there any tools that are good at combining device inventory managment with address management and assigning addresses to new devices according to configurable rules?
For example, if I need to add 50 new web servers and each web server needs a redundant pair of NICs sharing one IP plus. Plus allowing multiple IPs per device or multiple devices per IP (e.g. VRRP, HSRP).
Spreadsheets seem faster than web based systems if it takes multiple queries to get all the info you need. A flexible query system and integration of address management with server monitoring seems like it would be very useful, but I haven't come found anything yet.
Seriously, it's hugely useful. It's very nice to be able to show management that you not only have a config backup system for your network devices, but your backup system is also doubling as a change control system. It's at http://www.shrubbery.net/rancid . I tend to use something like webCVS with it, to let folks browse through the CVS configs (you will, of course, want to use authentication to restrict access to webcvs).