Slashdot Mirror
Bandwidth Accounting With Unix?
LegoB writes: "I am a student at a small, under-connected college. Despite our bandwidth woes, the administration is hesitant at buying a larger pipe, feeling that our current connection is being overly taxed by things like Napster, streaming media, and other non-educational (and non-constructive) traffic. Rather than have them start limiting certain applications, I would like to propose another alternative: bandwidth accounting. I'm hoping to find that Unix, in addition to being used as a router, can also be used as a bandwdith meter. What software packages do I need to track bandwidth by time, IP, and hopefully MAC address without massive kernel hacking?"
I have IPAudit running on two of my servers to keep tabs on internet usage. It sniffs the network and generates a dump text file with all the TCP/IP connections made during the program's runtime. It's files are easy to understand and parse and the processor usage isn't that high (on my 100Mb intranet, with 50% usage the process never goes beyond 25% on a PII 266MHz).
:)
A link to the Freshmeat page is here. I scoured Freshmeat for a userspace/rootspace solution for a bandwidth meter and IPAudit was the best because of it's simplicity. I personally prefer piping data into a perl program to parse the data than to let it become "Someone Else's Problem". The overhead is low and a parsing script isn't that hard to work out, the one I use (actually it's a suite of 2 programs) took 2 days to code and another week to tweak the filtering rules.
I also made a cute little web interface for the higher ups (computer illiterate) to browse through the user's usage - and it wasn't that hard to make. Oh, I don't release it 'cause it's a mess, one day I'll document it and release it, until then - sorry...
All browsers' default homepage should read: Don't Panic...
Some more links:
http://www.aspfree.co m/a uthors/chrisk/monitorsunnyline.asp
Old Slashdot story on Packet shaping....
http://slashdot.org/asksl ash dot/99/07/06/1433234.shtml
The joys of a search engine...
Malk-a-mite
Give up the idea of bandwidth accounting--you'll just give numbers to the fears.
Now hiring experienced client- & server-side developers
-- @rjamestaylor on Ello
http://www.ipmeter.com IP Meter. I haven't used it, but looks good.
Daniel
the url is http://www.comlink.apc.org/~moritz/ipac.html
at a former employeer we replaced a firewall appliance w/ a linux box in one of those cute mini cases (not rackmountable though) and set up ipac to gather stats every so often, and another box to periodically get those and graph them.... it mysteriously stopped working since i last checked, but it was working fine before hand, i probably screwed up something or other.
toast
Need a Catering Connection
the firewall/NAT functionality of the kernel and iptables utility will allow you to track the number of packets and nuber of bytes traveling across each port that you are monitoring.
You can also use the logging module to log particular firewall/NAT ruels to gain more detailed info.
For more info check out the netfilter home page.
Linux 2.2 does this, but not as nicely and without the logging functionality.
If at first you don't succeed, skydiving is not for you.
Ask it for totals according to address, port or protocol. Very simple to install : one package and one kernel compile option. The Debian package installs the couple of IPchains rules needed to make it work. No noticeable overhead on a DSL connection. And the new version even seems to build graphs ! It does not do user related accounting, but if your addressing policy links IPs to persons, the problem is solved ! IPac's homepage : http://www.comlink.apc.org/~moritz/ipac.html
If you have Cisco routers, you can enable something called NetFlow in IOS 12.0 or higher, which does this sort of accounting - search for cflowd, which is an open source data collection tool. But be careful, NetFlow can easily overload a Cisco router's CPU, and you need to have admin access to your Cisco router anyway.
;) My company (Orchestream) does QoS-management tools, as do others, which talk to Cisco routers. However, it's tough to justify modifying our product to talk to the Linux QoS stuff until there's enough volume of Linux routers out there. I would like to see us configuring Linux QoS-enabled routers, though.
i des/97-indenst97-diplom/stuttg97/tsld001 .htm
If you have a spare PC that can have Linux installed and be connected to the ethernet segment serving your Internet access router, something like ipaudit may be enough. It can monitor TCP/UDP ports, which you can't do with ipac.
If you have an ethernet switch serving the access router (quite likely), you will need to set up the switch to 'span' or 'mirror' the port serving the router to another port (serving your monitoring box). This just replicates (broadcasts) the traffic seen in and outbound on the access router's switch port, into the monitoring port.
Alternatively you could put a hub between the switch and the router, but your network manager is unlikely to be happy about this.
Commercial tools to do this are astonishingly expensive, by the way - there are things called RMON2 probes that do more or less what ipacct does, but with more features and SNMP-accessible MIBs for the results. Bandwidth management boxes such as Packeteer do something similar, but these are also quite expensive.
If any entrepreneur out there feels like doing embedded Linux or BSD boxes that monitor and maybe shape traffic (Linux's queuing features in 2.2 or later are very comprehensive), you would have quite a market. Even more so if you worked on cish (an open source emulation of the Cisco command line interface) so that standard QoS management tools could configure your box just like a Cisco router. Another useful standard to look at is RTFM (real-time traffic flow management) from the IETF, implemented by Netramet, which is supported by ipmeter.
Of course, I have an ulterior motive
Some useful links:
- cish - http://freshmeat.net/projects/cish/
- IPaudit - http://freshmeat.net/projects/ipaudit/
- IPmeter - http://www.ipmeter.com/
- NetraMet - http://www.auckland.ac.nz/net/NeTraMet/
- RTFM and other tools - http://www.mathematik.uni-stuttgart.de/~floeff/sl
- RTFM home - http://www.auckland.ac.nz/net/Internet/rtfm/
A RADIUS server might work in this situation as well. There are several free ones including:
Cistron and Livingston
Shawn
Not cheap, but very effective:
http://www.packeteer.com
Here we use two Linux based systems to monitor traffic, NTOP and MRTG
/~o etiker/webtools/mrtg/mrtg.html
NTOP stands for Network TOP and displays usage broken down by machine and protocol. I have successfully implemented this on RedHat 5.1 running on a 486 with 6 meg of RAM and a 500 meg HDD. I install the NTOP servers between the LAN and the router, connected to a hub where they can look at the traffic. Check out http://www.ntop.org for screenshots, etc.
MRTG is the Multi-Router Traffic Grapher. MRTG interrogates devices such as routers, switches and servers by using SNMP, and displays the results for a day, week, month and year on a webpage. For MRTG you need a slightly more meaty machine - I'm currently monitoring 12 sites every five minutes, using a P133 with 32M of RAM and a 1Gig HDD. (Mandrake 6.1 for this one). Site for this one is http://ee-staff.ethz.ch
I don't know if the above will be of any use - I think you'll have more luck with MRTG.
Good luck!
Matt (matt_brunton@hotmail.com)