Slashdot Mirror
Ask Slashdot: How To Monitor Your Own Bandwidth Usage?
Vrtigo1 writes "With many ISPs either already using bandwidth caps or talking about them, I was wondering how other Slashdot readers are keeping tabs on how much data is being transferred through their home Internet connections. None of the consumer routers I've used seem to make this information easily accessible. I'd like some way to see exactly how much data has been sent and received by the WAN port facing my ISP's modem so I can compare the numbers I get with the numbers they give me. I don't want to pay for their modem firmware updates and other network management traffic, so I'd like to see how the two numbers line up."
Tomato.
DD-WRT.
I use Tomato firmware on a WRT54G v2 router. It has many ways of viewing used bandwidth.
Many of the home routers now have this built into their firmware. My netgear does. Or for the Slashdot answer, build something. :)
Use the SSID "LINKSYS" or "NETGEAR" and it doesn't seem to matter.
Trolling is a art,
Install vnstat on your gateways and/or points where you want to monitor the traffic. It will monitor "per device" so it can be useful running it on a gateway so you can compare internal network traffic to external traffic.
Move to Australia.
Churn to Internode (http://www.internode.on.net/).
Download MUM (http://www.users.on.net/~johnson/internode/).
I am on Uverse and while they don't cap, I still like to know how much is coming and going from my connection. What can I say, I am addicted to information. Anyway, with a bit of wget and some perl, I pulled the up/down bytes from the web page of the Motorola 2-wire gateway/router/thingy (most any router will offer this in some form) and I pump that into Cacti for storage and graphing. Tada!
You are on entirely the wrong website if you want to talk about stock firmware.
And why wouldn't you pay for their overhead and data they choose to send to you for updates, etc?
Does your contract really say you don't?
Comment removed based on user account deletion
http://oss.oetiker.ch/mrtg/
http://www.pfsense.org
Set yourself up with a real Unix firewall, and get your info to the byte-level. Heck, you can even configure it to email you reports.
ClearOS reports this and will give you all the function you need. It is great for both seasoned and beginner Linux users. The alternative is to set it up yourself. It is free and will run on that old computer you have in your closet.
Tomato has WAN bandwidth usage built it so that should get you what you want. Beyond this I wrote an add-on to tomato (but should really work with almost any linux based firmware) to store bandwidth usage per IP or Mac Address on your local LAN to see who is using what. Detail are here:
http://soft-haus.com/wiki/index.php5?title=IPTables_Bandwidth_Monitor
some ISP's count cable ARP or pppoe dsl over head.
And counting at the router will not get all that.
Snapshot /proc/net/dev at various times, and look at the line for the interface in question. You can write easy little scripts to show it changing over the course of a few seconds, or over a month, or whatever.
The FritzBox ADSL modem/router has an menu where you can see your internet usage.
http://www.fritzbox.eu/en/index.php
Way to suck at websites, website.
Your ISP is likely not counting bytes that transfer through a connection to your modem. They are probably using a number of interesting tricks instead, probably mostly because whatever they bought into does something different. At a minimum counting packets and saying they are all MTU-sized would give different results and would eliminate the overhead of counting bytes.
If you are really, really nice about it, they might tell you what they are really measuring. But they probably will not. Even if you have a bandwidth cap in place, they probably aren't going give you detailed information about what they are measuring and how they are measuring it. Mostly, this would be for fear that you will use that information to figure out some way to circumvent it. In this case information certainly equals power - they have it and do not want you to have it.
So, while your router can count bytes with the right software, it probably isn't going to match up with what they say you are using, assuming they report it to you. My guess is your number will be lower, but it could go either way. In any event, the only number that means anything in your relationship with your ISP is their number. You will not be able to convince them that your number is "right" or "more correct" than their number.
Unless you need a number for your own management purposes - like finding out your neighbor creating 45% of the traffic on your connection - I'd say this is a pointless exercise.
If you're not comfortable/willing to install a custom firmware, the Netgear N600 has a meter built in.
I'd be particularily interested in knowing how one could query that information from the router without browsing to the router.
I'd love to have a utility that sits in a small corner of my desktop, just showing me how much bandwidth has gone through the router.
Does DD-WRT offer a way to do that?
http://tech.slashdot.org/story/11/03/14/1444213/ATampT-To-Introduce-Broadband-Caps
DDWRT on a Linksys WRT--54GL works just fine for me. I go to the router IP address, then click on status, then wan, and bingo, I know already that so far this month I've had May 2011 (Incoming: 1074 MB / Outgoing: 45 MB) ... and the information between the parenthesis is cut and pasted from the router information tab, currently open in the other chrome tab. The granularity isn't smaller than megabyte, but thats how my ISP measures caps and bandwidth anyway. A google search will point you to the DDWRT page, where you can download (for free) DDWRT. If you like DDWRT, you can donate money via paypal or moneybookers, or donate hardware (so they can create firmware for it).
I'm on Verizon, too, and they have no problem with me daisy-chaining my own router (DD-WRT) onto theirs. I agree that it might be fun to hook straight into their fiber modem with CAT-5 and skip the business of having coax and a second router in the loop, but it's their network and their modem. I'll get more huffy about it if/when I transition to IPv6 and don't want two layers of NAT between me and the network. In the mean time, though, it's trivial to shut off the transmitter for the router they provided and set up one that I can manage competently.
"Space Exploration is not endless circles in low earth orbit." -Buzz Aldrin
I used ipac-ng http://ipac-ng.sourceforge.net/ for many years, until its lack of maintenance caused it drop significantly behind newer Linux kernels. sigh. Worked brilliantly when it did work though.
It means running all data through a linux box, but this is a given for me as I always have a firewall box for iptables, so I can split off my public IPs and home network. But all a bit much for a home ADSL connection really.
Who's charging for router firmware updates? The ones I've found were free downloads.
"Who controls the past controls the future. Who controls the present controls the past." -- George Orwell
pfSense. Been running it on ALIX board for years. Love it.
http://www.pfsense.org/
lameness filter
I put a pair of RFC1149-to-Ethernet gateways between my border router and my cable modem.
I then estimate the number of packets by measuring the amount of poultry poop between the gateway devices.
I multiply this by an estimated average packet size and I have a pretty good estimate of the number of bytes transferred plus the number of bytes lost.
Unfortunately I'm still trying to figure out my packet-loss ratio. Once I've got that down I'll have a better handle on how much traffic is going in and out of the modem.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
This may be overkill for many home networks, but we use pfSense running on an about-8-year-old computer.
Besides for firewall, NAT and bandwidth reporting (per-IP and aggregate), we are running Squid/SquidGuard and a VPN connector.
CPU: Intel(R) Pentium(R) 4 CPU 2.80GHz (2793.01-MHz 686-class CPU)
RAM: 512 MB
I use BitMeter OS. http://codebox.org.uk/pages/bitmeterOs It's a nice piece of software that has a number of useful features, such as a stopwatch that measures the amount of bandwidth you are using over a short period of time. You can also query a database that has information on your bandwidth usage over time. You can export these data as a CSV. The only drawback is that, by default, the software writes to your drive every 1 second when it is in use. You can change this using command line options. I have it write once a minute personally.
Windows app: http://www.bwmonitor.com/ has worked great for me for a long time. Obviously, it's just for the current workstation not router.
in the form of "localhost pppd[15600]: Sent 136276607 bytes, received 1262955416 bytes." into the syslog, for example, when the connection closes - therefore it might not be what you are looking for.
You do it at your router. Easiest way is with a router that supports linux and use tomato as they have a whole bandwidth monitoring built in with good statistics.
We were all warned a long time ago that MS products sucked, remember the Magic 8 Ball said, "Outlook not so good"
My 5 year old FRITZ!Box keeps traffic statistics for the current day, last day, last week, and current and previous months, with the stock firmware. It did so when it ran as a DSL modem, and it does so now it's configured as a WLAN router connected to the cable modem. I kinda assumed that level of features was standard...
Stephan
I use darkstat http://dmr.ath.cx/net/darkstat/
I use a Linux router running nfsen on the internal interface. From there I can set filters that count flows, bytes, and packets in and out of the router. (I can also go back in later and look at who was doing what if the resulting graphs look funny.)
I don't expect the numbers that I get to match what my provider's say; I just expect that if they claim I am over, I will be able to confirm that (within certain loose percentages) and then figure out why I am over.
you should read everything on the internet as if it had "but I'm probably talking out of my ass" appended to it.
I'm swedish so caps aren't really of any concern to me. I do however monitor my bandwidth usage using SNMP + rrdtool with a small web page that shows bandwidth usage and some other statistics (including room temperature, system load and uptime for my home server).
I used to just use "rrdtool graph" to create images but I recently switched to using a jQuery-based client-side plotting library called flot since it produces my cleaner graphs and it also allows me to use AJAX to update only the data rather than push a whole new image on every update...
You could easily modify a solution such as this to also monitor total data transferred over a specified period of time (such as the current calendar month or the last 30 days).
Greylisting is to SMTP as NAT is to IPv4
I don't want to pay for their modem firmware updates and other network management traffic
How is that relevant? You think you have some say in the matter?
I have Cacti polling the byte counters on my Linux router interfaces, which then puts all that into an RRD file. I can then see usage by graph definitions created in Cacti.
But one very useful feature of Cacti is the '|sum:auto:current:2:auto|' feature (google this for further info), which does exactly what it says. Add this into any graph definition, and it will sum the entire data set values across the time frame that has been specified for the graph (obtained from the RRD file) and print you the sum of the values.
In other words, I can take my mouse, select the last 30 minutes of my bandwidth graph within Cacti, and it will tell me how much bandwidth I have used within this 30 minutes. Similarly, I can also select the last 30 days, and see my usage for the last month. In short, I can select -any- time period I want from my graph, and it will tell me how much data I have downloaded/uploaded during the selected time period. Very useful feature indeed (and one I don't think many know about).
If you are willing to replace your router, I highly reccomend the FreeBSD-based router software "PFSense". It runs on any X86 hardware, and combines the ease of use of a commercial router, with the highly advanced networking features of expensive routers, while running on any hardware you have (so if it breaks, you can just move your config files to another machine, boot it up, and begin running again). I am so confident in it, I deployed it at my workplace, a multi-million dollar business with about 75 users, and several WAN Connections. Recommending it because it includes a bandwidth meter, is like recommending an airplane because it has a reading light; That is only one of its many features and uses. http://www.pfsense.org/
Is it overheating?
When the sun shines directly on my old WRT54G it seems to hang. I moved it to an always shady spot and put a bit more space around it and it's been stable ever since.
For specific details, like how much my ISP thinks I've been using, and plans to charge me for, I go to their web page and bring up my account.
Then you're using too much.
If you just have a single Windows workstation, DUMeter works well. http://www.hageltech.com/dumeter/about
... and I run a cron job hourly that takes a snapshot of the network traffic (in and out) for that hour and then uses syslog to write the data to a log file. I know my hourly traffic for the past couple of years.
opennms - great does more than just bandwidth usage - maybe overkill
darkstat - works well for usage tracking though I wish the reporting was more human readable
bandwidthd - worked for me but seem to loose historical data
I used to use DD-WRT or Tomato, but I wanted a faster router/firewall with more features. so I built a Mini ITX router with the following.....
http://www.ipcop.org/ - a great high end firewall package.
http://m0n0.ch/wall/ --BSD based and solid as a rock.
http://www.pfsense.org/ if you want gobs and gobs of plugins and features. it's a fork of Monowall with more plugin support.
NOTE: some people consider plugins to be evil for a firewall. I find having to run 3 servers for a home network to be silly. So I run pfsense with a gajillion plugins for the features I want and a fileserver/app server on the inside.
Do not look at laser with remaining good eye.
Reboot your router on the 1st of every month (so you remember how, etc). Better to find out it doesn't boot when you're ready to fix it, rather than 2am some random day. That would imply the 1st is an excellent day to upgrade to the latest everything, just in case you missed a security advisory, etc.
Then, anytime, log in and "ifconfig" and look at the second to last line of the external interface (last line is a blank). Probably, you initially set up the firewall with eth0 plugged into the LAN and got it all set up, then plugged eth1 into the cablemodem or DSL modem or wireless gadget or whatever while reconfiguring eth0 to the old firewall's inside ethernet config. So probably "ifconfig eth1 | tail -2 | head -1" is all you really need. Assuming your email is working, have a cron job run that nightly or whatever and get an email. Or put it in a nice little script, or have the MOTD updated to contain that hourly, or whatever.
I do VLANs on the inside for the phones vs everything else (yeah for linux support of dot1q), and some traffic routes from the inside webservers to the phone web interfaces, so its much simpler to watch the "outside" than the "inside".
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
Again, folks, please moderate properly!! The parent is a troll, and possibly flamebait, but it is certainly not offtopic - despite stretching the meaning of the word bandwidth it is nonetheless about the subject of the article, and therefore cannot be offtopic!
This tagline was transcoded to result in at least one smirk. If you experience failure to smirk, please consult your Gen
Don't know if this has been mentioned, but there is a program called NetLimiter which records how much a computer is downloading, and even keeps track of what programs are downloading and how much they are downloading. This isn't really ideal for tracking how much is getting downloaded in total over your internet connection, especially if you have multiple computers, but it will allow you see how much each computer it is installed on is doing, if you want to see why you are downloading so much if that happens.
My home server is running Cacti querying my Airport Extreme basestation/router. It might be a bit much for the average user, so another option would be to find a Linksys or other type router that you can run Tomato on. It'll do this type of thing.
If your router supports SNMP (most do) you can use this script to keep track of the total traffic passing through an interface. It can be used as a stand alone script, or in conjunction with Nagios for alerting. I have been using this in a large scale production environment for a few months now, and its simple, but very effective. I also use it at home with great success!
http://code.google.com/p/checksnmptraffic/
I'm on Uverse too, and when I go to their bandwidth usage thing, it basically says that it's under construction and I don't have to worry about how much I'm using.
Presumably they're rolling it out slowly.
Putting moderation advice in your
Another option is to use the old PII u have sitting in the garage and set up a ClearOS box. (previously knows as Clarkconnect) http://www.clearfoundation.com/Software/overview.html Only takes a few minutes to set up, included linux firewall, web proxy report (great when u have teens at home), will handle the metering great plus give a ton of extra benefits.
1) They bill you, you pay, end of story.
2)So you Don't pay, no service, and bad credit.
3)So You Sue, you pay more and perhaps something good happens. Good luck.
"It does not matter Whether the rock hits the pitcher or the pitcher hits the rock, it's bound to be bad for the pitcher." -- Man of La Mancha
I know someone commented on this already. But PFSense has my vote. I have it running on a cheap Via mini itx board with a small 80GB hard drive and 256 MB of RAM. It is all you need for a home network. You can get into the hardware for less than $150 dollars and you get so much functionality.
OK, this is AC so you'll never see it.
But what I'm getting here is that there really are only two answers.
If you're an AT&T customer who's caps just started today, you have two options.
1. Go out and buy a new router.
Possibly install custom firmware on that router. (Tomato or DD-WRT)
Then connect the new router to the AT&T router and use the new one for all wired and wireless connections.
Use the new router to generate usage reports.
OR
2. "Use software to monitor the usage on each device and add them up."
DD-WRT on your router + ntop running on another machine. Ntop gives you all sorts of pretty graphics and stuff. Very easy to use.
sorry didn't meant to post as AC before... anyway:
DD-WRT on your router + ntop running on another machine. Ntop gives you all sorts of pretty graphics and stuff. Very easy to use.
Spoon not. Fork, or fork not. There is no spoon.
Step 1: Find an old computer, add a couple extra NICs (if it does not have one already)
Step 2: Load PFSense 2.0 RC1
Step 3: Use old wireless router as an AP.
Best decision I have made for my home network where it is common to burn through a wireless router in less than 6 months of use.
http://lmgtfy.com/?q=bandwidth+monitoring
So does the Asus RT-N56U.
I received one of these from the joint FCC-SamKnows bandwidth project. Its firmware has been customized to allow monitoring and reporting, but other features have been left alone. To the point, it has a "Traffic Meter" feature, with control and statistics functions that will keep track of monthly upstream or downstream bandwidth usage, or both. It has several options for keeping you from going over a set limit, including messages, flashing an LED, and a complete cut-off. My previous D-Link DGL-4300 was ridiculously expensive but had no such feature. When the SamKnows project is done in a few years I can re-flash this with other OpenRouter firmware like Tomato.
Right now it's telling me that I'm already 7.6 GB into my new AT&T cap, dammit....
I am using a Cradle point with two cellular modems connected to it for load balancing. One is a Verizon modem with 10GB a month cap and the other is an AT&T with 5GB a month cap. I don't trust them and need to know how much I am really using. I have not been able to see a good way to find this information out. Do any of you have any ideas?
You got the touch!
NetMeter. I have a older free version running on my desktop, but I think the new one is only a free trial. But it keeps my bandwidth logs nice and neat. Daily, weekly, monthly. Now, I need one that shows what programs are using what bandwidth.
No single raindrop believes it is to blame for the flood.
They will need to build in a Margin of error...
You will likely see that you are getting associated with traffic that is not for you... (All Broadcast traffic will likely be included.. EG ARP Traffic as well as any potential Multicast traffic that has not been properly setup)
Also depending on how they gather Stats you might see Network Management Traffic associated with your connection as well...
It would be interesting to see if Comcast is including their IPAD streaming feeds in your data caps or excluding it...
Who needs WiFi when we can have Packet Over Sheep! http://datacomm.org/PoS-InternetDraft.txt
I am with Cogeco Cable in Ontario, Canada. I have a 60GB Upload/Download Cap. They have a "feature" as part of their website that will allow you to check your bandwidth. The problem is it is only updated once a day (at midnight presumably). Considering 12MB/s on a 60GB cap, I can blow through a HUGH chunk of it, without knowing. Particularly if for instance I am downloading some pretty obscure garbage with few seeders, I might queue up a whole lot of stuff thinking it will be slowly downloading over a period of weeks... when all of a sudden some folks log on, and zip I suck it all down in an evening.
I accidentally doubled my cap one month because of that, @1.25$ per GB.
Having something that could read bandwidth on real time would be useful. Having something that would do that AND shut down my torrents once a certain threshold would also be nice.
Currently I try to just be careful and keep track in my head how many GB I have and will download. However with more software doing massive updates in the background this can be hard to manage, and will only get worse.
I use an old Cisco 2912XL-EN switch to connect my home network and collect SNMP stats from it. You can pick these up cheap on Ebay, but any enterprise class switch that supports SNMP will work. Gig switches will be pricier, though. I collect data from all the ports, so I can see both aggregate traffic on the firewall internal and external interfaces (on different vlans), as well as traffic from individual PCs, servers, or other IP devices.
Obviously this requires having a computer running all the time to do the polling, but I have a few of those already anyway. PRTG (for Windows) or MRTG (for *nix) are a couple of good free collectors, but I am sure there are many more as well.
Net Usage Item has been my weapon of choice for bandwidth use here in Australia. It's just a Firefox addon that grabs the information from the ISP and displays it as a handy bar in the browser. http://netusage.iau5.com/
Cable companies don't usually offer internet-only service and when they do they charge almost as much as if you had their overpriced television service.
That used to be the case. I am not promoting TWC since they are louts like the rest, but in LA they are finally advertising $29.99 cable modem service, no other crap required.
Http://www.switchtotwc.com
If you happen to be in a TIme Warner area.
.
I have a D-Link DIR-655 that I haven't taken the time to update from the factory firmware. No traffic usage monitoring at all in the stock firmware, so I grabbed a "UPnP Gateway Traffic Monitor" applet that runs in Windows.
I'm using this: http://www.d-bross.com/en/downloads/
The software seems totally lame, but apparently the key phrase to search for is "UPnP Traffic Monitor" and there are a couple little cruddy things that will monitor your WAN interface in some way.
A simple not-quite-perfect fix for monitoring traffic, says I.
We're no strangers to caps here in India. They're referred to as a "fair usage" policy. I used Networx for a while - it gave me excellent logs and graphs, until my ISP informed me that my uploads also count towards the cap. That's when I asked them to tell me where I could monitor my usage according to their specifications. Turns out that their site listed daily details of downloads and uploads - the best option simply because it tells me exactly how much I'm using according to the entity that's going to bill me.
Bitmeter is very nice. I use it on my gateway computer to measure the total in/out traffic for the household. It shows usage at hourly, daily and monthly intervals.
It seems to me that if the ISP is going to impose these caps they would be obligated to provide such tools for their customers to monitor their own usage.
If you're THAT worried about hitting your cap....or don't like it. Switch ISP's. Maybe if enough cap-placing ISP's lose business to companies that don't place caps on bandwidth, they'll get the idea.
I used to tell the comcrap sales people that hang out in every store I didn't want their service for a variety of reasons; too expensive, bad quality, monopolistic practices even down to "you plain won't give me service" (which was, in fact, 100% true as I'd been disqualified for even analog service) - the last year it's been the excuse of "I've got Verizon, STFU" - but here's something that might surpise you (or not), Verizon, being one company people seems to complain about...they have no usage limits on thier service. Big deal for DSL right? Try FiOS. Now I basically tell them "my current provider is twice the speed as your standard service and has no service limits" - they stay speechless...and trust me...when you can pull nearly 4 to 500 gigs of data off usenet a month and your ISP doesn't complain...you stay with 'em.
A friend of mine near Boston switched from comcrap to RCN solely for the fact that RCN does not throttle/cap users.
The real question is, why are you still on a provider that does?
Using a Firefox add-on.
#!/usr/bin/ruby
# I would have put this on github, but first it's horrible code and second I don't want my slashdot identity linked to my github one.
# create database 2wire; use 2wire;
# create TABLE readings (timestamp TIMESTAMP DEFAULT NOW(), port INT NOT NULL, txbytes BIGINT, rxbytes BIGINT, txpackets BIGINT, rxpackets BIGINT, txerrors BIGINT, rxerrors BIGINT, primary key `dateport` (timestamp, port));
series = `curl -s 'http://172.16.0.1/xslt?PAGE=C_2_0'`
(activeport, activeline) = [ nil, nil ]
data = Array.new
series.each_line {|line| /.*([0-9]*).*/ /rowlabel.*Port ([0-9]+) / then
if activeline == 4 then activeport = nil; activeline = nil end
if activeport then
line =~
$1 == "--" ? value=0 : value=$1.to_i
data[activeport].push value
activeline += 1
elsif line =~
activeline = 1
activeport = $1.to_i
data[activeport] = Array.new if data[activeport].nil?
end
}
1.upto(data.size - 1) {|port| # remember data is a zero-based array, but the ports from 2wire are positive integers
(txbytes, txpackets, txerrors, rxbytes, rxpackets, rxerrors) = data[port]
`mysql 2wire -e "INSERT INTO readings (port, txbytes, rxbytes, txpackets, rxpackets, txerrors, rxerrors) \
VALUES (#{port}, #{txbytes}, #{rxbytes}, #{txpackets}, #{rxpackets}, #{txerrors}, #{rxerrors})"`
}
du ~/pr0n/downloads/
What if your stuck with AT&T's uverse? They force you to use THEIR router. Even if you go with a business plan (which is a joke, had a customer want this and when they said I had to make their server DHCP I said "let's find another provider" and they went with Time Warner.
Anyhow, if I go double NAT that still breaks things like VPN and VOIP right?
My solution is pretty simple. A linux-based firewall gateway (pick your own flavor) running NTOP and Smokeping. Set NTOP to the internal interface and you get resolution of protocols and bandwith usage per internal client and aggregate as well. Smokeping to monitor several targets off your network for overall connection latency, jitter, and loss. I've actually used screenshots of these tools to go to the mat and wrestle with Qwest. Hard to argue with that kind of data.
Why not just pony up about $69/mo...and get a business connection for your home.
Yes that will 'fix' it but why should i be screwed as a home customer?
---- Booth was a patriot ----
I can't believe this is an ask slashdot question. There are countless free firewall solutions out there that easily do this. pfSense, IPCOP, m0n0wall, smoothwall, need I go on?
I am running "tomato" on a Linksys wrt54gl. It has good logging functions with some usage charting. I run between 15 and 25Gb a month. A 5Gb cap is a no-go for me. I'm quite scared that I'll get capped below my use.
I installed DD-WRT on my ASUS RT-N16 router (which I bought for $75 from newegg) and it provides a bandwidth graph for the current calendar month. It also makes the router actually work -- the ASUS firmware was *garbage*.
So I dug an old Catalyst switch out of the closet, put my cable modem on one port, my Time Capsule WAN port on another; put them on their own VLAN. I set another port to be the replication port such that all traffic on the cable modem is replicated to this port. I plugged this port into an unused ethernet port on one of my servers. I run MRTG on this port. I found that my bandwidth accounting more or less matched what my cableco was reporting. Unfortunately, in the process I also discovered that there were a dozen or so hosts in my neighborhood that were ARP-spamming so I started a tcpdump of packets that were not for me or from me and have been logging all of them. After 3 months of doing this, it would appear that these spurious ARP's account for 4% of my monthly bandwidth allotment. From this I deduce that the cableco is querying the counts directly from my cable modem and not any sort of upstream router. I'm currently collecting data and will present an accurate accounting to them when I get my first usage bill. I realize they will probably say something to the effect of that traffic being outside of their control but I will point out that everyone on my street or neighborhood or whatever the granularity of my head-end is paying for this traffic. At $2.00/GB of over-usage, multiplied by the number of people on my street who could potentially be over their monthly cap, they are raking in a tidy profit for what amounts to non-internet traffic. I figure the local media will be most interested in this math.
This is about 29 days worth:
-rw-r--r-- 1 root root 3858284544 2011-05-02 15:31 notmine.cap
tcpdump -n -r notmine.cap arp | wc -l
reading from file notmine.cap, link-type EN10MB (Ethernet)
47256871
$ bc -l
47256871*46
2173816066
That's about 2GB of ARP packets in nearly a month.
Configure MRTG on a linux, or Windows, box and SNMP on your router. If your router does not have SNMP by a used Cisco router.
I use an old Netscreen 5GT router, and PRTG (The freeware version) to monitor the interfaces. Gives pretty graphs :)
In a cable environment, what's usually measured is the actual bytes transferred to and from your cable modem. The measurement is recorded on the CMTS the cable modem connects through to provide a network connection. The CMTS numbers include bytes to/from devices (PCs, game consoles, smart phones, etc.) as well as a small amount of network overhead that's just between the cable modem and the CMTS. The overhead is typically a few K bytes per day. So the exact usage will never tie exactly to what's recorded on a home router, but it should be within +/- 5% at worst and typically run closer to 2% to 3%. The home router reported usage will be a little smaller than the CMTS side reported usage that the cable company will use for managing bandwith caps, but that can easily be compensated for by implementing a "grace buffer" of a few % over stated cap limits to account for variability. Whether or not this is done is another matter. But either way the actual variance is minimal.
Most home routers support SNMP, and SNMP shows you the packet and byte output counts on the WAN interfaces. It's pretty much guaranteed to be correct.
Just collect that and either shove it through MRTG, Cacti, or your hand-rolled scripts of choice.
$60 will get you 12/2 Comcast business, and $75 will get you 12/2 with a static IP.
http://business.comcast.com/internet/plans.aspx
I'm very happy using Mikrotik (mikrotik.com) products. You can buy a ~49$ license and install their software -RouterOS, a Linux derivative- on an old PC. You can also get one of their hardware products (routerboard.com), excellent choice if you care about power consumption. I highly recommend the RB750G wich is a small SOHO router in a nice plastic case. With RouterOS you can do a whole lot of interesting things, monitoring bandwidth usage is just one of these things.
I live in Canada and have a bandwidth cap. I've been using a Netgear UTM5 to log WAN usage and attempt to reconcile the UTM5's counts with my ISP's (Rogers). I have two observations:
The simply logs provided by routers (even a SOHO router like the UTM5) do not allow the drill down capability. For example, the UTM5 only provides detailed HTTP(S) logs; although it does keep aggregate counts for some other protocols.
My dilemma is that although I can determine how much HTTP(S) data my teens consume, I cannot determine what other applications they run are consuming. As a result, I haven't been able to completely get a handle on WAN usage and some months I am forced to block access to media streaming sites for the last few days of the month--so that we don't end up paying the ISP extra charges. My preference is to develop a better understanding of the traffic and then remove the offending applications from the LAN, but I simply don't have the data in hand to do that.
Not everyone rents a cable modem from their cable company. Therefore a firmware update becomes a moot point, I think.
Why can LinkSys, Belkin, and Dlink create smart routers that give stats to their home users?
Don't network commands on UNIX and the system monitor in Windows give you adequate statistics on how many packets are sent and received?
How do you separate out the Internet traffic from the phone and TV traffic from each of the many TVs in the home? It all goes over one cable. How can you cap Internet traffic, when the many TVs in the home use so much more bandwidth, collectively, than the Internet connection? Much better suggestion: cap the many TVs in the home, when mindless programs rob bandwidth from the Internet.
I didn't say you should live in the middle of nowhere. I say live where you want. Freedom.
The guy who said move close to work and ride a bike? That's not me. It's Steven Chu, Obama's energy secretary. In a speech at Harvard a couple months ago. And he said gas should be $7-$9. Watch it, I found it on youtube.
If you want the government to tell you where to live, how to live, well I disagree with that.
.
I have a Netgear 300 from SamKnows. It has it built into the firmware to monitor my traffic and report my bandwidth usage in a net graph. It can also kill it if I go over my comcast cap of 250 gigs a month. Which seems to be 20 days of netflix + hulu at around 5 hours a day.
Switch ISPs to one that doesn't meter you.
On a 64-bit machine (assuming you run Linux as your gateway), ifconfig will happily tell you exactly how much data has gone through it. (less useful on 32-bit, since the counter wraps). Eg:
eth0 Link encap:Ethernet HWaddr XXXXXXXXXXXXXX
inet addr:10.0.0.4 Bcast:10.0.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:48460422 errors:0 dropped:0 overruns:0 frame:0
TX packets:50200417 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:27186843235 (25.3 GiB) TX bytes:27203320607 (25.3 GiB)
:trollface:
... wonderful speed...
and I know that AT&T's usage can be found at myusage.att.com
As for other ISP's? No Idea. :D
My ADSL modem has a statistics page that provides statistics about each of its interfaces. Can't you get anything out of that?
Gargoyle-router is also pretty good, used it to set up quotas on my router and allowed me to manage 40gb/mo of internet between 6 people for a couple of years effectively. Works with WRT54GL and other common routers.
I don't want to pay for their modem firmware updates and other network management traffic
1. Most likely, their ability to provide management of the modem is in the terms of service. Even if you saw a significant difference, how would you prove it was firmware updates, and even if you could, would you be able to get reimbursed for the traffic?
2. Most likely, the modem updates and network management traffic is a small fraction of your total bandwidth usage (e.g. 20MB/month).
Most ISPs (well, those that don't have complex "this service is free, this is not" products) do accounting based on the interface counters of the virtual interface on the border router. To be able to separate the firmware and network management traffic, they would need to do DPI instead. The cost of the equipment upgrades to do this would have to be passed on to the consumer. Most likely, that would cost more than what the firmware updates and management traffic are costing you.
Yes, being able to see (without incurring more bandwidth use) how much you have used is useful, but not for the purpose you wanted ...
Without running a specific router, NetWorx is a pretty solid free option.
Automagically syncs instances across the network. Also, does some nifty things if you change networks a lot.
Though, it does have to be configured carefully, otherwise you end up missing traffic, or measuring locally.
No uncapped internet in NZ. And the best we can get is 80GB. You really do have to watch it.
Check it out at: http://netusage.iau5.com/
I use it on all my machines because in Australia we all have caps (almost).
It scrapes your isp's web page for their total of your usage so you know exactly how much you have left.
Get AdBlock or Privoxy and learn RegExp instead of listening to amateur advice from raving lunatics.
How many emails do you need?
Surely its cheaper to go the google apps way, and tie your domain to gmail , so you can have 1000s of accounts for a much cheaper rate with the added bonus of no need for a server.
Google apps free version is easy to setup.
point a special mx/name record to them on your domain and bingo.
Liberty freedom are no1, not dicks in suits.
How so? These 20 points (which I welcome you to disprove & good luck - you'll NEED it vs. YOUR "raving", minus a PHD in Psychiatry on YOUR PART no less, & yet YOU SEE FIT to dispense YOUR "amateur" wannabe '/.-SiDeWaLk-ShRiNk' b.s. quoted next):
"Get AdBlock or Privoxy and learn RegExp instead of listening to amateur advice from raving lunatics." - by Anonymous Coward on Tuesday May 03, @06:04AM (#36008540)
Sure, sure... (yea, right (not)): People are just going to instantly "learn regular expressions & javascript" to manage Adblock's code + rules themselves (lol, yea, right)!
vs.
Editing a HOSTS file!
(Which is something people on PC's most definitely ALREADY KNOW HOW TO DO & USE EASILY vs. what YOU stupidly suggest)!
And - again: Disprove these 20 points in favor of HOSTS files!
20++ ADVANTAGES OF HOSTS FILES OVER DNS SERVERS &/or ADBLOCK ALONE for added layered security:
1.) HOSTS files are useable for all these purposes because they are present on all Operating Systems that have a BSD based IP stack (even ANDROID) and do adblocking for ANY webbrowser, email program, etc. (any webbound program).
2.) Bad news: ADBLOCK CAN BE DETECTED FOR: See here on that note -> http://arstechnica.com/business/news/2010/03/why-ad-blocking-is-devastating-to-the-sites-you-love.ars
HOSTS files are NOT BLOCKABLE by websites, as was tried on users by ARSTECHNICA (and it worked, proving HOSTS files are a better solution for this because they cannot be blocked & detected for, in that manner), to that websites' users' dismay:
PERTINENT QUOTE/EXCERPT FROM ARSTECHNICA THEMSELVES:
----
An experiment gone wrong - By Ken Fisher | Last updated March 6, 2010 11:11 AM
http://arstechnica.com/business/news/2010/03/why-ad-blocking-is-devastating-to-the-sites-you-love.ars
"Starting late Friday afternoon we conducted a 12 hour experiment to see if it would be possible to simply make content disappear for visitors who were using a very popular ad blocking tool. Technologically, it was a success in that it worked. Ad blockers, and only ad blockers, couldn't see our content."
and
"Our experiment is over, and we're glad we did it because it led to us learning that we needed to communicate our point of view every once in a while. Sure, some people told us we deserved to die in a fire. But that's the Internet!"
Thus, as you can see? Well - THAT all "went over like a lead balloon" with their users in other words, because Arstechnica was forced to change it back to the old way where ADBLOCK still could work to do its job (REDDIT however, has not, for example). However/Again - this is proof that HOSTS files can still do the job, blocking potentially malscripted ads (or ads in general because they slow you down) vs. adblockers like ADBLOCK!
----
3.) Adblock doesn't protect email programs external to FF, Hosts files do. THIS IS GOOD VS. SPAM MAIL or MAILS THAT BEAR MALICIOUS SCRIPT, or, THAT POINT TO MALICIOUS SCRIPT VIA URLS etc.
4.) Adblock won't get you to your favorite sites if a DNS server goes down or is DNS-poisoned, hosts will (this leads to points 4-7 next below).
5.) Adblock doesn't allow you to hardcode in your favorite websites into it so you don't make DNS server calls and so you can avoid tracking by DNS request logs, hosts do (DNS servers are also being abused by the Chinese lately and by the Kaminsky flaw -> http://www.networkworld.com/news/2008/082908-kaminsky-flaw-prompts-dns-server.html for years now). Hosts protect against those problems via hardcodes
Are you guys asking for something that looks like the integral curve of your current network speed? And if so does anyone have a Linux applet for that?
try software like netpersec or dumeter. although this will only work if every PC on the network uses it religiously. also works if you have no router
Yes that will 'fix' it but why should i be screwed as a home customer?
People seem to want gigabit networks, unlimited transfer, and great customer service for $8/mo. And a pony.
The presence of the bandwidth caps on residential is an economic clue - it's what can be supported at those price levels. The home Internet service is offset to a degree by the crazy prices of cable TV.
The business class service is really what most Slashdotters want. Fast, no caps, no port bans, and you can get on the phone and talk to a guy editing the reverse zone files to put in your PTR records and be done in under 10 minutes. Shocker that this level of service costs more money to operate, right?
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
See subject-line above...
APK
http://ask.slashdot.org/comments.pl?sid=2120414&cid=36009336
Now, IF anyone's "king" here? It's YOU! LMAO - the king of RUNNING AWAY & backing up your "big words", lol...
APK
It might not be expensive where you're at, but at $80/month for 2mbps down and on a 3 year contract (in this area), that's a considerable increase over residential rates.
If it was the price you alluded to all over, yeah, many people could probably swing that (even me).
Just use good old GKrellM net monitor, it keeps the amount that you've downloaded and categorized by the dates