Slashdot Mirror
Ask Slashdot: Best Way To Monitor Traffic?
First time accepted submitter Shalmendo writes "My client needs to monitor traffic on his LAN, particularly going out to the internet. This will include websites like Facebook, Myspace, and similar, including from mobile devices. So far, based on the network education I have, I've concluded that it might be best to get a tap (And some kind of recording system with wireshark, probably a mini-barebone), or replace the existing Linksys router with a custom built mini barebone system with linux routing software and appropriate storage capacity etc to record traffic internally. (either way it looks like I will need to put together a mini barebone system for some purpose) My client is trying to protect his family from scammers and other unsavory types, and isn't savvy in this matter, so i'm doing it for him. What I need is a way to record the traffic at a singular point, like modem/router areas, or similar, and a way to scrape out Facebook, Myspace, and other messages. It also appears that the client's family is using iPhones and some game called 'words' which has message capability. Is it possible to scrape messages out of that game's packets, or are they obfuscated? Can I write a script? What software would you recommend? Linux routing OS? Can we sniff packets and drop them on the internal hard drive? or would a tap be better? How do I analyze and sort the data afterwards? my client needs easily read evidence (Such as text or screenshots) he can use as proof in discussion with his family to try and intercede in any potentially harmful transactions. In other words, how can I Achieve this goal? I have basic and medium training in computer networking, so I can make my own cables and such, but I've never worked on this exact kind of project before, and thought it might be better to query slashdot instead of do my own research from scratch. After days of discussion with the client, it's not plausible to put monitoring software in the devices on the network (due to legal issues and a few other factors), so I concluded a network tap or other device would be the best way to capture and study what's going on."
Oh it's realy easy. You just need about 800 offshore programmers, 200 solid state drives, hadoop, ruby on rails, cheese, bacon. Clearly your client has the funds.
Or maybe go and buy an internet security hardware appliance like Sonicwall or Watchguard and bill out 700 hours labor. It will take you less time to install one than writing that horrific maligned essay you chose to sully our pages with.
When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
Is that You?
You're going to need to install your scripts on the Verizon / AT&T point of presence to handle the iPhone / Words With Friends traffic molesting.
I think the NSA has the hardware in place, you'll simply need to rent some space on one of their servers.
'client'? And why does he need to know the content of every. single. message. that goes out on his network? Is this going to be like the talk with my kids when they say 'my friend has this girl he likes' kind of thing?
If you need to know what every message going out is, including the content of a (I assume) 'words with friends' game, maybe you should just unplug for a while and take a walk in the woods to clear your head. Then maybe speak to a psychiatrist for the paranoia issues.
Take all their devices, and get rid of the internet if he cant control them. When my kids started staying up later than I wanted I just shutdown the router from 10:30 pm to 8:00 am back in the day. Besides if they have I smart phones they can just get off the lan and onto the carrier circumventing any controls you put on the lan.
Seriously.
Logging traffic is not going to stop someone from doing something stupid, like falling for a scam.
Education is.
Harald
This is seriously a problem that starts and begins with the users. All the technology in the world isn't going to fix it. We don't even know if it's a family LAN or related to a family business. You won't be able to get the iPhone information if they are using a data network. There is so much wrong with this whole situation I don't even know where to start.
Fortigate will do what you need out of the box, paired with Fortianalyzer.
The bigger question is WTH you're doing with this. You can't put monitoring software on the devices, but you can look at every last bit they send and receive? Legal issues are a far bigger problem when data is in transit (as in flying across the network) than when it's at rest on the device. You won't even see everything, as a lot is TLS-protected and if it's a phone, it can bypass the fixed network entirely. I somehow doubt that he's making his wife and kids agree to an AUP that allows this sort of monitoring.
The truth about Scientology, Xenu, and you: Operation Clambake
An easy thing you could do is to set up a proxy on the network (such as Squid) and use DHCP to force all of the computers on the LAN to use it. It won't be foolproof unless you block any outbound web traffic that isn't coming from the proxy and that will maybe break things, but this is someone's house and not an IT shop so that's not a big deal.
After that, set up all the phones to use wifi and take the hit in battery performance, or else get everyone ipod touches instead of phones with a data plan. You can't get around the fact that he is paying for another data connection per handset from the phone company.
The *best* thing you could do is sit your friend down and advise him that the world is scary and that you can't shield your kids from everything, but you can certainly build a good rapport with them and answer questions about life when they come up.
My client is trying to protect his family from scammers and other unsavory types, and isn't savvy in this matter, so i'm doing it for him.
Then you're doing it wrong.
Quite frankly, extreme monitoring and filtering isn't going to work. Scammers will hide their words to avoid filters, so active filtering doesn't work. The exchanges are managed quickly, so scams (especially phishing scams) get your data instantly, so delayed review of activity isn't going to protect anyone, either, though it might make detection a bit faster. There is simple no hardware approach that will work.
If, as others have pointed out, your client is an overly controlling patriarch, he needs professional psychiatric help. If he's just paranoid and scared, he needs professional technical help, and that's where you should focus your efforts.
Educate him and his family on scammers' techniques and tactics, and security practices. Explain how the teenage daughter will be victimized and harassed, because that's just the nature of the assholes on the Internet. From a network perspective, make sure they have updated antivirus software, and maybe an active monitoring firewall to scan HTTP traffic for viruses. A basic scanner for the known threats, and education for the unknown threats, and the client will be far better off in the long run.
You do not have a moral or legal right to do absolutely anything you want.
I don't know. Can you?
Please read my Canon EOS tech blog at http://www.everyothershot.com
...is to drop the client. Seriously.
He wants Orwellian monitoring over his network that is not only unfeasible but would eventually prove completely ineffective. If he's this paranoid, what's going to happen when your kludge of a system inevitably misses a message or two and he decides that caused someone to fall victim to a scam? He's going to come after you with some shark lawyer and make your life incredibly annoying, that's what. In the end, his idea will not prevent scams and the like. It's only going to further a "big brother knows best and sees all" mentality. On top of that, it shows a frightening lack of trust in his family - both in their ability to "do the right thing" and in their general intelligence. Your best solution is to drop the client and not feed his totalitarian ego.
On the other hand, if this is really you wanting such a solution, the trust issues apply even moreso. Learn to EDUCATE instead of spy. You will have much better results.
And finally, if you're an ISP too clueless to do something on your own, GTFO Slashdot with your asking us how to spy on your customers. You should be ashamed of yourself.
tl;dr - Your plan is a bad idea all around...
"So after all this, you make my case for me. To end this stalemate, you must die..."
So, either you are clinically paranoid, and should probably address that issue before any technical ones...or you need to take a step back, relax, and realize you don't have control over everything. Your "client's" requirements are completely ludicrous, and even if you wrote a script for "him" to scrape messages out of Words with Friends, what about EA's Scrabble, or TextFree, or any of the 10,000 other iPhone/Android apps that can communicate privately between two parties?
My advice? Cancel your hardwired ISP, cancel all smartphones with network access, harden your doors, windows, and other points of entry and lock you and your family in your basement. There you go, no "unsavories" or "scammers" can ever access you or your family. I'm sure that will go over well with the wife and kids, but at least you're being upfront about it and not covertly spying on them through their electronic communication (which is what you *really* want to do).
When they object, tell them the other option (your little Napoleon complex and your in-home Echelon system), and be prepared for your, sorry your "friend's" wife to serve up some divorce papers.
Oh, that's right, you just want them to be "safe". Give us a break, even the most hardened Fox News or CNN watcher isn't really *that* scared of unsavory types messing with their lives, and if you are, please turn off the television and go for a walk in the park for a few hours.
For corporate traffic, Don't put a box in between that traffic. If it fails, everything is down. Get a TAP, as you hinted, but make sure to get one that fails 'open'. Then, run Ntop off the TAP port. If the TAP burns up, or port goes bad, you still have network access.
It sounds like your "client" is just wanting to basically monitor on his family, so in that case, get a 10/100 HUB (not a switch) to stick downstream of your modem. Plug in your linux box on port 1, and the router/modem into port 2. Don't put anything else on it because.. it's a hub. Run Ntop on the linux box.
Join the Slashcott! Feb 10 thru Feb 17!
It obviously depends on the laws to which your client is subject but, if there are "legal issues" in putting monitoring tools on "devices on the network," you may also find that there are similar restrictions, or at least hurdles to clear, in operating an interception capability as part of the network...
If it is just a private house, for members of a family, as the summary seems to suggest, chances are these will be minimal. If it will end up monitoring the nanny, cook or whatever other staff your client might have, you might need to have more robust procedures in place. In either case, it's worth checking it out if any part of your contract says "system will comply with applicable law" or anything like that — or just for your own peace of mind.
you just went full retard.
At least he can 'make his own cables and such'.
And not worth it. The couple of sarcastic comments that have started off the replys here are telling you this. The problem is you need to dump interesting data out of the packets, and there's no easy way to tell what is actually interesting. Also, this is a cryptographer's nightmare or dream depending on how you look at it. You're Charlie here, and that means you're the guy that everyone wants to defeat in this scenario. It's not going to yield much useful data since more and more communications on the 'net are switching to HTTPS. Also, I don't think you can fully appreciate the amount of storage this will require. I work with network video, and when I have to run a packet capture to do analysis, the problem is finding a storage medium to dump to that can handle the throughput. The only thing I can usually make feasibly work is a ramdisk. You can't do that from your linux embedded router. It just isn't going to happen. Now, I suppose you could only capture the headers of the packets. But again, that's not going to do you any good. You don't capture any of the payload then. Conclusion: Way more trouble than it's worth, and to do what you're talking about will cost a lot of money. Don't bother. Frankly, if you're client is that concerned about the traffic coming out of the house, wipe all the computers to remove any potential malware on them already, install a fresh OS, install your own keyloggers on the systems if it's the human element you don't trust, and be done with it. It's invasive as hell, but it's a lot less sinister, and easier, than trying to play the panopticon game.
I admit the scope of the project is overwhelming, and I've told my client that he's asking for an NSA quality project. I will direct him to this post and your replies to help him to better understand the nature of his requests. Also, it appears that my article was truncated before being posted, so some of the explanatory bits were cut off, although the core of the question is still there for the most part. And yes, this is an actual client, not myself. I already suspected what most of you were saying, and tried to tell him that, but computers are a big 'mystery box' to him, and I can't seem to nail stuff home on my own. (IF it was myself i would have all already solved this problem.) Also, I'm a little surprised at some of the hostility and non-seriousness i've seen here, but I suppose it is to be expected considering alot of the drama and arguing i've seen going on in other arguments. When I originally wrote the article, I did specify 'serious answers only please, I don't want to start an argument, but a bunch of random answers that are unrelated won't help me solve this problem' And to be more specific, it's a home network with a cable connection. (I obviously can't be too specific due to his need for anonymity to avoid 'alarming' his family to his clandestine monitoring intentions). He does reasonable cause for suspecting something is going on and just needs to have information available to aid him in making decisions about some unusual behavior. and yes, I know that you can't get 'screenshots' right off a client PC through a network, by screenshots i meant some kind of recreation of a visited website, or just text information in printable form off some kind of analyzer software. I really would like to solve this problem, but I agree it's an excessive project. He wants the moon without having to go there to get it, type of issue.
Finding a cheating spouse is way easier than that.
One of my acquaintances recently went through this. Evidence was *everywhere* on the computer, facebook, e-mail, etc.
When you save your password you are trusting everyone with access to that machine not to cheat and look at your profiles. Maybe you should change those passwords and not save the updated ones before you have a tryst.
Best thing ever: Judge ruled it was not unauthorized access for him to dump her e-mails to the printer because she had saved the password to the same computer that he had authorized access to, and since she saved the passwords so that IE would auto-enter them she had no expectation of privacy...
-nB
whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
If he wants proof he can use during a divorce, he should probably ask the judge to authorize a subpoena of facebook messages posted by the soon-to-be-ex, rather than engaging in clandestine surveillance which very well may be illegal, and cost him a lot more than he's going to win in any divorce settlement.
Google pfSense and set it as your firewall.
I am a Linux hobbyist and can comment on the Linux router option. Totally free if you have old hardware, but limited and will not cover all of your listed requirements.
This sits between my ISP's provided modem and my wireless router which serves the living room computer, bedroom, office, and a wireless laptop and phones using wifi.
I use Debian 6 on an old Semperon with 1 gig of RAM with two NICs. Overkill I know, substitute your hardware on hand and Linux needs here. It's nice having the option of a full desktop if you need it, but I usually ssh into it and have run it headless before. I have isc-dhcp-server installed.
For live viewing I open a terminal in Gnome or ssh and run screen split into a four-way window. Two screens run iftop- one for the external card and one for the internal card. The third window runs tshark for packet sniffing. You can export tshark's output into a log for examining of network traffic, sites visited, etc.
urlsnarf (part of dsniff) will also allow you to log sites (URLs) and it logs from all sources (phones, etc. as long as they are using the home network). This is proof against deleted browser history or content to confront someone suspected of illegal activity in the house, cheating spouses, crappy house-mates, etc. msgsnarf comes with dsniff and supposedly can log messenger traffic, but I have no experience with it.
Logkeys is a keylogger and will log anything as typed from the keyboard on the machine it is installed on. This won't work for phone logging obviously and conversations are one-sided.
If your client is jealous, paranoid, suspicious, or needing to protect themselves then a setup like this would work adequate with minor blind spots and annoyances. I'm just a hobbyist and have used these things (logkeys is good for saving school papers if your word processor crashes). No doubt there are even better options out there, but for someone who is not technical it may work well- as long as they know how to access logs, etc. on linux or you could aggregate it somehow.
You need to consider that these days people are starting to use HTTPS by default for things like Facebook. You won't be able to inspect the contents.
If it's scammers he is genuinely worried about, education will solve it, not monitoring (which will catch it too late, after the scam has already started).
Oolite: Elite-like game. For Mac, Linux and Windows
My client is trying to protect his family from scammers and other unsavory types, and isn't savvy in this matter, so i'm doing it for him.
combined with
After days of discussion with the client
LOL. If someone can't be educated in "days" then they simply can not be protected from themselves. By "unsavory types" I assume he means us /.ers, which makes it even funnier. Would you trust me with your 19 year old daughter? Thought so. Well, she'd probably kick my butt anyway so don't worry too much.
I must be the only guy in /. with little kids that click on every spam popup window and sign themselves up for anything because... they're little kids. That is why their monitor is in a public part of the house easily viewed about 5 feet from my home office desk. My wife and I have caught them doing all kinds of ridiculous stuff and have (mostly) calmly used those events as "teachable moments", with excellent results. We've caught them watching remarkably inappropriate youtube videos, applying to work at the local Culvers (he was only 7 at the time), installing all kinds of spyware toolbars and stuff (whats more evil than a kids TV show that only exists to sell toys? I know, a kids game that only exists to install spyware! ). I'm pretty close to wiping his machine and installing debian, but people keep buying him windoze only "educational software" to my intense annoyance.
Also I must be the only guy with elderly relatives with a known proven tendency to fall for telemarketing pitches (clean your furnace ductwork for $400? Hearing aid for $5000?).
There are reasons to block/track/examine/log things beyond trying to catch the wifey cheating with the pool boy, in fact keeping a really close eye on little kids and elders is being a nice civilized responsible guy, not a jerk. In comparison "easily read evidence" and "use as proof" is simply being a jerk.
I will suggest that printing this ask /. out and giving it to the client will probably be extremely educational for the client. Probably this is one of those "the client is a little overbearing and I need some backup in arguing with him" situations. We should demand a cut of the proceeds from the consultant; maybe a tithe to the EFF would be appropriate?
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
A few things:
Better firewalls, including even the lowly dd-wrt and the now-defunct Snapgear, support syslog so you can capture and create your own custom reports, and dd-wrt reports total bandwidth usage on a daily, monthly and and annual basis and will retain that info until you do a reset (or until it runs out of NVRAM). It can come in very handy if your ISP claims you hit your bandwidth cap.
Another thing you might want to try is IMFIREWALL/WFilter in monitoring mode to see which users are doing what on your network. What is required is to either put a port on your switch (connected to your gateway/firewall) in either promiscuous mode or a two-way mirror to the port that connects to the firewall.
http://www.imfirewall.us/WFilter.htm
It will report the number of hits to instant messaging, streaming, social networking, porn, gambling, stock trading, and any other criteria you can think of configuring. You can also put it in filter mode so it will basically kill any requests that you disapprove of, but in monitoring mode you can create custom reports of who is doing what.
Other firewalls will include these features as integrated, but some vendors (Cisco, Sonicwall) won't sell you the complete feature set for a flat price; they nickle and dime you because it's more profitable, and when the unit dies, good luck transferring those purchases.
You might want to check out m0n0wall as well, and get a good syslog app so you can capture detailed logs and create your own detailed status reports.
The Christian Right is Neither (Christian nor right). See: Matthew 23, Matthew 25, Ezekiel 16:48-50
"and some game called 'words' which has message capability"
So the guy wants to wire tap everything they use, period. Even a freaking games? Most of those games already filter "bad words".
One thing you can do is set all their DNS servers to use OpenDNS's FamilyShield. It will do a pretty good job of filtering bad sites/etc at the DNS level.
As for logging, I wouldn't. That just sounds like not only violating your family's privacy (okay so they're under-age? That may be okay) but should anyone else happen to use the device and have no clue the things they were typing or doing were being recorded could pose a big issue.
It's a thought anyhow.
While I'm not a troll by any means, the level of hostility and such has led me to feel it would be a good idea to apologize to everyone for having wasted their time with a ridiculous inquiry. Trolling was never my intention, but it appears I may have done so unintentionally by asking to be informed by people that are experts of many fields, and intelligent and well educated, so you all have what apology I can offer. And I'm quite serious. I don't think I can really say anymore, so I'll leave it at that, link my client to this article, and let him judge for himself.
Wow. I wonder if that is actually the longest summary ever posted to /.
And to be more specific, it's a home network with a cable connection. (I obviously can't be too specific due to his need for anonymity to avoid 'alarming' his family to his clandestine monitoring intentions). He does reasonable cause for suspecting something is going on and just needs to have information available to aid him in making decisions about some unusual behavior.
In other words, he thinks his wife is having an affair and has seen some "unusual" transactions on the credit card or caller ID numbers, and is trying to gather "evidence" to use against her.
If you're the client, this is a hugely bad idea and could get you in very, very big legal trouble.
If you're not the client, then this is still a hugely bad idea and could get you in legal trouble.
Probably the second best move for you is to contact the wife and let her know what you've been asked to do. You might even get a larger paycheck out of it that way.
But finally, the first best move for you is to contact a qualified divorce/family law lawyer in your jurisdiction. Because before you take any further moves, you want to know which ones will get you sued and which ones will get you in jail.
Okay, you find it interesting. Look at any corporate Firewall and monitoring system and you have your answers. Hell I have an O'Reilly book from the very early 90s on TCP/IP security that covers all of the topics you need to know. The technology is nothing new, the only real variations are in how the logs are stored and parsed.
-The wise argue that there are few absolutes, the fool argues that there are no probabilities.
I do most of what you are inquiring about with a linux computer that has 2 NIC's. I have a custom build on openSUSE called net-tap. With a large hard drive, you can perform a tcp dump in a rolling manner, so the oldest is overwritten by the newest. You need to break the dump into reasonably sized bits and you will probably need to do a good bit of manual analysis.
That is the technical side. Now, ethically, this is a bad idea. I capture packets for clients all the time, but my analysis is only concerned with network performance. I can see scenarios where this would be acceptable, but they are few and far between.
I also concur that you are in over your head. People like you are pricing people like me out of the market. You don't know what your doing, but your clients have no idea.
"If your think a professional is expensive (me), you should try hiring an amateur (you)", is meant to be sarcastic, not instructive.
Cheap storage VM.