Slashdot Mirror
Ask Slashdot: Best Way To Monitor Traffic?
First time accepted submitter Shalmendo writes "My client needs to monitor traffic on his LAN, particularly going out to the internet. This will include websites like Facebook, Myspace, and similar, including from mobile devices. So far, based on the network education I have, I've concluded that it might be best to get a tap (And some kind of recording system with wireshark, probably a mini-barebone), or replace the existing Linksys router with a custom built mini barebone system with linux routing software and appropriate storage capacity etc to record traffic internally. (either way it looks like I will need to put together a mini barebone system for some purpose) My client is trying to protect his family from scammers and other unsavory types, and isn't savvy in this matter, so i'm doing it for him. What I need is a way to record the traffic at a singular point, like modem/router areas, or similar, and a way to scrape out Facebook, Myspace, and other messages. It also appears that the client's family is using iPhones and some game called 'words' which has message capability. Is it possible to scrape messages out of that game's packets, or are they obfuscated? Can I write a script? What software would you recommend? Linux routing OS? Can we sniff packets and drop them on the internal hard drive? or would a tap be better? How do I analyze and sort the data afterwards? my client needs easily read evidence (Such as text or screenshots) he can use as proof in discussion with his family to try and intercede in any potentially harmful transactions. In other words, how can I Achieve this goal? I have basic and medium training in computer networking, so I can make my own cables and such, but I've never worked on this exact kind of project before, and thought it might be better to query slashdot instead of do my own research from scratch. After days of discussion with the client, it's not plausible to put monitoring software in the devices on the network (due to legal issues and a few other factors), so I concluded a network tap or other device would be the best way to capture and study what's going on."
Oh it's realy easy. You just need about 800 offshore programmers, 200 solid state drives, hadoop, ruby on rails, cheese, bacon. Clearly your client has the funds.
Or maybe go and buy an internet security hardware appliance like Sonicwall or Watchguard and bill out 700 hours labor. It will take you less time to install one than writing that horrific maligned essay you chose to sully our pages with.
When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
Is that You?
You're going to need to install your scripts on the Verizon / AT&T point of presence to handle the iPhone / Words With Friends traffic molesting.
I think the NSA has the hardware in place, you'll simply need to rent some space on one of their servers.
'client'? And why does he need to know the content of every. single. message. that goes out on his network? Is this going to be like the talk with my kids when they say 'my friend has this girl he likes' kind of thing?
If you need to know what every message going out is, including the content of a (I assume) 'words with friends' game, maybe you should just unplug for a while and take a walk in the woods to clear your head. Then maybe speak to a psychiatrist for the paranoia issues.
Take all their devices, and get rid of the internet if he cant control them. When my kids started staying up later than I wanted I just shutdown the router from 10:30 pm to 8:00 am back in the day. Besides if they have I smart phones they can just get off the lan and onto the carrier circumventing any controls you put on the lan.
Seriously.
Logging traffic is not going to stop someone from doing something stupid, like falling for a scam.
Education is.
Harald
Fortigate will do what you need out of the box, paired with Fortianalyzer.
The bigger question is WTH you're doing with this. You can't put monitoring software on the devices, but you can look at every last bit they send and receive? Legal issues are a far bigger problem when data is in transit (as in flying across the network) than when it's at rest on the device. You won't even see everything, as a lot is TLS-protected and if it's a phone, it can bypass the fixed network entirely. I somehow doubt that he's making his wife and kids agree to an AUP that allows this sort of monitoring.
The truth about Scientology, Xenu, and you: Operation Clambake
An easy thing you could do is to set up a proxy on the network (such as Squid) and use DHCP to force all of the computers on the LAN to use it. It won't be foolproof unless you block any outbound web traffic that isn't coming from the proxy and that will maybe break things, but this is someone's house and not an IT shop so that's not a big deal.
After that, set up all the phones to use wifi and take the hit in battery performance, or else get everyone ipod touches instead of phones with a data plan. You can't get around the fact that he is paying for another data connection per handset from the phone company.
The *best* thing you could do is sit your friend down and advise him that the world is scary and that you can't shield your kids from everything, but you can certainly build a good rapport with them and answer questions about life when they come up.
My client is trying to protect his family from scammers and other unsavory types, and isn't savvy in this matter, so i'm doing it for him.
Then you're doing it wrong.
Quite frankly, extreme monitoring and filtering isn't going to work. Scammers will hide their words to avoid filters, so active filtering doesn't work. The exchanges are managed quickly, so scams (especially phishing scams) get your data instantly, so delayed review of activity isn't going to protect anyone, either, though it might make detection a bit faster. There is simple no hardware approach that will work.
If, as others have pointed out, your client is an overly controlling patriarch, he needs professional psychiatric help. If he's just paranoid and scared, he needs professional technical help, and that's where you should focus your efforts.
Educate him and his family on scammers' techniques and tactics, and security practices. Explain how the teenage daughter will be victimized and harassed, because that's just the nature of the assholes on the Internet. From a network perspective, make sure they have updated antivirus software, and maybe an active monitoring firewall to scan HTTP traffic for viruses. A basic scanner for the known threats, and education for the unknown threats, and the client will be far better off in the long run.
You do not have a moral or legal right to do absolutely anything you want.
I don't know. Can you?
Please read my Canon EOS tech blog at http://www.everyothershot.com
...is to drop the client. Seriously.
He wants Orwellian monitoring over his network that is not only unfeasible but would eventually prove completely ineffective. If he's this paranoid, what's going to happen when your kludge of a system inevitably misses a message or two and he decides that caused someone to fall victim to a scam? He's going to come after you with some shark lawyer and make your life incredibly annoying, that's what. In the end, his idea will not prevent scams and the like. It's only going to further a "big brother knows best and sees all" mentality. On top of that, it shows a frightening lack of trust in his family - both in their ability to "do the right thing" and in their general intelligence. Your best solution is to drop the client and not feed his totalitarian ego.
On the other hand, if this is really you wanting such a solution, the trust issues apply even moreso. Learn to EDUCATE instead of spy. You will have much better results.
And finally, if you're an ISP too clueless to do something on your own, GTFO Slashdot with your asking us how to spy on your customers. You should be ashamed of yourself.
tl;dr - Your plan is a bad idea all around...
"So after all this, you make my case for me. To end this stalemate, you must die..."
So, either you are clinically paranoid, and should probably address that issue before any technical ones...or you need to take a step back, relax, and realize you don't have control over everything. Your "client's" requirements are completely ludicrous, and even if you wrote a script for "him" to scrape messages out of Words with Friends, what about EA's Scrabble, or TextFree, or any of the 10,000 other iPhone/Android apps that can communicate privately between two parties?
My advice? Cancel your hardwired ISP, cancel all smartphones with network access, harden your doors, windows, and other points of entry and lock you and your family in your basement. There you go, no "unsavories" or "scammers" can ever access you or your family. I'm sure that will go over well with the wife and kids, but at least you're being upfront about it and not covertly spying on them through their electronic communication (which is what you *really* want to do).
When they object, tell them the other option (your little Napoleon complex and your in-home Echelon system), and be prepared for your, sorry your "friend's" wife to serve up some divorce papers.
Oh, that's right, you just want them to be "safe". Give us a break, even the most hardened Fox News or CNN watcher isn't really *that* scared of unsavory types messing with their lives, and if you are, please turn off the television and go for a walk in the park for a few hours.
It obviously depends on the laws to which your client is subject but, if there are "legal issues" in putting monitoring tools on "devices on the network," you may also find that there are similar restrictions, or at least hurdles to clear, in operating an interception capability as part of the network...
If it is just a private house, for members of a family, as the summary seems to suggest, chances are these will be minimal. If it will end up monitoring the nanny, cook or whatever other staff your client might have, you might need to have more robust procedures in place. In either case, it's worth checking it out if any part of your contract says "system will comply with applicable law" or anything like that — or just for your own peace of mind.
you just went full retard.
At least he can 'make his own cables and such'.
And not worth it. The couple of sarcastic comments that have started off the replys here are telling you this. The problem is you need to dump interesting data out of the packets, and there's no easy way to tell what is actually interesting. Also, this is a cryptographer's nightmare or dream depending on how you look at it. You're Charlie here, and that means you're the guy that everyone wants to defeat in this scenario. It's not going to yield much useful data since more and more communications on the 'net are switching to HTTPS. Also, I don't think you can fully appreciate the amount of storage this will require. I work with network video, and when I have to run a packet capture to do analysis, the problem is finding a storage medium to dump to that can handle the throughput. The only thing I can usually make feasibly work is a ramdisk. You can't do that from your linux embedded router. It just isn't going to happen. Now, I suppose you could only capture the headers of the packets. But again, that's not going to do you any good. You don't capture any of the payload then. Conclusion: Way more trouble than it's worth, and to do what you're talking about will cost a lot of money. Don't bother. Frankly, if you're client is that concerned about the traffic coming out of the house, wipe all the computers to remove any potential malware on them already, install a fresh OS, install your own keyloggers on the systems if it's the human element you don't trust, and be done with it. It's invasive as hell, but it's a lot less sinister, and easier, than trying to play the panopticon game.
I admit the scope of the project is overwhelming, and I've told my client that he's asking for an NSA quality project. I will direct him to this post and your replies to help him to better understand the nature of his requests. Also, it appears that my article was truncated before being posted, so some of the explanatory bits were cut off, although the core of the question is still there for the most part. And yes, this is an actual client, not myself. I already suspected what most of you were saying, and tried to tell him that, but computers are a big 'mystery box' to him, and I can't seem to nail stuff home on my own. (IF it was myself i would have all already solved this problem.) Also, I'm a little surprised at some of the hostility and non-seriousness i've seen here, but I suppose it is to be expected considering alot of the drama and arguing i've seen going on in other arguments. When I originally wrote the article, I did specify 'serious answers only please, I don't want to start an argument, but a bunch of random answers that are unrelated won't help me solve this problem' And to be more specific, it's a home network with a cable connection. (I obviously can't be too specific due to his need for anonymity to avoid 'alarming' his family to his clandestine monitoring intentions). He does reasonable cause for suspecting something is going on and just needs to have information available to aid him in making decisions about some unusual behavior. and yes, I know that you can't get 'screenshots' right off a client PC through a network, by screenshots i meant some kind of recreation of a visited website, or just text information in printable form off some kind of analyzer software. I really would like to solve this problem, but I agree it's an excessive project. He wants the moon without having to go there to get it, type of issue.
Finding a cheating spouse is way easier than that.
One of my acquaintances recently went through this. Evidence was *everywhere* on the computer, facebook, e-mail, etc.
When you save your password you are trusting everyone with access to that machine not to cheat and look at your profiles. Maybe you should change those passwords and not save the updated ones before you have a tryst.
Best thing ever: Judge ruled it was not unauthorized access for him to dump her e-mails to the printer because she had saved the password to the same computer that he had authorized access to, and since she saved the passwords so that IE would auto-enter them she had no expectation of privacy...
-nB
whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
If he wants proof he can use during a divorce, he should probably ask the judge to authorize a subpoena of facebook messages posted by the soon-to-be-ex, rather than engaging in clandestine surveillance which very well may be illegal, and cost him a lot more than he's going to win in any divorce settlement.
My client is trying to protect his family from scammers and other unsavory types, and isn't savvy in this matter, so i'm doing it for him.
combined with
After days of discussion with the client
LOL. If someone can't be educated in "days" then they simply can not be protected from themselves. By "unsavory types" I assume he means us /.ers, which makes it even funnier. Would you trust me with your 19 year old daughter? Thought so. Well, she'd probably kick my butt anyway so don't worry too much.
I must be the only guy in /. with little kids that click on every spam popup window and sign themselves up for anything because... they're little kids. That is why their monitor is in a public part of the house easily viewed about 5 feet from my home office desk. My wife and I have caught them doing all kinds of ridiculous stuff and have (mostly) calmly used those events as "teachable moments", with excellent results. We've caught them watching remarkably inappropriate youtube videos, applying to work at the local Culvers (he was only 7 at the time), installing all kinds of spyware toolbars and stuff (whats more evil than a kids TV show that only exists to sell toys? I know, a kids game that only exists to install spyware! ). I'm pretty close to wiping his machine and installing debian, but people keep buying him windoze only "educational software" to my intense annoyance.
Also I must be the only guy with elderly relatives with a known proven tendency to fall for telemarketing pitches (clean your furnace ductwork for $400? Hearing aid for $5000?).
There are reasons to block/track/examine/log things beyond trying to catch the wifey cheating with the pool boy, in fact keeping a really close eye on little kids and elders is being a nice civilized responsible guy, not a jerk. In comparison "easily read evidence" and "use as proof" is simply being a jerk.
I will suggest that printing this ask /. out and giving it to the client will probably be extremely educational for the client. Probably this is one of those "the client is a little overbearing and I need some backup in arguing with him" situations. We should demand a cut of the proceeds from the consultant; maybe a tithe to the EFF would be appropriate?
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
A few things:
Better firewalls, including even the lowly dd-wrt and the now-defunct Snapgear, support syslog so you can capture and create your own custom reports, and dd-wrt reports total bandwidth usage on a daily, monthly and and annual basis and will retain that info until you do a reset (or until it runs out of NVRAM). It can come in very handy if your ISP claims you hit your bandwidth cap.
Another thing you might want to try is IMFIREWALL/WFilter in monitoring mode to see which users are doing what on your network. What is required is to either put a port on your switch (connected to your gateway/firewall) in either promiscuous mode or a two-way mirror to the port that connects to the firewall.
http://www.imfirewall.us/WFilter.htm
It will report the number of hits to instant messaging, streaming, social networking, porn, gambling, stock trading, and any other criteria you can think of configuring. You can also put it in filter mode so it will basically kill any requests that you disapprove of, but in monitoring mode you can create custom reports of who is doing what.
Other firewalls will include these features as integrated, but some vendors (Cisco, Sonicwall) won't sell you the complete feature set for a flat price; they nickle and dime you because it's more profitable, and when the unit dies, good luck transferring those purchases.
You might want to check out m0n0wall as well, and get a good syslog app so you can capture detailed logs and create your own detailed status reports.
The Christian Right is Neither (Christian nor right). See: Matthew 23, Matthew 25, Ezekiel 16:48-50
While I'm not a troll by any means, the level of hostility and such has led me to feel it would be a good idea to apologize to everyone for having wasted their time with a ridiculous inquiry. Trolling was never my intention, but it appears I may have done so unintentionally by asking to be informed by people that are experts of many fields, and intelligent and well educated, so you all have what apology I can offer. And I'm quite serious. I don't think I can really say anymore, so I'll leave it at that, link my client to this article, and let him judge for himself.
And to be more specific, it's a home network with a cable connection. (I obviously can't be too specific due to his need for anonymity to avoid 'alarming' his family to his clandestine monitoring intentions). He does reasonable cause for suspecting something is going on and just needs to have information available to aid him in making decisions about some unusual behavior.
In other words, he thinks his wife is having an affair and has seen some "unusual" transactions on the credit card or caller ID numbers, and is trying to gather "evidence" to use against her.
If you're the client, this is a hugely bad idea and could get you in very, very big legal trouble.
If you're not the client, then this is still a hugely bad idea and could get you in legal trouble.
Probably the second best move for you is to contact the wife and let her know what you've been asked to do. You might even get a larger paycheck out of it that way.
But finally, the first best move for you is to contact a qualified divorce/family law lawyer in your jurisdiction. Because before you take any further moves, you want to know which ones will get you sued and which ones will get you in jail.
Okay, you find it interesting. Look at any corporate Firewall and monitoring system and you have your answers. Hell I have an O'Reilly book from the very early 90s on TCP/IP security that covers all of the topics you need to know. The technology is nothing new, the only real variations are in how the logs are stored and parsed.
-The wise argue that there are few absolutes, the fool argues that there are no probabilities.