Slashdot Mirror
Ask Slashdot: Best Way To Monitor Traffic?
First time accepted submitter Shalmendo writes "My client needs to monitor traffic on his LAN, particularly going out to the internet. This will include websites like Facebook, Myspace, and similar, including from mobile devices. So far, based on the network education I have, I've concluded that it might be best to get a tap (And some kind of recording system with wireshark, probably a mini-barebone), or replace the existing Linksys router with a custom built mini barebone system with linux routing software and appropriate storage capacity etc to record traffic internally. (either way it looks like I will need to put together a mini barebone system for some purpose) My client is trying to protect his family from scammers and other unsavory types, and isn't savvy in this matter, so i'm doing it for him. What I need is a way to record the traffic at a singular point, like modem/router areas, or similar, and a way to scrape out Facebook, Myspace, and other messages. It also appears that the client's family is using iPhones and some game called 'words' which has message capability. Is it possible to scrape messages out of that game's packets, or are they obfuscated? Can I write a script? What software would you recommend? Linux routing OS? Can we sniff packets and drop them on the internal hard drive? or would a tap be better? How do I analyze and sort the data afterwards? my client needs easily read evidence (Such as text or screenshots) he can use as proof in discussion with his family to try and intercede in any potentially harmful transactions. In other words, how can I Achieve this goal? I have basic and medium training in computer networking, so I can make my own cables and such, but I've never worked on this exact kind of project before, and thought it might be better to query slashdot instead of do my own research from scratch. After days of discussion with the client, it's not plausible to put monitoring software in the devices on the network (due to legal issues and a few other factors), so I concluded a network tap or other device would be the best way to capture and study what's going on."
Oh it's realy easy. You just need about 800 offshore programmers, 200 solid state drives, hadoop, ruby on rails, cheese, bacon. Clearly your client has the funds.
Or maybe go and buy an internet security hardware appliance like Sonicwall or Watchguard and bill out 700 hours labor. It will take you less time to install one than writing that horrific maligned essay you chose to sully our pages with.
When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
Is that You?
I suppose too many /.'ers have been found guilty of not RTFA. Instead, they are going to start loading the entire article into the summary section.
You're going to need to install your scripts on the Verizon / AT&T point of presence to handle the iPhone / Words With Friends traffic molesting.
I think the NSA has the hardware in place, you'll simply need to rent some space on one of their servers.
I say bullshit. Your "client" is probably trying to snoop on his wife and kids. Paranoid types like him are often controlling, abusive and should be avoided at all costs. Step away and do not work with people like him.
'client'? And why does he need to know the content of every. single. message. that goes out on his network? Is this going to be like the talk with my kids when they say 'my friend has this girl he likes' kind of thing?
If you need to know what every message going out is, including the content of a (I assume) 'words with friends' game, maybe you should just unplug for a while and take a walk in the woods to clear your head. Then maybe speak to a psychiatrist for the paranoia issues.
Or would ti be better to recommend your client a visit to psychiatrist. He/she seems to be rather paranoid and doesn't even trust his own family.
Just asking and talking with your family about the issue should be way enough.
Let us know how that divorce settlement goes.
Take all their devices, and get rid of the internet if he cant control them. When my kids started staying up later than I wanted I just shutdown the router from 10:30 pm to 8:00 am back in the day. Besides if they have I smart phones they can just get off the lan and onto the carrier circumventing any controls you put on the lan.
You know nothing about technology, quit screwing up the bidding market. Problem solved.
Seriously, is anyone employable any more?
Install cameras behind each user.
Seriously.
Logging traffic is not going to stop someone from doing something stupid, like falling for a scam.
Education is.
Harald
This is seriously a problem that starts and begins with the users. All the technology in the world isn't going to fix it. We don't even know if it's a family LAN or related to a family business. You won't be able to get the iPhone information if they are using a data network. There is so much wrong with this whole situation I don't even know where to start.
"it's not plausible to put monitoring software in the devices on the network (due to legal issues and a few other factors)" but it's totally fine to go ahead and try and record all communications over the network. Given the already-long post, the author should have mentioned whether they are in a one-party-consent wiretap state.
Technical plausibility is one issue, yes. Legality is another issue. Then there's just the ethicality of the matter. In my opinion - in typical slashdot form - the uber-nerd remains 95% focused on issue #1, 5% on the second, and 0.00% on the third. You functioning sociopaths are all the same: you think you can do whatever you want because laws are for the everybody else and if you want it it must be best because after all a priori you can do no wrong - you're just so smart, after all.
Fortigate will do what you need out of the box, paired with Fortianalyzer.
The bigger question is WTH you're doing with this. You can't put monitoring software on the devices, but you can look at every last bit they send and receive? Legal issues are a far bigger problem when data is in transit (as in flying across the network) than when it's at rest on the device. You won't even see everything, as a lot is TLS-protected and if it's a phone, it can bypass the fixed network entirely. I somehow doubt that he's making his wife and kids agree to an AUP that allows this sort of monitoring.
The truth about Scientology, Xenu, and you: Operation Clambake
nuff said?
clearos should be a complete gateway
http://www.clearfoundation.com/
DHCP, DNS and NTP Server ... lots of features...
OpenVPN
PPTP Server
Multi-WAN
Bandwidth Manager
MySQL Server
And you are his do-boy. Quit while you're ahead; leave your client to his "family", go get a real job or an actual education, and stop crowd-sourcing solution architecture to /.
I'm sorry but I thin you are going about this the wrong way
Dumping network traffic isn't going to help anyone in this situation. What are you going to do inspect every packet? Will you be onsite 24/7? Does the guy have the savvy to understand the traffic when you are not there? The best you will achieve is figure out what went wrong long after it went wrong, and that assumes you know something has gone wrong, and have the skill to spot it.
Much better solution would be to install a decent AV, keep the AV and OS up to date, educate the users that clicking "yes" to everything is a bad idea, stay off dodgy sites. Behave sensibly. Use pre paid credit cards that can't go negative if they need online cash. Get them to call you if they are unsure of something, or it looks suspicious.
Your "client" doesn't even know what kind of phones his "family" is using.
Still, one of the best FS I've read on /. for a while. Gave me a laugh.
A "client" *cough* of mine.
LOL
These kind of requirements sure as hell don't sound like someone trying to prevent scamming. My guess would be trying to catch a cheating/suspected cheating spouse. That's some heavy duty monitoring your client has asked for.
you just went full retard.
An easy thing you could do is to set up a proxy on the network (such as Squid) and use DHCP to force all of the computers on the LAN to use it. It won't be foolproof unless you block any outbound web traffic that isn't coming from the proxy and that will maybe break things, but this is someone's house and not an IT shop so that's not a big deal.
After that, set up all the phones to use wifi and take the hit in battery performance, or else get everyone ipod touches instead of phones with a data plan. You can't get around the fact that he is paying for another data connection per handset from the phone company.
The *best* thing you could do is sit your friend down and advise him that the world is scary and that you can't shield your kids from everything, but you can certainly build a good rapport with them and answer questions about life when they come up.
What you are describing basically sounds like what NGFW (Next Generation Firewalls) solve. These are standard firewalls, but add more "smarts" to them, like detecting certain applications, telling you which users access them and when. So you'll want something inline to do it properly.
A lot of traffic to the web may also be going over an SSL connection, so you would probably need an SSL module in-line to basically man-in-the-middle all the computers on the network and snoop the traffic.
Check out the NSS report (costs money to buy the report) on NGFW appliances.
Its not what it is, its something else.
"my client needs easily read evidence (Such as text or screenshots) he can use as proof in discussion with his family to try and intercede in any potentially harmful transactions." You don't need "proof" in a real discussion. Also, by the time you've captured and read any proof, it's already too late to "intercede harmful transactions". Translation: "I casually mention 'client' so many times I probably don't have one. How do I spy on my family without the need to actually talk to them?" (Also: Isn't (currently-plummeting) Facebook and others moving towards default-encryption?)
Gather all of the electronic devices, then smash them with a sledgehammer.
My client is trying to protect his family from scammers and other unsavory types, and isn't savvy in this matter, so i'm doing it for him.
Then you're doing it wrong.
Quite frankly, extreme monitoring and filtering isn't going to work. Scammers will hide their words to avoid filters, so active filtering doesn't work. The exchanges are managed quickly, so scams (especially phishing scams) get your data instantly, so delayed review of activity isn't going to protect anyone, either, though it might make detection a bit faster. There is simple no hardware approach that will work.
If, as others have pointed out, your client is an overly controlling patriarch, he needs professional psychiatric help. If he's just paranoid and scared, he needs professional technical help, and that's where you should focus your efforts.
Educate him and his family on scammers' techniques and tactics, and security practices. Explain how the teenage daughter will be victimized and harassed, because that's just the nature of the assholes on the Internet. From a network perspective, make sure they have updated antivirus software, and maybe an active monitoring firewall to scan HTTP traffic for viruses. A basic scanner for the known threats, and education for the unknown threats, and the client will be far better off in the long run.
You do not have a moral or legal right to do absolutely anything you want.
This is for a home / family network?
Has Facebook turned on SSL by default yet? I know that Twitter has, and Facebook has the option, not sure if they've thrown it on by default yet?
In any case, if they haven't, I imagine that it is coming, and then sniffing out contents of messages will not be so simple. You'd have to install a man-in-the-middle service with a fake SSL certificate and install said fake certificate as trusted on all of the client machines. (Good luck doing that on the iPhone.) And that's just to be able to see them in clear text. If you're trying to scrape them out, you're going to be constantly fighting with Facebook every time they change up their interface. Are you going to be tasked with updating this every time a new social service or game comes along?
It seems like the better approach may be to just have them learn some basic Internet safety.
Most of those apps will be using SSL encryption and thus your idea of a "tap" will not work. You need something like Pearl Echo, that puts a small client on each PC that's hidden. That will capture everything you want. But you say he can't put that on the PC's, which then leads me to believe this is in some way, an illegal thing he's trying to do...
As for cell phones and other devices, if they are on wifi, you could drop an Untangle box in there and get a good amount of reporting, also that will work from unencrypted sites on the normal LAN clients...
It looks like your client has a limited budget. Check products or services like Astaro Security Gateway (http://www.astaro.com) or zscaler (http://www.zscaler.com).
If you can ensure the mobile devices in your home use only your wifi to access the internet then a firewall / proxy / ips system like pfsense could work for you. It would require you to dedicate a system, many are available in formats not much bigger than your existing DSL or Cable modem. IDS/IPS from Snort, easy overview with ntop, filtering with whatever sort of oversight you want.
There should not be any legal issues if the family members consent to the monitoring. If they do not, stay away from this one.
I think your solution is user education, honestly. Your time will be better spent. All your monitoring will do is show them very clearly how they were scammed, not prevent it.
If any of those services use SSL, you cannot record the traffic you want from the network. There are too many varieties of services that they use, so if you capture only facebook and words, then you missed something else. If you capture everything, then you have so much data that you will never be able to sort it out. And no matter what you do on the network, the iphones would bypass it when they are on 3G.
Whatever solution you come up with, I would get 2-3 quotes from other people on the same project before you start. The price should point out that technology is not the solution here.
One happy family
I don't know. Can you?
Please read my Canon EOS tech blog at http://www.everyothershot.com
...setup a network tap between the router and the modem (buy separate ones if they don't have them already) leading to a PC with two network cards and a few TBs of hard drive space. Run Wireshark to capture and analyse the packets.
Haha, it sounds so easy when put like that, network packet analysis is a massive PITA - there is no convenient way to monitor everything sent over a network connection, and it may just be worth burning a nice big hole in your client's pocket to get that message across to them - the massive amount of time you'll spend picking through all the traffic, figuring out how to decipher it all, then actually reading everything you find - if you can bill by the hour it's virtually a license to print money!
Asking a site which users are generally known for disliking censorship and wiretapping about monitoring advice.
Made my day.
...is to drop the client. Seriously.
He wants Orwellian monitoring over his network that is not only unfeasible but would eventually prove completely ineffective. If he's this paranoid, what's going to happen when your kludge of a system inevitably misses a message or two and he decides that caused someone to fall victim to a scam? He's going to come after you with some shark lawyer and make your life incredibly annoying, that's what. In the end, his idea will not prevent scams and the like. It's only going to further a "big brother knows best and sees all" mentality. On top of that, it shows a frightening lack of trust in his family - both in their ability to "do the right thing" and in their general intelligence. Your best solution is to drop the client and not feed his totalitarian ego.
On the other hand, if this is really you wanting such a solution, the trust issues apply even moreso. Learn to EDUCATE instead of spy. You will have much better results.
And finally, if you're an ISP too clueless to do something on your own, GTFO Slashdot with your asking us how to spy on your customers. You should be ashamed of yourself.
tl;dr - Your plan is a bad idea all around...
"So after all this, you make my case for me. To end this stalemate, you must die..."
So, either you are clinically paranoid, and should probably address that issue before any technical ones...or you need to take a step back, relax, and realize you don't have control over everything. Your "client's" requirements are completely ludicrous, and even if you wrote a script for "him" to scrape messages out of Words with Friends, what about EA's Scrabble, or TextFree, or any of the 10,000 other iPhone/Android apps that can communicate privately between two parties?
My advice? Cancel your hardwired ISP, cancel all smartphones with network access, harden your doors, windows, and other points of entry and lock you and your family in your basement. There you go, no "unsavories" or "scammers" can ever access you or your family. I'm sure that will go over well with the wife and kids, but at least you're being upfront about it and not covertly spying on them through their electronic communication (which is what you *really* want to do).
When they object, tell them the other option (your little Napoleon complex and your in-home Echelon system), and be prepared for your, sorry your "friend's" wife to serve up some divorce papers.
Oh, that's right, you just want them to be "safe". Give us a break, even the most hardened Fox News or CNN watcher isn't really *that* scared of unsavory types messing with their lives, and if you are, please turn off the television and go for a walk in the park for a few hours.
Spying without search warrant is illegal. If you want to know what's your family up with internet, just ask them, talk with them.
If you can't have open discussion with them, it is already too late and better option is to get a better realtionship ( or have a deep look in the mirror yourself ).
I'll let Tom know.
it's not plausible to put monitoring software in the devices on the network (due to legal issues and a few other factors)
... so I'm wanting help building a monitoring soultion for the devices on the network (and not realizing that it's the same damn thing legally)
You can use a Linux box with two network interface cards and configure it to be a router. Then I would put it between the wireless access point/Router and the DSL modem/cable modem or whatever.
http://unixfoo.blogspot.com/2008/02/how-to-configure-linux-machine-as.html
You can use ethereal or TCP dump or whatever to record the traffic.
http://www.ethereal.com/
Analyzing it will be a pain because there will probably be a lot. I recommend giving all the devices you want to monitor static ip addresses, so you can ignore traffic from other machines that you don't want to monitor.
You will still have a *lot* of traffic. If the app obfuscates the traffic, they might use encryption, and you will probably not be able to do much about this at your skill level. If you root the device, and can figure out where the trusted certificates for the certificate authorities are, you can make your own certificate authority and then conduct SSL man-in-the-middle attacks and decrypt that traffic. This is quite possibly more complicate than you can figure out on your own though.
One thing that might be good is to get the blacklists of malicious sites from google or something similar. Then, you could at least make something that looked for traffic to those sites.
Anyway, good luck.
Walk away from this one. Whatever system you put into place isn't going to do what they want, and then you're the one getting the phone calls and nasty emails. There are bigger issues afoot here, and you don't need to be a part of them. No amount of money is going to be worth getting into this quagmire.
For corporate traffic, Don't put a box in between that traffic. If it fails, everything is down. Get a TAP, as you hinted, but make sure to get one that fails 'open'. Then, run Ntop off the TAP port. If the TAP burns up, or port goes bad, you still have network access.
It sounds like your "client" is just wanting to basically monitor on his family, so in that case, get a 10/100 HUB (not a switch) to stick downstream of your modem. Plug in your linux box on port 1, and the router/modem into port 2. Don't put anything else on it because.. it's a hub. Run Ntop on the linux box.
Join the Slashcott! Feb 10 thru Feb 17!
If you can set up your gateway to export Netflow data, you get excellent data for tracking your traffic (connection metadata) without all the bulk of keeping a full copy of the traffic.
There's a large number of tools available for collecting, analyzing and otherwise dissecting collected Netflow data, with a good number most likely available via your favorite free Unix-like operating system's packages collection. My favorite combo is to set up an OpenBSD box as the gateway, have it export traffic data via the pflow(4) facility and do the collection and analysis bits somewhere via nfdump/nfsen (see eg nfsen.sourceforge.net for info).
There are various resources available within direct reach of web search, but I would also recommend taking a look at Michael W. Lucas' book Network Flow Analysis for a nice treatment of Netflow in general (it uses flow-tools, but most of what he writes will be useful in the context of other tools too).
-- That grumpy BSD guy - http://bsdly.blogspot.com/
The "client" is most likely a husband trying to catch his wife cheating or wife trying to catch the husband cheating, hence the need to grab the "evidence" as the poster put it. This client has probably already tried going through emails and such with no success and is looking to have something that will get info before it can be deleted, if it even exists in the first place.
This is a thinly veiled attempt to get help in stalking someone's family. He's checking up on someone's wife (or husband) or kids. He suspects something untoward is going on and just wants evidence. This has nothing to do with 'protecting' anyone. I'm honestly surprised the editors put this one up.
It obviously depends on the laws to which your client is subject but, if there are "legal issues" in putting monitoring tools on "devices on the network," you may also find that there are similar restrictions, or at least hurdles to clear, in operating an interception capability as part of the network...
If it is just a private house, for members of a family, as the summary seems to suggest, chances are these will be minimal. If it will end up monitoring the nanny, cook or whatever other staff your client might have, you might need to have more robust procedures in place. In either case, it's worth checking it out if any part of your contract says "system will comply with applicable law" or anything like that — or just for your own peace of mind.
Use pfsense or Smoothwall. I personally like pfsense better, and it has better support for newer hardware, but Smoothwall has better graphs for what you're looking for.
And this used to be a nerd site.
Captcha: Notifies
Hire me or give me $$$ and I'll show you in details. Easy way and get all traffic easy in graphic form.
you just went full retard.
At least he can 'make his own cables and such'.
And not worth it. The couple of sarcastic comments that have started off the replys here are telling you this. The problem is you need to dump interesting data out of the packets, and there's no easy way to tell what is actually interesting. Also, this is a cryptographer's nightmare or dream depending on how you look at it. You're Charlie here, and that means you're the guy that everyone wants to defeat in this scenario. It's not going to yield much useful data since more and more communications on the 'net are switching to HTTPS. Also, I don't think you can fully appreciate the amount of storage this will require. I work with network video, and when I have to run a packet capture to do analysis, the problem is finding a storage medium to dump to that can handle the throughput. The only thing I can usually make feasibly work is a ramdisk. You can't do that from your linux embedded router. It just isn't going to happen. Now, I suppose you could only capture the headers of the packets. But again, that's not going to do you any good. You don't capture any of the payload then. Conclusion: Way more trouble than it's worth, and to do what you're talking about will cost a lot of money. Don't bother. Frankly, if you're client is that concerned about the traffic coming out of the house, wipe all the computers to remove any potential malware on them already, install a fresh OS, install your own keyloggers on the systems if it's the human element you don't trust, and be done with it. It's invasive as hell, but it's a lot less sinister, and easier, than trying to play the panopticon game.
It sounds to me like either you're either dishonest in your submission or your client client told you a load of crap and you believed him. Why would someone (with good intentions) who wants to monitor his family's Internet activities be worried about legal stuff? Perhaps the husband believes his wife is cheating on him and is trying to put together some kind of proof?
Detection is not prevention, OpenDNS for network DNS resolution, Web of Trust on the endpoints, and antivirus on all clients will give good protection on the cheap.
I have only every used smoothwall but others seem to like PfSense better. great at getting a high and low level view of traffic on your network. I say simple but there is some configuring involved and you'll need a separate box with 2 NICs. it can be a low end system though nothing fancy, something like 3-5 GB of space and 256-512 MB of RAM would do you fine.
I admit the scope of the project is overwhelming, and I've told my client that he's asking for an NSA quality project. I will direct him to this post and your replies to help him to better understand the nature of his requests. Also, it appears that my article was truncated before being posted, so some of the explanatory bits were cut off, although the core of the question is still there for the most part. And yes, this is an actual client, not myself. I already suspected what most of you were saying, and tried to tell him that, but computers are a big 'mystery box' to him, and I can't seem to nail stuff home on my own. (IF it was myself i would have all already solved this problem.) Also, I'm a little surprised at some of the hostility and non-seriousness i've seen here, but I suppose it is to be expected considering alot of the drama and arguing i've seen going on in other arguments. When I originally wrote the article, I did specify 'serious answers only please, I don't want to start an argument, but a bunch of random answers that are unrelated won't help me solve this problem' And to be more specific, it's a home network with a cable connection. (I obviously can't be too specific due to his need for anonymity to avoid 'alarming' his family to his clandestine monitoring intentions). He does reasonable cause for suspecting something is going on and just needs to have information available to aid him in making decisions about some unusual behavior. and yes, I know that you can't get 'screenshots' right off a client PC through a network, by screenshots i meant some kind of recreation of a visited website, or just text information in printable form off some kind of analyzer software. I really would like to solve this problem, but I agree it's an excessive project. He wants the moon without having to go there to get it, type of issue.
Get a Palo alto firewall. You can filter by application, and even make firewall rules like "allow reading of facebook, but disallow posting", or even "disable attachments".
Of course, you didn't exactly specify budget...
If there are iphone/android phones involved, all they have to do in turn wi-fi off and they do an end-run around all of your fancy logging.
What's next? "My client has an urgent need to dispose of a number of black trash bags, the content of which are roughly human-sized. What would be the most efficient way of doing this? His family must not find out."
squid as a mitm ssl proxy? but like so many previous commenters... why? other than messing w/ a roommate (ala http://www.ex-parrot.com/pete/upside-down-ternet.html) this is really useless. but hell, billables are billables!
Get a router compatible with tomato firmware, install tomato, and then install rpcapd on it (no need to compile from source, there are standalone binaries out there compiled for your router's CPU). Then use wireshark to monitor and capture the traffic. After that you can take your pick of software to parse the pcap files.
Careful with names containing L slashdot.org/~AiphaWolf_HK slashdot.org/~AlphaWoif_HK slashdot.org/~AiphaWoif_HK
Because I would not touch that project for less than 5 figures plus an ongoing support contract of at least very high 4 figures or low 5 figures.
I am highly suspect of the "protect his family from scammers" and the "monitor and record all outgoing traffic"
If he is really interested about protecting his family from scammers then educating every in the home that "everything on the internet is a scam unless you personally know the person" is all that is needed.
Finally, if a lot of ipads and iphones are involved, your system is completely worthless as turning off wifi will disable your system completely for that unit. 3G on their ipads and iphones will bypass everything you can think of doing unless you force a VPN back to the home so that all traffic goes through there and refuse to share the admin password on the devices.
Do not look at laser with remaining good eye.
Install IDS (SNORT). Sniff for what info you are looking for. Cacti is nice for bandwidth monitoring.
Very clear author wants to monitor his kids social lives...
Won't he be surprised when he finds out it's you that's having an affair with his wife!
I didn't have to do nearly the amount of stuff you are asking for. But I did throw in a spare box I had laying around and installed Untangle to manage / monitor the kids playing those damn club penguin virus sites and the like. It did the job and then some. But I do agree with the rest of the posters here .. this is almost impossible to do and way .. way .. unnecessary.
They have a whole army of people trying to do this, and yet some stuff still gets through.
Felt the need to post because I don't see any good advice for this guy. I understand the clients need to protect his family from Internet based crap, even if he is being a little over-zealous - that just means he needs some education, not psychiatry.
If this were my client, I would first empathize with him, and ask him what is real concerns are. Then explain that monitoring everything that goes in or out of a residential Internet hookup is cost-prohibitive, not just the technology, but the man-power to manage it all - it probably doesn't make sense. But we can take reasonable precautions to protect the family from bad stuff through a little technology and a little knowledge. Let's use some basic protection (with OpenDNS - free) to filter web content and get some education on what scams and phishing looks like, then extoll the virtues of anti-virus, monitoring and security subscriptions. A good router like Fortinet / Sonicwall would also help you log, analyze and control to some extent what is allowed to flow in and out and when.
See, now you've just built a little recurring revenue opportunity. Bill him a flat monthly fee for offering to make sure everything is up to date and the network is as secure as one can reasonably expect. Just put some limitations into your contract i.e. X hours of on-site support per month.
I once achieved this on web traffic for a large corporation back in the days where internet @ work was "new" and pr0n was the main "misuse" in working hours.
I proposed to do it as ethical as it could be done, so we agreed about obfuscating domains, the idea was to educate users that were "new" to internet, so the administrator would only get notice about a "violation of terms". (using regex for the usual++ pr0n and other stuff related terms).
There was no actual "snoop", no logging, just a hint on who to talk to "use internet wisely and stop fooling around in working hours".
If i had a request like the one in this "Ask Slashdot" i would just tell the guy it can't be done, or at least, i wouldn't do it since it's not ethical at all.
you just went full retard.
I'll second that. All he has to do is sign onto his kids' facebook account and request a full data dump from time to time. But I have the feeling he wants his family to not be aware of his little scheme.
My advice to the submitter- this is a sketchy situation. The guy is asking for some really heavy corporate-grade network monitoring. A lot of these services run encrypted or at least somewhat secured traffic, for and good luck sniffing shit if they use https for facebook, google, and youtube. The guy is either far too paranoid and needs a Shrink, or he's up to No Good, or he's just drank a lot of Kool-Aid and needs to be sold on a much lesser and more appropriate monitoring solution.
So much of this post reeks of dodgy.
Sounds decidedly odd that so called 'client' needs screenshots and text messages as 'proof' against his own family, yet it's for online 'scammers' and it's his own LAN, but it's your having legality troubles? Monitoring 'words' app.... oh heavens.
Sounds to me like a jealous paranoid partner.
So much crazy in the world.
Google pfSense and set it as your firewall.
I am a Linux hobbyist and can comment on the Linux router option. Totally free if you have old hardware, but limited and will not cover all of your listed requirements.
This sits between my ISP's provided modem and my wireless router which serves the living room computer, bedroom, office, and a wireless laptop and phones using wifi.
I use Debian 6 on an old Semperon with 1 gig of RAM with two NICs. Overkill I know, substitute your hardware on hand and Linux needs here. It's nice having the option of a full desktop if you need it, but I usually ssh into it and have run it headless before. I have isc-dhcp-server installed.
For live viewing I open a terminal in Gnome or ssh and run screen split into a four-way window. Two screens run iftop- one for the external card and one for the internal card. The third window runs tshark for packet sniffing. You can export tshark's output into a log for examining of network traffic, sites visited, etc.
urlsnarf (part of dsniff) will also allow you to log sites (URLs) and it logs from all sources (phones, etc. as long as they are using the home network). This is proof against deleted browser history or content to confront someone suspected of illegal activity in the house, cheating spouses, crappy house-mates, etc. msgsnarf comes with dsniff and supposedly can log messenger traffic, but I have no experience with it.
Logkeys is a keylogger and will log anything as typed from the keyboard on the machine it is installed on. This won't work for phone logging obviously and conversations are one-sided.
If your client is jealous, paranoid, suspicious, or needing to protect themselves then a setup like this would work adequate with minor blind spots and annoyances. I'm just a hobbyist and have used these things (logkeys is good for saving school papers if your word processor crashes). No doubt there are even better options out there, but for someone who is not technical it may work well- as long as they know how to access logs, etc. on linux or you could aggregate it somehow.
I allready KNEW alot of what you're telling me..it's excessive, unecessary, paranoid, etc...but I needed to get a second opinion anyway, for the sake of sanity. (Obviously sanity seems to be at a short supply somewhere in all of this), so I linked my client to this article so he can see for himself how excessive his requests are. I agree that monitoring at nework like this is way over the top, that somone needs to get booked into an asylum, (Probably me for even thinking of posting this to slashdot lol), and the whole 'legal' issue thing is him being a nutcase over the leagal matters like he usually is (but not to this extreme). Unfortunately I am the only source of real tech advice for him, so I have to at least address the problem instead of saying 'I refuse to handle your case' or something. I need some reply for him at least. I do understand the situation he's in, but it's nothing like an affair. I know this family well, and this is more of an 'old friend of the family' type of client than a paying one, although i'm sure i'd get some amount of renumeration, but it's also an interesting scholastic exercise in advanced networking. Obviously it's not exactly feasible, like I originally thought, but I thought maybe I had missed a development since I went to college, and wanted to see what the experts had to say. (It's really telling when most of the replies tell me to book myself into an asylum lol)
You know how to make your own network cables?
Has the wrong consultant
You need to consider that these days people are starting to use HTTPS by default for things like Facebook. You won't be able to inspect the contents.
If it's scammers he is genuinely worried about, education will solve it, not monitoring (which will catch it too late, after the scam has already started).
Oolite: Elite-like game. For Mac, Linux and Windows
you are welcome.
Yes, his wife cheated on him, but she probably won't leave him, and his daughter lost her virginitiy ages ago.
My client is trying to protect his family from scammers and other unsavory types, and isn't savvy in this matter, so i'm doing it for him.
combined with
After days of discussion with the client
LOL. If someone can't be educated in "days" then they simply can not be protected from themselves. By "unsavory types" I assume he means us /.ers, which makes it even funnier. Would you trust me with your 19 year old daughter? Thought so. Well, she'd probably kick my butt anyway so don't worry too much.
I must be the only guy in /. with little kids that click on every spam popup window and sign themselves up for anything because... they're little kids. That is why their monitor is in a public part of the house easily viewed about 5 feet from my home office desk. My wife and I have caught them doing all kinds of ridiculous stuff and have (mostly) calmly used those events as "teachable moments", with excellent results. We've caught them watching remarkably inappropriate youtube videos, applying to work at the local Culvers (he was only 7 at the time), installing all kinds of spyware toolbars and stuff (whats more evil than a kids TV show that only exists to sell toys? I know, a kids game that only exists to install spyware! ). I'm pretty close to wiping his machine and installing debian, but people keep buying him windoze only "educational software" to my intense annoyance.
Also I must be the only guy with elderly relatives with a known proven tendency to fall for telemarketing pitches (clean your furnace ductwork for $400? Hearing aid for $5000?).
There are reasons to block/track/examine/log things beyond trying to catch the wifey cheating with the pool boy, in fact keeping a really close eye on little kids and elders is being a nice civilized responsible guy, not a jerk. In comparison "easily read evidence" and "use as proof" is simply being a jerk.
I will suggest that printing this ask /. out and giving it to the client will probably be extremely educational for the client. Probably this is one of those "the client is a little overbearing and I need some backup in arguing with him" situations. We should demand a cut of the proceeds from the consultant; maybe a tithe to the EFF would be appropriate?
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
Looking at the above replies, I may be reiterating previous comments So be it - it just means that more than one person had the same enlightened thoughts. If your client wants to monitor all the traffic coming into and out of his LAN, then good luck to him. Honestly, what kind of paranoid person wants that much control over their family members? Moreover, what kind of person wants to suck you into their paranoid, evil, misshapen worldview? There is likely, in my opinion, something seriously flawed with their thought processes. That and/or he is probably lying to you about his real reasons. Saving the family from the evils of the internet? You can't protect a person from all of the evils of the internet through blacklists. The internet comes at you from all sides. Monitoring software will not do anything except bolt the doors after the horses have fled. Proper education about social engineering and decent values imbued by proper parenting will do far more than packet filtering. The only thing that "client" is suggesting doing is teaching the people on his LAN to be afraid. Of him. Maybe his wife is chatting up an ex high school flame. Maybe his kids are hanging out with ne'er do wells, doing drugs or just reading books not on his approved list. His family problems should never require your technological solutions. Getting involved in something like this is akin to walking into the monkey cage at the zoo. The only thing that is going to happen is that you are going to be clawed, abused, shrieked at and covered with monkey byproducts. You have to draw an ethical line somewhere.
HP ML110 Gen7 server running PFsense and various packages available for PFsense.
"If any question why we died, Tell them because our fathers lied."
By screenshot I was referring to from the analyzer software or something similar.
To clarify, I have studied formal networking at a local college. Alot of my article was truncated, apparently because it was too wordy, but at least the core of the article is there. My comment in regards to my education was to help everyone understand that i'm not a redneck hilbilly that's never set up a subnet before.
The easiest way, assuming your switching infrastructure supports it, is to setup a port mirror of the interface(s) that go to your firewall or gateway device. Send the traffic out to an interface that goes to a device that listens in promiscuous mode. You could buy an expensive appliance, like a Network Instruments Gigastor, or you could build a server with decent I/O (faster SATA disks on a decent array would actually suffice in most situations, but you could go with SAS... SSD would probably be overkill, plus if there is any situation where you would run through the lifetime write operations, full-time packet capture would be one of them). On this server, you can run TShark (the command-line version of Wireshark) as a service or daemon (for Windows or Unix-type systems, respectively). You can setup it up for a FIFO system where it will capture all packets and overwrite the oldest as necessary. I have setup a large number of similar devices for a decent sized enterprise, and we can store between three and seven days worth of traffic, depending on the volume.
If you should choose to use TShark, setup a file share to the capture files (SMB or NFS) and you can use the full Wireshark application to analyze the files.
If this person is primarily concerned with malicious sites/data you might get some traction with installing DansGuardian and a transaparent Squid proxy on a linux box serving as the site's gateway. Direct all HTTP/HTTPS traffic through the proxy and enable DansGuardian's selection of malware sites. It won't see/find everything but I've found it can be good at preventing browsers and apps that fetch web content from getting to malicious sites, and its relatively low-impact and easy to setup (as opposed to a TAP port w/ PCAP dumps going to an IDS, which is possible albeit significantly more complicated and costly).
Boo! Boo!
This seems like a big overarching project that isn't going to be possible. It reminds me of a request that I got from my client: He wanted to be able to block his employees from wasting time on Facebook. I told him that I could block sites easily enough, but it's not foolproof and a savvy enough user may be able to get around the blocks. The client then explained that he *didn't* want Facebook blocked, because his employees were involved with social networking campaigns and they needed to be on Facebook. He just wanted them blocked from *wasting time* on Facebook.
Networking monitoring, filtering, and blocking are not that smart. You theoretically *can* capture every bit going through a router, but it's going to be such an unruly amount of data that it'll be functionally worthless. For the amount of time you'd spend sorting through all of the data for a single user, it would be less time consuming to stand over your employee's shoulder all day and watch what he's doing. You can filter based on various things, but you will never block every scammer, every virus, every porn site, or every waste of time. Or no... that's not right, you can block all of those things, but it means effectively cutting the network cable and denying all access to the Internet.
This is one of those things where, as the expert, it's not your job to fulfill your client's request. It's your job to explain to him why his request is misguided, and offer some solutions that might help him. You can block access to particular sites, for example. If he doesn't want his kids on Facebook, that's not hard to accomplish. If he doesn't like his kids using Words with Friends, you can turn on parental controls and deny the kids the rights to install applications on their phones. You can provide advice and educational resources to avoid scammers.
If he's dead-set on monitoring, then try to narrow the field a bit-- what exactly is he looking for? You could probably set up a system that gives him a list of all web sites visited from his home, for example, but giving him the content of all interactions is a bit more difficult. It also doesn't prevent his children from using the Internet at a friend's house or at school. He can set up email accounts for his children where he has access and can monitor their email, but he can't prevent them from creating/using other email accounts.
So the take away message here is that what he's asking for is unreasonable and paranoid. He can't collect or block everything that he wants to, and even if he did, there are ways that his children could probably circumvent his blocking/monitoring. And anyway, it's kind of... well... crazy and creepy. Focus on giving him a few tools to prevent the worst: install antivirus software and educate everyone on safe internet practices. You can also try blocking stuff, but if you remember being a kid at all, you should realize that they're just going to get around the blocks.
A few things:
Better firewalls, including even the lowly dd-wrt and the now-defunct Snapgear, support syslog so you can capture and create your own custom reports, and dd-wrt reports total bandwidth usage on a daily, monthly and and annual basis and will retain that info until you do a reset (or until it runs out of NVRAM). It can come in very handy if your ISP claims you hit your bandwidth cap.
Another thing you might want to try is IMFIREWALL/WFilter in monitoring mode to see which users are doing what on your network. What is required is to either put a port on your switch (connected to your gateway/firewall) in either promiscuous mode or a two-way mirror to the port that connects to the firewall.
http://www.imfirewall.us/WFilter.htm
It will report the number of hits to instant messaging, streaming, social networking, porn, gambling, stock trading, and any other criteria you can think of configuring. You can also put it in filter mode so it will basically kill any requests that you disapprove of, but in monitoring mode you can create custom reports of who is doing what.
Other firewalls will include these features as integrated, but some vendors (Cisco, Sonicwall) won't sell you the complete feature set for a flat price; they nickle and dime you because it's more profitable, and when the unit dies, good luck transferring those purchases.
You might want to check out m0n0wall as well, and get a good syslog app so you can capture detailed logs and create your own detailed status reports.
The Christian Right is Neither (Christian nor right). See: Matthew 23, Matthew 25, Ezekiel 16:48-50
You might think this isn't likely to come up, but you have to bear in mind you're not just intercepting his family's communications by doing this but any guests and also the communications of anyone who is communicating with his family. I'm assuming by your reference to the NSA that you're working with someone from the United States and this makes things tricky. Many people mistakenly believe, "well I paid for it, I can do what I like with it", but this is not the case, particularly with communications services. A lot of states have "two party consent" wiretap laws, which means even if hypothetically he could consent on behalf of his children (which is debatable), he can't consent on behalf of the persons they may be communicating with.
Let's say he were to take evidence from this into school and say: "My child is being bullied!", the question would be how do you know? Also if he were to discover anything serious (grooming etc), what he discovered may not be admissible as evidence as fruit of the tainted tree. Also you may wish to see a lawyer, because you may also be committing an offence installing this.
The other thing is that an intelligence source the well is going to dry up pretty fast the moment he presents any evidence to his family. He also better have discussed this with his wife, because he certainly can't consent for her and her reaction to being spied on may be somewhat awkward. If I know teenagers, their reaction is not going to be the one he'd hoped, they'll be very very very angry and the lesson he's trying to impart will likely be lost.
Greetings,
As a network engineer for a major financial trading company I've some experience in this area. I've also served as a network engineer for several companies in various fields (Internet Service Provider, Professional Services Vendor, Extremely Large Retail (Borders... I'll miss you.). In my experience traffic monitoring becomes a key requirement of any efficient & secure organization and a key responsibility of any qualified network engineer.
Depending on context traffic monitoring has several definitions. You (or your boss) appears to be headed in the direction of security and/or packet intercept. This is one of those projects that is rarely implemented well. Furthermore there are major legal and privacy concerns. Before you proceed further, I recommend you receive written confirmation from your employer that his employees (or family in this case) is notified of the scope and depth of monitoring. In my opinion if you do so without this confirmation, you are morally and professionally just as responsible for any abuses that may occur.
Let's begin with some of the options that you have available to you.
SNMP - The most basic network monitoring tool, supported by most devices out there. For example, a Cisco router or firewall is polled by a SNMP monitoring application, showing interface usage as a function of packets per second or total throughput in both directions. Not really what you want to do here but any discussion of "traffic monitoring" should start here.
Netflow - Netflow is set up in a similar manner. A Netflow supporting device is configured to send a record of traffic conversations to a collector and/or analyzer. This could be a router, switch or firewall. This begins to provide some of the information that you are looking for. Flows are packets matched with the same source, destination and ports. Netflow provides valuable information for this reason. What ports are in use? What are my most common destinations? Who is my bandwidth hog? An analyzer might also include DNS look ups as a feature, so a Facebook destination address shows up as Facebook's DNS in a reporting chart or export spreadsheet.
To go any deeper than that, your looking at packet intercept, which can be done in a few different ways.
Hardware:
I'm assuming that you don't have a Cisco 6500 or Nexus 7000, so simply buying a $30,000 packet intercept blade and sliding it in is out of reach. You appear to be much more familiar with software (and comfortable with those options) so I won't try to steer you away from that. I'm only going to briefly cover your hardware choices. These may or may not provide you with the information your looking for. For example, depending on the application even the internal messaging component you mentioned could be encrypted and the information gibberish.
Firewall - The simplest and easiest "appliance" you can buy is a next generation firewall. Such as a model sold by Sonic Wall. The TZ Network Security Appliance Series has a lot of useful features, including DNS intercept, filtering, packet intercept, built in netflow collector & analyzer, etc.. I haven't used the packet intercept features myself, so I can't tell you exactly what information can be accessed or in what format.
Specialized Appliance - An appliance specialized for packet intercept and analysis, other than the Cisco packet intercept models, I haven't used anything else so I won't mislead you with guesses or half truths. I will say that generally these are going to relatively large financial investments.
Software:
Proxy - Maybe your cheapest and/or best bet. Implementing a web proxy on a server (such as the open source Squid project) should give you most of the information you are looking for. DNS, content analysis, packet intercept and "scamming protection". At Borders, each of our stores ran a Squid proxy server for internal traffic, and public traffic went through a pair of McAfee proxy appliances (oh how I hated them).
People like you keep me in business. You like to setup a kludge and we sell a proper enterprise solution after your client realizes your kludge is total shit.
Recommend your client purchase Websense. Don't worry; you'll get plenty of billable hours installing and configuring it.
There's a bunch of comments talking about how impossible this is, but it's really not that difficult. It costs less than a few million USD to do it across an entire Enterprise using software (and hardware) specifically designed for the purpose. I think I could put together a cheap version for a family-type scenario using freely available software and hardware lying around the house. Companies like Vericept (aquired by Trustwave), Vontu (acquired by Symantec), and even NIKSUN all play in this specific space.
Check out the xplico project. It's basically the successor to dsniff. It's free and opensource. It will do a lot of what the commercial products do, but without the bells-and-whistles like case and workflow management, access control, audit, FIPS compliancy, etc.
Install a Linux system with a simple HTTP proxy service enabled using a hostname that is accessible both via their local home network (WiFi) and publicly (i.e. cell network), configure the iPhones to use said proxy, allow authenticated connections to the proxy, install xplico on the proxy system. You'll be able to see who goes where depending on the granularity of the protocol dissector. For example, with SMTP, it will decode the To, From, Subject, etc headers. HTTP has the Host, URI, etc. I think xplico also support various IM protocols -- including Facebook. But I haven't really looked into it recently. The framework is extensible, so if you have the resources you could probably build a protocol dissector for the Words with Friends application (but I bet it just uses HTTP). To top it off, I think xplico even has a nice web-UI for the management piece.
If you wanted to take it a step further, you use something like described on this wiki article to set up an SSL-in-the-middle. You can install a trusted CA certificate on the iPhones using the iPhone Configuration Utility. Then you can use that CA to sign certs for domains on the fly, legitimately decrypting all the traffic. This will require quite a bit of CPU, though. I bet if you created the certificates with a far out expiration you'd eventually have a nice cache-hit-ratio.
That said, the post sounds a bit too much like, "please do my job for me!"
You're running into most of the important traffic taking place via HTTPS. So in addition to the problems already inherent in trying to grab this much data in any remotely useful fashion (onto the hundreds of dollars of HDD space), you're also going to have to require a MITM SSL stripper. SSLSniff or DSniff probably don't entirely have the functionality you're looking for, but may be a good place to start looking.
Anyway, I'm otherwise with everyone else on this: likely impossible (on a civilian budget anyway), and depending on reasoning for wanting it implemented, likely also unethical.
Use a Cisco switch, configure SPAN for the port that his router is plugged into : http://www.cisco.com/en/US/products/hw/switches/ps708/products_tech_note09186a008015c612.shtml
Then run ColaSoft Capsa http://www.colasoft.com/capsa/ on the monitoring port.
Works like a charm - you can even setup traps and alarms for anything you want.
...unless your family is the mob.
You are asking for ways to detect criminals inside your own family.
I once was hired by someone who had to get rid of something like what you are talking about. You would be in the position of this person's ex-husband. He installed taps on gmail, put in a keylogger and was able to do things like read all her mail, know all her passwords immediately after they were changed, harass her privately (phone) and publicly (various defaming websites), and monitor the children's work even popping up messages on their PC saying time to do your homework, quit posting on facebook, etc. even when he was not living with them (he had a house down the street to hack in from and spy on them).
Look, you are a piece of shit and I hope you never come back. What you are talking about is absolutely criminal. You are going to detect scammers by sniffing the local lan? Oh maybe you want to catch the IPs people are messaging from? Maybe you should just move the fuck out and get a life, Loser with a big fucking L.
I think you could effectively do what you need with a simple PC, two network cards and untangle installed. You can set it up to block web sites and content, protocols, and it even has an attack blocker which is updated automatically (re:SNORT) It can automatically generate reports and email it to your friend which will show (via IP address) who did what to whom and when. It will take 4-10Hours (estimated) to set it all up. I occasionally have to set this up for a client when internet usage gets out of hand to figure out why. I don't work for untangle, or represent them, but as best I could interpret, this may be your most practical option. Unless your client feels it's worth about a grand to fulfill their goals, I'd recommend installing a reliable ADSL router, securing it, and managing their expectations. Good luck.
"and some game called 'words' which has message capability"
So the guy wants to wire tap everything they use, period. Even a freaking games? Most of those games already filter "bad words".
One thing you can do is set all their DNS servers to use OpenDNS's FamilyShield. It will do a pretty good job of filtering bad sites/etc at the DNS level.
As for logging, I wouldn't. That just sounds like not only violating your family's privacy (okay so they're under-age? That may be okay) but should anyone else happen to use the device and have no clue the things they were typing or doing were being recorded could pose a big issue.
It's a thought anyhow.
As has been pointed out, deep packet inspection of everything isn't realistic.
You might start by logging websites visited, either with local monitoring or using open dns.
-Dave
While I'm not a troll by any means, the level of hostility and such has led me to feel it would be a good idea to apologize to everyone for having wasted their time with a ridiculous inquiry. Trolling was never my intention, but it appears I may have done so unintentionally by asking to be informed by people that are experts of many fields, and intelligent and well educated, so you all have what apology I can offer. And I'm quite serious. I don't think I can really say anymore, so I'll leave it at that, link my client to this article, and let him judge for himself.
Ethical arguments notwithstanding:
Put up a Checkpoint firewall with IPS software blade. Get their smallest appliance, configure to suit. It'll set you back a couple grand. You can block by site, IP range, and other stuff, and the IPS will go a long way toward protecting you from bad guys. Don't allow any inbound session initiation, and filter outbound traffic by port. This covers layer 3, 5 and 7 methods.
Get a Telemate Netspective Webfilter. The only port you'd have to passively scan is the outbound port of the switch to which it all attaches, which I presume would also be a WAP, or network of WAPs. This will give you the ability to block by category, and the categories are updated automatically by subscription. It'll set you back another several grand.
Both of these involve recurring annual licensing, support and subscription costs. Bite the bullet and pay up. They'll accomplish at least most of what you seek.
After using a few boxed solutions including several mentioned above (ClearOS, Untangle, Smoothwall, Zentyal, & pfSense) I ended up going with ClearOS. They all have their up's and downs, most of them have paid services that they will offer you which with a little configuring and time you can accomplish yourself.
From what I can tell your going to be interested in something more then just a simple router/firewall my suggestion is grab one of the free ones available build a low end machine with two nic's and have fun.
For some more reading take a look at this:
http://en.wikipedia.org/wiki/List_of_router_or_firewall_distributions
I settled with ClearOS because I wanted to have a full server at my disposal and it was lite on hardware requirements where as Zentyal in particular were heavy on server resources now while my home router is old it isn't by any means a slouch, dual processor Athlon MP 2200+, 2G Ram, 120G 7200rpm, (10 internets if you can guess the maker/model of the mobo /.) I have held that box at 80mbps inbound constant for days on end downloading..... "TPS reports" yeah that's it, without showing any marked reduction in its performance, since install its only been shutdown twice for hardware maintenance(upgrades) before I shut it down the last time the up-time counter showed 240day's.
After all of that I personally think Clear is more then likely something your client could easily use to monitor traffic to and from the network, utilizing some of the built-in features or adding in something like ntop for ultra detailed logs of everything going on anywhere on the network. Your client could easily access the the logs by going to an internal web page and reading the logs at his leisure, a fair warning though if you go the ntop route the amount of logging is immense when I said everything is logged I mean it you can easily have several gigs of logs per day if the network is heavily trafficked.
What I used to love about slashdot was the discussions that would result from articles and questions. But now everyone just jumps down the submitter's throat (though part of that is deserved in this case, especially coming from a brand-new account) if the question isn't phrased properly. Yes, censorship is bad, clandestine monitoring is bad, we should all trust each other, etc. but we all know that isn't the case. Only by offering solutions can we help improve our collective level of problem-solving. For example, I'm already thinking about several ways around this: is there any way to stop a user from using a VPN (or use your monitoring solution to impersonate one), and how are you going to deal with SSL traffic?
This sounds obsurd, but the guy needs to install video cameras pointed at all this computers. If its truly educating his family that is his goal, the sheer obtrusiveness of this idea will prove a point and make family members careful. And if they truly are ignorant of possible threats and do something that compromises security, then they can go over the footage together. Should be easy to install, fairly cheap and get the point across. How did this make front page?
RSA has an appliance called netwitness, that can basically record and reassemble any traffic that passes. i've seen a presentation of the tool and it looks like it is very, very advanced. i don't know what it costs but i think it is very expensive.
Paired with Hexamail for eMail is pretty secure with a lot of logging options for both programs.
I'm not going to get in the argument about too much monitoring vs trusting etc. When the stupid laws say that I, as an IT Admin, can go to jail if I let some perv in Sales gets away with distributing naked pics of his teenage step-daughter, damn straight I'm going to find the best way to protect myself and the company.
After it was set up, I spend less than an hour a week admin time.
Sometimes, just knowing that there is a ton of logging going on, employees are almost too scared to try to pull anything, which takes a whole level of complexity out of my day to day work. Exceptions are made for World events, news stories, March Madness, etc so people don't feel oppressed. It's been 7 years since there has been a single employee that has made a conscious effort to search for porn. 7 years. (sure there are pop ups on certain legit sites that the advert infected the machine, but that's bound to happen sometimes.)
I understand how a locked down Internet makes it difficult to get work done, which is why there are a lot of things we don't block. Make no mistake, we are logging every bit of it.
Wow. I wonder if that is actually the longest summary ever posted to /.
if you are looking to record facebook conversations and much more you could do it from the network but that is inefficient.
just use a product called refog it will record key strokes, passwords, take screenshots, and then upload them to an FTP site of y our choice.
there is even a mobile version http://www.refog.com/phone-spy/
this is a simple project and should remain simple
And to be more specific, it's a home network with a cable connection. (I obviously can't be too specific due to his need for anonymity to avoid 'alarming' his family to his clandestine monitoring intentions). He does reasonable cause for suspecting something is going on and just needs to have information available to aid him in making decisions about some unusual behavior.
In other words, he thinks his wife is having an affair and has seen some "unusual" transactions on the credit card or caller ID numbers, and is trying to gather "evidence" to use against her.
If you're the client, this is a hugely bad idea and could get you in very, very big legal trouble.
If you're not the client, then this is still a hugely bad idea and could get you in legal trouble.
Probably the second best move for you is to contact the wife and let her know what you've been asked to do. You might even get a larger paycheck out of it that way.
But finally, the first best move for you is to contact a qualified divorce/family law lawyer in your jurisdiction. Because before you take any further moves, you want to know which ones will get you sued and which ones will get you in jail.
Get a small Astaro box. It should do all you need. Looks like they were just bought by Sophos, but they are maintaining the product lines.
http://www.sophos.com/en-us/products/unified/utm.aspx
The best way would probably be to install GPS transmitters on all vehicles, and then write some code to plot them all on a map, but the price and privacy issues would probably prove this too difficult.
A second option would be to point video cameras at a stretch of road and then use some image recognition software to get the information you need.
Another option would be to get a group of small children, have them play the game where they pick a color or model of car and count how many they say, and then just tally their results.
Use WireShark and Print All The Packets!
I will not be pushed, filed, stamped, indexed, briefed, debriefed or numbered. My life is my own.
Download the free edition; it'll be all you'll ever need.
http://www.untangle.com/store/get-untangle/
Cheers!
You'll never be able to filter the scammers completely no matter what you try. If you can't detect a scammer right away yourself, doing so afterwards by processing log files won't change that, you'll still get scammed. At best you'll be able to filter 99% or so of SPAM email and some known malware and viruses. Expecting a mini-barebone to be able to handle any serious internet filtering is also not realistic. Stuff that will filter even a minimum of multi protocol internet access, requires quite a lot of CPU power and plenty of real-time access to internet databases to check traffic/files for malicious content.
Either yank the Internet plug, or make sure your client gets educated on scams, malware and such. Education and common sense have stopped more scammers, malware and such than all firewalls and virusscanners combined.
I was promised a flying car. Where is my flying car?
I have kids and have not even attempted to do anything like this. What I do is discuss with them the dangers and possible pitfalls of online interaction. The bottom line is "Do not converse with ANYONE on the Internet that you do not know in person. Come to me if you need an exception to this rule.". I also make sure that computers are in a public space int he home. No kid is allowed a computer in their bedroom.
Monitoring connections is pretty easy. Assuming there's even a modest budget behind this project, I'd recommend upgrading to a decent firewall with robust monitoring/logging built in. I use Sonicwall NSA appliances with log servers running their analytical tools, but that space is crowded with many good alternatives.
Your client will never be able to prevent his family from being scammed, though. Sure, you can block phishing sites, etc. by subscribing to various blacklists, but scams rely on the victim's credulity and that exists outside of your control as a network admin.
Actually pulling out data from within those connections simply isn't going to happen. It's not even remotely practical in too many ways. You will only disappoint your client if you ever imply that you might be able to give them access to that sort of data. Be completely honest about the limitations of what you're capable of. You can always block certain traffic types, but if you're dealing with mobile clients, they can sidestep those blocks by disabling the wifi connection and just riding on the cellular.
Let us know how that divorce settlement goes.
Well, he could be the divorce *lawyer*... Just saying.
In other words, his daughter is camming with boys and he wants both fap fodder and a plausible story to cover his ass in case he gets caught?
That's just a theory but no matter how you slice it this is a client you don't want.
My client needs to monitor traffic on his LAN, particularly going out to the internet. This will include websites like Facebook, Myspace, and similar, including from mobile devices.
It also appears that the client's family is using iPhones and some game called 'words' which has message capability.
After days of discussion with the client, it's not plausible to put monitoring software in the devices on the network (due to legal issues and a few other factors),
Ok - first of all, story submitter, no offense but at this time, you're out of your depth. Also, I wanted to point out that things above does not make sense - we are talking about the client's family, as in their children, right? Because, if that's the case, then there are no legal issues with installing software on their devices, since they technically own it (even if they provided them gratis to their children).
Basically, you won't get the level of knowledge you would need to implement anything this client needs by just reading slashdot posts. If your client understands that this will be a major learning evolution for you, then perhaps this could be worthwhile.
squid (or similar proxy) + splunk
Okay, you find it interesting. Look at any corporate Firewall and monitoring system and you have your answers. Hell I have an O'Reilly book from the very early 90s on TCP/IP security that covers all of the topics you need to know. The technology is nothing new, the only real variations are in how the logs are stored and parsed.
-The wise argue that there are few absolutes, the fool argues that there are no probabilities.
That's really all you need, Wireshark and a managed switch (business-grade) that can replicate the data stream from one port (or VLAN) to the other one.
And then you send the raw data packets to whoever wants to check these things, after a couple of hours they'll get bored and can the whole idea.
Problems you'll encounter:
- FB/iPhone/MySpace/E-Mail... data is (or should be) encrypted, you can't read it unless you do some really nasty things like set up your own CA, generate certs for all individual domains and then proxy SSL connections through your own, which the client then also has to accept (which if there is no link for SSL exceptions (which is common in apps) to the user (such as in a browser) won't happen). It's easily detectable and easy to avoid unless you literally route ALL traffic 0/0:1-65535 through a proxy which logs and sanitizes it.
- Although these days this kind of interception is possible, a simple bare bone Linux box won't do. At the level you're describing (SSL proxies and wireshark continuously logging) you'll need a disk at least 4 times as fast as your internet connection (an SSD will do) and large enough to hold the data (including frames and a bunch of other "junk" like ICMP packets) analyze it, structure it and re-write it until you're ready to view it (easily 10 GB/24h for simple household traffic). You're easily looking at a quad core or 8-core system if not a cluster.
Custom electronics and digital signage for your business: www.evcircuits.com
The bad news is the NSA is likely the only group that has the technology to do this sort of monitoring, even for your home network.
The good news is that by simply mentioning a few select keywords on the internet, they will gladly do this monitoring for you for free.
So many comments and none of them really answering OP's question. First: Yes, OP needs to ensure that what he's asking for is actually what he wants to do. Now, OP: How about using Open Source IDS/IPS? Something like Bro (http://www.bro-ids.org) could be a good option. It's completely scriptable and keeps track of general information (number of connections, what IP addresses are talking to what others, etc.), but where it really shines is that it alerts on "weird" traffic and since it's scriptable, you can write your own protocol inspection code to look at network streams on the fly and only pull out what matters. To implement this kind of system, I'd put a linux/bsd box inline acting as the network's gateway so everything on the network outbound goes through it, enable routing (linux: add net.ipv4.ip_forward=1 and net.ipv6.ip_forward=1 to /etc/sysctl.conf, bsd: add net.inet.ip.forwarding=1 and net.inet6.ip6.forwarding=1 to /etc/sysctl.conf), configure the firewall as needed (NAT and what have you), and set bro up to look at the traffic. Then I'd define very clearly what traffic I thought was "interesting" and warranted looking into. That traffic I would write some inspection code for and wait for alerts (which can be formatted however you please -- they're just text).
Finally:
Should an I[DP]S be used for oppression? No. Should this type of solution even be implemented at all on a home network? I think that's an issue that can only be answered by the client. Remember: anything can be used for good or evil. Make sure that anything you build and sell is going to be used for good (as much as you can ensure such a thing, of course). Talk to your client. I have a feeling that training for dealing with social engineering will go a lot further than a custom-engineered DLP system.
It sounds like your client may have a hard time dealing with something like Wireshark or any of the other port/traffic monitoring methods I've seen mentioned. I'll suggest looking at OpenDNS as a way to give some protection against "unsavory" sites as well as some degree of reporting in their "Stats" section.
It's far simpler and more efficient to implant electronic monitors within the organisms, directly.
A cheap way to do it is with DD-WRT. If you get a router that has a USB on it, you can load DD-WRT and then use opkg to install tcpdump. Use that to dump traffic captures to the USB hard disk and download them later to analyze with wireshark.
You won't be able to capture HTTPS traffic easily. The problem is it gets encrypted on the device before it is sent to the router to go to the Internet. There are ways around this but it would most likely require additional configuration (Setting up proxying on the devices) to work.
"My client needs to monitor traffic on his LAN" .. that just plain wrong. Maybe she 'wants' to monitor traffic on his LAN becaus he thinks that could mean anything to him if he knows/controls what LAN-users are doing, which social sites they are using etc., but maybe you tell him that he is just misinformed.
best regards
Ok. You have beaten this guy down already and told him very straight what you think about him. He has made a mistake, apologized and learned something valuable.
I feel pity for him. Anyone else?
To clarify, I have studied formal networking at a local college. Alot of my article was truncated, apparently because it was too wordy, but at least the core of the article is there. My comment in regards to my education was to help everyone understand that i'm not a redneck hilbilly that's never set up a subnet before.
With all due respect, as someone else who's had a formal networking education, there's a massive difference between setting up a subnet and performing full blown packet capture/analysis.
You could try ClearOS. There is a community edition that you could install. I think it will do most of the things you want. http://www.clearfoundation.com/
Have the customer start using Chrome; Let Google do the 'monitoring' for you.
wont work for smart phones though..
I do most of what you are inquiring about with a linux computer that has 2 NIC's. I have a custom build on openSUSE called net-tap. With a large hard drive, you can perform a tcp dump in a rolling manner, so the oldest is overwritten by the newest. You need to break the dump into reasonably sized bits and you will probably need to do a good bit of manual analysis.
That is the technical side. Now, ethically, this is a bad idea. I capture packets for clients all the time, but my analysis is only concerned with network performance. I can see scenarios where this would be acceptable, but they are few and far between.
I also concur that you are in over your head. People like you are pricing people like me out of the market. You don't know what your doing, but your clients have no idea.
"If your think a professional is expensive (me), you should try hiring an amateur (you)", is meant to be sarcastic, not instructive.
Cheap storage VM.
Clearly no. How about you tell him to hire someone who knows what the fuck they're doing?
Does anyone have a less malicious, less illegal, less profit-driven way to do this at a level that doesn't violate civil liberties?
I have kids who are well-internet-educated. I trust them. But I also want to be able to see what's trying to leave my network. I'm a hardcore security guy, but I have better things to do than spend my free time setting up netflow on my Tomato-USB router. I use OpenDNS as a first line of defense (kids are still young enough to be more likely to find porn by accident rather than on purpose, but I know that won't last.) and I have their internet connections cut off at night so they'll go the hell to bed.
I don't want to spy on my kids conversations, but I reserve the right (and make this abundantly clear to them) to see where they're going and what they're doing. As they get older that will fade a little, especially if we can maintain the level of trust we have today. I want non-intrusive but effective ways to keep tabs on goings on without being a dick.
Thoughts?
Nagios will allow you to monitor multiple interfaces in multiple machines, will send you alarms when thresholds are reached or exceeded and will provide you with an excellent platform if you whish to monitor anything else in the future. Will run in any Linux or unix and even you can get FAN (Fully automated ngaios) as a virtual machine that requires almost no configuration to get running.
Wow.
All this typing, bitching, and complaining.. All you guys needs are some virtual rocks.
Give UNTANGLE a try for free (untangle.com). Get a computer with two Ethernet ports and setup untangle.
It works well and logs and/or blocks traffic based on your selections. It may not have all the features you are looking for, but I would try it. It works well for a number of my clients.
PFsense and ntop.
tpcdump -i $OUTGOING_INTERFACE -w $HOME/capture_file -s 65000
Then tell everyone who he is monitoring to use a VPN.
Forget about trying to monitor everything. It's not possible. Just set him up with OpenDNS and have it block 'unsavory' websites for him. Beyond that you're going to need to invent HAL... and we all know how that turned out.
This suddenly looks like a stoopid school prank. Has anyone else thought of anagrams?
Ref: anagram solver
Yep. Hilarious.
With each breath in, a flower somewhere opens; with each breath out, a flower withers away. In between lies beauty.
For the environment and conditions you are describing you may want to consider a UTM appliance, like those put out by Fortinet.
Fortinet has some nice small office wireless-type routers that can manage/filter/allow/block web traffic as well other kinds of internet traffic (e.g. IM/proxy/peer-to-peer). (Wireless traffic can be even filtered/blocked from accessing the internal network as well.) It has reporting capabilities that can viewed and email reports out or can even be dumped to a logging server for later analysis.
The up-front cost for a unit may be a bit steep (for a average home user) and there's a yearly subscription (after the first year). But these unit are pretty much âoeset up once and forget about itâ. (Though, you may need to get help setting it up the first time and tweaking the settings to get it running the way your client wants.)
I don't work for Fortinet, but do use their products at work and at home.
Having had dealings with several Brethren families this absolutely rings true. Historically they have shunned the internet, radio, newspapers and television. However they still need to run their (normal) family businesses and doing business as much as possible with other Brethren only gets them so far. They now "bend" the rules a bit to allow them to operate in today's market. So they'll have a home or office network of computers but only one with an internet connection, tightly controlled so it can only access industry websites and those of suppliers and customers. The need to satisfy religious beliefs while having a minimal grasp of technology means that they will invariably contract someone (preferably another Brethren even if they are only slightly more tech savvy) to supply this infrastructure for them without really understanding the complexities or legalities involved.
Put a bunch of monkeys on the router, and have each one count the packets for each port #. Or you could use the distributed monkey model, where each workstation and server has a dedicated monkey.
Vote monkeys into Congress. They are cheaper and more trustworthy.
Let's assume that you could do this without going to jail, without any ethical dilemmas, or any trouble sleeping at night. You have several options:
To make this work, you'll need to drop in a small linux router box between the modem and the wireless router. This will run all the software for filtering, etc. As some people have mentioned, you'd need to get a femtocell box also if you wanted to log cellphone data. There will be legal issues with this, since you can't guarantee that it only services inside your "client"'s home.
As for the setup, you'll want to run a transparent proxy, not squid. You can check on the hacking forums for ones that even perform man-in-the-middle attacks on ssh, to log that traffic also. This should already be raising some red flags as to why this is a Bad Idea (tm). These tools will most likely already be set up to log facebook messages. If not, you can write some NLP software to sort through captured data.
Honestly, I think this post was allowed through not because of the content, or looking for actual answers, but rather to test the ethical waters. Most posters know of ways to do this, but will first raise the ethical/legal issues involved.
What you want can be done, but don't expect it to be user friendly unless you're willing to part with a fair amount of money and bring in a few programmers for that bit.
So, please don't tells us ... but please, if you're going to all this trouble ... I hope you've got something to protect that is a lot bigger than what you describe here.
Anyone who feels s/he needs to do all that monitoring of -family- member(s) has already lost them.
Evidence doesn't convince someone to stop doing what they want (& may have a human right) to do.
How much $$$ would it take to join forces with such a despicable person?
I don't respect what you seem to be doing, as I believe it may be -both- unethical & ineffective.
Because the "client" is a guy trying to catch his wife cheating on him and will soon land himself in divorce court. The "legal" issue is that wiretapping laws prevent him putting snoopware on to her machine(s) without her consent. Run away from this project as fast as you can. Unless you are working for the NSA, the first rule of data capture is that you JUST have the permission of the people whose data you're capturing. Bad things happen to techs who skip this step.
His question reminds me of one of my own. I have an office environment who is a client who'd like to have an idea of what kind of Internet traffic and bandwidth their workers are using. Stuff like is Legal using Facebook all day and how many megs are they using doing so? Who is the guy streaming music all day and is he affecting the FTP upload of large graphic marketing materials? They have a Cisco router and 10megs bandwidth but want a better picture of how it is being used. I picture some software running against the Cisco logs and making it understandable.
Something pretty and easy to read. :)
http://www.ipcop.org/ Bootable Linux ISO installed on some dinosaur PC hardware. 2 NICs, Inside & Outside. Put behind the Linksys/Netgear/router. Install the Squidguard add on. Transparent Proxy that watches ALL HTTP (TCP port 80) traffic and HTTPS (TCP 443) URL's. transparent for all outbound recording. Designed network bottleneck for all outbound Internet traffic. Configure DHCP client on the outside, and DHCP server to the inside. So all clients receive DHCP services from IPCop. All free.
How long can a troll go?
With each breath in, a flower somewhere opens; with each breath out, a flower withers away. In between lies beauty.
As soon as they find out they're being monitored, they'll just turn off WiFi and use 3G instead. Good luck monitoring that.
But seriously...
1) if you're trying to build this from scratch and are asking slashdot - it's not going to work, and
2) what ever happened to simply stating the rules and expecting everyone to follow them? It sounds like this guy is a serious control freak.
...rather than answering the question, I'm going to tell you that you're doing it wrong.
Your client needs to educate his family. Spying on them isn't going to protect them or teach him anything he wants to know. He should start by educating himself - this has the added benefit that he might realise how pointless this approach is.