Slashdot Mirror
ISPs Using "Deep Packet Inspection" On 100,000 Users
dstates writes "The Washington Post is reporting that some Internet Service Providers (ISP) have been using deep-packet inspection to spy on the communications of more than 100,000 US customers. Deep packet inspection allows the ISP to read the content of communications including every Web page visited, every e-mail sent and every search entered, in short every click and keystroke that comes down the line. The companies involved assert that customers' privacy is protected because no personally identifying details are released, but they make money from advertisers who use the information to target their online pitches. Deep packet inspection is a significant expansion over tools like cookies in the ability to track a user. Critics liken it to a phone company listening in on conversations."
..., ssh, pgp all the time!
DNSSec and opportunistic IPSec should put an end to the snooping and throttling once and for all.
Thats it, I say webservers move to SSL only transactions. All other plaintext transmissions should get encrypted at the endpoints transparently. Then when the government whines about not being able to find the terrorists they can blame datamining companies that paid for their election campaign. Then they can make a law that forces a back-door, which would create a need for some nifty-ass steganography which would lead to massively excessive processor and network overhead (encryption and steganography respectively) for the most basic of transactions which would lead to NSA funded algorythms to find these hidden messages which would. . .holy shit it's almost 10AM, I need to hit the sack.
If you are about to mod me down, keep in mind that this post was most likely sarcastic.
If ISPs are monitoring traffic so closely, doesn't that make them more responsible for what people are using their service for? Namely piracy.
ISPs have always been notorious for secretly compressing your images, caching your traffic, proxying stuff, slipping their own content into your web pages, etc. They look at the contents of your mail, since you can't spoof from anyone to anyone via their servers. How is this different, other than some joker gave it an ominous sounding name like 'Deep Packet Inspection' ?
I want to delete my account but Slashdot doesn't allow it.
Let's start turning over rocks in the private lives of telcom CEO's and see what scurries out. I'm sure they won't mind, it's in the interests of an open society and free debate, don'cha know.
Kwisatz Haderach
Sell the spice to CHOAM
This Mahdi took Shaddam's Throne
Never mind that it's evil, or that it's a great step to losing their common-carrier status.
Never mind that it's a true violation of privacy.
Never mind that I block cookies pretty well and I run with NoScript most of the time and I don't see very many ads, and besides, half of the time I'm inside my employer's VPN.
But even more than that, I have seven other users in my household, half of them teenagers. If they want to sniff all of my NAT-ed packets coming out, they're going to discover that I'm a geek who has four Facebook sites, likes art and hates it, plays Runescape incessantly (the 10-year-old), likes the Wiggles, and works as a beauty consultant. So go ahead and hand me the ad for the latest XBox game (I hate games). Offer my kids server hardware, and see if you can get my wife to click on fun games to play with the Backyardigans. Oh, wait, you already do. It's called "not targeting advertising", and it's free.
So what we have is a thoroughly broken high-cost borderline-illegal absolutely-unethical service offered to advertisers in a difficult economic period. By people who we all hate a lot, and who will rapidly become targets for everything from blocking to legislative action to you name it.
I knew there would be some kind of career move for spam kings in the future. I just thought it would pay better.
I predict a less than stellar outcome for these idiots, and they deserve every painful moment.
Isn't this the real issue with clogging 'tubes'? How can the government and ISPs keep up with the computational resources needed to continue this as we demand greater and greater amounts of bandwidth? OK, so they could only inspect http traffic, rather than say, bittorrent traffic, but OMG what happens when 'terrorists' start communicating with other protocols?
I pay for a dedicated server (essentially colo but they provide the hardware) from a company with a decent AUP. I put linux on the server and run squid on a non-standard port, allowing connections from localhost only. Then from the machine I'm surfing from I tunnel into the squid server. Say squid is running on port 1234 and sshd is running on 4567:
ssh -f -N -L 1234:localhost:1234 -p 5678 my.squid.server.com
Configure firefox to use a proxy to localhost:1234 and all traffic is encrypted to the squid server.
Of course, I could just use Tor, which is great, but can be slow. In fact, you could run a tor server on your colo machine and have all tor traffic bounce off of the server, which would be pretty fast if you leave tor running as a daemon and dedicate a decent amount of bandwidth to the tor network.
A squid eating dough in a polyethylene bag is fast and bulbous, got me?
It's illegal for anyone to open mail not intended for them. The same should be done for electronic communication.
And if I hear one libertarian say we need less laws, I'll puke. It's as if they though they had a magic wand and all the troubles of the world would disappear by removing government. Unfortunately, the world hasn't worked that way since we left the caves 12,000 years ago.
---Technology will liberate us if it doesn't enslave us first.
If these are the ISPs (as opposed to the visited web sites) doing the spying, then how are the advertising companies involved supposed to deliver the content? Are they going to use the same "deep packet" method to inject the advertising? If the advertising delivery is away from that deep packet inspection, then how do they identify which user was interested in penis enlargement products vs. which user was interested in replica watches? Or are the ISPs going to lock-in the IP address, now?
now we need to go OSS in diesel cars
Comment removed based on user account deletion
The difference is that in the first case, the data passes through a dumb machine that compresses, caches, etc. The result is cached like it is expected (RFC 2616 is pretty clear about that), even though it is done transparently. No need to keep logs about who downloaded what.
In this case, the data is explicitly mined, by a company interested in building a profile of each user. It doesn't say it is limited to web traffic only, only that "Nor does NebuAd record a user's visits to pornography or gaming sites or a user's interests in sensitive subjects -- such as bankruptcy or a medical condition such as AIDS.", which I doubt both on technical grounds and because it is a market and someone will want to take advantage and "The company said it processes but does not look into packets of information that include e-mail or pictures." which I think is in contradiction with other parts of the article and even if they didn't, it's a matter of time before they do.
Basically, it's the intent that counts. The ISP can intercept everything they want because they're in the middle. When they start doing so for reasons that are not part of maintaining the communications as specified (like forwarding, maybe firewalling and proxying depending on the conditions), alarms should go off.
GPG 0x1B479C78
If you do this in the EU. Packet pauyloads are off-limits without court order. You may not even store them.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Search for info on heartburn... get some post cards advertising the latest antacid. Search for info about Lasik eye surgery... gee handy flyers about your local providers appear.
You get the idea. If I were selling a service and an ISP offered to sell me names and addresses based on keyword searches, why wouldn't I buy that list?
This issue is a bit more complicated than you think.
Ever get the feeling the the Internet just isn't worth it anymore?
Government's idea of a balanced budget: take money from the right pocket to balance...oh who am I kidding?
standing up for our rights is the answer. unfortunately, corporations listen only to once voice, money, so hit them where it hurts.
Cancel your internet, refuse to pay your bills... boohoo, then you won't have internet? you won't have internet anyway, if they get their way.
You think these guys don't like BitTorrent, wait until everyone starts a process to spider the web to obfuscate where the fleshies are really browsing at and run that 24/7 to overload their deep-packet inspection devices.
Apocalypse Cancelled, Sorry, No Ticket Refunds
Time has shown that nobody will protect your privacy besides yourself. It's time for ALL Internet traffic and ALL phone traffic to be encrypted with an option to get SSL keys for each machine or phone from trusted authorities in different countries. This way a particular person asserting privacy is not labeled a terrorist, Comcast can not selectively block bittorrent, Chinese firewall is out of business and phone companies do not need immunity for spying on subscribers. IPV6 will have to be adopted anyway in the next 10 years and it included encryption, so the time is right to make both switches at once with little extra IT overhead.
7. Go directly to Federal-pound-me-in-the-ass-prison for postal fraud. Do not pass go, do not collect $200.
Seriously, if the USPS, UPS or Fedex started doing this can you imagine the outrage? Yet somehow it's ok to do it with electronic communications? WTF?
I want peace on earth and goodwill toward man.
We are the United States Government! We don't do that sort of thing.
The government may have the resources to break strong encryption in real time, but even the largest ISP's do not. So maybe now the FreeS/WAN project no longer sound like tinfoil-hatted paranoiacs when they push opportunistic encryption at every node. Everything gets encrypted automatically and transparently when talking between two OE nodes, regardless of the protocol.
This was their goal, but hostility and forking ensued when most people really wanted to just have an IPsec implementation on Linux. OE is still a good idea, though, and that's what they're focusing on now.
The obvious design win would be if Linksys and Netgear built OE into their consumer grade firewall/routers. Then everyone would have it, not even know it, and when large site operators started deploying it on their network edges, massive amounts of crypto would start traversing the Internet, and no one would be bothered by it.
That's really the key to good system design: add complexity, but don't bother the end user -- it's not his problem.
Tired of FB/Google censorship? Visit UNCENSORED!
Funny, while loading this page I got a "bandwidth cap warning" from my ISP, stealthily inserted into the page (Rogers Cable).
I expect nothing less from the despicable scam shop that is Rogers, but it's still kind of creepy.
For me, it's not a huge deal because I run a number of geographically diverse servers, I can VPN or proxy my traffic through any combination of them, should the need arise. Like any invasion of privacy, I'm not concerned about the marketing uses, it's the inevitable abuse that scares me, either by ISP staff sniffing passwords, or script kiddies rooting the monitoring systems (and/or the idiot sysadmin's PC).
The thing is, at this point I've given up on common sense. Things will continue to get more and more ridiculous until we reach a breaking point... the bubble will burst and there will be backlash against these invasions of privacy, but only when the common fool finally realizes their life is being tarnished by the practice.
Until then, we'll continue to be labeled as paranoids with our tinfoil hats.
-Billco, Fnarg.com
I just checked NebuAd's Privacy policy:
NebuAd products do collect and use the following kinds of anonymous information:
Now that's way out of line for an ISP to collect, let alone send to an ad agency.
We may be able to do something about this.
We run SiteTruth AdRater, which rates advertisers. We have a Firefox extension which displays a rating icon for each ad served. When an ad link goes by, and it's not in the browser cache, the extension contacts our server for a rating of the advertiser. So we collect, over time, a list of advertisers for various ad systems. We're not collecting data about users; we're interested in advertiser behavior. (You can read the source code for the plug-in, so there's no mystery about what we're doing.)
We're not currently tracking NebuAd, Front Porch, or Phorm ads; we've been focusing on the bigger players. It looks like we need to be tracking this behavior. If anyone can find ad links from those services, please post the ad link here, or mail it to "info@sitetruth.com". We need some examples so we can modify the plug-in to recognize them.
If we can collect sufficient information about this class of advertisers, we may publish their customer list, which would be useful for boycott purposes. Thanks.
its called tor.
I have a bit of history with two large service providers in the US. While I have not been involved directly with the deep packet inspection teams, I have had direct contact with all of them and helped them design networks using this technology. The technology was never sold to upper management as a way to track our users and target ads to them. It was never intended to capture a web page hit that was directed at a specific company to see what that consumer was interested in. Instead, it was always meant to monitor users (and more importantly, user aggregates) and determine what kind of traffic they were sending.
It was, and is, always about the network profile. If they find out that 10% of the traffic on the network is VoIP traffic, they want to design the network shift this traffic to have lower latency.** If they find out that 50% of the traffic is BitTorrent, they may put rules in place around such services. In my opinion, the service providers that I have dealt with do not have the technology in place to target down to the user. Also, they do not appear to be developing this technology.
**Some can argue that providers are instinctively evil and want to destroy this traffic, but I'm not going to fight this here.
Disagreeing with me does not mean you get to mod me troll.
Some of us do not use Google mail or Google desktop search for exactly the reasons you give.
Statesman
Fedex and UPS open your packages to look at what you are shipping so they can sell that data to advertisers?
rather they're searching through it looking for things that look suspiciousDid you even bother to RTFA? Wait, dumb question around here. This has nothing to do with looking for 'suspicious activity'. The ISPs in question are allowing third-party companies to build profiles of their users by spying on their traffic in order to do targeted advertising.
I want peace on earth and goodwill toward man.
We are the United States Government! We don't do that sort of thing.
Every datacom box supplier is developing DPI features for their products. The main driver is not targeted marketing, but QoS. When you're able to identify traffic on the application layer, it gives you a lot of extra options in determining how to route the traffic.
This way you can decide to route P2P traffic flows on best effort basis, but "over-the-top" video (eg. Youtube) flows you route through a higher quality connection. This improves user satisfaction.
That's the idea anyway, saying it's for targeted advertising sounds quite paranoid to me.
So which ISPs are doing this? What can we do to protect our selves? It sounds like it's "enabled" by a cookie placed there by your ISP or NebuAd? Would Adblock and/or PeerGuardian be enough? Implementing blocking at the home router level? What can home users actually do?
It'd be nice at least to know who's actually participating in this so we could know who to avoid.
No one authorized ISPs to inspect packets for any purpose.
However if they provided their service at the same price google offers gmail in exchange for authorization to inspect packets, I'm sure there would be lots of people willing to take the deal. And I think whoever modded you insightful was on crack.
You could have 10,000 domains that share a common cert provided by the hosting provider. It does squat for authentication but it does prevent snooping.
With ISPs starting to snoop, suddenly this has real value.
Combine this with 3rd-party SSL-enabled DNS, and you've got some reasonable countermeasures.
Your ISP will know you talked to dns.ssldnsprovider.com over an encrypted channel and then immediately carried on a series of conversations with 1.2.3.4 over port 443, but he won't know which of the thousands of web sites hosted by 1.2.3.4 you talked to.
Dns.ssldnsprovider.com will know you looked up the address for www.freetibetnowdammit.com but not much else.
You will be presented with a certificate for www.somebigwebhostingprovider.com that mismatches www.freetibetnowdammit.com, but freetibetnowdammit.com will explain why and say not to worry about it, as will all the other hosts residing on 1.2.3.4.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
The quick fix to this is web-sites all allowing https, ssl, and vpn connections to them. That will end deep-packet inspection, leaving only a list of web-pages visited available. gMail already allows https, but you have to ask for it.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
It's my understanding that most shippers won't open a package unless they already suspect something or are required to by law.
If they suspect dangerous goods, they may open it to protect their planes and other packages.
If it's at a customs location, they may inspect items if they are acting on behalf of customs agents.
If they suspect illegal material, if their lawyers are smart they will get the cops or courts involved before they open the package.
What they don't do is just snoop for the hell of it, if they did, their reputation would be in shatters and there is too much competition in that industry to withstand the bad press. Unlike some industries *cough*localisps*cough*.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
The best way to generate a groundswell against these systems is for websites to warn their uers if they are on an ISP that does this. For those in the UK worried about the 'phorm' spying system, Richard Clayton has extracted some technical information from them here: http://www.lightbluetouchpaper.org/2008/04/04/the-phorm-webwise-system/ and Gavin Jamie already has a prototype Phorm detector here: http://www.mythic-beasts.com/~gjamie/