Slashdot Mirror


Tool To Allow ISPs To Scan Every File You Transmit

timdogg writes "Brilliant Digital Entertainment, an Australian software company, has grabbed the attention of the NY attorney general's office with a tool they have designed that can scan every file that passes between an ISP and its customers. The tool can 'check every file passing through an Internet provider's network — every image, every movie, every document attached to an e-mail or found in a Web search — to see if it matches a list of illegal images.' As with the removal of the alt.binary newgroups, this is being promoted under the guise of preventing child porn. The privacy implications of this tool are staggering."

35 of 370 comments (clear)

  1. Probably just for P2P by clang_jangle · · Score: 5, Informative
    FTFA:

    Here's how CopyRouter would work, according to the company's slide show: A law enforcement agency would make available a list of files known to contain child pornography. Such files are commonly discovered in law enforcement raids, in undercover operations and in Internet searches that start with certain keywords (such as "pre-teens hard core"). Police officers have looked at those files, making a judgment that the children are clearly under age and that the files are illegal in their jurisdiction, before adding them to the list. Each digital file has a unique digital signature, called a hash value, that can be recognized no matter what the file is named, and without having to open the file again. The company calls this list of hash values its Global File Registry.
    Whenever an Internet user searched the Web, attached a file to an e-mail or examined a menu of files using file-sharing software on a peer-to-peer network, the software would compare the hash values of those files against the file registry. It wouldn't be "reading" the content of the files -- it couldn't tell a love note from a recipe -- but it would determine whether a file is digitally identical to one on the child-porn list. If there were no match, the file would be provided to the user who requested it. But if there were a match, transmission of the file would be blocked. The users would instead receive another image or movie or document, containing only a warning screen.
    The makers of CopyRouter claim that it can even be used to defeat encryption and compression of files in the Internet's Wild West: the peer-to-peer file-sharing tools such as Gnutella and BitTorrent.

    This will cause huge latency issues and cost beaucoup bandwidth. ISPs would be shooting themselves in the foot if they did this with all traffic. OTOH, I could see laws requiring such tools for P2P traffic -- in fact that may well be inevitable, with the **AA's "ruling class" status these days.

    --
    Caveat Utilitor
    1. Re:Probably just for P2P by negRo_slim · · Score: 4, Insightful

      This will cause huge latency issues and cost beaucoup bandwidth.

      A soft touch with this would yield far better results depending on your intent. I would imagine an ISP that is sick and tired of certain traffics could utilize a system like this to start taking a closer look. Catch a few token users and then you have a excuse to throttle/monitor/block at will. I mean think of the children! What worries me is that with so many computers doing the bidding of people other than their owners, who knows what kind of traffic is being exchanged. Seems like an easy way for law enforcement to take a closer look at an individual... I've come across very questionable images via Google from rather inane, yet obscure, search queries. You could be one Russian rickroll away from the authorities and those around you having some nasty suspicions in their head.

      --
      On the Oregon Cost born and raised, On the beach is where I spent most of my days
    2. Re:Probably just for P2P by Anonymous Coward · · Score: 4, Informative

      The parent is an example of typical slashdot idiocy. ISPs aren't common carriers. Though my karama will end up a smoking crater for breaking with the established GroupThink, so I'm making this post anonymously.

      The immunity ISPs currently enjoy in the US come from various other safe harbor laws (i.e. Â230; DMCA). The constant slashdot drone of "ohhh.. ISPs can't suppress my free speech: common carrier common carrier!" is both entirely incorrect and dangerous, since it causes the geek squad to under-estimate the risks and the importance of things like net neutrality.

    3. Re:Probably just for P2P by Hyppy · · Score: 5, Interesting

      Even better... What happens if you send traffic to a user with one of the "bad files" in it? They don't need to have a connection open in order for you to send a jpeg to them. Even if the user's computer simply drops the unknown data, the ISP will pick it up in their scan. If all the software does is scan the hash values of images transferred over common protocols, I seriously doubt that it goes and checks to see if the user actually REQUESTED it before crying foul.

      One step further: make a file that has the same hash value of a "bad" file. This is trivial, especially if the file doesn't need to be valid for any application. If all that is checked is a hash of the traffic, then the actual contents of the file are meaningless.

      So, this software will allow law enforcement to ruin your life (any implication crime involving sex and/or kids will do that, guilty or not), by simply seeing an unknown party send you a block of unintelligible data that happens to have the same hash as "pr0n." Great.

      Anyone up for making an automated hash-spoofing packet forger? I'm sure something similar has already been done. With the speed of current connections, one could probably get the entire human race indicted for child pornography in under a week.

    4. Re:Probably just for P2P by Fulcrum+of+Evil · · Score: 4, Funny

      "Well Mr. Smith's ISP reports he downloads copies of "Playboy's College Girls". Is this really the man you want to be your next state represenative???"

      What, am I going to not vote for him because he watches boring porn?

      --
      "We returned the General to El Salvador, or maybe Guatemala, it's difficult to tell from 10,000 feet"
    5. Re:Probably just for P2P by dat+cwazy+wabbit · · Score: 5, Insightful

      You would still lose the election.

    6. Re:Probably just for P2P by Pax681 · · Score: 5, Funny

      You would still lose the election.

      but would he lose his erection?

    7. Re:Probably just for P2P by CodeBuster · · Score: 4, Insightful

      The makers of CopyRouter claim that it can even be used to defeat encryption and compression of files in the Internet's Wild West: the peer-to-peer file-sharing tools such as Gnutella and BitTorrent.

      What are they going to do? Detect and Man in the Middle every single connection attempt that goes through their router? The file sharing tools will simply upgrade to stronger encryption, such as AES, and harden the connection handshaking against MITM attacks (perhaps by introducing public key infrastructure with well known key server(s)). It was my understanding that the present crop of file sharing tools provide obfuscation (ROT13 and the like) and not real encryption to set the bar just high enough to prevent packet inspection. However, it would not be difficult to implement stronger encryption methods (if they haven't done so already), should that prove necessary. In fact, the CopyRouter folks are at a distinct disadvantage in any encryption arms race since MITM and other cryptanalysis techniques are much more computationally expensive than the encryption itself AND the users outnumber the routers by thousands or even tens of thousands to one. The NSA might more credibly claim to be able to do this, but they have acres of underground super computers consuming as much electrical power as a small country, so I am very skeptical when anyone claims to be able to "defeat encryption" and doubly so when a private company mentions it as a bullet point in their power point presentation. It is more likely that this is a private company trying to sell a pig in a poke to ISPs and governments who don't inspect the merchandise to carefully or don't know any better.

    8. Re:Probably just for P2P by DerekLyons · · Score: 5, Insightful

      The parent is an example of typical slashdot idiocy. ISPs aren't common carriers. Though my karama will end up a smoking crater for breaking with the established GroupThink, so I'm making this post anonymously.

      Yet, for all your noise and handwaving - you fail to establish that an ISP isn't a common carrier.

    9. Re:Probably just for P2P by Baton+Rogue · · Score: 5, Insightful

      I think he's referring to MD5 Collisions where you can make a completely different file that matches the same MD5 hash of another file.
      But if all they are doing is comparing hash files, couldn't you just as easily change the resolution of the file, or insert a couple different bits around to change the file slightly, which ends up with a completely different hash?

    10. Re:Probably just for P2P by svank · · Score: 5, Insightful

      But if all they are doing is comparing hash files, couldn't you just as easily change the resolution of the file, or insert a couple different bits around to change the file slightly, which ends up with a completely different hash?

      Yup. That, along with good encryption, means the bad guys get around this easily, while innocent bystanders are caught up by hash collisions.

    11. Re:Probably just for P2P by Baton+Rogue · · Score: 4, Interesting
      After I RTFA, I got my answer.

      Encrypted files on the peer-to-peer network could not be decrypted by CopyRouter, but the company claims it can fool the sender's computer into believing that the recipient was requesting an unencrypted and uncompressed file.

      So basically what they do, is if your bittorrent client requests the files in encrypted format, they intercept that and instead request them unencrypted. They aren't decrypting the file, they are just asking for an unencrypted transmission of the file. If the file is in an encrypted zip file, then there is no way that they could see the actual files being transmitted.

    12. Re:Probably just for P2P by Fluffeh · · Score: 4, Insightful

      If I was a person who wanted to get this thrown out the window, I wouldn't look at trying to convince people that it is bad. I wouldn't look at how it could be abused. I would much rather be looking at how to misuse it myself. I mean what better way to show potentially how bad a system is than to get into the "black list of hashes" and add some. Add lots. Like a real lot. Every email suddenly gets a warning message with details of why. Yes it was hacked. Yes the public outrage will be huge. It would be so huge that it would end up getting shit-canned pretty damned quickly.

      Best way to get anyone to get rid of something is to make them hate it. All my email blocked today? You bastards! Turn that thing off.

      --
      Moved to http://soylentnews.org/. You are invited to join us too!
    13. Re:Probably just for P2P by PopeRatzo · · Score: 4, Insightful

      If my ISP told my opponent what porn i watch, they'd be sued.

      Your ISP doesn't care about your stroke material.

      This is all about P2P, the RIAA and collecting data for government and marketing purposes. Don't kid yourself that your ISP is so broken up about the possibility of sketchy porn traveling their network.

      Just today I read an article quoting telecom execs about how SKYPE and other VOIP applications are going to make us less safe from terrorists. It's about profit and control, nothing more nothing less.

      --
      You are welcome on my lawn.
    14. Re:Probably just for P2P by PopeRatzo · · Score: 5, Insightful

      but the company claims it can fool the sender's computer into believing that the recipient was requesting an unencrypted and uncompressed file.

      That's not hostile, much. As is common in our corporatocracy, here's a company that starts from the assumption that their customers are their enemy. So now we're going to pay our ISPs to "fool" our computers. Some "customer service" huh?

      No thank you.

      How about this: We pay you, and you give us bandwidth and stay the fuck out of our business. If we're using too much bandwidth, then spell it out in our contract and charge us more, so we can choose to give our business to someone else.

      --
      You are welcome on my lawn.
    15. Re:Probably just for P2P by TapeCutter · · Score: 5, Informative

      "My ISP is AT&T. They're not a common carrier?

      The AC is correct in what he is saying about common carriers. Check out the registered company name of your ISP and I will wager that it is not AT&T but rather a subsiduary of AT&T (ie: a seperate company in the eyes of the law).

      This is how the telco's in Australia with common carrier status get around the rule against sniffing the line, eg: Australia's "Telstra" is not an ISP but "Telstra Big Pond" is an ISP. Since common carrier rules are international I dare say AT&T do exactly the same thing.

      --
      And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
  2. Huh? by LoRdTAW · · Score: 4, Insightful

    "The tool can 'check every file passing through an Internet provider's network -- every image, every movie, every document attached to an e-mail or found in a Web search -- to see if it matches a list of illegal images.' "

    How exactly is this going to be accomplished? The equipment cost must be staggering and would consume allot of power. Way to conserve electricity, I thought we were trying to reduce the amount of power the Internet consumes. Does also this remove the common carrier status of ISP's?

    I hope this never comes to fruition.

    1. Re:Huh? by fred+fleenblat · · Score: 5, Interesting

      TFA says they're going to use hash values. This will take a stateful packet inspection filter to catch, but the amount of state is only enough do the hash, and they can throw it away if it doesn't match anything on the blacklist.

      While hashing seems easy enough to get around, I think the real thing they're looking for is a repeated pattern of someone sending blacklisted images. If you send/receive thousands of images, there's a good chance that you'll screw up and maybe a dozen of them won't get resampled (or use some other trick) to change the hash value. you'll pop up on a screen someplace, they'll get a search warrant, and you are busted.

    2. Re:Huh? by maugle · · Score: 4, Informative

      Every time this topic comes up, someone posts something about how this could remove the common carrier status of ISPs.

      Repeat after me: ISPs do not have common carrier status.

  3. Brilliant Digital Entertainment? by Anonymous Coward · · Score: 4, Informative

    Wasn't that the Aussie spyware company attached to Sharman Networks/KaZaA?

    Before it got raided, I mean?

    I call shenanigans.

  4. One question by MathFox · · Score: 4, Insightful

    Can it decrypt SSL/SSH in real time?

    --
    extern warranty;
    main()
    {
    (void)warranty;
    }
    1. Re:One question by whoever57 · · Score: 4, Informative

      Can it decrypt SSL/SSH in real time?

      Exactly. They claim that the can search "every document attached to an e-mail .. -- to see if it matches a list of illegal images. Apparently, they have never heard of SMTP-TLS, POP3S, etc.. Or perhaps they have and they are just like many others -- selling snake oil.

      --
      The real "Libtards" are the Libertarians!
    2. Re:One question by unlametheweak · · Score: 4, Informative

      No. RTFA. CopyRouter merely pretends to be a server and tells the client the client to send data unencrypted. Bittorent just needs to upgrade it's encryption mechanisms.

  5. You know, it really makes me wonder... by genw3st · · Score: 5, Insightful

    ... what is going to prevent this proverbial snowball from building into a full-blown avalanche? I guess it has already become one to some extent... I can't recall a time in history when the WORLDS rights and privacy were as stripped and neglected as it is now, and then everyone suddenly got their right to privacy and freedom back. Despite its amazing capabilities, technology sure has put us into an interesting position when in the hands of people like "Brilliant Digital Entertainment" ... yeah, real brilliant. Crackheads.

  6. Won't work. by Xtense · · Score: 4, Informative

    Ok, on really simple protocols, like HTTP or FTP, maybe - but most, if not all, p2p traffic is safe, i think. This is of course because of the chunky nature of transmission - you can't really tell what part of the file went through your pipe just by looking at it, and since parts are sent at random, you cannot rebuild the file with your chunks without guiding information, be it a torrent file, a list of parts for emule, or whatever else there is. And you need the whole file to get your hash-check. That's one. Two: encryption totally kills the effort, as the ISP can in no way examine your file without interfering with your transfer, and SSL exists solely to protect you from this.

    Even if my line of thinking is really misguided here, this would require lots and lots of processing power - i mean, on a routing line with a hundred users on one end, it's thousands of hash-checks to be made for every stupid rebuilt file - both processes of course painfully CPU-eating, unless you want false-positives, since you didn't bother to use a proper hashing algorithm.

    All in all, this looks to me like a terrible waste of money.

    --
    "We are the music makers, and we are the dreamers of dreams [...]."
    1. Re:Won't work. by Anonymous Coward · · Score: 5, Interesting

      They claim they can scan Gnutella and BitTorrent.

      Gnutella I don't know, but BitTorrent, almost certainly.

      The common forms of BitTorrent encryption uses a "shared secret". The shared secret for BitTorrent is a 20-byte key known as the "infohash". This infohash is ALSO used as the unique hash to uniquely identify a given set of files. So its ALWAYS given to the tracker, and if the tracker isn't using SSL, that means its in the clear.

      Making the encryption in BitTorrent almost laughably insecure. It's good enough to block non-stateful packet filters. It's not good enough to prevent people from listening in.

      As for getting a file hash with BitTorrent, that's even easier.

      It does it for them.

      The ".torrent" file contains a list of hashes. They don't even need to look at the file contents.

      I dunno about other P2P systems, but BitTorrent is definitely not safe from this.

  7. Re:useless by Mr_Tulip · · Score: 4, Funny

    shhh.. don't tell the government..

  8. Easily gotten around by Anonymous Coward · · Score: 4, Insightful

    Time to make a utility that puts a file into an encrypted 7Zip archive, with the password stored in some reversable encryption method (encrypt the password with all zeroes as a key 1 million to 2 million times), so it would take x CPU seconds on some hardware to decrypt it.

    This would allow files to still go across the net without requiring passwords or keys, but prevent utilities like this from just passively obtaining traffic, just due to the CPU cycles involved.

    Of course, just stuffing a password in the comments field works too, but with a decent text parser, it can be extracted.

    Its just more of the same cat and mouse game. The real crooks will not be affected while Joe ISP User will lose his privacy even more.

  9. Evil by Anonymous Coward · · Score: 5, Informative

    According to the Wikipedia entry on Australian copyright law "[...]Brilliant Digital Entertainment in Australia were raided for copyright violations[...]" in 2004.
    It looks like someone switched sides but taking a closer look they only seem to be in charge of the adware that came with Kazzaa, so I guess they were always evil.

  10. This is Fantastic by pnotequalsnp · · Score: 4, Insightful

    This is fantastic, since the amount of money required by an ISP to implement this will sink them. This will filter all "idiot" ISPs, who think they are rulers of the internet.

  11. A better use for this technology... by thenewguy001 · · Score: 4, Insightful

    is to have ISPs scan all downloading files to make sure they do not contain malware or viruses so we don't have so many botnet zombies around the web from idiots opening britneysex.exe

  12. One answer by Willbur · · Score: 4, Insightful

    Can it decrypt SSL/SSH in real time?

    According to the article they use man-in-the-middle attacks. This is probably quite easy if the server is using self-signed certs.

  13. Child porn is perfect for framing people by Jimmy_B · · Score: 4, Insightful

    The problem with all the hysteria around child pornography is that it's too easy to frame someone. A little research, five minutes alone with your computer, and an anonymous phone call are all someone needs to ruin your life and reputation.

    Let me be perfectly clear: Even if you're completely innocent, this is a serious threat to you. If someone decides to frame you, you won't be able to prove your innocence, and it won't matter even if you can. That's unacceptable. Yes, child porn is bad, but a society where anyone can anonymously destroy anyone else is much, much worse.

  14. Big Daddy knows best by farbles · · Score: 4, Insightful

    You know what? In a dozen years of actively surfing porn, I've never encountered kiddie porn in the wild. This great big threat to all mankind so severe that we all need to put woolly pullovers over all our electronic gear and filter all telecommunications is simply and plainly crap. It's a ruse.

    There are some people who want to control everyone else. They want to control what you see, what you hear, and as much as is humanly possible, what you think. They want to monitor us all (but not themselves, of course) and make us all cookie-cutter little clones who all think the same harmless little thoughts and are all scared of their authority.

    F * U * C * K them.

    Anyone telling you this sort of "protection" is necessary is deluded or a liar. Either way, such people should be ignored or in extreme cases, put somewhere they cannot bring harm to others.

  15. The last page (4) of the article reveals the truth by mrpacmanjel · · Score: 4, Insightful

    "...Internet service providers could easily be seen by the public as "overreaching," making it harder to get public support for efforts of law enforcement. What's needed, said the group's executive director, Grier Weeks, is for cops to investigate the leads they already have..."

    and

    "The Department of Justice and all 50 attorneys general are sitting on a mountain of evidence leading straight to the doors of child pornography traffickers," Weeks said. "We could rescue hundreds of thousands of child sexual assault victims tomorrow in America, without raising any constitutional issues whatsoever. But government simply won't spend the money to protect these children. Instead of arrests by the Federal Bureau of Investigation, the child exploitation industry now faces Internet pop-ups from the Friendly Bus Investigators. That was always the fundamental difference between the Biden bill and the McCain bill. Biden wanted to fund cops to rescue children. McCain wanted to outsource the job."

    This my friends is about the money! The U.S. Government and Brilliant Digital (ironic business name!) both know this won't work. Brilliant Digital see this as a market to exploit and make millions of dollars. The U.S. Government get a "cheap" way of "dealing" with child pornography and a perception from the general public as "something being done".

    I'm sure the Government know about Brilliant Digital's dubious past but the percieved "benefits" are too good to miss.

    It's a win-win for both parties!

    I have children myself and I find developments like this horrifying.

    Someone does not become a paedophile by looking at images on the internet, it's deeper and more complex then this - blocking content will not cure the problem or reduce related crimes in any way.

    The last quoted paragraph sends chills down my spine and really makes me angry.
    Children can be rescued if the funding is available but a company like Brilliant Digital will recieve the funding instead and the problem is never solved - people are made richer instead.

    I really mean Think of the children