Slashdot Mirror
Tool Detects "In-Flight" Webpage Alterations
TheWoozle writes "In a follow-up to a recent story about ISPs inserting ads into web pages, the University of Washington security and privacy research group has teamed with the International Computer Science Institute (ICSI) to develop an online tool to help you identify if your ISP is inserting ads or otherwise modifying the web pages you request."
If that isn't desirable, do a patch to Apache that creates a header that holds a hash of the content.
The hash gets calculated once for static content, which is usually the bulk of the traffic, no? So
not too big of a hit.
Browser sees content. Browser sees hash. Browser compares the two...
--
Censored by Technorati and now, Blogger too!
Do ISPs really do this? I've never really noticed anything like this.
When was the last time I saw an ad of a rival to Verizon in my verizon dsl line, I wonder.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
Do we sue the ad folks for inserting ads and stealing content? I mean, in just about any other medium this would wind up in court overnight as copyright and stolen content and so on. But now we have a circumvention tool to detect it...so are we going to get sued under DMCA like nonsense for attempting to circumvent the ad insertion?
The only change I can believe in is what I find in my couch cushions.
I like UW and their tools. I think they've done wonderful work. Paint.NET is fun, easy, and I love that they are still working on it.
Who/what is able add to your pages:
Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong fix.
ISPs intercepting, altering results from online security tool
The only surefire protection against Microsoft infections is abstinence. - The Onion
We (the authors of the page) will be answering questions in this thread.
Test your net with Netalyzr
What if the ISP is simply putting the web-page in its own frame, and the advertisement in a second frame? Unless you add the ability for web-pages to dictate that they should not be in frames, this one can't really be trapped for like that. The ISP could create its own hash for the served web-page that holds the frames.
Ben Hocking
Need a professional organizer?
No need for thousands of "All good in Kalamazoo" & "Up to date in Kansas City" posts.
I don't read ACs: If a post isn't worth so much as a nom de plume to its author then I wont bother either.
make a package that can be used as a simple drop-in to a website to detect this. If enough websites implement something that alerts users that the webpage was altered, isp will be forced to stop doing this.
A friend of mine had a similar problem with his webpages. They were on a free host (rolls eyes). I wrote a script for him to store special tags to denote the beginning and the end of his webpage content. After the webpage was loaded, a script erased everything and replaced all the html with his marked content. Ta-da, no ads!
If you want to be stricter, encode your webpage content with base64 to make sure the ads don't intrude your precious content.
[an error occurred while processing this directive]
I guess the next step is for the ISP to add their right to do this into your agreement with them.
A certain ISP in Canada delt with this not long ago...
I've wondered about this for a while as a way to defeat XSS attacks but would be adding some sort of ability to sign the content in a HTML response be beneficial here? You could use your SSL cert to simply add a signature response body for content transmitted over http. I way to inform the browser to expect the signature that the ISP can't strip out may be problematic though.
The XSS idea would be to have the ability to have multi-part responses from the web server. The browser would put the page together from each part in order but only parts that contained a valid signature from your domain cert would have scripts and such executed by the browser. Then we don't have to worry about escaping scripts and such in the output content.
they're not talking about the ISP hosting the web page, they're talkign about your ISP adding ads to random sites that you visit. client-side, not server-side.
This whole subject doesn't make any sense to me. #1 90% of computer users.
#2 Because there's no other choice for most users when it comes to ISP's.
It is by my will alone my thoughts acquire motion; it is by the juice of the coffee bean that the thoughts acquire speed
Well, I hear that in Soviet America they have to use the ISP decided by their corporate overlords as the only option if they want high speed internet, so they do not have any other option but to use the crappy ISPs for a very high price in that case.
1. You're hosting ISP may not do this, but the ISPs of the people who view your webpage may. How would you know? Are you going to sign up for an account with every ISP in existence and test each one for yourself?
2. There are plenty of people who would never know: people who use adblocking software, for one. In any event, many commercial webpages are so overrun with advertising anyway, how would you know that one in the crowd was inserted by your ISP and not original to the page?
The subject doesn't make sense to you because you're not running a small ISP. The potential revenue vs. the loss of a few savvy customers might be a trade-off you'd be willing to make if the upside were big enough.
Insisting on "correct" English is like saying that there is only one, definitive recipe for chili.
I always thought it was funny that I get tons of MicroSoft ads when viewing slashdot, now it all makes sense with my MSN as my ISP they are inserting those nasty ads everywhere.
3. What kind of moron continues to use Microsoft products?
4. What kind of moron continues to use AOL?
5. What kind of moron uses Alexa?
If you can answer 3-5, then you already know the answers to your own questions.
I can think of one way to do it - but it wouldn't be too hard for a determined ISP to defeat:
Step 1: Calculate md5sum of webpage, store in separate location.
Step 2: Include on the webpage some javascript to md5sum itself and compare this to md5sum in known location. Issue an alert if it differs.
Step 3: Profit!
Of course, this is awkward for dynamically generated pages and if the ISP is happy to mess around with the page to insert ads, they're probably also happy to mess around with any javascript which detects it coming down the line. Does this method solve that?
It's not the host ISP that's inserting the ads, It's the "Client" ISP, for example Joe Smith buys a computer and buys high speed internet from "ECI" the Evil Cable ISP. Joe Smith visits Bob's Website, Bob, who hates ads never put any on his webpage, and instead makes his money through online sales of his product. Now Joe loads up Bob's webpage to purchase a widget from Bob, and he sees Ads all over Bob's Website. Bob who has GHI (Good Highspeed ISP) visits his website and there's no ads. ECI is putting the Ads on Bob's website. and collecting all the revenue from those ads. Profiting off of Bob's Website.
It seems that everyone is concerned about downstream modification, and is completely ignoring the possibility of upstream modification. What if Sprint started modifying upstream http-posts to start a more viral ad distribution system? Not only would they be able to target their customers, they would also be able to target the customers of anyone who could read the post!
This is the reason that we need to push for network neutrality. When the only choices are between a giant douche which alters content and a turd sandwich which alters content, the customer ends up screwed in the end.
Karma: SELECT `karma` FROM `users` WHERE `userid`=138474;
We've seen a couple cases of NebuAdd, one other that looks interesting, and a fair amount of addblocking/firewall software (eg, ZoneAlarm does some modifications)
We are waiting for the Slashdot and DIGG deluges to pass, however, before we have a more detailed analysis.
Test your net with Netalyzr
Actually... theoretically, would this be somehow "between" client-side and server-side? I'm having a hard-time classifying this type of interception.
It is by my will alone my thoughts acquire motion; it is by the juice of the coffee bean that the thoughts acquire speed
I've got this awesome new Firefox plugin called ISPBlock Plus. It blocks all the packets your ISP is sending you in-flight. No more ads!
Well, unfortunately for me I want broadband access, and in my area I'm stuck with only one choice for an ISP. So, whether my ISP inserts ads or not I really only have two choices: 1) Learn to like it* 2) Learn to live with no internet access.
* - Of course there is always choice 1b) Learn to use Firefox with the Adblock Plus plugin (which I already do - and it works great!). This is a choice that I'm sure many others will point out.
And yes I've called several other broadband providers and none of them have any plans to provide service to my area.
These guys actually want as much traffic as they can get to get a good idea of what isps are doing what. Go ahead, click online tool. It's pretty nifty.
If you are about to mod me down, keep in mind that this post was most likely sarcastic.
Years ago on one April Fool's day, I got a list of ad sites (from the usual /etc/hosts files out there), then got the internal DNS server to resolve them to a server that served up the company logo instead (for all possible url paths).
:). Nope I didn't get fired or even reprimanded - plus even better - I was saving company bandwidth (remember this was years ago)... Nobody complained about the lack of ads from ad.doubleclick.net and gang.
FWIW, seemed only one person noticed that the forbes page they loaded somehow had the company logos everywhere
I toyed with the idea of substituting ads with reminders (meeting at 2pm, or "you have been on slashdot for 2 hours!") and other more useful information.
Lastly, I don't think their naive hashing thing checks if you are altering the images - the content may remain unchanged, but linked to contents may change (they aren't checked from what I see), so it doesn't work for my scenario where different ads are substituted for the unaltered URL.
That said, I'm still curious on:
1) How many ISPs would bother modifying traffic from those 7 destinations they are testing.
2) What the various laws around the world say about this.
3) What those laws say about "sponsored internet access" where an ISP gives a cheaper package/plan where the ads are substituted with the ISPs advertisers with the risk of some corrupted info.
4) What those laws say about "streamlined internet access" where an ISP provides a package/plan where ads and other crap are removed (or modified) for their customer.
If the pages you create have a copyright notice comment in them, I am pretty sure the ISP's could not modify them without your permission. I am not a lawyer, but it seems right to me.
..why not just use SSL?
I can understand how this wouldn't help with hosting ISPs who insert ads into their own customers' pages, but if you're worried about your readers' ISPs modifying your pages, SSL seems like a no-brainer.
What's the downside? It can't still be CPU, can it? It's 2007 now, and processing power is ridiculously cheap/fast.
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
Or maybe just the kind of moron who won't even RTFS.
(IANAL)
And they are claiming a new tool to do something which a universally-deployed tool has been doing for ten years now.
------------
Software development blog
Considering that the customers that are likely to notice this are probably the same customers that use the most bandwidth, I can see and ISP doing this as a tactic to shed thier least profitable customers.
If it's plain ol' http, anything can be modified on the fly,
including hiding posts to blogs.
All blogs should be https, in fact, all websites should
be https, even if you have to roll your own SSL cert.
You are being MICROattacked, from various angles, in a SOFT manner.
Man-in-the-middle
Are you pretending to be mentally challenged in order to troll, or do you really not understand even after having it explained to you a little further up the page? It is not the developer's ISP, or the hosting ISP that is doing this! It is the ISP of the people looking at the page. So, you left out a step in your patented eyeball method: signing up for every ISP in existence and loading your page, to see if that particular ISP does it.
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
Hey, Christian: judge not lest ye be judged. I don't think Christ said anything along the lines of "Thou shalt mock the stupid, for they are humorous, and will not understand your taunts."
If you post on a site with a lot of atheists and agnostics while shilling for your Christian music, and then turn around and act un-Christian, expect to be called on it.
This is a war however which we can make damn difficult by using virus-like mutation techniques, so that every checker looks different: force THEM to solve the AV defender arms race.
As long as the actual API used by the Javascript is common enough that the ad-injectors can't recognize and block our code by keeing in on the API calls rather than the overall Javascript.
The proper solution, adding integrity checking to all HTTP, seems like its not happening.
Test your net with Netalyzr
I understand the first knee jerk reaction people have with the concept of ISPs sticking ads on content. Kind of a NIMBY thing. However if you think about it, people have been making money out of other people's content for a long time in the Internet. What's a search engine really but a way to generate ad revenue by organizing other people's content? This reminds me of when content providers were complaining that search engines' tactics of deep indexing their sites was allowing the users to bypass their home pages (where the ads were hosted). I understand why ISPs are going to be moving in this direction. Just compare the market valuations of the big 'Internet' plays with those of the large ISPs. Why should Google's market cap. be 158.38B and Comcast's 88.20B? Google and other companies like it honestly have done very little but leech off of other people's content (Search, News, Groups, Images, Videos, etc.). Their business model is to provide ads. (content providers: where is the outrage here?). What type of company would you miss more if it were to disappear tomorrow (search engine vs last mile ISP)? Comcast and the other providers in comparison have made a massive investment in infrastructure that they have to both innovate and maintain. Look at Verizon and FIOS as an example. Do you see Google spending B's to roll out fiber? To some degree they have been left out. They are are in an industry where their product has become a commodity where people make decisions largely based on price. These technologies give these companies an opportunity to change the nature of the game. Sure its greedy, that's capitalism, but its not evil. Nor is it illegal if done properly. Its an agreement between two parties, one the provider, the other the consumer who wants a FREE/cheap (cost) Internet. If you want 'free' Internet you are going to have to pay for it somehow. Either through the government through taxes, or though something like ads. Personally I prefer the ads, if I do not like them I can pay (time or $$) for the option to not have them.
Doesn't just using HTTPS as the protocol to retrieve pages at URLs make the server sign the code, and encrypt it so no middlemen can change it "in flight"? I guess if the HTTPS server is controlled by the ISP, the server just signs the altered pages. But what kind of downstream test can stop that?
--
make install -not war
Could someone find out that you are using Adblocker this way?
Well if we block changes then how will my /. duplicate removal software work?
Except that in the contract, as governed by the terms and conditions, you probably assigned your ISP right to insert ads or otherwise modify your content in exchange for hosting and serving the content (in addition to any fee you pay for the service).
This is effectively what my former ISP, Club Internet, was doing when it insisted on inserting JavaScript into the header of every page, in order to display advertising.
The very existence of your content on the server is dependent on the terms of the contract and uploading the content happens later than the date the contract became effective, I think that you would have a very hard time attacking your ISP.
The ISP will simply point out the terms and conditions you agreed to, and say "stick to the terms and conditions, or you are free to take your business elsewhere".
Beef>