Slashdot Mirror

As someone who has been using squid to block ads and other annoyances since at least 1996 or 1997 (i.e. since animated gif ads became commonplace) I have to say that squid is no longer of much (if any) use in blocking ads.

That capability was a casualty of https everywhere - generally a good thing but with some unfortunate collateral damage.

With https, squid only sees a single CONNECT, it doesn't see individual URLs or requests, so is unable to filter any of them.

I rely mostly on uMatrix and uBlock these days....and spend far too much time fucking around with Stylus (which is useful not only for making sites readable by undoing cretinous web-designer abominations but also to, e.g., force "display: none !important;" on all sports and celebrity crap on newspaper web sites) and Tampermonkey

i thought merging REs was standard practice by now. i've been doing it since long before I started using SpamAssassin, when I was still mostly using postfix body_checks and header_checks.

here's some of my anti-spam stuff.

the scripts are old, but pretty close to what i actually still use today to generate postfix body/header checks and spamassassin rules.

they're not packaged software you can just install and use - think of them as examples of a particular approach to managing anti-spam rulesets.

BTW, note that with SpamAssassin, fewer and larger rules require less CPU time to run, but reduce the likelihood of multiple matches if there are multiple spammy phrases in an email - max one match per rule. this is why the scripts are configured to generate max of 500-character rule lines, when SA can easily handle 5000 or more characters per line. also, shorter lines are easier to read when debugging problems, and each rule is generated with a unique identifier so I can see which rules are matching for each msg

AdBlock is probably the single most useful Mozilla extension.

I had been using Squid for years with an ad-blocking add-on. After my home server's hard drive crashed, I reloaded Gentoo onto the replacement drive, but left off Squid and configured Adblock on my other computers (running a mix of WinXP, Linux, and Mac OS X). It's worked so well that I'll probably just keep things this way. It's especially handy for my notebook, as it saves me from having to switch proxy settings every time I move between home, work, and elsewhere. It's relatively easy to port settings between machines, but you can kill >90% of ads with maybe half-a-dozen rules that take a minute to key in.

Adblock 0wnz j00.

Am I the only one who had to copy and paste the text from the second linked page into a word processor to view the story?

In Mozilla, I got a Sprint ad in a huge box which overlaid the story text, making it impossible to read. I tried hovering over and such to see if it had a 'click to hide' option, but nothing. I'm not clicking the ad itself.

No problems here with Mozilla 1.4...but this might have something to do with it.

Great, we get a bunch of moralizing from a bunch of jackasses who allow FUI banner ads on their site.

Banner ads? I didn't see any banner ads...

However if that's annoying you, you pay and get rid of the advertisement!

...or you set up an ad-filtering proxy server and get rid of the ads that way.

Since the ads are inserted at the ISP theres no way to even block them

To insert them, it has to alter the HTML returned by a webserver as it passes through...essentially, it's a proxy server that munges content to insert ads. There are other proxy servers (WebWasher comes to mind as an example) that do the opposite--they examine the HTML they receive and alter it to remove ads, scripts, and other nastiness. Since your browser will also issue an HTTP GET request for the extra ads, ad-filtering proxies that work by redirecting certain types of requests (Squid can do this) should still be effective as well...you just need to create another rule to block your ISP's ad server.

(The popups appear when you click a "next page" or "previous page" link in the article, so Mozilla must be treating it as a requested popup. In addition to a whitelist of sites that are allowed to throw popups, Mozilla needs a blacklist of sites that are never allowed to throw popups.)

Or just install Mozilla which has pretty decent popup prevention

My ad-filtering proxy (updated block list available through this page) blocks the Flash ad they try to send. Editing the URL in the address bar brings you back to the Flash ad (which gets replaced with a 1x1 transparent GIF by the proxy). You also need to remove the cookies set by Salon and block them from sending any more cookies (the same page came up fine in Lynx when I told it to not accept Salon's cookies). Select Edit|Preferences, select "Cookies" under "Privacy & Security," and click "Manage Stored Cookies." Check "Don't allow removed cookies to be reaccepted later," scroll through the list of cookies, and remove the ones set by Salon (I found two, sent by salon.com and www.salon.com).

There's no way to 'block' flash ads in Mozilla yet, and Yahoo keeps throwing up this damn huge Oracle/IBM ad on the my.yahoo.com page I have.

squid-redir lets you block anything from anywhere, based on the URL. This rule, for instance, blocks all Flash at the Motley Fool:

//.*.fool.com/.*.swf BLANK

It substitutes a 1x1 transparent GIF for the Flash. Something similar would work elsewhere...if you want to cut off all Flash from all sites, you can do that:

//.*.swf BLANK

It works on any system that can run Squid and Perl, and it'll work with any browser (I usually use IE, though I also have Lynx, Konqueror, and iCab available). More info and the block list I'm currently using are available here. Here are the Yahoo-related rules I'm currently using:

//.*yimg.com/.*/main.*.gif $1
//.*yimg.com/.*/yahoo.gif $1
//.*yimg.com/i/.* $1
//.*yahoo.com/serv.* BLANK
//.*yimg.com/.*.js NULLJS
//.*yimg.com/.*/adv/.*.gif BLANK
//.*yahoo.com/adv/.* BLANK
//images.yahoo.com/promotions/.* BLANK
//.*yimg.com/.*\.(gif|jpg) BLANK
//java.yahoo.com/.* BLANK
//promotions.yahoo.com/promotions/.*gif BLANK
//.*.geo.yahoo.com/toto NULLJS

That is exactly what fair use is all about.

As for your sig WRT ad-blocking...if I didn't have to worry about third parties following my every click, maybe I'd consider shutting down Squid. When I go to fubar.com, I've consented for fubar.com to send content (including potentially harmful scripts) to my computer. That consent doesn't extend to DoubleClick, Aureate, or other third parties (note the previous remark about usually not blocking ads served up by a website's server...if fubar.com has its own banner, it'll usually get through). Also, what about the people who use Lynx...do you consider them to be without scruples because their browser will never display that inane "punch the monkey" banner?

Interesting to note that Kuro5hin.org has started a "pay for no ads" version of their site. I doubt it is going to be very succesfull though: most people don't really mind banner ads that much.

...and those of us who are sufficiently annoyed by banner ads to do something about it have already taken measures to block them anyway

Except now ads will be actually part of the content, so it'll be a lot harder to filter them out...

Rather than just get a blank rectangle where a banner should be, you might end up with blank javascript pop-ups, which'll be more annoying then a banner ad could ever be, IMO.

squid-redir includes a replacement script that closes pop-ups as soon as they open. Since it rewrites URLs, it can substitute this script whenever a known pop-up site is called.

Likewise for redirect links. If the actual link to the article you want to read requires that you step through, say, DoubleClicks site in order to see the ad and hence, retrieve the URL, then you'll be effectively blocked from content if your blocking proxy prevents you from visiting doubleclick.

If you look at the average DCLK URL, you'll often find the true destination URL tacked on somewhere at the end. By stripping out DCLK's garbage, you can go straight to your destination without pinging DCLK. squid-redir ought to be able to handle this; I've set mine to strip out the navigation bar at the top of AOHell-hosted websites. As an example, here's the pattern squid-redir uses for that fix:

//(members|hometown).aol.com/ //members.aol.com/_ht_a/*

This is a simple pattern that only adds "_ht_a/" to the URL. Similar manipulation is possible for other sites as needed.

[WebWasher] can filter out Javascript cued on opening/closing windows, remove pop-ups entirely, and reclaim space which would have been used by banner ads.

Actually, squid can do some (all?) of these things. My former company used to use a squid proxy, and they'd configured it to automatically remove popups from a number of well known annoying sites (Tripod, GeoCities, etc.)

I used WebWasher for a while...it's really good at figuring out what's probably an ad, but it also tends to be a bit too aggressive in blocking JavaScript (sometimes, Windows Update wouldn't work while WebWasher was running). I put Squid on my Linux server at home a few months back, along with a Perl-based redirector script from http://taz.net.au/block. It's been nearly as effective as WebWasher, without impairing the function of certain sites like WebWasher sometimes does. If an ad slips through, it happens because the site that serves it up isn't in the known list of ad sites; I check the logs, add the new site, and no more ads come through from them ever again. (The list I'm currently using is available at http://salfter.dyndns.org/redir.)

Now I'm trying to get this setup running on the NT server at work. Squid itself is up and running (the homepage for the NT port of Squid 2.1 is at http://www.phys-iasi.ro/users/romeo/squidnt.htm, but their download link is bad...try ftp://ftp.tsu.tula.ru/pub/windows/squidnt21a.zip instead); now I need to get Perl going and then see if the redirector will work the same under NT as it does under Linux. (I'll probably have to wait until everybody's gone, in case the Perl installer decides it needs to reboot the server...and even if it doesn't, some other needed updates will want to reboot the server anyway.)

Don't get me wrong, I'm sensitive about privacy too, which is why I have doubleclick.net cookies blocked.

Merely denying cookies, I suspect, isn't going to hit anybody's wallet. It's still a good idea from a standpoint of them not being able to follow you as you move about the Web, but the people who make money from banner ads are probably counting the number of downloads of a particular banner. As long as your browser is still grabbing the ads, all they need to do is crunch their server logs to determine who gets paid.

To stop this, you'll need an ad filter. Squid-redir works well under anything that can run Squid. If you're looking for something a little easier to set up and you're running Win9x, there's WebWasher.

(I have a more comprehensive list of sites for squid-redir to block here.)

_/_
/ v \
(IIGS( Scott Alfter (remove Voyager's hull # to send mail)
\_^_/

Evidence: Notice the sharp increase in bright, flashing advertising recently? Like the "B12" ad I am viewing right now...

Actually, I hadn't. Ad-blocking software is a wonderful thing. :-) (Here's a list of sites to block with the aforementioned software.)

_/_
/ v \
(IIGS( Scott Alfter (remove Voyager's hull # to send mail)
\_^_/

I actually used to [redirect ad sites to localhost in /etc/hosts], but a number of sites stopped working completely for me - instead of loading a broken image or whatever for the banner, I got a full page error message, and no web page.

I've been using squid for a few months to filter out ads and keep cookies from being set; it's worked really well. It hasn't broken any sites that I can recall, and it's cut out most of the clutter and third-party cookies. You still need to check periodically for third-party cookies as new ad servers are put online, but I've gotten most of the current sites loaded into it. It even strips out the annoying host-navigation frames put up with sites hosted by the likes of Xoom and AOHell.

Here's some info on configuring squid as an ad-blocker. My list of blocked sites is here. (I've tweaked the redirector script to support a NULLHTML tag that causes a file containing "<html></html>" to be returned...it's a simple hack, and I don't know squat about Perl.)

_/_
/ v \
(IIGS( Scott Alfter (remove Voyager's hull # to send mail)
\_^_/

I wouldn't block ads out if they weren't so annoying. It's impossible to read text on a screen with some annoying animation hovering over it.

There are still plenty of ways for web sites to intermediate themselves into the value chain. Say, for example, slashdot started its own shopping mall. Instead of them putting whatever products someone paid to have in there available, they'd have only products that the Slashdot team thought were cool. So long as /. never breaks that trust, people would buy stuff.

What network cards do you guys use? That cool new toy you just covered, where can I get it? Oh looky, I could buy it from the place you linked to but who would I rather trust with my personal details?

I already got that with Junkbuster! (though I still have to stop and think about why a site might act funny if Junkbuster's eating the site's cookies...)

Try squid-redir...it doesn't mangle pages as badly as Junkbuster.

_/_
/ v \
(IIGS( Scott Alfter (remove Voyager's hull # to send mail)
\_^_/

My blocklist is available here, and via anonymous rsync at rsync://cloudmaster.com/redir/redir
Assuming you already have squid up and running, you can just

• mkdir /var/squid/blocker
• echo "redirect_program /var/squid/blocker/squid.redir" >> /etc/squid.conf
• rsync -v rsync://cloudmaster.com/redir/* /var/squid/blocker
• cd /var/squid/blocker
• make

I'm already using squid to block banner ads _because_ if I'm not looking for ads. I suppose this'd be OK if I were searching for exactly what was being advertised, but I have a feeling the ads wil be marginally relevant.
As a side note, isn't this kinda what lycos does? The banner ads are usually somewhat related to your search...