Slashdot Mirror
AVG Fakes User Agent, Floods the Internet
Slimy anti-virus provider AVG is spamming the internet with deceptive traffic pretending to be Internet Explorer. Essentially, users of the software automatically pre-crawl search results, which is bad, but they do so with an intentionally generic user agent. This is flooding websites with meaningless traffic (on Slashdot, we're seeing them as like 6% of our page traffic now). Best of all, they change their UA to avoid being filtered by websites who are seeing massive increases in bandwidth from worthless robots.
For anyone that happens to run a site behind an F5 BigIP, here's a nice little IRule to nuke this horrible crap from orbit.
rule IRULE_block_avg-prefetch { ::avg_useragents [list \
when HTTP_REQUEST {
set
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" \
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)" \
"User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" \
"User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)" \
]
if { ![HTTP::header exists "Accept-Encoding"] } {
if { [matchclass [HTTP::header User-Agent] equals $::avg_useragents] } {
reject
}
}
}
- U
I use AVG on a couple machines. I didn't really think about the traffic tracking piece of this when I saw it working, I just thought about it slowing me down, increasing bandwidth use, etc. and I turned it off.
I know most people don't mess with defaults - and I'm not defending them as far as the agent thing and all that - but it was easy to do.
On the negative side my avg icon in the systray has a big exclamation over it like something is really wrong - when I know it's just because I turned off a piece of functionality I don't want to use.
It's hard to believe that's how Micronians are made. Why don't we see it right now by having you both kiss one another?
AVG was once a good product. Then, it got bloated and started eating up kernel memory voraciously. It was impossible to play games with it running in the background, especially Crysis (skip the jokes, my system could handle it maxed once I replaced AVG with Avast!). Now, with this development, I'll be sure to replace AVG with Avast! on all of my machines, not just my gaming one.
Colin Dean Go a year without DRM
Avast.
It's not just for Talk-Like-A-Pirate Day any more!
Slightly disreputable, albeit gregarious
You can actually install AVG 8 without the 'Safe Search' feature that crawls websites (it's essentially a BHO/Firefox extension). Even if you already have AVG 8, you can uninstall it and reinstall:
At a Command Prompt window, type /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch
c:\downloads\avg_free_stf_xxxxxxxxxx.exe
where c:\downloads\avg_free_stf_xxxxxxxxxx.exe is the full path of your AVG 8 installer.
Go somewhere random
Has anyone else noticed that AVG 8 is also DOG SLOW on their PC? My computer is from 2001 and ran fine with 7.5, but 8.0 is unusably slow. Every time an application is opened it takes forever for AVG to scan it and let the app open. This combined with this linkscanner bullcrap has caused me to switch. I doubt I'll ever go back.
I'm a longtime user of AVG. Version 7 was reasonably lightweight, effective and (most importantly to me) unobtrusive.
Unfortunately, version 8 is a different story. After Grisoft forced me to upgrade in May, suddenly AVG became a nagging resource hog. Nightly scan times rocketed from about an hour to over six hours - a scheduled scan that started at 2am would still be going at 8:30am. I have been able to reduce this time somewhat by changing the scan settings (e.g., don't scan inside compressed archives), but it's still slow.
Most annoyingly, their new "LinkScanner" and "SafeSurf" features slowed my browser to a crawl. I didn't want these, since I already use FireFox with the AdBlock and NoScript extensions. I tried to simply disable LinkScanner, but then AVG constantly bothered me with nagging warnings that my computer "was not fully protected". After a little digging, I found that it was possible to uninstall the feature entirely with the following command:
avg_free_stf_xxxx.exe /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch
(Substitute "avg_free_stf_xxxx.exe" in the above command with the name of your setup file.)
This improved my browser performance, and eliminated the warnings.
I'm still (grudgingly) using AVG, but I will switch if/when I find a better alternative.
It's not really the load -- it's throwing off our internal metrics so we don't know what readers are actually interested in. We like numbers, and messing with our stats annoys us.
I have an updated version of this redirect to AVG, based on info I've been gathering over the last 2 weeks from Webmaster World, El Reg, and of course Pixelbeat. Here is the rule set I am using now:
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1; SV1\)$" [OR]
RewriteCond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1;1813\)$"
RewriteCond %{REQUEST_METHOD} ^GET$
RewriteCond %{HTTP_REFERER} ^$
RewriteCond %{HTTP:Accept-Encoding} ^$
RewriteCond %{HTTP:Accept-Language} ^$
RewriteCond %{HTTP:Accept-Charset} ^$
RewriteRule ^.* http://www.avg.com/?LinkScannerSucks [R=301,L]
I have the check for "GET" method in there so that the earlier "User-Agent: ..." version of linkscanner will still get redirected. See, that version does a HEAD request first, most likely to check for a redirect. So we allow that HEAD request to pass, since it is small any ways. But the GET request that follows will still get redirected. We want to redirect the maximum amount of traffic we can to AVG, to drive the point home.
This filter is also more selective, by also checking for the non-existance of Accept-Language and Accpet-Charset we make absolutely sure we are not redirecting a valid user. No web browser out there would fail to set all 3 of these, so we can be absolutely sure this is crap coming from a linkscanner.
I also decided to use a permanent redirect, in hopes that linkscanner caches this and it will reduce the number of repeat hits from the same user? Not sure if that is the case or not.
Someone in this thread asked if these rules work in the main Apache config file instead od using .htaccess. I don't use .htaccess on my servers either, and these rules reside in our main Apache config file. So the answer is yes, it will work in BOTH places.
I hope by now that AVG realizes the futility in their continuing to change how linkscanner acts to try and hide it from us. We will simply continue to work together as a community of server admins to block this crap and send it right back at them!
If you right-click on a component in the AVG User Interface, you can select 'Ignore Component State'. That way the component is turned off, but the AVG icon doesn't show anything wrong.
Hope this helps...