Slashdot Mirror
AVG Fakes User Agent, Floods the Internet
Slimy anti-virus provider AVG is spamming the internet with deceptive traffic pretending to be Internet Explorer. Essentially, users of the software automatically pre-crawl search results, which is bad, but they do so with an intentionally generic user agent. This is flooding websites with meaningless traffic (on Slashdot, we're seeing them as like 6% of our page traffic now). Best of all, they change their UA to avoid being filtered by websites who are seeing massive increases in bandwidth from worthless robots.
For anyone that happens to run a site behind an F5 BigIP, here's a nice little IRule to nuke this horrible crap from orbit.
rule IRULE_block_avg-prefetch { ::avg_useragents [list \
when HTTP_REQUEST {
set
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" \
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)" \
"User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" \
"User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)" \
]
if { ![HTTP::header exists "Accept-Encoding"] } {
if { [matchclass [HTTP::header User-Agent] equals $::avg_useragents] } {
reject
}
}
}
- U
Avira.
Posts not to be taken literally. Almost everything is sarcasm.
I use AVG on a couple machines. I didn't really think about the traffic tracking piece of this when I saw it working, I just thought about it slowing me down, increasing bandwidth use, etc. and I turned it off.
I know most people don't mess with defaults - and I'm not defending them as far as the agent thing and all that - but it was easy to do.
On the negative side my avg icon in the systray has a big exclamation over it like something is really wrong - when I know it's just because I turned off a piece of functionality I don't want to use.
It's hard to believe that's how Micronians are made. Why don't we see it right now by having you both kiss one another?
AVG was once a good product. Then, it got bloated and started eating up kernel memory voraciously. It was impossible to play games with it running in the background, especially Crysis (skip the jokes, my system could handle it maxed once I replaced AVG with Avast!). Now, with this development, I'll be sure to replace AVG with Avast! on all of my machines, not just my gaming one.
Colin Dean Go a year without DRM
Must be a slow news day...This story's been around for nearly 2 weeks. AVG will probably keep changing the useragent with every few updates to annoy Admins and stats sites...
Laters Sol "Have you found the secrets of the universe? Asked Zebade "I'm sure I left them here somewhere"
Avast.
It's not just for Talk-Like-A-Pirate Day any more!
Slightly disreputable, albeit gregarious
LinkScanner, the component they're talking about, works in Firefox as well - so no, using Firefox does not 'keep you safe'.
Nor is this about the users of the thing in the first place - either they like its functionality (security theatre-advance warning blabla) and leave it on, or they don't and they switch it off.
This is about the poor, poor admins who are suddenly seeing bogus traffic and omgosh it's spoofing user agents at that! .. repeatedly*
*changes his user agent to 'cry more, Taco' in FF and hits F5
You can actually install AVG 8 without the 'Safe Search' feature that crawls websites (it's essentially a BHO/Firefox extension). Even if you already have AVG 8, you can uninstall it and reinstall:
At a Command Prompt window, type /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch
c:\downloads\avg_free_stf_xxxxxxxxxx.exe
where c:\downloads\avg_free_stf_xxxxxxxxxx.exe is the full path of your AVG 8 installer.
Go somewhere random
Has anyone else noticed that AVG 8 is also DOG SLOW on their PC? My computer is from 2001 and ran fine with 7.5, but 8.0 is unusably slow. Every time an application is opened it takes forever for AVG to scan it and let the app open. This combined with this linkscanner bullcrap has caused me to switch. I doubt I'll ever go back.
I've been using Avast! Home Edition for a while now, no complaints.
I'm a longtime user of AVG. Version 7 was reasonably lightweight, effective and (most importantly to me) unobtrusive.
Unfortunately, version 8 is a different story. After Grisoft forced me to upgrade in May, suddenly AVG became a nagging resource hog. Nightly scan times rocketed from about an hour to over six hours - a scheduled scan that started at 2am would still be going at 8:30am. I have been able to reduce this time somewhat by changing the scan settings (e.g., don't scan inside compressed archives), but it's still slow.
Most annoyingly, their new "LinkScanner" and "SafeSurf" features slowed my browser to a crawl. I didn't want these, since I already use FireFox with the AdBlock and NoScript extensions. I tried to simply disable LinkScanner, but then AVG constantly bothered me with nagging warnings that my computer "was not fully protected". After a little digging, I found that it was possible to uninstall the feature entirely with the following command:
avg_free_stf_xxxx.exe /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch
(Substitute "avg_free_stf_xxxx.exe" in the above command with the name of your setup file.)
This improved my browser performance, and eliminated the warnings.
I'm still (grudgingly) using AVG, but I will switch if/when I find a better alternative.
It's not really the load -- it's throwing off our internal metrics so we don't know what readers are actually interested in. We like numbers, and messing with our stats annoys us.
avast! antivirus Home Edition is FREE to use but it is necessary to register before the end of the initial 60 day trial period. To register, click here. Following registration you will receive by E-mail a license key valid for a period of 1 year. After you have downloaded and installed the program, the license key must be inserted into it within 60 days. The registration process is very easy, and it will take you only a couple of minutes.
Also Avira has been getting more and more annoying over the years, it's practically adware now.
So now it looks like it's either AVG with the browser plugins removed or MoonAV (which is FOSS):
http://www.moonsecure.com/
(It used to have a problem where you'd need to remove the Windows service manually after uninstalling, they might have fixed it though.)
"When information is power, privacy is freedom" - Jah-Wren Ryel
I love AVG for the free scanner it provides but ...
Safesearch: It doesn't work.
Somehow I ended up on one of those "Your computer is infected..." sites
while trying to dl their crap. So for fun I went back to the referrer page
(google) and sure enough, it was marked as safe.
I second Avast, it's free for home use, and has very reasonable commercial license terms. Plus it gives you one code for all machines, no need to chase 20 different keys like you do with Norton etc. And the key is good for the whole license period; before I used to loose at least 10 % of licenses to crashes or borked installs, and getting new ones from Norton was like pulling wisdom teeth on a grouchy alligator.
I'm aging rapidly, I bought a new game and had no idea if my machine was good for it.
Hah! Checking my addons in FF3, and on AVG Safe Search 8 it says "Not compatible with Firefox 3.0". Awesome :-)
I installed AVG on my mother-in-law's machine because she had an expired trial version of some other AV software. It was great for a while, but they must've had a change in direction/managment. Because all of a sudden they started with popups to get a full paid version of the software - even uninstalling the product didn't fix it. I had to surgically extract crap from the registry and program files folder to finally get rid of it. Avast or ClamWin for me - no more AVG.
90% of everything is crap. Also, crap is relative.
They are attempting to help their customers at the expense of everybody else on the Internet. If I understand the article, they're pre-scanning every possible URL on a page. In essense they're clicking every possible link before you do.
For instance I searched for "avg" on google and counted the number of "href=" appearances on the resulting page. It happened to be an even 100. AVG is visiting ALL of of those HREFs in the background. A user will click on only one.
I would assume their scanner is smart enough to remove duplicates HREFs and do some other smart things. But still, this is a terrible idea. I guess we all have to go buy more servers and bandwidth so the anti-virus people can make a living now?
Google, as other search engines, not only obey robots.txt but also quite clearly identify themselves a GoogleBot and connect from an IP address registered to Google.
Another company that's particularly bad is Cyveillance, they also regularly spider sites very aggressively (redownloading the same content repeatedly even tho it hasn't changed), and they try to spoof their user agent.
If you mail them to complain, they will claim to remove your sites from their spider if you give them the IPs, but they lie... They will continue spidering your sites, but from a different IP range which is still traceable to them.
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
While all other /.ers are complaining that ClamWin is useless I want to bring some points :
- ClamWin has a built-in plug-in to scan incoming mail in outlook.
- ClamWin is easy to call from scripts and is a nice thing to add to the commands that are launched by your favourite bit-torrent client once a file is completed (I use this on my linux based torrent downloading/file server machine)
- ClamWin has plug-ins for FireFox : SafeDownload, Download Scan, Download Statusbar all let you launch the scanner of your choosing once a download finishes. ClamWin Antivirus Glue is another solution, but one has to manually update the minimal supported version (the plugin is set to support up to 1.5 although it works with more modern versions).
So, although ClamWin isn't continuously scanning in background, it can cover most of the usual entry points. (Although I don't know about plugins for Thunderbird and Microsoft file server).
For those who like to test newer bleeding edge software : WinPooch software can launch a scan when ever an executable is opened - it's almost as good as an on demand scanner.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
I have an updated version of this redirect to AVG, based on info I've been gathering over the last 2 weeks from Webmaster World, El Reg, and of course Pixelbeat. Here is the rule set I am using now:
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1; SV1\)$" [OR]
RewriteCond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1;1813\)$"
RewriteCond %{REQUEST_METHOD} ^GET$
RewriteCond %{HTTP_REFERER} ^$
RewriteCond %{HTTP:Accept-Encoding} ^$
RewriteCond %{HTTP:Accept-Language} ^$
RewriteCond %{HTTP:Accept-Charset} ^$
RewriteRule ^.* http://www.avg.com/?LinkScannerSucks [R=301,L]
I have the check for "GET" method in there so that the earlier "User-Agent: ..." version of linkscanner will still get redirected. See, that version does a HEAD request first, most likely to check for a redirect. So we allow that HEAD request to pass, since it is small any ways. But the GET request that follows will still get redirected. We want to redirect the maximum amount of traffic we can to AVG, to drive the point home.
This filter is also more selective, by also checking for the non-existance of Accept-Language and Accpet-Charset we make absolutely sure we are not redirecting a valid user. No web browser out there would fail to set all 3 of these, so we can be absolutely sure this is crap coming from a linkscanner.
I also decided to use a permanent redirect, in hopes that linkscanner caches this and it will reduce the number of repeat hits from the same user? Not sure if that is the case or not.
Someone in this thread asked if these rules work in the main Apache config file instead od using .htaccess. I don't use .htaccess on my servers either, and these rules reside in our main Apache config file. So the answer is yes, it will work in BOTH places.
I hope by now that AVG realizes the futility in their continuing to change how linkscanner acts to try and hide it from us. We will simply continue to work together as a community of server admins to block this crap and send it right back at them!
If you right-click on a component in the AVG User Interface, you can select 'Ignore Component State'. That way the component is turned off, but the AVG icon doesn't show anything wrong.
Hope this helps...