Slashdot Mirror
AVG Fakes User Agent, Floods the Internet
Slimy anti-virus provider AVG is spamming the internet with deceptive traffic pretending to be Internet Explorer. Essentially, users of the software automatically pre-crawl search results, which is bad, but they do so with an intentionally generic user agent. This is flooding websites with meaningless traffic (on Slashdot, we're seeing them as like 6% of our page traffic now). Best of all, they change their UA to avoid being filtered by websites who are seeing massive increases in bandwidth from worthless robots.
For anyone that happens to run a site behind an F5 BigIP, here's a nice little IRule to nuke this horrible crap from orbit.
rule IRULE_block_avg-prefetch { ::avg_useragents [list \
when HTTP_REQUEST {
set
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" \
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)" \
"User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" \
"User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)" \
]
if { ![HTTP::header exists "Accept-Encoding"] } {
if { [matchclass [HTTP::header User-Agent] equals $::avg_useragents] } {
reject
}
}
}
- U
Avira.
Posts not to be taken literally. Almost everything is sarcasm.
I use AVG on a couple machines. I didn't really think about the traffic tracking piece of this when I saw it working, I just thought about it slowing me down, increasing bandwidth use, etc. and I turned it off.
I know most people don't mess with defaults - and I'm not defending them as far as the agent thing and all that - but it was easy to do.
On the negative side my avg icon in the systray has a big exclamation over it like something is really wrong - when I know it's just because I turned off a piece of functionality I don't want to use.
It's hard to believe that's how Micronians are made. Why don't we see it right now by having you both kiss one another?
This is not AVG doing this, it is the AVG IE toolbar. And since this is running in the IE context it is debatable if it should not use the IE user agent.
If you use Firefox or disable the toolbar it is a non issue. The issue to me is I can't figure out how to install AVG without this toolbar, or how to remove it.
AVG was once a good product. Then, it got bloated and started eating up kernel memory voraciously. It was impossible to play games with it running in the background, especially Crysis (skip the jokes, my system could handle it maxed once I replaced AVG with Avast!). Now, with this development, I'll be sure to replace AVG with Avast! on all of my machines, not just my gaming one.
Colin Dean Go a year without DRM
Must be a slow news day...This story's been around for nearly 2 weeks. AVG will probably keep changing the useragent with every few updates to annoy Admins and stats sites...
Laters Sol "Have you found the secrets of the universe? Asked Zebade "I'm sure I left them here somewhere"
Avast.
It's not just for Talk-Like-A-Pirate Day any more!
Slightly disreputable, albeit gregarious
LinkScanner, the component they're talking about, works in Firefox as well - so no, using Firefox does not 'keep you safe'.
Nor is this about the users of the thing in the first place - either they like its functionality (security theatre-advance warning blabla) and leave it on, or they don't and they switch it off.
This is about the poor, poor admins who are suddenly seeing bogus traffic and omgosh it's spoofing user agents at that! .. repeatedly*
*changes his user agent to 'cry more, Taco' in FF and hits F5
AVG has become more obnoxious recently than it used to be anyway, but I think this is the straw that broke the camels back for me. Can some nice slashdot user suggest a new (free) antivirus for me to use on my windows box?
Use Linux and your antivirus paranoia will end.
You can actually install AVG 8 without the 'Safe Search' feature that crawls websites (it's essentially a BHO/Firefox extension). Even if you already have AVG 8, you can uninstall it and reinstall:
At a Command Prompt window, type /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch
c:\downloads\avg_free_stf_xxxxxxxxxx.exe
where c:\downloads\avg_free_stf_xxxxxxxxxx.exe is the full path of your AVG 8 installer.
Go somewhere random
Has anyone else noticed that AVG 8 is also DOG SLOW on their PC? My computer is from 2001 and ran fine with 7.5, but 8.0 is unusably slow. Every time an application is opened it takes forever for AVG to scan it and let the app open. This combined with this linkscanner bullcrap has caused me to switch. I doubt I'll ever go back.
hate replying to myself, but didn't notice this before - it works in FF2, not in FF3.
Disabling it FF-side: Tools > Add-ons > AVG Safe Search > Disable /nokarma-anon
I've been using Avast! Home Edition for a while now, no complaints.
The Mozilla part at the beginning is the standard IE user agent. IE has been falsifying their UA as Mozilla since the beginning, originally because Netscape was the top dog, and Microsoft wanted to make sure that it worked with sites that sniffed the UA only worked with Netscape.
I'm a longtime user of AVG. Version 7 was reasonably lightweight, effective and (most importantly to me) unobtrusive.
Unfortunately, version 8 is a different story. After Grisoft forced me to upgrade in May, suddenly AVG became a nagging resource hog. Nightly scan times rocketed from about an hour to over six hours - a scheduled scan that started at 2am would still be going at 8:30am. I have been able to reduce this time somewhat by changing the scan settings (e.g., don't scan inside compressed archives), but it's still slow.
Most annoyingly, their new "LinkScanner" and "SafeSurf" features slowed my browser to a crawl. I didn't want these, since I already use FireFox with the AdBlock and NoScript extensions. I tried to simply disable LinkScanner, but then AVG constantly bothered me with nagging warnings that my computer "was not fully protected". After a little digging, I found that it was possible to uninstall the feature entirely with the following command:
avg_free_stf_xxxx.exe /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch
(Substitute "avg_free_stf_xxxx.exe" in the above command with the name of your setup file.)
This improved my browser performance, and eliminated the warnings.
I'm still (grudgingly) using AVG, but I will switch if/when I find a better alternative.
It's not really the load -- it's throwing off our internal metrics so we don't know what readers are actually interested in. We like numbers, and messing with our stats annoys us.
avast! antivirus Home Edition is FREE to use but it is necessary to register before the end of the initial 60 day trial period. To register, click here. Following registration you will receive by E-mail a license key valid for a period of 1 year. After you have downloaded and installed the program, the license key must be inserted into it within 60 days. The registration process is very easy, and it will take you only a couple of minutes.
Also Avira has been getting more and more annoying over the years, it's practically adware now.
So now it looks like it's either AVG with the browser plugins removed or MoonAV (which is FOSS):
http://www.moonsecure.com/
(It used to have a problem where you'd need to remove the Windows service manually after uninstalling, they might have fixed it though.)
"When information is power, privacy is freedom" - Jah-Wren Ryel
I love AVG for the free scanner it provides but ...
Safesearch: It doesn't work.
Somehow I ended up on one of those "Your computer is infected..." sites
while trying to dl their crap. So for fun I went back to the referrer page
(google) and sure enough, it was marked as safe.
I second Avast, it's free for home use, and has very reasonable commercial license terms. Plus it gives you one code for all machines, no need to chase 20 different keys like you do with Norton etc. And the key is good for the whole license period; before I used to loose at least 10 % of licenses to crashes or borked installs, and getting new ones from Norton was like pulling wisdom teeth on a grouchy alligator.
I'm aging rapidly, I bought a new game and had no idea if my machine was good for it.
Hah! Checking my addons in FF3, and on AVG Safe Search 8 it says "Not compatible with Firefox 3.0". Awesome :-)
I installed AVG on my mother-in-law's machine because she had an expired trial version of some other AV software. It was great for a while, but they must've had a change in direction/managment. Because all of a sudden they started with popups to get a full paid version of the software - even uninstalling the product didn't fix it. I had to surgically extract crap from the registry and program files folder to finally get rid of it. Avast or ClamWin for me - no more AVG.
90% of everything is crap. Also, crap is relative.
They are attempting to help their customers at the expense of everybody else on the Internet. If I understand the article, they're pre-scanning every possible URL on a page. In essense they're clicking every possible link before you do.
For instance I searched for "avg" on google and counted the number of "href=" appearances on the resulting page. It happened to be an even 100. AVG is visiting ALL of of those HREFs in the background. A user will click on only one.
I would assume their scanner is smart enough to remove duplicates HREFs and do some other smart things. But still, this is a terrible idea. I guess we all have to go buy more servers and bandwidth so the anti-virus people can make a living now?
Google, as other search engines, not only obey robots.txt but also quite clearly identify themselves a GoogleBot and connect from an IP address registered to Google.
Another company that's particularly bad is Cyveillance, they also regularly spider sites very aggressively (redownloading the same content repeatedly even tho it hasn't changed), and they try to spoof their user agent.
If you mail them to complain, they will claim to remove your sites from their spider if you give them the IPs, but they lie... They will continue spidering your sites, but from a different IP range which is still traceable to them.
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
Ok. It's run by Jews in a secret conspiracy to take over the World using sharks with frickin' lasers and gorgeous fembots with a penchant for evil.
Score:5, Informative ?
Some readers (and AVG) don't get why this is "slimey". It's worse than slimey -- it's outrageous.
We provide a web service for serious scientists, and each query to our system requires a LOT of computational and database resources. We're not talking about delivering up static results or a simple database query here, we're talking about launching jobs that run for several seconds to several minutes. A given page might have dozens of these links. So a scientist who asks an reasonable question would spend a few seconds of our server's resources. But then AVS comes along, and could launch dozens of searches that might potentially use an HOUR of CPU time.
Most of these links would never be clicked, because they're not what the scientist is interested in. But AVS, being blind and dumb, hits every one of them.
If this goes on unchecked, we're going to have to install some elaborate traps, at great cost to us, to try to detect AVS's scans based on behavioral patters. For example, no scientist would ever click on links in quick succession, because she/he wouldn't have time to read the results. But this will cost us tens of thousands of dollars in programmer resources.
AVS, you suck. Your holier-than-thou attitude is disgusting. What you're really doing is sucking off the resources of other companies in order to improve your own profits. You're throwing the cost of the criminals onto the shoulders of innocents.
While all other /.ers are complaining that ClamWin is useless I want to bring some points :
- ClamWin has a built-in plug-in to scan incoming mail in outlook.
- ClamWin is easy to call from scripts and is a nice thing to add to the commands that are launched by your favourite bit-torrent client once a file is completed (I use this on my linux based torrent downloading/file server machine)
- ClamWin has plug-ins for FireFox : SafeDownload, Download Scan, Download Statusbar all let you launch the scanner of your choosing once a download finishes. ClamWin Antivirus Glue is another solution, but one has to manually update the minimal supported version (the plugin is set to support up to 1.5 although it works with more modern versions).
So, although ClamWin isn't continuously scanning in background, it can cover most of the usual entry points. (Although I don't know about plugins for Thunderbird and Microsoft file server).
For those who like to test newer bleeding edge software : WinPooch software can launch a scan when ever an executable is opened - it's almost as good as an on demand scanner.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
After some checking logs today - the beauty of this mess, is that linkscanner doesn't send accept-encoding and it also seems to 'support' the caching header in a quite hilarious manner.
If your homepage is 100k, browsers will see a page maybe 15k in size, linkscanner sees a page 100k in size.
If you regularly update and set a low/negative expires, then a browser will see the page once (when they visit it), whereas linkscanner seems to re-download the page every time it sees a link to it.... combined with a page that is SEO optimized, and you can see insane bandwidth usage.
*IF* page scanner avoided re-downloading pages with "don't cache" set (since it's bloody pointless), AND supported gzip encoding - then I wouldn't be quite as pissed as I am. Honestly, this is not only a bad idea, it's half-assed coding on top of that.
I have an updated version of this redirect to AVG, based on info I've been gathering over the last 2 weeks from Webmaster World, El Reg, and of course Pixelbeat. Here is the rule set I am using now:
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1; SV1\)$" [OR]
RewriteCond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1;1813\)$"
RewriteCond %{REQUEST_METHOD} ^GET$
RewriteCond %{HTTP_REFERER} ^$
RewriteCond %{HTTP:Accept-Encoding} ^$
RewriteCond %{HTTP:Accept-Language} ^$
RewriteCond %{HTTP:Accept-Charset} ^$
RewriteRule ^.* http://www.avg.com/?LinkScannerSucks [R=301,L]
I have the check for "GET" method in there so that the earlier "User-Agent: ..." version of linkscanner will still get redirected. See, that version does a HEAD request first, most likely to check for a redirect. So we allow that HEAD request to pass, since it is small any ways. But the GET request that follows will still get redirected. We want to redirect the maximum amount of traffic we can to AVG, to drive the point home.
This filter is also more selective, by also checking for the non-existance of Accept-Language and Accpet-Charset we make absolutely sure we are not redirecting a valid user. No web browser out there would fail to set all 3 of these, so we can be absolutely sure this is crap coming from a linkscanner.
I also decided to use a permanent redirect, in hopes that linkscanner caches this and it will reduce the number of repeat hits from the same user? Not sure if that is the case or not.
Someone in this thread asked if these rules work in the main Apache config file instead od using .htaccess. I don't use .htaccess on my servers either, and these rules reside in our main Apache config file. So the answer is yes, it will work in BOTH places.
I hope by now that AVG realizes the futility in their continuing to change how linkscanner acts to try and hide it from us. We will simply continue to work together as a community of server admins to block this crap and send it right back at them!
Probably not a brilliant idea to use a 301 redirect in there.
The original code was using a 307 (Temporary Redirect) so that when people come back having got rid of the AVG software, they'll be able to get the original page.
Otherwise, sounds good!
If you right-click on a component in the AVG User Interface, you can select 'Ignore Component State'. That way the component is turned off, but the AVG icon doesn't show anything wrong.
Hope this helps...