Slashdot Mirror
Chinese Security Vendor Qihoo 360 Caught Cheating In Anti-virus Tests
Bismillah writes: China's allegedly largest security vendor Qihoo 360 has fessed up to supplying custom versions of its AV for testing according to an investigation by Virus Bulletin, AV-Comparatives and AV-Test. "On requesting an explanation from Qihoo 360 for their actions (PDF), the firm confirmed that some settings had been adjusted for testing, including enabling detection of types of files such as keygens and cracked software, and directing cloud lookups to servers located closer to the test labs. After several requests for specific information on the use of thirdparty engines, it was eventually confirmed that the engine configuration submitted for testing differed from that available by default to users."
The company submitted 360 Total Security with Bitdefender enabled to the antivirus test firms. It was very highly rated. The 360 TS and TSE base products let you enable Bitdefender and Avira engines, but does not come with them pre-enabled. They also have a version that comes with Bitdefender enabled called 360 TSE Enhanced. This is what was submitted, as I understand this issue. I'm not convinced that there was any "trickery". It more than likely was poor communication between the firms.
I am not shocked, but I am confused. Why would they give bad software to their customers, but give good software to the testers? The marginal cost of software is zero. So, if they have good software, why don't they give it to their customers? Can someone please explain how any of this makes sense?
It's really easy to "detect" everything so you get a high detection rate. It's really hard to do so without a ton of false positives.
Very few of the tests out there check for false positives, so it is easy to game the results. You could never ship the product to customers that way because you'd drown in support calls from customers complaining about programs not work, broken websites, etc.