Slashdot Mirror
Chinese Security Vendor Qihoo 360 Caught Cheating In Anti-virus Tests
Bismillah writes: China's allegedly largest security vendor Qihoo 360 has fessed up to supplying custom versions of its AV for testing according to an investigation by Virus Bulletin, AV-Comparatives and AV-Test. "On requesting an explanation from Qihoo 360 for their actions (PDF), the firm confirmed that some settings had been adjusted for testing, including enabling detection of types of files such as keygens and cracked software, and directing cloud lookups to servers located closer to the test labs. After several requests for specific information on the use of thirdparty engines, it was eventually confirmed that the engine configuration submitted for testing differed from that available by default to users."
Qihoo has been a joke in China for a long time. They finally made their way to the international platform. Good.
A Chinese.
Any sufficiently sophisticated Chinese security security product to be of any use will either be compromised by the Chinese government "in the interest of domestic social harmony" or for national security/military/espionage.
If the test is checking for non-virus files like keygens it sounds like the test is broken. AV software should detect things that are harmful to your computer, not things that software vendors don't like but are otherwise harmless.
I'm not surprised they ship with keygen detection off in China.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
The company submitted 360 Total Security with Bitdefender enabled to the antivirus test firms. It was very highly rated. The 360 TS and TSE base products let you enable Bitdefender and Avira engines, but does not come with them pre-enabled. They also have a version that comes with Bitdefender enabled called 360 TSE Enhanced. This is what was submitted, as I understand this issue. I'm not convinced that there was any "trickery". It more than likely was poor communication between the firms.
Not shocked at all
I am not shocked, but I am confused. Why would they give bad software to their customers, but give good software to the testers? The marginal cost of software is zero. So, if they have good software, why don't they give it to their customers? Can someone please explain how any of this makes sense?
I am not shocked, but I am confused. Why would they give bad software to their customers, but give good software to the testers? The marginal cost of software is zero. So, if they have good software, why don't they give it to their customers? Can someone please explain how any of this makes sense?
It's really easy to "detect" everything so you get a high detection rate. It's really hard to do so without a ton of false positives.
Very few of the tests out there check for false positives, so it is easy to game the results. You could never ship the product to customers that way because you'd drown in support calls from customers complaining about programs not work, broken websites, etc.
Probably because the customers don't want keygens to flag unless there's an actual Trojan?
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
Very few of the tests out there check for false positives, so it is easy to game the results.
I see. In that case, shouldn't the story be "AV Tests are Stupid" rather than "Chinese Company Sort of Cheats on a Test Designed to Make Cheating Easy"?
All the major testing houses check for false positives alongside detections, but perhaps they decided more false positives would still look better on benchmarks than a lower detection rate.
How can I believe you when you tell me what I don't want to hear?
Very few of the tests out there check for false positives, so it is easy to game the results.
I see. In that case, shouldn't the story be "AV Tests are Stupid" rather than "Chinese Company Sort of Cheats on a Test Designed to Make Cheating Easy"?
No, the testing organizations here are competent. It is the "let's have the intern do an antivirus review" articles in publications having no particular reputation in security matters that should be treated with suspicion.
The major American AV vendors announced a joint task force today to respond to these results.
When asked how they would ensure that corporate members of the task force would be held accountable for this sort of cheating, their spokesperson responded with the following:
"Accountable for cheating? No, no, no, the point of the task force is to keep from getting caught like this."
Never underestimate the power of stupid people in large groups.
I am not shocked, but I am confused. Why would they give bad software to their customers, but give good software to the testers? The marginal cost of software is zero.
The good software is not theirs, it is Bitdefender's, and it does not have a zero marginal cost unless they steal it. That would not be unknown, of course, but this company may be too large, and have big enough aspirations, for that not to be an option.
I also tend to agree with those who suspect they are selling to customers who don't like to be reminded that using keygens is risky.
All the major testing houses check for false positives alongside detections, but perhaps they decided more false positives would still look better on benchmarks than a lower detection rate.
It's not that they don't claim to test for false positives... It's that their FP testing tends to be... rudimentary.
To be fair, I haven't worked with these specific test houses. I have, however, worked closely with some very well-known and trusted test labs. Perception and reality don't line up very well
Almost all the anti-virus companies get caught cheating on these tests, so yes, they are basically worthless.
Probably because the customers don't want keygens to flag unless there's an actual Trojan?
For me this is true of all security software. Why do they flag keygens if there isn't an actual Trojan? It's supposed to be security software, not anti-piracy software.
And it is probably why pirated software are the main attack vector. Can't be scanned
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
if they ask the vendor for a version to test (and money? then the test is suspect.
world was created 5 seconds before this post as it is.