Slashdot Mirror

← Back to Stories (view on slashdot.org)

Chinese Security Vendor Qihoo 360 Caught Cheating In Anti-virus Tests

Posted by Soulskill on from the hand-in-the-virus-jar dept.

Bismillah writes: China's allegedly largest security vendor Qihoo 360 has fessed up to supplying custom versions of its AV for testing according to an investigation by Virus Bulletin, AV-Comparatives and AV-Test. "On requesting an explanation from Qihoo 360 for their actions (PDF), the firm confirmed that some settings had been adjusted for testing, including enabling detection of types of files such as keygens and cracked software, and directing cloud lookups to servers located closer to the test labs. After several requests for specific information on the use of thirdparty engines, it was eventually confirmed that the engine configuration submitted for testing differed from that available by default to users."

63 comments

  1. Is this shocking? by Anonymous Coward · · Score: 0

    Not shocked at all

    1. Re:Is this shocking? by ShanghaiBill · · Score: 1

      Not shocked at all

      I am not shocked, but I am confused. Why would they give bad software to their customers, but give good software to the testers? The marginal cost of software is zero. So, if they have good software, why don't they give it to their customers? Can someone please explain how any of this makes sense?

    2. Re:Is this shocking? by tippen · · Score: 3, Informative

      I am not shocked, but I am confused. Why would they give bad software to their customers, but give good software to the testers? The marginal cost of software is zero. So, if they have good software, why don't they give it to their customers? Can someone please explain how any of this makes sense?

      It's really easy to "detect" everything so you get a high detection rate. It's really hard to do so without a ton of false positives.

      Very few of the tests out there check for false positives, so it is easy to game the results. You could never ship the product to customers that way because you'd drown in support calls from customers complaining about programs not work, broken websites, etc.

    3. Re: Is this shocking? by AvitarX · · Score: 1

      Probably because the customers don't want keygens to flag unless there's an actual Trojan?

      --
      Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
    4. Re:Is this shocking? by ShanghaiBill · · Score: 1

      Very few of the tests out there check for false positives, so it is easy to game the results.

      I see. In that case, shouldn't the story be "AV Tests are Stupid" rather than "Chinese Company Sort of Cheats on a Test Designed to Make Cheating Easy"?

    5. Re:Is this shocking? by pushing-robot · · Score: 1

      All the major testing houses check for false positives alongside detections, but perhaps they decided more false positives would still look better on benchmarks than a lower detection rate.

      --
      How can I believe you when you tell me what I don't want to hear?
    6. Re:Is this shocking? by Capt.Albatross · · Score: 1

      Very few of the tests out there check for false positives, so it is easy to game the results.

      I see. In that case, shouldn't the story be "AV Tests are Stupid" rather than "Chinese Company Sort of Cheats on a Test Designed to Make Cheating Easy"?

      No, the testing organizations here are competent. It is the "let's have the intern do an antivirus review" articles in publications having no particular reputation in security matters that should be treated with suspicion.

    7. Re:Is this shocking? by Capt.Albatross · · Score: 1

      I am not shocked, but I am confused. Why would they give bad software to their customers, but give good software to the testers? The marginal cost of software is zero.

      The good software is not theirs, it is Bitdefender's, and it does not have a zero marginal cost unless they steal it. That would not be unknown, of course, but this company may be too large, and have big enough aspirations, for that not to be an option.

      I also tend to agree with those who suspect they are selling to customers who don't like to be reminded that using keygens is risky.

    8. Re:Is this shocking? by tippen · · Score: 1

      All the major testing houses check for false positives alongside detections, but perhaps they decided more false positives would still look better on benchmarks than a lower detection rate.

      It's not that they don't claim to test for false positives... It's that their FP testing tends to be... rudimentary.

      To be fair, I haven't worked with these specific test houses. I have, however, worked closely with some very well-known and trusted test labs. Perception and reality don't line up very well

    9. Re:Is this shocking? by TheCastro1689 · · Score: 1

      Almost all the anti-virus companies get caught cheating on these tests, so yes, they are basically worthless.

    10. Re: Is this shocking? by Anonymous Coward · · Score: 0

      This. If an AV test has a scoring and ranking system, and publishes these results -- and it includes keygens amongst the test "viruses", AV vendors should boycott it. I can well imagine that being the case, and if one of the "cheats" here was to flag keygens, then it was probably done for a reason.

    11. Re: Is this shocking? by LinuxIsGarbage · · Score: 1

      Probably because the customers don't want keygens to flag unless there's an actual Trojan?

      For me this is true of all security software. Why do they flag keygens if there isn't an actual Trojan? It's supposed to be security software, not anti-piracy software.

    12. Re: Is this shocking? by AvitarX · · Score: 1

      And it is probably why pirated software are the main attack vector. Can't be scanned

      --
      Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
    13. Re:Is this shocking? by gl4ss · · Score: 1

      if they ask the vendor for a version to test (and money? then the test is suspect.

      --
      world was created 5 seconds before this post as it is.
  2. Finally by Anonymous Coward · · Score: 2, Interesting

    Qihoo has been a joke in China for a long time. They finally made their way to the international platform. Good.

    A Chinese.

    1. Re:Finally by Anonymous Coward · · Score: 5, Interesting

      Chinese here too.

      360 is no "joke" in all seriousness. They are bullies, really badass bullies.

      They "kidnapped" hundreds of thousands of terminals (PC/Phone/browser) by disguising themselves as a "security guard", telling users what is bad and what is good, and then blackmail developers and websites to bribe them to get into their "good" list.

      My company has a website that only shows text and picture news and contact info and stuff. One day 360 decided to reported our website as "security threat" and show warning on ALL 360 browsers (which is A LOT).

      We contacted them, they told us to put "a security script" into our server. Once they confirmed the script is in place, they re-score our website to 100-OK, without asking us to modify/patch anything.

      What that script does (thankfully it's PHP so it's naturally "open source") is scanning our whole www directory, upload whatever info they want, and even modify our code whenever they like.

      Oh, and they also labeled my company's phone number as scam in their "smartphone guard", even though we've been using it for years.

    2. Re:Finally by Anonymous Coward · · Score: 0

      AC, A Chinese, surely.

    3. Re:Finally by The+MAZZTer · · Score: 2

      You should put the PHP script on a copy of your website that you only serve to 360. It would seem to be a tactic they approve of.

    4. Re:Finally by Anonymous Coward · · Score: 1

      Yes, that's exactly what I did.

    5. Re:Finally by ITRambo · · Score: 0

      How do we know you're not part of a competing firm, like Symantec? Posting as AC doesn't give us any reason to believe that your statements are true and plenty to doubt their veracity. I use 360 TS. If the company genuinely sucks, I'd like evidence of it, not anecdotes. Please post some links that can verify your comments. Have there been any incidences of Qihoo bullying firms outside of China? Are they trying to be a 1990's Microsoft-style company?

    6. Re: Finally by Anonymous Coward · · Score: 0

      Evidence? Are you new to the internet?

    7. Re:Finally by Anonymous Coward · · Score: 0

      Qihoo 360 is shit malware. Dump it! Why is this being discussed as legitimate software?

    8. Re:Finally by LordLimecat · · Score: 2

      How about the fact that if you think the NSA does some crazy malware stuff with Flame and Stuxnet, at least they tend to confine it to foreign political targets. China has probably the largest censorship and MITM infrastructure in the world, and actively uses it to pull average citizens into a government run botnet to DDOS western sites.

      Not to mention that any sufficiently large business needs to have the explicit blessing of the powers that be in China.

      All of that combined means you would have to be crazy to trust Qihoo; the FSB-affiliated Kaspersky is more trustworthy. Installing Qihoo gives one of the most technically competent, politically repressive organizations in the world root access to your computer. That more than anything is sufficient reason to not use them.

      Call me when Symantec has close ties to a government that denies the Tianenmen Square massacre and actively represses search results on it.

    9. Re:Finally by AK+Marc · · Score: 1

      So it's the same as McAfee and Norton?

      --
      Learn to love Alaska
    10. Re: Finally by Anonymous Coward · · Score: 0

      McAfee and Norton are well known western brands. How can you compare them with Chinese communist software? Ok, both the software sucks, take up an insane percentage of your cpu power till you cannot do anything useful, behaves like malware, fails to alerts you to any virus at times, cause you lots of problems, but look, they are well known western software so they mustbe better.

    11. Re: Finally by AK+Marc · · Score: 1

      Just be more clear. One's made by white people. That one's ok. The other's made by yellow people. That's not ok. Got it. No need to pretend anything else.

      --
      Learn to love Alaska
  3. Isn't "Chinese Security Vendor" an oxymoron? by swb · · Score: 2

    Any sufficiently sophisticated Chinese security security product to be of any use will either be compromised by the Chinese government "in the interest of domestic social harmony" or for national security/military/espionage.

    1. Re:Isn't "Chinese Security Vendor" an oxymoron? by Anonymous Coward · · Score: 0

      Like any sufficiently sophisticated American product will be compromised by the American government "in the interest of national security".

    2. Re: Isn't "Chinese Security Vendor" an oxymoron? by Anonymous Coward · · Score: 1

      The difference being here in America we can take our government to task for such infractions, and we do, and even our "tech giants" fight back against government meddling. Go try that in China. Drop me a line and I'll be happy to help remove the boot from your ass after you find out how far it gets ya.

    3. Re: Isn't "Chinese Security Vendor" an oxymoron? by Anonymous Coward · · Score: 0

      The difference being here in America we can take our government to task for such infractions, and we do, and even our "tech giants" fight back against government meddling. Go try that in China. Drop me a line and I'll be happy to help remove the boot from your ass after you find out how far it gets ya.

      ROFLMAO Oh you typed this response with a straight-face and were serious. Oh my goodness!

    4. Re: Isn't "Chinese Security Vendor" an oxymoron? by Anonymous Coward · · Score: 0

      'Merica! Fuck Yeah!

    5. Re: Isn't "Chinese Security Vendor" an oxymoron? by sonicmerlin · · Score: 1

      Ah yes we "take them to task" with stern words and a disapproving glance.

    6. Re:Isn't "Chinese Security Vendor" an oxymoron? by BVis · · Score: 1

      There's no rule of law in the USA either, if you have enough money and your skin is the right color.

      --
      Never underestimate the power of stupid people in large groups.
    7. Re:Isn't "Chinese Security Vendor" an oxymoron? by AmiMoJo · · Score: 1

      You are just projecting US thinking onto the Chinese government. They have little interest in turning AV software into a trojan, because they don't want or need to spy on their citizens that way. They have more direct means, and prefer censorship over mass spying because it's cheaper and easier.

      Unlike the US, China does have an interest in keeping its citizens safe so doesn't break their security software.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    8. Re:Isn't "Chinese Security Vendor" an oxymoron? by LordLimecat · · Score: 1

      I dont think you really have any idea in how the MSS is different than the NSA.

      Lets start with the fact that the MSS gives no craps, they straight up block sites like Google who dont play the censorship game, and they inject malicious javascript into millions of citizens sessions to enact a government-run DDOS of foreign sites.

      The things the NSA does that are violations of our principles are extra-ordinary. The things that the MSS does on that scale are ordinary, expected, and well documented.

    9. Re: Isn't "Chinese Security Vendor" an oxymoron? by LordLimecat · · Score: 1

      I suppose you're not familiar with the genesis of the phrase "illegal flower ceremony" or the history of internet censorship in China.

    10. Re: Isn't "Chinese Security Vendor" an oxymoron? by Anonymous Coward · · Score: 0

      HAHA.

      You're really naive if you think that's the case.

      A company doesn't do what the US wants it to.... will have its operations made a living hell by a thousand legal ways through the government.

      Imports and exports stopped and inspected at great cost. Federal permits reinspected. Constant tax audits. Passport issues for its officers. Raids backed by warrants. Impounds. Etcetera.

      This isn't just speculation. I seen this shit firsthand. Please try your jingoism else where. Sing about the land of the free in an Indian Reservation or something. It would be a bit more believable.

  4. Useless, lazy testers by Anonymous Coward · · Score: 0

    If your software turns off a bunch of stuff by default and your competitors doesn't, then the test will of course be better for your opponent. Unless they are going to have two sets of tests, one with default settings and one with both configured for the test, then the test is unfair.

    I once had to do due diligence for security gateway products that my company was looking to acquire. We didn't buy the company that had the best product using the default settings. We bought the company that had the best product (highest detection, fewest false positives).

  5. Broken test? by AmiMoJo · · Score: 3, Insightful

    If the test is checking for non-virus files like keygens it sounds like the test is broken. AV software should detect things that are harmful to your computer, not things that software vendors don't like but are otherwise harmless.

    I'm not surprised they ship with keygen detection off in China.

    --
    const int one = 65536; (Silvermoon, Texture.cs)
    SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    1. Re:Broken test? by Anonymous Coward · · Score: 0

      AV software tends to detect keygens because historically these things have been a package of three things: 1) a list of keys 2) a virus, worm or some other malware and 3) a flashy UI that installs the virus and selects a random key for display.

    2. Re:Broken test? by Anonymous Coward · · Score: 0

      Keygens and cracks are popular virus-aquisition vectors. Wanna know the best way not to get malware? Stop pirating shit.

    3. Re:Broken test? by Anonymous Coward · · Score: 1

      What to know the real best way not to get malware? Stop buying software.

      Seriously, pirated software has been proven to have a lower infection rate than commercial software.

    4. Re:Broken test? by AmiMoJo · · Score: 2

      Sure, sometimes keygens are trojans as well, but those are covered under the heading "virus". Most anti-virus software also detects perfectly harmless keygens these days, supposedly to "protect" the user from "accidentally" generating a key and pirating software.

      I use some keygens for old software that can't be bought any more. It would be lost to the world without those keygens. I even had keys for some of it, e.g. a Windows 98 serial that was stuck (with a non-removable sticker) to the side of an ancient PC case long ago sent to the dump, and which I now want to install in a VM to play some old games that don't work on Windows 7.

      I don't want my AV software deleting those perfectly safe files, thanks. I'm already paranoid enough to run them in a disposable VM anyway.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    5. Re:Broken test? by Anonymous Coward · · Score: 0

      Seriously, pirated software has been proven to have a lower infection rate than commercial software.

      Care to backup your BS assertion with a citation?

    6. Re:Broken test? by tlhIngan · · Score: 1

      Sure, sometimes keygens are trojans as well, but those are covered under the heading "virus". Most anti-virus software also detects perfectly harmless keygens these days, supposedly to "protect" the user from "accidentally" generating a key and pirating software.

      Actually, most keygens people run into are infested with malware - Trojans and viruses and all that. Usually they're wrapped with a "dropper" application - run the keygen, and the dropper downloads the malware then launches the keygen.

      The reason for this is infecting installations is a bit more difficult these days - since a lot of software is already downloadable the companies behind them sign the executable. So when you launch the installer, Windows pops up the nice message about the file and it's all signed and everything. Of course, since keygens are rarely signed, if they've been altered it's impossible for the user to tell.

      The money involved in the malware trade is sufficient enough that they basically crowd out the sites that actually offer clean keygens.

      Cracks, too. At least with keygens you can reasonably run them in a VM to get a serial number without infecting your PC, but cracks have to be run on the live installation, making them an ideal target for malware authors.

      Stuff like drive-by-downloads generally aren't used much - between enhanced browser security, elimination of Java or Flash plugins, it's a lot harder to spread malware. But a good keygen or crack for a popular application and you can easily spread CryptoWall around and get $500 from a lot of users.

    7. Re:Broken test? by Anonymous Coward · · Score: 0

      You are naive. Anything is possible when money or favour changes hands. About a decade ago, a Microsoft dev wrote a little tool called MSPrivateFolder. It allowed you to put a password on a folder. He released it into the wild without going through proper MS channels. MS Enterprise customers had a fit that employees could protect folders and files that were secure from the company's eyes. About a month or two later, Norton AV helpfully flagged the file as a virus and deleted it. BAM! It wasn't a virus, nor was it malware. It did one specific job, but that job was causing problems for MS customers, so Symantec helpfully "fixed" the problem.

    8. Re:Broken test? by Anonymous Coward · · Score: 0

      It is a BS assertion, however there has been a recent trend that commercial software contains unwanted components that cause problems for end users and that pirates have been stripping those bits of when they could.
      I'm not sure if this is going to be the future though. It's still true that running keygens is a sure-fire way to get a virus and I expect that going forward, malicious components will be ever more tightly integrated with the legitimate part of the software. At some point they will become impossible to remove.
      I think the most future-proof advice would be to use free software for things that need your data like office productivity software, and to use a separate computer or game console for things that don't like games. Which is painful to admit as a long-standing PC gamer and a fan of the PC as a gaming platform precisely because it allows you to have one box that does it all with all the advantages of that.

  6. Not really an issue by ITRambo · · Score: 3, Informative

    The company submitted 360 Total Security with Bitdefender enabled to the antivirus test firms. It was very highly rated. The 360 TS and TSE base products let you enable Bitdefender and Avira engines, but does not come with them pre-enabled. They also have a version that comes with Bitdefender enabled called 360 TSE Enhanced. This is what was submitted, as I understand this issue. I'm not convinced that there was any "trickery". It more than likely was poor communication between the firms.

    1. Re:Not really an issue by Lodlaiden · · Score: 1

      I was trying to understand the problem. Unless it's an up-sell product, which seems to be what you indicated, I would expect those items to be turned on by default.

      --
      Suborbital [spaceflight] is the special olympics of spaceflight. - Rei
    2. Re:Not really an issue by tnk1 · · Score: 2

      Right. There's no issue with them putting their best foot forward if this is something you can get with the basic product.

      However, if you have to enable these features AND you have to pay for them, that's a different product. The danger is that the reviewers rate their "basic" product as a top-rated AV product. Then people flock to get this basic product over the basic offerings of other AV companies who did not rate as well, but might well have a better "basic" product.

      It's basically bait and switch, and probably fraudulent. It seems like every crime in China can get you executed or sent to a camp, so the fact that so many Chinese companies work this way makes me think that China itself has a very different view of what makes up a fraudulent practice.

    3. Re:Not really an issue by Anonymous Coward · · Score: 0

      Why do testees and testers need to communicate in the first place? If testers can't afford to anonymously pay for the retail version of the software they're testing, then their review is not worth my attention.

    4. Re:Not really an issue by Anonymous Coward · · Score: 1

      Which testing organisation are you regularly paying to write unbiased reviews?

      Thought so.

    5. Re:Not really an issue by ITRambo · · Score: 1

      There is no upsell, that I can see, when there is no charge for either product. How can there be a bait and switch when it's free? "Here's a nice AV. But wait, here's a better one. Gonna cost ya... nothing."

    6. Re:Not really an issue by Anonymous Coward · · Score: 0

      I pay for testing when I pay for a magazine that contains a software review. I'm glad to hear that you thought so.

    7. Re:Not really an issue by Anonymous Coward · · Score: 0

      Consumer Reports perhaps you've heard of them?

    8. Re:Not really an issue by Em+Adespoton · · Score: 1

      Which raises the question: Why do they have two products that are free? One that they market, and one that they test, and pawn off as the marketed item?

      The problem here is that they were submitting one product for testing, and using the certification gained by that testing to represent another product.

      My guess is that this was done so that the product they distribute in China is 100% Chinese, but they get the one that's essentially BitDefender certified to raise acceptance.

    9. Re:Not really an issue by AK+Marc · · Score: 1

      Or the have two products that are free because one is more "secure" with more false positives, and the other is more "permissive" because some people only want "hits" when it's a real virus, not the more generic hits when it detects a nonvirus, like a kegen. As for the mixup for which was provided, did the reviewer use a native Chinese speaker to discuss the versions and which is delivered? It may have been a simple miscommunication on the default config, not malicious.

      --
      Learn to love Alaska
  7. Corparate security hijinx by Guy+From+V · · Score: 0

    I keep imagining some scenario like in Blade Runner and the crazy Chinese eye-doctor or something.

  8. In other news.. by BVis · · Score: 1

    The major American AV vendors announced a joint task force today to respond to these results.

    When asked how they would ensure that corporate members of the task force would be held accountable for this sort of cheating, their spokesperson responded with the following:

    "Accountable for cheating? No, no, no, the point of the task force is to keep from getting caught like this."

    --
    Never underestimate the power of stupid people in large groups.
  9. Qi-hoooooo oooo ooo by Anonymous Coward · · Score: 0

    (sung to Yahoo theme)

  10. It's about spying on the users by Anonymous Coward · · Score: 0

    >Can someone please explain how any of this makes sense?

    It sounds like it is a steath product designed to dig into activities of the customers or some sub-group. Perhaps some users, perhaps via altered router DNS, are directed to product updates containing alternate version. It's likely one piece of a larger scheme.

  11. In fairness? They "did me right"... apk by Anonymous Coward · · Score: 0

    They took out a 'false positive' of a program I did, & quickly (email direct excerpt):

    From: Alexander Kowalski [apk4776239@hotmail.com]
    Sent: Wednesday, September 10, 2014 0:48
    To: support
    Subject: 1 False positive of 3 now gone (Qihoo360) only Comodo & NOD32 remain now (interesting points on last one)... apkâ

    Thank you.

      My program for populating custom hosts files does MORE than any single solution for added speed, security, reliability, & even anonymity than any other solution out there, for the GOOD of end users!

      (Whom the best in the business currently in MalwareBytes, per this test http://www.av-test.org/en/news... verified it as safe code + doing its job better than any other program of its kind & recommend it as "best of breed" here on their hpHosts MalwareBytes hosts data page at the top of it -> http://hosts-file.net/?s=Downl...

      APK

      P.S.=> Let me know when the false positive is removed please (not sure if you meant they already HAVE been is all - so asking for confirmation)... apk

    ---

    From: support@360safe.com
      To: apk4776239@hotmail.com
      Subject: RE: False positive: Why are you calling a program of mine a "bad" file? apkâ
    Date: Tue, 9 Sep 2014 12:30:23 +0000

    Dear Alexander,

    Thank you for your support to 360safe.

    We sincerely appreciate your help of improving our products and services. We make every effort to avoid false-positive results in our service, and proper actions have been taken. We are sorry for the inconvenience.

    So again, thank you and please feel free to contact us anytime you have any question or suggestion about our product.

    Kind regards,

    Jay

    Qihoo 360 Support Team

    Email: support@360safe.com