Slashdot Mirror

← Back to Stories (view on slashdot.org)

Baidu Forced To Withdraw Last Month's ImageNet Test Results

Posted by timothy on from the please-reconsider dept.

elwinc writes: Back in mid-May, Baidu, a computer research and services organization in Mainland China, announced impressive results on the ImageNet "Large Scale Visual Recognition Challenge," besting results posted by Google and Microsoft. Turns out, Baidu gamed the system, creating 30 accounts and running far more than the 2 tests per week allowed in the contest. Having been caught cheating, Baidu has been banned for a year from the challenge. I believe all competitors are using variations on the convolutional neural network, AKA deep network. Running the test dozens of times per week might allow a competitor to pre-tune parameters for the particular problem, thus producing results that might not generalize to other problems. All of which makes it quite ironic that a Baidu scientist crowed "Our company is now leading the race in computer intelligence!"

5 of 94 comments (clear)

  1. Chinese Hyper-competitiveness by Anonymous Coward · · Score: 4, Interesting

    This reminds me of some Chinese ball screws (rotary to linear motion components) my company once ordered from Alibaba. The companies had pictures, drawings and put together quotes for parts but then delivered samples that were just totally totally useless. Some of these 'precision' parts looked like they had been made with a file. It just didn't make any sense that they would waste their and our time on such clearly incompetent products.

    But when you go there you realise the problem. It is basically an economy in a state of hyper competition. There is so much competition that people will just try anything get ahead, completely oblivious to the wider problem or goal they are trying to solve. You can see that in how the government had to rationalise the solar industry because nobody could make any money. They are just really really crazy competitive.

    The trouble though is that there are now many good Chinese engineers who know what they are doing but are still hyper competitive. I really don't know how us westerners with our 40hr work weeks, healthcare and pensions are going to eventually compete with that until we too are faced with the desperation of trying to escape from abject poverty along with 1 billion other people.

    1. Re:Chinese Hyper-competitiveness by JeffOwl · · Score: 4, Interesting

      You need the buyers to stand up to that kind of crap. These guys screwed (sorry) you once, black list them. I had a frustrated guy in procurement say "You could make tons of money in x business by just doing what you say you are going to do." When it came down to it, he was not always allowed to make the smart decision and go with a reliable company's bid if it wasn't close to the bottom.

      Part of the problem we see in China is that they spawn new companies like rabbits. One will eventually go under doing shady stuff only for the same people to come back, selling the exact same crap under a different name. We had this happen and you could look at the product and see that they were using the exact same molds, same defects.

  2. Chinese cheat by mi · · Score: 5, Interesting

    People growing up under oppressive governments have much fewer problems with cheating — because cheating government is a fair game. It rubs off — and the attitude is quickly extended to non-governmental institutions large and even smaller ones.

    This is not "racism" — ex-Soviets like myself often have the same problem... A cheating Western student fears (or used to fear) the shame of being exposed. A Chinese — or a Soviet — fears merely getting caught. Like a speeding ticket — there is no shame in driving fast, only in being stopped by "the bear".

    China today uses drones to catch cheaters — America had not felt the need for such measures. Perhaps, it was a foolish attitude, because we the immigrants bring all our traits to the "wonderful tapestry of diversity", not just the good ones...

    Anybody dealing with Chinese companies (or Russian ones, if you can find any), ought to be careful and not depend merely on trust.

    --
    In Soviet Washington the swamp drains you.
    1. Re:Chinese cheat by ShanghaiBill · · Score: 2, Interesting

      The rules of life are not "whoever dies with the most toys wins"

      But the rules are "whoever doesn't eat loses". Historically, this was not about accumulating toys, but basic survival. During the Great Leap Forward, 30 million Chinese starved to death. There were another few million deaths from starvation and violence during the Cultural Revolution. Both of those were caused by government abuse and incompetence. Even today, China has a hereditary class system, that denies hundreds of millions of people access to housing, healthcare, and even the right to send their children to public schools. When the system treats you like that, you feel no obligation to be honest in return.

      Striving to live one's life with integrity may not be easy, but then nothing that is really worth doing ever is.

      That is a very western attitude. You should not project your another cultural values onto a culture that does not share them. To the Chinese, it is not 'integrity" that is honorable, but loyalty to your family. So if you cheat to help your family, or abuse your authority to give all the plum government jobs to your nephews, that is not only tolerated, but admired.

  3. Re:WHAT! by ShanghaiBill · · Score: 3, Interesting

    it's not called cheating there.

    Except it wasn't "there". It was "here". Baidu's main research lab, where this AI research was done, is in Sunnyvale, California, and many employees there are American citizens.