Baidu Forced To Withdraw Last Month's ImageNet Test Results

elwinc writes: Back in mid-May, Baidu, a computer research and services organization in Mainland China, announced impressive results on the ImageNet "Large Scale Visual Recognition Challenge," besting results posted by Google and Microsoft. Turns out, Baidu gamed the system, creating 30 accounts and running far more than the 2 tests per week allowed in the contest. Having been caught cheating, Baidu has been banned for a year from the challenge. I believe all competitors are using variations on the convolutional neural network, AKA deep network. Running the test dozens of times per week might allow a competitor to pre-tune parameters for the particular problem, thus producing results that might not generalize to other problems. All of which makes it quite ironic that a Baidu scientist crowed "Our company is now leading the race in computer intelligence!"

Comments

WHAT!

[war4peace]

Chinese company caught cheating? NO WAY!
Seriously though, raise your hand if you're surprised.

Re:WHAT!

[retchdog]

it's not called cheating there. the ideograms translate roughly to "auspicious cooperation with jade dragon of opportunity."

Re:WHAT!

[NotDrWho]

I think that's from Confucius's 65th Rule of Acquisition.

Re:WHAT!

[jandersen]

Chinese company caught cheating? NO WAY!
Seriously though, raise your hand if you're surprised.

I'm not surprised, but not for the reason I suspect you have in mind: "Because Chinese are just so and so...".

However, it does not really surprise me that we see this from some Chinese companies. China is a developing nation, and they are still relatively new to the way companies play it in the West - not that we are in fact more honest in the West, we have just learned how and when to be dishonest in a way that doesn't make as much noise. I mean, just think of large corporations that avoid paying tax or buy cheaply from sweat-shops employing child-labour. There is no shortage of examples.

But there is another thing in it: lack of regulation. It should not be a surprise to anyone that when there is too little regulation, the most ruthless will feel entitled to bully others - the free, unregulated market can never work to the benefit of everybody, because there will always some, that ruthlessly go for maximising their own short-term advantage, and and that behaviour pushes out the competition and creates monopolies. It is perhaps ironical that this argument is exactly analogous with the argument against Communism: "people are selfish, so if they don't have a reason to work harder, most won't".

We see this in all developing countries, but perhaps most tragilcally in Russia, where they tried to go from tightly regulated Communism to a kind of laissez-faire Capitalism overnight and got horribly burned. And they ended up with the same kind of masters as before, because scum always rises to the top of the pond.

Re:WHAT!

[war4peace]

" I mean, just think of large corporations that avoid paying tax or buy cheaply from sweat-shops employing child-labour."

When all else is equal...
All companies have these kind of skeletons in the closet. Chinese companies simply seem to have some more on top of those which everyone else lovingly owns.
Lack of regulation might be a reason when seeing this internally (within the country) - and then again, when all else is equal... But here we're talking about international events, and that's where you see companies A, B, C playing by the rules and company D (Chinese, more often than not) trying to cheat its way in.
No more than a couple weeks ago there was a story here on /. about Chinese students taking exams instead of the ones who should. After many, many, MANY such stories over the years one can't help but develop a stereotype.
From fake $ITEM to cheating in competitions, China seems on top of the world as count of occurrences.

I googled "chinese cheating": got 22.6M results, top results are about exam cheating.
I also googled "americans cheating", got 14.8M results, top results are about marital cheating.
Of course, this might not mean much, but it's a start. Anyone wants to send a research grant my way? :)

Chinese Hyper-competitiveness

[Anonymous Coward]

This reminds me of some Chinese ball screws (rotary to linear motion components) my company once ordered from Alibaba. The companies had pictures, drawings and put together quotes for parts but then delivered samples that were just totally totally useless. Some of these 'precision' parts looked like they had been made with a file. It just didn't make any sense that they would waste their and our time on such clearly incompetent products.

But when you go there you realise the problem. It is basically an economy in a state of hyper competition. There is so much competition that people will just try anything get ahead, completely oblivious to the wider problem or goal they are trying to solve. You can see that in how the government had to rationalise the solar industry because nobody could make any money. They are just really really crazy competitive.

The trouble though is that there are now many good Chinese engineers who know what they are doing but are still hyper competitive. I really don't know how us westerners with our 40hr work weeks, healthcare and pensions are going to eventually compete with that until we too are faced with the desperation of trying to escape from abject poverty along with 1 billion other people.

Re:Chinese Hyper-competitiveness

[JeffOwl]

You need the buyers to stand up to that kind of crap. These guys screwed (sorry) you once, black list them. I had a frustrated guy in procurement say "You could make tons of money in x business by just doing what you say you are going to do." When it came down to it, he was not always allowed to make the smart decision and go with a reliable company's bid if it wasn't close to the bottom.

Part of the problem we see in China is that they spawn new companies like rabbits. One will eventually go under doing shady stuff only for the same people to come back, selling the exact same crap under a different name. We had this happen and you could look at the product and see that they were using the exact same molds, same defects.

Chinese cheat

[mi]

People growing up under oppressive governments have much fewer problems with cheating — because cheating government is a fair game. It rubs off — and the attitude is quickly extended to non-governmental institutions large and even smaller ones.

This is not "racism" — ex-Soviets like myself often have the same problem... A cheating Western student fears (or used to fear) the shame of being exposed. A Chinese — or a Soviet — fears merely getting caught. Like a speeding ticket — there is no shame in driving fast, only in being stopped by "the bear".

China today uses drones to catch cheaters — America had not felt the need for such measures. Perhaps, it was a foolish attitude, because we the immigrants bring all our traits to the "wonderful tapestry of diversity", not just the good ones...

Anybody dealing with Chinese companies (or Russian ones, if you can find any), ought to be careful and not depend merely on trust.

Re:Chinese cheat

[gstoddart]

You know, when your parent society doesn't value honesty, and everybody around you is cheating ... you're a fool to think there's any value in being that one guy who says "gee, I should be honest here".

In situations in which it's a liability to be honest, only suckers are honest.

And in governments who have spend decades saying "there is no higher power than the state", if the state is rampantly corrupt, "integrity" is a relative term.

Give it a few more years, and you'll discover that integrity in America is a much more malleable concept than you realize -- in fact, it's probably already there.

The mentality of "it's OK as long as I don't get caught" isn't a new thing to humanity.

Or easier still

[Virtucon]

They'll just go in and steal the research from another competitor and call it their own. Cheating and espionage are familiar bedfellows.

We know what Baidu is, thankyouverymuch.

[captaindomon]

Baidu isn't just "a computer research and services organization", they're the Chinese version of Google. They're a massive company with eight billion USD in revenue last year. The headline is either misleading or completely clueless.