Baidu Forced To Withdraw Last Month's ImageNet Test Results

elwinc writes: Back in mid-May, Baidu, a computer research and services organization in Mainland China, announced impressive results on the ImageNet "Large Scale Visual Recognition Challenge," besting results posted by Google and Microsoft. Turns out, Baidu gamed the system, creating 30 accounts and running far more than the 2 tests per week allowed in the contest. Having been caught cheating, Baidu has been banned for a year from the challenge. I believe all competitors are using variations on the convolutional neural network, AKA deep network. Running the test dozens of times per week might allow a competitor to pre-tune parameters for the particular problem, thus producing results that might not generalize to other problems. All of which makes it quite ironic that a Baidu scientist crowed "Our company is now leading the race in computer intelligence!"

Comments

WHAT!

[war4peace]

Chinese company caught cheating? NO WAY!
Seriously though, raise your hand if you're surprised.

Re:WHAT!

[jandersen]

Chinese company caught cheating? NO WAY!
Seriously though, raise your hand if you're surprised.

I'm not surprised, but not for the reason I suspect you have in mind: "Because Chinese are just so and so...".

However, it does not really surprise me that we see this from some Chinese companies. China is a developing nation, and they are still relatively new to the way companies play it in the West - not that we are in fact more honest in the West, we have just learned how and when to be dishonest in a way that doesn't make as much noise. I mean, just think of large corporations that avoid paying tax or buy cheaply from sweat-shops employing child-labour. There is no shortage of examples.

But there is another thing in it: lack of regulation. It should not be a surprise to anyone that when there is too little regulation, the most ruthless will feel entitled to bully others - the free, unregulated market can never work to the benefit of everybody, because there will always some, that ruthlessly go for maximising their own short-term advantage, and and that behaviour pushes out the competition and creates monopolies. It is perhaps ironical that this argument is exactly analogous with the argument against Communism: "people are selfish, so if they don't have a reason to work harder, most won't".

We see this in all developing countries, but perhaps most tragilcally in Russia, where they tried to go from tightly regulated Communism to a kind of laissez-faire Capitalism overnight and got horribly burned. And they ended up with the same kind of masters as before, because scum always rises to the top of the pond.

Re:WHAT!

[war4peace]

" I mean, just think of large corporations that avoid paying tax or buy cheaply from sweat-shops employing child-labour."

When all else is equal...
All companies have these kind of skeletons in the closet. Chinese companies simply seem to have some more on top of those which everyone else lovingly owns.
Lack of regulation might be a reason when seeing this internally (within the country) - and then again, when all else is equal... But here we're talking about international events, and that's where you see companies A, B, C playing by the rules and company D (Chinese, more often than not) trying to cheat its way in.
No more than a couple weeks ago there was a story here on /. about Chinese students taking exams instead of the ones who should. After many, many, MANY such stories over the years one can't help but develop a stereotype.
From fake $ITEM to cheating in competitions, China seems on top of the world as count of occurrences.

I googled "chinese cheating": got 22.6M results, top results are about exam cheating.
I also googled "americans cheating", got 14.8M results, top results are about marital cheating.
Of course, this might not mean much, but it's a start. Anyone wants to send a research grant my way? :)

Or easier still

[Virtucon]

They'll just go in and steal the research from another competitor and call it their own. Cheating and espionage are familiar bedfellows.

Re:Chinese cheat

[gstoddart]

You know, when your parent society doesn't value honesty, and everybody around you is cheating ... you're a fool to think there's any value in being that one guy who says "gee, I should be honest here".

In situations in which it's a liability to be honest, only suckers are honest.

And in governments who have spend decades saying "there is no higher power than the state", if the state is rampantly corrupt, "integrity" is a relative term.

Give it a few more years, and you'll discover that integrity in America is a much more malleable concept than you realize -- in fact, it's probably already there.

The mentality of "it's OK as long as I don't get caught" isn't a new thing to humanity.