Baidu's Supercomputer Beats Google At Image Recognition

catchblue22 writes: Using the ImageNet object classification benchmark, Baidu’s Minwa supercomputer scanned more than 1 million images and taught itself to sort them into about 1,000 categories and achieved an image identification error rate of just 4.58 percent, beating humans, Microsoft and Google. Google's system scored a 95.2% and Microsoft's, a 95.06%, Baidu said. “Our company is now leading the race in computer intelligence,” said Ren Wu, a Baidu scientist working on the project. “I think this is the fastest supercomputer dedicated to deep learning,” he said. “We have great power in our hands—much greater than our competitors.”

Comments

Great power

[Anonymous Coward]

I'm not sure an improvement of .5 percent on image cataloging is really that impressive to get not one but two greats...

Note to Slashdot Editors

[Malenx]

This is actually News for Nerds.

I'm curious how much difference in computational power was thrown at training these by Google, Microsoft, and Baidu, though it's going to be great to watch how these continue to evolve.

Just Guess

[Greyfox]

After all the news stories from the past couple of years, it seems like you could just guess "Yeah, that's a penis" and be correct about half the time. Seems like most people if you give them a camera, they're going to take a picture of a penis with it. And subsequently post that picture to the internet somewhere.

Bad summary

[Anonymous Coward]

The summary is written to imply that Google/MS have error rates in the 90's, while the competition only has about 5% error. The values got inverted - Google/MS also have error rates around 5%, but are behind by fractions.

Your maths is off...

[ZG-Rules]

As a pedant, I need to point out that the improvement is 0.24%

"The system trained on Baidu’s new computer was wrong only 4.58 percent of the time. The previous best was 4.82 percent, reported by Google in March. One month before that, Microsoft had reported achieving 4.94 percent, becoming the first to better average human performance of 5.1 percent."

Also why are the numbers reversed to quote success rates for Google and Microsoft in the summary on Slashdot - it would have been much clearer if the actual numbers in the article (which were all error rates) were quoted!

Re:Your maths is off...

[rudy_wayne]

Also why are the numbers reversed to quote success rates for Google and Microsoft in the summary on Slashdot - it would have been much clearer if the actual numbers in the article (which were all error rates) were quoted!

Because this is Slashdot and it is required that all stories be written as poorly as possible.

Baidu's new computer was wrong only 4.58 percent of the time. The previous best was 4.82 percent, reported by Google in March.

If Google is only wrong 4.82% of the time then why is it whenever I search for an image I get thousands of pictures that have absolutely nothing to do with what I am searching for?

Re:Your maths is off...

[bouldin]

My main question is how damn stupid are the humans they are using to have a >5% error rate looking at pictures.

This was the first thing I thought after reading the summary, too. I had to dig into a paper about the 2014 ImageNet challenge, but here is the likely answer:

The most common error that an untrained annotator is susceptible to is a failure to consider a relevant class as a possible label because they are unaware of its existence.

My second question was, if humans failed to label the images correctly, how did they get a correct label in the first place?

The methodology they used just to label the images is impressively sophisticated. Briefly, they crowdsourced through Amazon Mechanical Turk. A first person would draw bounding boxes around individual items in each image, then additional people would classify the items in each box. Only when a majority of labelers agreed on a label did they consider the label correct.

Your monthly algorithm tweak brought to you by...

[Idarubicin]

Okay, so we have a benchmark where the bog-standard human being scores 94.9%.

Then in February (that's three months ago), Microsoft reports hitting 95.06%; the first score to edge the humans.

Then in March, Google notches 95.18%.

Now it's May, and Baidu puts up a 95.42%.

Meh. Swinging dicks with big iron are twiddling with their algorithms to squeeze out incremental, marginal improvements on an arbitrary task.

“Our company is now leading the race in computer intelligence,” said Ren Wu, a Baidu scientist working on the project. ... “We have great power in our hands—much greater than our competitors.”

I presume that next month it will be IBM boasting about "leading the race" and being "much greater than their competitors". The month after that it will be Microsoft's turn again. Google will be back on top in August or so...unless, of course, some other benchmark starts getting some press.

Huh? This is not a very powerful computer.

[Chalnoth]

The computer has 72 processors and 144 GPU's. That's tiny. Seriously tiny. Sure, GPU's are powerful, especially for image processing. But the larger computers these days are running tens to hundreds of thousands of processors in parallel.

For example, assuming each shelf has 2 processors and 4 GPU's, and they can fit 12 shelves into a single rack, that's a total of 2 racks. Compare that to this image of one of Google's datacenters, where you can see dozens of racks, each containing 14 shelves by my count. And that's just one row. These are gigantic warehouses, with row upon row of racks.

The level of processing power claimed here is closer to the level of a university processing cluster. The larger scientific clusters can be ten or a hundred times larger, and it's not clear just how big private datacenters are.

So overall I'm very, very skeptical. There's a very good chance that they fudged the data somehow to make theirs appear better. But if it is better, well, there's no reason why Google and Microsoft couldn't easily outcompete them in short order.