Google Announces Image Recognition Advance

Rambo Tribble writes Using machine learning techniques, Google claims to have produced software that can better produce natural-language descriptions of images. This has ramifications for uses such as better image search and for better describing the images for the blind. As the Google people put it, "A picture may be worth a thousand words, but sometimes it's the words that are the most useful ..."

Comments

Re: Maybe help the blind?

[stigmato]

You are standing in an open field west of a white house, with a boarded front door. There is a small mailbox here.

Re:what would be useful

[Morose]

Not only is there a program for this, but it's free. It's called VisiPics (http://www.visipics.info/index.php?title=Main_Page). I use it to organize my photo collection. Not only can it do scaling, but it can check other similarity factors as well. Not sure if it scales to millions, but I've used it with 20,000 images before.

Enjoy.

Re:Siri, describe the world around you.

[CreatureComfort]

Your torch goes out.

You are eaten by a Grue.

actually automatic picture caption generator

[slew]

Not as "advanced" in image recognition as advertised.

Basically they took the output of a common object classifier and instead of just picking the most likely object (which is what a typical object classifier looks for), it leaves in in a form where multiple objects are detected in various parts of the scene. Then they train a neural network to create captions (by giving it training pictures with associated captions).

According to the paper, it sometimes apparently generates a reasonable description. Other times it reads in picture of a street sign covered with stickers and emits a caption like "refrigerator filled with lots of food and drink".

Actually the most interesting thing about it is the LSTM-based Sentence Generator that is used to generate the caption from the objects. LSTM's are notoriously hard to train and they apparently they borrow some results from language translation techniques to attempt to form intelligible sentences.

This is all very googly-researchy in that they want to see what the limits of pure data driven machine learning are (w/o human tuning). This is not a however much of an advance in image recognition as it is an advance in the language for caption construction.

Re:But can it generate an image from words ...

[Tablizer]

Yes, it's called "Googling for images". (You didn't say "original".)

Re:what would be useful

[ArcadeMan]

However, because of lossy compression, you might want to keep an image that is slightly lower resolution but still has a better overall image quality.

Re:what would be useful

[mister_playboy]

And runs the comparison over several thousand files (or even hundreds of thousands, or millions)

Ah yes... the joys of Internet Art collecting!

you know, for the cats. on the internet.

[Thud457]

Next they need an AI that can describe the smells in a image.
Then you can finally tell if someone on the internet is a dog.