Searching with Images instead of Words

johnsee writes "A computer vision researcher by the name of Hartmut Neven is developing ingenious new technology that allows the searching of a database by submitting an image, for example, off a mobile phone camera. Imagine taking a photo of a street corner to find out where you are, or the photo of a city building to see its history"

Comments

Is A Picture Worth A Thousand Words?

[fembots]

Tell me, which is easier? Upload this image and try to find out where you are via this Visual Google, or enter the street name (street sign in the photo says "Queen Street") in Text Google?

The article also mentioned this thing should start small, like a movie guide, so is it easier to upload a 2K "I,Robot" billboard photo, or just enter "I,Robot" in Google on your cell phone?

As long as human input is still required (i.e. you need to submit something), I don't think this is going to be popular. However, if you have a Oakley that automatically takes photos of what you see and feeds you the location details, that'll be something.

Re:Is A Picture Worth A Thousand Words?

[hyu]

Actually, what I look forward to is searching with a picture of Waldo. Maybe I can finally find him.

Re:Is A Picture Worth A Thousand Words?

[jabex]

It does seem to have some functionality though. Let's say for example, this holiday I received a Thing(tm) as a present. I could take a picture or two of the Thing, and it may be easier to figure out what the it is.

Of course, for some reason I think it would be difficult to make Visual Google function that well... the only way I would get results for my Thing would be if someone already knew what it was, and defined it for the search engine.

Re:Is A Picture Worth A Thousand Words?

[tomhudson]

Pretty nifty for the functionally illiterate ... except that if they can't read the signs, they won't be able to read the answer either.

Of course, it could just return the picture with a big red arrow saying "you are here" (or should it be "U R HERE"?)

Seems to me that just offering a mapping function via cell locators would be more popular.

Re:Is A Picture Worth A Thousand Words?

[orangesquid]

The real killer app will be taking out hotornot's MeetMe funding (you know, if you want to actually talk to someone you click on, you have to pay money); you just google for their picture, and find them that way :)

Re:Is A Picture Worth A Thousand Words?

[Anonymous Coward]

Actually, what I look forward to is searching with a picture of Waldo.

What I fear is people searching with a picture of goatse.

Re:Is A Picture Worth A Thousand Words?

[Rei]

Also, there's the whole "vaporware" issue. The scale of this programming task is staggering; it's not only image recognition, but image *searching*. Just look at how poor OCR does with handwriting (and sometimes even pre-printed text). Generalized image recognition is orders of magnitude harder than recognizing a small set of print characters lined up in nice rows and clustered into words, and image searching is beyond that.

He can claim he's developing whatever he wants, but I'll believe it when I see it. It reminds me too much of how many AI researchers in the 60s were convinced that by the 90s computers would regularly converse with humans and be able to reason like them.

Re:Is A Picture Worth A Thousand Words?

[Anonymous Coward]

Totally agreed. But we're talking about tecnology here, instead of "ease of use", aren't we? I personally think, that, if thats really capable of doing what it claims to, we might be really close of having those Oakleys yelling "This isnt your fiance's house!"...

Re:Is A Picture Worth A Thousand Words?

[mOoZik]

I agree. I can't imagine how this can be pulled-off in the near term. How will the image processor find that image in the database? How will data be entered into the database to begin with? How will the software understand what you're referring to in a particular picture, whether it happens to have a car, a building, a person, a billboard, and other stuff in the same picture. Either I am too stupid or I underestimate the technology that can make this available soon.

Re:Is A Picture Worth A Thousand Words?

[Anonymous Coward]

I imagine it won't work at all in the winter season when it is snowy.

Re:Is A Picture Worth A Thousand Words?

[Stealth Potato]

Actually, I think I read once (might have even been on /.) about another potential application for a similar technology, which seemed much more useful than this. The idea involved using images to search, say, a parts database. If you were holding some unidentified doohickey in your hand, and you needed to know what it was so you could find a replacement, you could sketch a rough outline of the object, and the sketch would be used to search through the design information in the database (say, CAD drawings and whatnot). Limited application, sure, but it's a more immediately realistic goal than full photographic image recognition. (Especially when, as you point out, the human processing is more practical in those sorts of situations.)

Re:Is A Picture Worth A Thousand Words?

[RancidBeef]

That reminds me of a story I heard once about some military system (using a neural net) that looked at satellite or aerial photos and selected the ones that had tanks. They fed it some photos with and without tanks. It scored almost perfectly. They decided to run another test and went out and took more pictures. This time it failed miserably. They eventually found out in the original set of pictures with tanks, it had been either sunny or cloudy (can't remember which). In the pictures without tanks it was the other way. All the software was doing was indicating whether it was cloudy or sunny. It didn't give a rip about whether a tank was in the picture.

So, if you take a picture of a street sign in front of a museum, is it going to search for a museum, architecture that is similar to the museum building, or your location? Or the weather condition based on the sky in the background? If they can pull this off, I'll be very impressed.

Re:Is A Picture Worth A Thousand Words?

[Q2Serpent]

Read this post.

Re:Is A Picture Worth A Thousand Words?

[YrWrstNtmr]

Interesting idea, but what's missing is context. Where did this doohickey come from? Is it an alternator arm from a Ford, or a tape transport arm from a VCR? Same basic shape, vastly different scale.
Alternator arm from a Ford or a Chevy? 1/4" difference in length would negate its use on one or the other.

If I know the context (VCR model 110FC, or 2001 Ford Focus), then it would be much easier to just go directly to an illustrated parts breakdown or CAD repository for that model. If I don't know the context, I'd have to be a very precise artist to even get vaguely close. And close usually isn't good enough.

I agree, though, that its a more realistic goal than full image recognition, but wonder at the true usefulness.

Re:Is A Picture Worth A Thousand Words?

[Rei]

Comparing the wavelets of your images has little to do with trying to identify a 3d scene - especially with the degree of certainty you'd need with any of the named applications.

Re:Is A Picture Worth A Thousand Words?

[Takumi2501]

True, there are situations where it's just as easy (as far as the user is concerned, at least) to search for a string of text, but I can see practical applications.

They mentioned in the article about using facial recognition for identifying criminals with a photo. I'd be pretty impressed if they managed to pull that off. Mind you, I don't even want to think about the privacy issues it would raise.

Re:Is A Picture Worth A Thousand Words?

[_ph1ux_]

well, maybe not more efficient for you - but could be a HELL of a lot more efficient for visual AI.

The human mind is already exactly this - a visual google. We have a DB of contextual knowledge that is accessed and triggered through a few sensory inputs - smell, touch, sight etc...

here we could have a robot that could 'see' be able to get contextual information based on what it is seeing.

Say you have it walk into a room - and the room is filled with a bunch of objects, the AI could scan the room, then determine approproate courses of action/manipulation for each object. It would see a chair, and could then receive instructions on how to sit in it. How to use the sink in the corner etc.

As long as these AI share a backend DB, when one robot learns to use a thing - that knowledge could be shared and associated with all like-things. So even though the chair looks different than the original seed chair, the useage is always the same - such that all contextual information would be attached to all images that match the criterion.

The next step - is modifying the "definition" of an object, based on the need or situational dynamics.

A chair in one dynamic is used for sitting - in another its used for climbing up higher - in another a even a weapon.

The real value here is to be able to pull contextual information from Object based reality, as this is a leap for machine AI to do what the human mind has mastered.

Re:Is A Picture Worth A Thousand Words?

[theLOUDroom]

Also, there's the whole "vaporware" issue. The scale of this programming task is staggering; it's not only image recognition, but image *searching*. Just look at how poor OCR does with handwriting (and sometimes even pre-printed text). Generalized image recognition is orders of magnitude harder than recognizing a small set of print characters lined up in nice rows and clustered into words, and image searching is beyond that.

Yeah, but this doesn't have to be any more accurate that say.....Google. And the computer doesn't actually have to "comprehend" the image like a human, it just has to recognize something close to it.

Personally I think it'd make a great reseach topic for a grad student studying DSP.

Here's how I would do it:

• First, you downsample any ridiculously large images. (This can cut your processing and bandwidth costs signifcantly.)
• Then, you convert the color space to HSV. (Brightness and shape information is probably more important than colors.)
• Next, you might choose to normalize some or all the channels. (You want a bright picture of something to show up when you submit a dark picture, a picture taken with a diffent camera, etc.) You probably also want to reduce the color depth to something easy.
• So you've got this image in a reasonable shape, now you need to extract something to search with (as well as create a database key for the image you were just given). The thing I'd try here would be various mathematical transforms (Fourier, etc). I bet you could have some luck with using wavelets.
• Now you've got a 3 dimensional matrix of coefficients. Thing is, you probably don't want all of them so you can throw out all but the most important ones. So you keep only the first N coeffiecients from each channel.
• Take all those coeffients and apply a weighting/importance to them. Use them in chunks, so your seach can be handled with a tree.

There are some tricks you'd want to develop, like being able to find an image that of something that's been offset, or finding close ups from wide shots, etc but I think it's worth a shot.

Re:Is A Picture Worth A Thousand Words?

[theLOUDroom]

Forgive me replying to myself, but I hadn't realized that the subject of this article was talking about searching for TEXT with images. I must admit that's a harder problem.

My method really only encompasses searching for images with images. You could add text by searching through ALT tags, and processing the text on a page which contains a given image.

A lot of pages with a picture of the eiffel tower would have the word "Paris" on them, for example.

Re:Is A Picture Worth A Thousand Words?

[Rei]

But that's not at all what this person is claiming that his software will do. He's claiming that *you*, the user, can take a picture of anything, and it will find information about what you take the picture of. That's an almost unimaginably hard task to even come close at. Unless of course, by taking a picture of barney, you want information on eggplant, e-coli, and neptune. ;)

Distinguishing one street corner from the next, regardless of angle, requires full understanding of the 3d geometry underlying the scene and the ability to reverse-calculate from the 2d user-supplied image the 3d geometry that formed the scene behind the picture, then comparing the 3d geometries on a "close enough" basis - plus the ability to handle objects that move and change.

Re:Is A Picture Worth A Thousand Words?

[ChrisPee]

Tell me, which is easier? Upload this image and try to find out where you are via this Visual Google, or enter the street name (street sign in the photo says "Queen Street") in Text Google?

Try doing that in China*, and then get back to us. Oh, and don't forget to take your Chinese keyboard. And a manual on how to use it. And sufficient experience to distinguish between similar-looking ideograms.

* But not on one of the few streets where signs are printed in both Chinese and English; that's cheating.

Re:Is A Picture Worth A Thousand Words?

[Anonymous Coward]

You don't read enough of the computer vision literature. Image clustering and retrieval is currenly being done, and works quite well. It has nothing to do with technology, it is all about statistics.

Re:Is A Picture Worth A Thousand Words?

[aug24]

I disagree. It's 'pretty easy' to find flat surfaces in a picture, and almost all buildings are made of those. Then 'all' you have to do is search a database for a matching set of planes. Not absurdly difficult, and prolly much easier than a general handwriting reader or face recognition.

J.

Re:Is A Picture Worth A Thousand Words?

[Savant]

Bullshit. There's nothing "easy" about extracting 3D plane data from a photograph. In fact, I can't claim to know of any techniques that allow you to do that without either manual interaction or multiple photographs (using stuff like stereo matching etc) and I work in the field. Even if you had a laser scan from a single perspective, it could be rather challenging to search a large database of laser scans and find the ones that match the original scanned object, and that's when you have depth data to play with.

Re:Is A Picture Worth A Thousand Words?

[Threni]

>Tell me, which is easier?
>As long as human input is still required (i.e. you need to submit something), I
>don't think this is going to be popular

You clearly need human input to describe what you want to look for.

It doesn't matter if something is easy or popular, just that it performs a useful function. Whether or not you think it's easier to type "I, robot" is neither here nor there.

Re:Is A Picture Worth A Thousand Words?

[Threni]

Sounds like they just needed to feed it more data. Eventually they'll have tanks both with and without clouds, and an absence of tanks both with and without clouds.

Re:Is A Picture Worth A Thousand Words?

[aug24]

Please note the inverted commas: 'easy'. I was speaking relatively.

I meant in comparison to things like faces, which the parent poster was commenting (and I agree) are very hard.

Here's how you do it: Look for slightly curved lines. Examine curvature (due to lens) to infer distance. Convert to 3d model of lines. Look for lines which are parallel. Infer planes.

[I'm a physicist and mathematician by the way. Your terminology may vary.]

Justin.

Re:Is A Picture Worth A Thousand Words?

[sweet reason]

Tell me, which is easier? Upload this image and try to find out where you are via this Visual Google, or enter the street name (street sign in the photo says "Queen Street") in Text Google?

my office is on queen street. what google query will tell you what city i am in?

Re:Is A Picture Worth A Thousand Words?

[Savant]

Lens curvature? Nowhere near precise enough even in the ideal image where the surface was facing dead on to you; remember we're working to the fairly crude constraints of pixel data (think integers all the way), and if determining distance from curvature worked at all it would work only with lines large enough to have visible curvature. In addition to all this, accurate line detection from photos can be a bit hit and miss anyway, which is going to make the error in your calculations impossible to work with without the manual interaction I already mentioned. This isn't within a thousand miles of being a practical method.

My field's computer geometry, and I'm working on commercialising a fairly advanced bit of research which does this kind of thing (albeit with a fair bit of manual interaction and multiple photos). We're nowhere near finding a way to automate plane extraction from a single photo.

Re:Is A Picture Worth A Thousand Words?

[aug24]

Ta for the info... interesting stuff. I've been doing a lot of work with motion sensors and stuff, so I'm familiar with the problems of having no continuous data (both in time and space). As pixel densities and processing speeds go up, the problems you detail should be alleviated though.

I'd still bet that your problem is solved (whether by algorithms or processing power) before human face recognition is solved ;-)

J.

Better yet...

[punkass]

..imagine being the guy who has to photograph EVERY STREET CORNER IN THE WORLD.

Re:Better yet...

[Antonymous Flower]

That guy is you. And me. Why hire someone to populate the database when the people who use this service will do it for them?

Re:Better yet...

[Daxx_61]

Hmm. I'm thinking that might take a while. He's gonna need a lot of coffee...

Re:Better yet...

[weenis]

on the other hand .... meet a lot of high class women that way!

Re:Better yet...

[kd5ujz]

This might be the case. Mapquest has a company do something simular, drive around and take notes on EVERY intersection, ramp, road construction, and whatever else. They do it slowly, and I imagine its boreing as hell.

information here

Re:Better yet...

[L7_]

"Can you see me now?"

Re:Better yet...

[kurzweilfreak]

"Can you see it now? Good!"

Re:Better yet...

[Anonymous Coward]

already done in many large cities in western europe. not just every corner, every address. the french yellow pages is one way to see the images.

Well of course

[Anonymous Coward]

We all know the 1000x gain you get using pictures over words.

hmm,

[Tibor the Hun]

but what if the camera tells me that there's nothing to see there, and to please move along?

how will i know which way to go?

Re:hmm,

[Anonymous Coward]

how will i know which way to go?

I'm sure Microsoft can tell you.

The possibilities!

[Anonymous Coward]

*Uploads porn*

Now, machine, find me a hooker!

Re:The possibilities!

[RancidBeef]

Yeah, I want something that works in the reverse. I want to type "medium-sized-breasts, redhead, no tattoos" and have it find the matching photos.

Re:The possibilities!

[Anonymous Coward]

That's a great idea ... maybe they should call it ... images.google.com ...
P.S. Turn off SafeSearch Filtering

Re:The possibilities!

[RancidBeef]

I don't think images.google.com is quite *that* good. Also, it won't look on -ahem- local files...

Re:The possibilities!

[hunterx11]

Google's image search, however, sucks.

Fleck's image recognition

[2advanced.net]

So when do you combine this with Fleck's nude recognition algorithms to provide a service that can identify a person by partial nude picture?

The possibilities are endless!

Re:Fleck's image recognition

[plover]

So is this Fleck's law, then?

"No technology is sufficiently advanced until it is used for pr0n."

Re:Fleck's image recognition

[rbarreira]

This paper demonstrates an automatic system for telling whether there are naked people present in an image.

So it's not "identifying a person by a nude picture", it's identifying pictures which contain nude people...

Re:Fleck's image recognition

[nEoN nOoDlE]

Bah! That link is USELESS without pics!

dewdz!#@

[Anonymous Coward]

omg no way!

Oops!

[Anonymous Coward]

Oops!
The article you requested can not be found. Please use the search below to find what you are looking for.

Why not use GPS Technology?

[xmas2003]

While this looks pretty cool, I'm confused by the examples provided in the writeup - "Imagine taking a photo of a street corner to find out where you are, or the photo of a city building to see its history" since GPS technology would probably be a better enabler for those specific applications.

Re:Why not use GPS Technology?

[bwcarty]

My thoughts exactly. It's not like we don't have vehicles with GPS guidance systems already.

Besides, how would weather, day/night conditions, etc. affect image recognition? If a major landscape/skyline change occurs (think pre/post World Trade Center collapse), what impact would it have on the ability of the visual recognition system?

Re:Why not use GPS Technology?

[neverkevin]

Or why not just look at the street signs to find out where you are? If the street corner is in a database it is probably in an area that is developed enough to have street signs.

Re:Why not use GPS Technology?

[Anonymous Coward]

Yeah, but this poor guy has to write a thesis about SOMETHING to get that piece of paper that entitles him to debt and precarious employment!!

Re:Why not use GPS Technology?

[nacturation]

Imagine you're going through photos of your latest vacation and you find one of a street corner which you snapped while on a drinking binge. Since, in your drunken stupor, you don't remember where it was, you can just submit it to find out the history of the building and perhaps discover other famous people who have similarly vomited in that vicinity.

Re:Why not use GPS Technology?

[OECD]

or the photo of a city building to see its history"

Substitute "person" for "building". (Is that a cameraphone, officer?)

Re:Why not use GPS Technology?

[Anonymous Coward]

Wouldn't this technology just be useful for was it states "searching of a database by submitting an image".

People are storing more and more digital image information, with digital camera and videos. Maybe after 20 years of digital imagery finding all the pictures of a certain individual, especially if you haven't supplied proper IPTC info, might be a challenge.

Re:Why not use GPS Technology?

[eln]

Well, you'd have a much harder time doing searches about the World Trade Center...

Re:Why not use GPS Technology?

[xmas2003]

I forgot to add to my post above that GPS encoding as part of the JPEG EXIF header had been a standard for some time and a handful of high-end DLSR's have this capability today - this will become more prevelent - heck, as part of E911, your cell phone camera already has GPS info, but I don't know first hand if those add the tags to the EXIF header, but would be trivial to do.

alek

Re:Why not use GPS Technology?

[drgonzo59]

I thought of that too. Why not use GPS instead. It's a problem of machine learning and classification. Given a picture of a street corner, what features make that street corner in any light and weather conditions different than other hundreds of thousands of corners. Also, what about the angle at which the image is taken?

The database to work will have to understand what 3D objects are (at least in the specific domain) and have an idea of what features of the object are important (like signs for example, so it will need a very good OCR system then too). That becomes a knowledge representation issue.

There have been many projects like this before attempted. But until a computer knows what a "chair" is, or what "statue" or a "tree" is, it will just not work right. To have a computer understand concepts though is a much larger and more interesting accomplishment.

Re:Why not use GPS Technology?

[radish]

heck, as part of E911, your cell phone camera already has GPS info
Actually no, it doesn't. Even in the US (where E911 is relevant) the vast majority of phones (even brand new ones in the shops now) don't have GPS. There's some kind of weird urban legend floating around that they all do, or it's some kind of federal requirement. It's not. The requirement is that _if_ the phone has GPS, it should be available to 911 services.

Given how well my handheld GPS works (or rather, doesnt) in the city, I wouldn't hold out much hope for a tiny phone buried in my pocket under several layers of clothes while I ride the subway being able to get a useful signal anyway.

Re:Why not use GPS Technology?

[snorklewacker]

Or submit a picture of your vomit to discover similar celebrity vomit. Or celebrities that look like your vomit.

Re:Why not use GPS Technology?

[Profane MuthaFucka]

This is like an answer to a Jeopardy category. Alex, I'll take "Ways to profit off a vomiting drunk for $500".

Re:Why not use GPS Technology?

[Bigby]

I think the article should have said:

Imagine taking a photo of a chick and finding out what her phone number is.

Re:Why not use GPS Technology?

[bug506]

GPS would be useful in some situations (if you want to know about a general area), but for the example of taking a "photo of a city building to see its history", GPS itself would not be sufficient.

GPS can provide a location, but it can't pinpoint what you are looking at. This is the case even with compass data indicating which direction you are pointing your device--what if there are two things in your line of site from that perspective? (Do you want information about the building, or do you want information on the kiosk in front of the building?)

Also, this is more generic than these examples anyway. What if I want information on the building and then on a street performer in front of the building? I could take a picture of the building, read about it, then take a picture of the street performer and read about him/her. GPS wouldn't be sufficient to tell me about the street performer, because he or she might move around the area.

Re:Why not use GPS Technology?

[MissTuxie]

"Imagine taking a photo of a street corner to find out where you are"

gee, I wonder what happened to asking someone on the street.

Oh, I forgot, this is /. Talking to people is taboo.

Re:Why not use GPS Technology?

[operagost]

What is:

1. Get drunk and vomit.
2. ???
3. Profit!

Re:Why not use GPS Technology?

[Anonymous Coward]

you could simply have the gps coordinates embedded into the picture for future reference, invisible when you look at the picture of course.

Man on man

[sulli]

the pr0n industry is going to love this.

Re:Man on man

[2advanced.net]

"Man on man"? Man oh man, Freud would love this!

Re:Man on man

[macsox]

i assume the title of this was supposed to read "Man oh man" -- unless you were suggesting a new type of porn. in which case, i'm pretty sure they're way ahead of you.

Re:Man on man

[Anonymous Coward]

> "Man on man"

In Soviet Russia, is other way around!

Re:Man on man

[Anonymous Coward]

Believe me, there is no shortage of man on man porn.

Link in the article is wrong

[gwjgwj]

The article you requested can not be found. Please use the search below to find what you are looking for.

Or a photo of...

[dmorel]

Imagine taking a photo of a street corner to find out where you are, or the photo of a city building to see its history
or a photo of your wife to see...
oh never mind.

New Query Language

[Tablizer]

Enter search criteria: (.)(.)

Re:New Query Language

[Anonymous Coward]

Enter search criteria:(.)(.)

Showing search results for: "male asses" ...

Re:New Query Language

[Tablizer]

[Enter search criteria:(.)(.) ] Showing search results for: "male asses" ...

You mean "Pimply Male Asses..." Or perhaps siamese Goatse's

Re:New Query Language

[Anonymous Coward]

More interesting: V

Re:New Query Language

[Cap'n Steve]

You idiot, everyone knows boobs are written like this: (.Y.)

Goverment uses...

[mpatmcg]

I'm sure the govt. will have a lot of uses for this. We'll never hear about this again.

Scary Results

[The_Rippa]

I'm a beta tester for this product and have gotten some scary results. For instance, I was on vacation in Yellowstone and took a photo of Old Faithful with my camera phone. I submitted it and it gave me back search results for tubgirl!

difficult for slashdotters to find p0rn

[brer_rabbit]

it's going to be difficult for slashdotters to find p0rn with this search engine. Not only must you have sex, but you've got to take a picture of the act too...

Good idea but....

[DrinkingIllini]

how much harder is it to just use a regular text search for the restaraunt, movie, building, etc. that you want info on? It's like voice dialing on a cell phone, good idea, but it's about ten times faster and more effective to either dial or scroll to the name you want to call manually.

Out of business

[Anonymous Coward]

Imagine submitting a picture of your wife and you find out who she've been messing around with. Now that's something that will put private investigators out of business.

Re:Out of business

[slart42]

>Imagine submitting a picture of your wife and you find out who she've been messing around with. Now that's something that will put private investigators out of business.

right, because that information is currently readily available on the net, there just isn't a way to search it

un-possible

[siliconcenturion]

the implications for the adult entertainment industry are staggering, to say the least

Cell Phones

[Anonymous Coward]

This could be the one thing I would actually use my cell phone camera for.

If I have a mobile phone..

[the_skywise]

Isn't it more efficient to use the GPS in it to tell me where I am than to submit an image?

Seriously it's a good thing but the uses are somewhat limited. What if you don't have a digital image or a photograph to be scanned? How would you translate the image in your minds-eye into something searchable by the PC? (yah, mindlink... I know...)

Re:If I have a mobile phone..

[radish]

I believe a larger percentage of phones have cameras than GPS. But yes, for the examples given GPS would be more useful. For some other uses though (like menu translation) GPS wouldn't help at all.

It's going to be boobs

[Nom du Keyboard]

Everyone is going to try searching on a pair of boobs to see what they get. I can set it now:

1: Take picture of current date's frontside archtecture.
2: Submit to search.
3: Reply: You can do better than that. Try her older sister.

i can't wait for this!

[macsox]

i'm going to rush out today and buy a nokia phone so that i can have this functionality the instant it is available in 2038.
i mean, really. isn't this one of the main roadblocks to having a robot that can operate independently -- object recognition? i mean, if spam bots (the world's most advanced robots) get tripped up trying to read the word 'cat' behind some wavy lines while they're signing up for a hotmail account, you really think that i'll be able to photograph a car in 5 years and get info about it? highly improbable. a worthy goal, yes. but not likely in the near or near distant future.

Re:i can't wait for this!

[soricine]

Image recognition is immensely more difficult than people seem to think. And yet every few weeks someone is claiming to be just around the corner from a system that can easily identify the contents of an image.

When your brain 'recognises' what it is looking at, it is doing a lot more than just comparing two images (as in the street-corner example from the article). Your brain simply doesnt operate in terms of bitmaps.

The fact that he is basing his hyper-vaporous product on facial-recognition software should set of alarm bells. Facial-recognition in a real-world context has consistently failed to be of any use at all, although it may work fine under lab conditions.

If all the money invested so far hasn't made a computer able to successfully recognise a subset of the visual field (faces), why should I believe in a machine that is able to recognise practically anything?

Um...

[torinth]

OPTION 1:
Take picture of street corner with camera phone. Connect cameraphone to Internet connection. Upload to wherami.cooldatabases.com. Wait 30 seconds for processing. Get location.

OPTION 2:
Pull out $400 GPS with map software.

OPTION 3:
Read street signs. Check index of road atlas.

Yeah. Option 1 sounds awesome...

Re:Um...

[Anonymous Coward]

If the took option 1. But instead uploaded the picture to a site that could tell you where you're by a picture of the ceiling. Your mom would always know where she's at in the morning.

Where am I?

[Anonymous Coward]

[face lit by phosphor glow]

[eyes fixated straight ahead]

[slight dribble of drool running from corner of mouth]

> You are at www.hornygeeksluts.com.

And we have a winner...

[Momoru]

Sounds like we may have a winner for Wired's 2005-2010 Vaporware awards.

Visual Google?

[Rick and Roll]

Visual Google? Try visual semantic web. It identifies something and figures out where to go to find answers to it.

Also, I think the building and street corner thing would work a lot better with GPS than a camera.

The most interesting thing I saw in this article is that he plans to roll out a first version in about a year. Besides that, it's interesting research, but stuff we've already heard about.

I would definitely like to experiment with this sort of system.

First step toward augmented reality?

[CompSurfer]

Think this is the first step toward augmented reality?

It's also possible to search with music.

[Future Man 3000]

I've heard of more than one service in development for returning music titles against a song clip you play or hum into a phone or microphone.

Even though satellite/digital radio will reduce the market for this kind of thing, because each displays the artist name and track title, there are still plenty of opportunities to get a song stuck in your head that you don't recognize. A surprising number of people find out about music when it's used as the background tune in TV commercials, for example.

umm

[sometwo]

How could taking a picture of a street corner possibly tell you where you are? Don't most look alike? There are other, better technologies for telling you where you are, such as GPS or even just looking at the nearest street sign and typing the name of the corner into the map application on your phone.

Biological Identification/Classification

[AngryElmo]

Actually a practical use for this sort of technology is taking a photo of an animal, insect or flower/tree when on a field excursion or hike and getting back its taxonomy. Or in Australia, taking a photo of some of the wildlife to see how many painful ways you can die if it bites/stings you...

Re:Biological Identification/Classification

[katenysh]

or taking a picture of someone you have picked up at a club to make sure they are not lying about what sex they are

iDating

[El_Smack]

Or taking a picture of someone and finding out their history.
click
"Whoa Dude!, she's been on 4 amature Pr0n sites!"

Re:iDating

[Threni]

> Whoa Dude!, she's been on 4 amature Pr0n sites!

amature = not mature, or not professional?

Perhaps a better use...

[jfengel]

On a run the other day I found a waterfowl that I'd never seen before in the lake. It took some doing to figure out what it was. If I could have snapped its picture on my cell camera and gotten an identification (a hooded merganser, it turned out, after some digging), that would have been cool.

I can't imagine how well this would work, since orientation fools things pretty easily. But I imagine that if it were available, I'd find a lot of unidentified objects to look up. (A cooking magazine I read has a "what is this gadget?" column every month.)

Yes, Imagine taking a picture

[Nom du Keyboard]

Imagine taking a photo of a street corner to find out where you are,

Yes, imagine that.

1: Take picture with ultra-modern all-features camera phone of building while lost in city.
2: Submit to search system.
3: Search system queries phone's built-in GPS for position information.
4: Search system sends back retrieved GPS location.
5: Customer is absolutely blown away and immediately sends back picture of self signing virtual 10-year contract at Early Adopter prices.
6: Profit!

Stalkers...

[Chordonblue]

...are gonna love this too. Take a picture of the girl you like and do a search. This has some scary connotations I'm afraid.

Re:Stalkers...

[That's Unpossible!]

...are gonna love this too. Take a picture of the girl you like and do a search. This has some scary connotations I'm afraid.

Straw-man.

Stalkers already use Google. It's a lot easier to stalk someone with text than with pictures. What are the chances your image search would actually turn up anything for your average Jane Q Public?

Re:Stalkers...

[kaustik]

Not in the near future, of course. I can't image that taking a picture of most of anything would produce valid results in the near future. However, this type of photographic facial recognition is already being reseached and developed for things like bank robberies and terorism. I can picture this taking off to the point where it applies to the general public...

Re:Stalkers...

[Anonymous Coward]

Stalkers already use Google. It's a lot easier to stalk someone with text than with pictures. What are the chances your image search would actually turn up anything for your average Jane Q Public?

Not necessarily - try stalking someone with a name like "Smith." A photo search would speed things up considerably.

err... not that I would know personally, or anything...

Re:Stalkers...

[ashot]

where is the straw man argument here?

On today's search engines...

[wcitechnologies]

The most searched term is 'sex' or something like that. It'd be interesting to see the "search queries" for a picture-based search tool. I bet there would be a lot of stick figures behaving badly.

What to expect...

[Anonymous Coward]

Imagine taking a photo of a street corner to find out where you are

User: "Hmmm, the sign says 5th and Main. Let's find out where I am..." *click* *send*
Phone: "You... are... at... fifth... and... main... streets."
*smash*

Heh.

[sulli]

I'm sure it would work well on that kind of pr0n too.

Robot potential?

[CaptRespect]

Maybe this can be used for robots to recognise stuff or something like that.

bogus alarm

[Anonymous Coward]

this is just a bunch of money trolling by someone

there's no way these matches would actually be useful -- it's way too complex -- most places look just like other places, even in the same city.

this reminds me of the article recently were they were looking for funding for super-pens and super-paper. you can draw a picture of a calculator, and then actually use the buttons and do math. yeah, right... can I draw an oscilloscope and look at RF signals?
can I draw a microscope and use that?

it's all just flashy snake oil looking for funding, just like this photo search

Re:bogus alarm

[myukew]

sure you can. it's called imagination. it was around for millenia, you should know about it...

Finding places

[SamMichaels]

If I need to find a specific location, I'll send a text message to 46645 (GOOGL). Then, I'll use the street signs or the navigation system in my car.

If I'm completely lost, the only way object recognition would work is if I'm in an area with a lot of recognizable features...like a city...in which case I'd just ask somebody. I doubt taking a picture of a bush when I'm lost in the middle of nowhere will be helpful (see: car navigation system).

Hartmut Neven and hundres of other researchers...

[dioscaido]

This seems less like a technology article and more like an advertisement for Hartmut Neven himself. Yes, he's built a 'google for images'... But how does it perform? How exactly is it 'ingenious'? What sets his project apart from the handful of people at almost every University with a Computer Vision research department that is tackling the problem. The problem of matching images is well known, and very difficult to solve. Even in my grad school (BU), which has a small number of computer vision grad students, there are two different research projects on this very topic.

What I want to know?

[blackmonday]

Can my Powerbook be prompted to surf porn sites when the iSight catches me pulling down my pants?

Re:What I want to know?

[Man in Spandex]

Of course it can!

* iSight power on. Please stand by.
* Scanning..........
* Individual scanned. Penis exposed to local environment. Analyzing......
* No pubic hair. Missing one testicle. Age approximatively 14. Procede to launch....
* Launching Safari @ yahoo.com
* Echo "Enjoy"

Totaly Handy

[theManInTheYellowHat]

Way to figure out the name of the chick you saw on flashyourrack.com!!!

IBM has a related technology

[farrellj]

Back in the 1990s, I used to develop Kiosk Software using IBM's Audio-Visual Connection language. The next generation of that software was called Ultim-Media Builder. At the same time, IBM came out with an extension to DB2 that would allow you to do queries against a database of images...you could say "Find all pictures with a red ball and a tree", and it would find them in the database by "looking" at the pictures, not because of any captions or notes. I never heard if that tech became well know, or if it even still exists. I just thought it was very cool at the time.

ttyl
Farrell

Duh

[dfj225]

"Imagine taking a photo of a street corner to find out where you are..."

Imagine reading that street sign you just took a photo of to find out where you are.

Re:Duh

[Xconnect]

Hey, that's what I'd do if the street signs are in a language that I don't know!

As a person who loves technology advancements...

[Anonymous Coward]

...I have to say this technology sounds utterly worthless and doomed for failure.

Kind of Scary

[TooTechForYou]

This could end up being bad. If it expands enough it could reconize people's faces. Imagine some creep at the mall taking pictures of girls with his camera and uploading the picture and being able to find them by a blog or school database.

Re:Kind of Scary

[Anonymous Coward]

I suppose if some creep can do this so can the police that protect us. It will make you fell safe at night knowing that just from your picture that anyone can know your history.

Re:Kind of Scary

[eyegee88]

Why ? It would need a database which includes that person's face in a pretty high resolution to begin with. This technology, even if it seems promising, is quite overhyped. Assembling a database of street corner pictures in different angles all over the world (disregarding urban evolution and day/night status.. ..yes, there is quite a difference...) is a never-ending cycle which seems unfeasible. Unfeasible in many ways. Jokes aside, (I assume your post aims to be sounding funny) i am pretty convinced that besides the fact that the topological data that has to be assembled and gathered (we are talking pictures here, not GPS input) can not be achieved in a way that a photo-search engine can be even remotely effective, the sheer idea that people can be identified by any picture to be just ludicrous and best kept to cyberpunk-alike scenario's. The slashdot privacy-police surely will have a field day with this opinion. I hereby call this project "Duke Nukem Forever 2: the photogenic farce" cant wait till 2049... smiling faces now... -ig88

Too much clutter

[gleam_mn]

So you take a picture of a building on a street corner... how does the engine decide whether you're searching for:
1) your location
2) the history of the building
3) dog related info since there is a dog taking a leak over by the building
4) p0rn because of the good looking lady looking out her window wondering why you're taking a picture of her
5) violent crime since her over protective husband saw you and is wondering why you're taking pics of his wife

Most pictures could easily have lots of extra clutter in them which would confuse the search engine...

I'm just waiting for object recognition

[digitalgimpus]

That IMHO is the real prize in imaging right now.

I show an image of a car, and the computer knows, the make, model.

I show an screenshot of a TV show where they remove the product name/brand from the product... it can ID the product.

Facial recognition is not to bad at this point (though it seems lots of the pioneers are going under). Nobody seems to have successfully applied it to objects.

I think that has much more use... think about it:
1. Indexing and searching images/video
2. Explaining TV to the blind or impaired. More than just dialog, it could explain what the scene is.

Imagine being able to open up something like iPhoto, and just search for "dolphin" to find your picture of you at sea world. No need to add keywords, or anything. It's all automatic.

IMHO that will make tons of information accessible to many people.

Information is only useful if you can find it.

Re:I'm just waiting for object recognition

[madstork2000]

I agree. Object recognition is where the human compture (i.e brain) is FAR superior to any silicon based computer at this point. It seems people in this thread are thinking pretty small. Visual Google, will have its uses but the technology will revolutionize the way cmputers and humans interact someday.

This image searching would make computing far more intuitive. In the far future, it may also lead to computers able to learn/recall in the same way as humans, (via visual memory). I believe that when this "skill" is mastered by computers we wil have computers that can easily pass a turing test. Whether or not they are truely intelligent will still be up in the air.

This will also lead to a better human / computer interface. Imagine wearing a pair of those fancy glasses that have a computer screen displayed on the lense. If you have a camera that snaps an image of what your looking at and then does a real-time lookup it'd be pretty sweet. of course this pipe dream also depens on ubiquitous high-speed wireless access to the internet.

Even further down the road is an implant that reads the signal from your optic nerve (I know its been done already on Cats) and use the raw signal from the eye for the lookup.

The possiblities for human enhancement are really endless if we can get computers and people to be more "compatible".

-MS2k

Poor Example

[EndingPop]

The "street corner" example is a poor choice. I saw a story on here a while back (no idea exactly how long ago) about a visual search engine, but this one was for finding things that have already been designed. If an engineer needs a special kind of widgit he can search for the general shape of it and the engine could bring up known (and probably patented) examples that fit the description. This would decrease time spent designing something in house if you could more cheaply buy the design off some other company. Also, it could make it easier to determine what designs are already protected by patents.

dupe?

[Anonymous Coward]

i cannot find it but this is very similar to a story from within the past year

Um

[HarveyBirdman]

This of course assumes that every location on the planet is stored somewhere from multiple angles. I guess you can scale in software.

oh, something new (!?)

[l3v1]

[...]is developing ingenious new technology that allow[...]

Sorry, please somebody explain me what is so ingeniously novel in this technology. In the digital image compression, pattern recognition and indexing research area answering example-based queries on an image or video database is anything but novel. And that includes color-, histogram-, texture-, object-, depth-, even motion-based indexing and query/retrieval (no secrets here, many conferences every year).

Don't get me wrong, I have nothing against commercial (or else) implementations of decades-old research resutls. Just don't call them ingenious and new please.

Re:oh, something new (!?)

[funkmeister]

I have also come across papers on this topic going back at least 10 years. I am real suprised that no one has commercialized this technology. What's been holding it back?

Imgseek.

[Adhemar]

Imagine you're a photographer. Professional or hobbyist, I don't care. You have made thousands of pictures; they all are on your hard drive.

Imagine you're lazy. (Maybe you don't have to imagine that.) You don't want to describe your photos, you don't want to label them. The only metadata associated to your photos is date and time.

Imagine you're looking for a particular photo. You know where you'va taken it, you know what is on it, you can remember the subject, the color shades, etc. You just can't remember exactly when you took that picture. How do you search for it?

Well, you quickly make a drawing in which you try to (sort of) replicate colors and shapes. And you let your computer search for "similar" graphics.

Such software exists already (for quite some time). There's a beta Free software project (GNU licenced) called imgseek . Current version: 0.8.4. I haven't tried it, I don't know how good it is. But this screenshot looks impressive.

Re:Imgseek.

[zonker]

actually this is more useful to copyright owners wanting to find infringers. i've heard playboy already uses digicmarc watermarking. this might be the next logical extension, finding pictures that look similar to find images that people have modified to attempt watermark removal...

on the flipside, it would be helpful to regular folks who have a picture kicking around that they want to find out what it is or where they got it from. i do a lot of photoshop work and have an enormous collection of images that i've collected over the years that i couldn't attempt to figure out where i got them. an image search would be very useful...

Re:Imgseek.

[Anonymous Coward]

That looks amazingly cool. I'm off to try it right now.

Re:Imgseek.

[Rei]

Very cool, but completely different. You take your picture of the street corner that youw ant identified from a slightly different angle, and all the lines will be wrong. 3d messes with things like that; imgseek gets to work only in 2d. And it even has trouble with that - it only gives you a rough approximation.

All imgseek does is compare the wavelets - not even remotely applicable to identifying a 3d scene with a non-predetermined 2d image.

Still, a neat program. :)

Re:Imgseek.

[garaged]

It's quite good I've used for like 2 years, since the first time i knew it i loved it Great piece of not so known app

Re:Imgseek.

[Anonymous Coward]

That's interesting. So if I drew a picture of someone with a 12 inch cock tearing open a 12 year old, it would find porn for me?

Re:Imgseek.

[Anonymous Coward]

your IQ must be about 12. And when you have kids of your own (if anyone would get involved with a piece of scum like you long enough), you would want to use technology like that to hunt down and destroy people who use oversized genitalia to destroy and torture little 12-year old girls. Hope the cops can keep track of you and your ilk.

Your post isn't even a little bit funny

Re:Imgseek.

[raindog2]

There are lots of choices, really. I wrote a perl script (using Image::Magick) about 5 years ago that did the same thing as this, and the awesome GQView image manager has had visual image matching for nearly as long. On the Windows side, I'm pretty sure ThumbsPlus has a feature like that, and I'm surprised they didn't build it into XP since they were supposed to be all digital-photo-friendly.

of course, the first use of this would be...

[jxyama]

...for the pr0n directory, by uploading the pics of various parts of the body... to identify the actress...

Images AND Words

[Anonymous Coward]

How about Images and Words

Re:Images AND Words

[Anonymous Coward]

Mod parent up. Dream Theater is a great band.

Even better yet...

[killmenow]

...imagine a beowulf cluster of

Ahh, nevermind...

Re:Man OH man

[kiddailey]

Being a geek, locked away at your desk staring at a computer, what exactly are you going to be taking pictures of to search for?! ... wait. Don't answer that.

WHAT THIS WILL DO FOR PORN!!!!!!!!

[Anonymous Coward]

Imagine submitting a pic that you like, and not only getting more like it ---- ah forget it, getting more like it is the best part :P

use this for a dictionary

[Anonymous Coward]

french-to-english is easy because both written languages are latin-based. this would be useful for chinese/korean-to-english.

iDating -- The Nightmare

[Anonymous Coward]

click
"Whoa Dude! She used to be a dude!"

Image search

[TheToon]

This is nothing new, we had this 10 years ago in OS/2 ffs :)

Great idea!

[Anonymous Coward]

I always wondered if that were possible.
Imagine if you could search for a sond.

You could humm a tune from your head into google (or kazaa) and BAM! you have you're song. No more guessing games.

What about the context?

[amichalo]

So I am at work in downtown Fort Worth, TX USA.

I take a photograph of a building that is under construction here and submit it to the "search engine". So what might I be wondering?
- what street conrner am I at?
- who owns/is building/will occupy?
- what materials is it made of?
- how is it being constructed?

Same goes for anything. A flower, a person, an object. Without context, search results will be across the board.

Huh, now that you mention it, same problem with Google today. Just do a seach for "building".

Same idea, Different application...

[SkippyTPE]

Using this same idea, why couldn't you submit a picture of a given person/place/thing and the search engine return return all the other pictures it knows of that same person/place/thing. I realize that this is likely exponentially more complex, but it could stem the tide of data about to be lost to the billion+ pictures out there named DSCXXXX.jpg or MVCXXXX.jpg....

Re:Same idea, Different application...

[bluenirve]

So wait, I give it a hot girl, and it gives me thousands? Sign me up!

Re:Trolls: A Unique Social Movement?

[Zen Punk]

Other issues were raised with such surprising regularity that one must conclude that virtually all Malicious trolls have both erectile dysfunction and suffer frequent enuresis. The average IQ of this group was 83.

Truly eye-opening...

Oh yeah.... sure....

[Ancient_Hacker]

It seems unlikely we've discovered the algorithm that will compare two scenes and not get fooled by different angles, perspectives, saturation, angle of sun, street lighting on/off, fog, rain, snowcover, traffic changes, etc.... It may be quite a few decades before it's cheaper to use a computer for this than to have a real human being look at the picture and go "hmmm, looks like a Asian scene, those mopeds look Laotian, and that guy has a bag from Taco Bell-- must be the capital!

All right, poo-pooers...

[ChuckleBug]

Jeez, everyone's jumping all over this. "Why not read the street signs?" "What about GPS? This is stupid!" So the submitter cited some lame examples. Join the 0.05% percent of slashdotters and RTFA. He cites ideas like taking a picture of a cafe and getting a food review, or taking a picture of a French menu and getting a translation.

Maybe it could never work, but in principle, this isn't a bad idea.

Yawn

[SmokeHalo]

Imagine taking a photo of a street corner to find out where you are
Or, you could just read the street names from the signs and drop them into MapQuest...

News for Nerds, indeed

[lildogie]

The only technical challenges he discussed were database size/scope and what resolution to use vs. relevant details.

I was hoping to find out how, exactly, images could be normalized, and the normalize images indexed in a way to avoid O(N) searches.

Differences in perspective seem to me to be an enormous hurdle, not to mention shadows that change from hour to hour. Nothing about these challenges was even mentioned in the article

Looks to me like this guy is trolling patent sharks.

Even Better Application

[vertinox]

This could be used as an anti-beer goggle via facial recognition for the swinger types, which would tell you if the person you are interested in has been modded down by previous interlopers after waking up to find they were sorely mistaken.

Even if they can't do that... It would be just good to find out where the hell you are as you are fleeing down the street in your undies the morning after.

Not really new

[SCHecklerX]

Didn't IBM have this ability several years ago? I remember seing ads in magazines that let you search for images based on their shape.

How to find things you don't have?

[DigitalCrackPipe]

So guys can now find... more single guys looking for p0rn? Hmm, I think you often do searches for things you don't already have, so you wouldn't have a picture.

whew close one

[Anonymous Coward]

I took a picture of the prostitute down the street and it found pharmacy's that had results for genital warts.

Thanks dude

Star Trek Tech

[kdark1701]

Just like on Star Trek! "Computer, who is that humanoid three spaces from the left?"

could be dangerous

[briancnorton]

And what kind of search results do you get back when you sumbit this picture

Now imagine _THIS_ !!!

[Anonymous Coward]

> Imagine taking a photo of a street corner to find out where you are

Or maybe you could just _LOOK_ at the street signs instead. For the blind, we'll put braille placards down low.

Of course...

[giantsfan89]

... you could always write a perl program to let people think they are controlling the city building's lights, when infact they're just controlling pictures in a database. ;-)

Re:Of course...

[xmas2003]

Ummmmm ... now I wonder who the heck could do something like that ... and if they did (and bazillions of people were turning those lights on and off), do you think anyone walking by the building (including reporters) would wonder why the lights never change?!? ;-)

Better suggestions for examples

[tgrigsby]

Imagine taking a photo of a street corner to find out where you are, or the photo of a city building to see its history.

Lame. Better yet:

1. Identify edible as opposed to poisonous plants when hiking.

2. Identify a part you took off your car during a doomed DIY tune-up session.

3. Identify a part that was lying on the ground after you closed up the computer case that you just *swear* wasn't there when you started.

4. Identify the odd substance in the bowl from the back of the fridge...

5. ID your blind date, see if Joe Walsh is looking for him/her/it.

6. And for die-hard nerds, ID that big shiny ball in the sky on those rare occasions when you tear yourself from the computer and wander through the front door when it's still daytime.

Of course it's worth!

[Anonymous Coward]

*Normally*, it's much easier to point an object, click a button to take a photo and send it to the server, and then wait for the image search tool's answer, than to pause to *think* about a good string for the search tool, write it, and then send it.

And also

[Heem]

While this is quite a cool idea, what we really need is an audio search, where as someone could whistle or hum a few bars of that tune that is stuck in their head, and the search results could tell you what it is, where to buy or download it, the lyrics, etc.

Botany.

[burnttoy]

This would be a fabulous tool for bontanist, gardeners and plant lovers everywhere!

Take a snap of something and search to find out exactly what it is (and in my case can I eat it!). Combined with mobile camera phone technology this could be quite amazing.

Final proofing is, of course, done by you yourself. If you eat the wrong mushroom don't say I didn't warn you!

Its been done before - Informix had this

[rjstanford]

I don't know how well it was done, but I remember many demonstrations and other interesting presentations from Informix (now owned by IBM) covering this as - IIRC - a blade plugin to their 9.x series Universal Server. You could submit a picture of, say, a wheel and it would figure out what index picture it was matching and then, of course, retrieve other related information. Pretty cool, but I don't know of anyone who came up with a production case for it.

One of the big goals of this kind of technology was to be able to help small-town doctors get more access to fully indexed medical texts. Imagine if you track down some odd bacteria in a patient, grab a picture of it from your microscope, and immediately have it linked to information from all of the various public health agencies, CDC warnings, et cetera. Pretty damn cool, IMO. Even if you could do a set of probable matches, with the system saying that its one of the following, that would cut down a ton of time in a potentially life-saving way.

Ah, nostalgia. I still miss Informix - yet another part of the proof that a solid marketing department will outdo solid technology any day of the week. Still being used a bunch too, but its a bit like having some embarrasing disease like Herpes, or using COBOL - nobody ever wants to admit it in public.

Too easy

[rbarreira]

That 2nd one's easy - http://archiv.radio.cz/nato/fotogalerie1/bush.jpg

product manuals

[LuxFX]

This needs to be hooked up into a service that archives product manuals. Take a picture of your TV/cell phone/microwave/etc. and this service would be able to give you a PDF of your lost manual.

No more hunting for model numbers (which I've found are not often included somewhere on the actual product).

The value of image searches

[fileslinger]

The real value to me of searching with images would not be either for directions or movies, but when one needs to find a particular image or an image of a particular thing. For instance, I do trademark research, and in order to search the USPTO database of logos, you have to be able to describe the image in terms of their specific and limited vocabulary and numeric codes. This is amazingly time-consuming and sometimes quite difficult when the image is a non-standard shape and not representational. If I just plug in the image and say 'Find all the ones that look like this,' it would be a heck of a lot easier.

Re:Hartmut Neven and hundres of other researchers.

[ashot]

exactly my thoughts. if only we had a slashdot with 'smarter' editors..

Already been done for porn

[jesser]

The porn industry is, as usual, one step ahead. MiltonSoft's Thumbnail Gallery Finder lets you search a large database of porn galleries for copies of an image you have. It recognizes images even when they are cropped or resized, and it sometimes recognizes that two photos from the same set are "similar" even though they are not the same photo. It's a great solution for the "incomplete photoset" problem. It even comes with a Firefox extension (which I wrote) that lets you right-click an image to find more galleries containing it.

take THAT intellectual-property!

[evilmousse]

muwahahahaaa

writing is speech
pictures are speech (this)
code is tshirt is music is speech (decss)
insert numerous other examples (my laziness)

good luck continuing to legislate speech as property based upon the medium, the inevitability is that they're all communication and increasingly interchangable. i'd start looking for another way to designate what can be proprietary and not NOW.

Re:Trolls: A Unique Social Movement?

[rbarreira]

Thanks for the link. Didn't know that article, I'll be reading it :)

Matching Type

[ballookey]

This would be helpful to a designer. I can't memorize the names of the 13,000+ fonts on our server, but it'd be great if I could scan a sample that we need to match and have the computer tell me which font it is. It beats the hell out of playing 20 questions with some type foundry's website only to get a result that's completely off.

Re:Matching Type

[Anonymous Coward]

The 13,000 are licensed, right? :-)
Problem for any uber-font-matching database has been how closely you can match without having to have the IP, or font data (aka software).

You should be trying myfonts.com and identifont.
After that you need humint. You can try typophile.com and the font fans there will be happy to show off. I've probably got 2,000 I can recall from my own type work.

On a similar note...

[Takuryu]

Here in Japan, almost every new cellphone is equipped with a bar-code reader capable of reading "regular" bar-codes, as well as 2D "matrix" codes (QR Codes being the standard). You simply take a photo of a bar-code, and it translates it into text into the browser address bar. While not "searching" per se, but now almost every advertisement seen in a magazine, on a train, etc. has a QR code printed on it. It's much easier to snap the photo and be blitzed to a webpage than to either jot down (or try to remember) a URL like http://www.denso-wave.com/qrcode/aboutqr-e.html
(and that URL about QR Codes is an easy one to remember).

While image search technology has its uses (identifying things without "written" landmarks), I believe advertising isn't one of them.

snap pix

[Doc Ruby]

If all you have is a cameraphone, with its crappy alphanumeric "keypad" it's easier to point and click the phone than to enter the text. Especially since text entry will often require reentry, after typos, while point/click pix will probably work every time, and be minimally frustrating to reshoot if needed. I commonly snap pix with my Treo 600 rather than "type" reminders, even though its QWERTY keypad is superior to most phones. I'll be even happier when I can just point a finger, snap another finger, and get structured results from a search engine, but this image search engine will help pave the way for that more integrated interface.

Online demo

[Yurian]

Something pretty similar has already been done here. They have an impressive online demo to play with.
It's called Visual Google - You give it a picture, and it returns all the frames in a movie that contain the picture. The clever bit is that it uses a kind of text-retrieval inspired method to do it, so the processing time is essentially zero (just an inverted index lookup, with "visual words")

Work for Google

[mrmeval]

So we can find that special picture with all the curvy bits in the right places.

Intersting but hard to do I think

[KeoniHI]

The dada base woould have to havee a photo of every place in the world and that is just about imposable!

Princeton has something similar (sorta)

[tonsofpcs]

http://shape.cs.princeton.edu/search.html allows you to search a database of 3d models by submitting text, 2d [orthagonal] sketch(s), or a 3d [isometric] sketch, or even a 3d model file. Note that at this time these sketches are drawn by the user at search time, but there is nothing to say they can't use a border detection algorithm to accept image input. Also, once you have some results, you can select 'find similar shape'.

IconSurf

[Pseudonymus Bosch]

IconSurf also searches with images. It is a collection of the favicons of lots of sites. You choose whichever you fancy.

Wavelet transforms

[Peter Cooper]

Washington U has an interesting software project along similar lines. It can index thousands of pictures, and then recall them based on your crude drawings. Very cool and I think this tech is already appearing in one open source image management tool.

What would it retrieve?

[Ninwa]

Websites with an image similar to the one you've submitted? The location of the building you've taken a picture of? There would have to be some extremely smart image recognition and comparison algorithms going into this. I wonder if those algs could be used to benefit AI, I would imagine they would have to be that good for this to work properly. -n

next step

[fikx]

I like this as an idea. We base so much on using text to deal with information. Humans don't even think entirely in words.I picture concepts myself more than I think of a nice paragraph about them. This is just getting computers to be able to keep up.
I'm also just tired of the assumption that we have to generate MORE text to allow us to use other types of information. meta data in genral is adding more TEXT to text, pictures, etc. when we already have enough of the stuff! text isn't the only way to represent information. It isn't even close to the most efficient way to do it!
Oh, and speaking of metadata, if computers can process images, maybe we could use pictures as metadata! it works for Movies. one poster describes the whole movie sometimes: type, actors, characters, etc.

Give me your face!

[Chordonblue]

Are you really part of the Slashdot community and DON'T believe this will eventually happen?

Remember: At some point when exabyte storage is commonplace, this is be all too easy. And I don't think it will be the gov't who will do it. It will be a result of the same kind of marketing genius that makes you use discount cards at the supermarket to get a deal. "Just give us a picture of your face - you'll get a deal..." You watch.

One thing builds upon another - don't believe this sort of thing doesn't have organizations drooling...

Finally!

[LHSPomPom]

A real, live Pokedex!

Explanation of How It Works - Kohonen Neural Nets

[TwistedPants]

Kohenen Neural Nets are probably the basis for this... I discovered this article only a short time ago. The author does some pretty impressive things with his client. I can't find the source he linked to anymore, as it's dead :( Self Organising Maps (another name for Kohonen NN's) are pretty good at figuring out stuff like this.

Image Search, huh?

[Anonymous Coward]

If this is the opposite of Google Image Search (put text in, get pictures. Often porn.), then the opposite result should be true.

Put image in, get porn.

"Imagine" doing things efficiently instead

[Anonymous Coward]

Neither of the "imagine this" suggestions make any sense. As far as practical application of technology to find out where you are, how about a GPS service in that phone to show you a map of where you are. For that matter the GPS service could just as easily locate landmarks around you and allow you to read their history.