Slashdot Mirror

← Back to Stories (view on slashdot.org)

Searching with Images instead of Words

Posted by CmdrTaco on from the blessing-for-those-who-can't-spell dept.

johnsee writes "A computer vision researcher by the name of Hartmut Neven is developing ingenious new technology that allows the searching of a database by submitting an image, for example, off a mobile phone camera. Imagine taking a photo of a street corner to find out where you are, or the photo of a city building to see its history"

34 of 207 comments (clear)

  1. Is A Picture Worth A Thousand Words? by fembots · · Score: 4, Insightful

    Tell me, which is easier? Upload this image and try to find out where you are via this Visual Google, or enter the street name (street sign in the photo says "Queen Street") in Text Google?

    The article also mentioned this thing should start small, like a movie guide, so is it easier to upload a 2K "I,Robot" billboard photo, or just enter "I,Robot" in Google on your cell phone?

    As long as human input is still required (i.e. you need to submit something), I don't think this is going to be popular. However, if you have a Oakley that automatically takes photos of what you see and feeds you the location details, that'll be something.

    --
    Rock that crushes, Paper & Scissors that don't matter.
    1. Re:Is A Picture Worth A Thousand Words? by hyu · · Score: 2, Funny

      Actually, what I look forward to is searching with a picture of Waldo. Maybe I can finally find him.

    2. Re:Is A Picture Worth A Thousand Words? by jabex · · Score: 2, Informative

      It does seem to have some functionality though. Let's say for example, this holiday I received a Thing(tm) as a present. I could take a picture or two of the Thing, and it may be easier to figure out what the it is.

      Of course, for some reason I think it would be difficult to make Visual Google function that well... the only way I would get results for my Thing would be if someone already knew what it was, and defined it for the search engine.

      --
      Like Teddy with an elephant gun.
    3. Re:Is A Picture Worth A Thousand Words? by orangesquid · · Score: 2, Funny

      The real killer app will be taking out hotornot's MeetMe funding (you know, if you want to actually talk to someone you click on, you have to pay money); you just google for their picture, and find them that way :)

      --
      --TheOrangeSquid Is it any wonder things seem so awry? We swim in a sea of confusion and don't have to think to survive
    4. Re:Is A Picture Worth A Thousand Words? by Rei · · Score: 4, Interesting

      Also, there's the whole "vaporware" issue. The scale of this programming task is staggering; it's not only image recognition, but image *searching*. Just look at how poor OCR does with handwriting (and sometimes even pre-printed text). Generalized image recognition is orders of magnitude harder than recognizing a small set of print characters lined up in nice rows and clustered into words, and image searching is beyond that.

      He can claim he's developing whatever he wants, but I'll believe it when I see it. It reminds me too much of how many AI researchers in the 60s were convinced that by the 90s computers would regularly converse with humans and be able to reason like them.

      --
      We're practicing our labials.
    5. Re:Is A Picture Worth A Thousand Words? by Stealth+Potato · · Score: 2, Interesting

      Actually, I think I read once (might have even been on /.) about another potential application for a similar technology, which seemed much more useful than this. The idea involved using images to search, say, a parts database. If you were holding some unidentified doohickey in your hand, and you needed to know what it was so you could find a replacement, you could sketch a rough outline of the object, and the sketch would be used to search through the design information in the database (say, CAD drawings and whatnot). Limited application, sure, but it's a more immediately realistic goal than full photographic image recognition. (Especially when, as you point out, the human processing is more practical in those sorts of situations.)

    6. Re:Is A Picture Worth A Thousand Words? by RancidBeef · · Score: 2, Insightful

      That reminds me of a story I heard once about some military system (using a neural net) that looked at satellite or aerial photos and selected the ones that had tanks. They fed it some photos with and without tanks. It scored almost perfectly. They decided to run another test and went out and took more pictures. This time it failed miserably. They eventually found out in the original set of pictures with tanks, it had been either sunny or cloudy (can't remember which). In the pictures without tanks it was the other way. All the software was doing was indicating whether it was cloudy or sunny. It didn't give a rip about whether a tank was in the picture.

      So, if you take a picture of a street sign in front of a museum, is it going to search for a museum, architecture that is similar to the museum building, or your location? Or the weather condition based on the sky in the background? If they can pull this off, I'll be very impressed.

  2. Better yet... by punkass · · Score: 4, Funny

    ..imagine being the guy who has to photograph EVERY STREET CORNER IN THE WORLD.

    --
    "Nobody owns the fucking words man." - James Dean
  3. Fleck's image recognition by 2advanced.net · · Score: 5, Funny

    So when do you combine this with Fleck's nude recognition algorithms to provide a service that can identify a person by partial nude picture?

    The possibilities are endless!

    --
    2advanced.net - Business Quality Hosting
    1. Re:Fleck's image recognition by rbarreira · · Score: 2, Insightful

      This paper demonstrates an automatic system for telling whether there are naked people present in an image.

      So it's not "identifying a person by a nude picture", it's identifying pictures which contain nude people...

      --

      The AACS key is NOT 0xF606EEFD628B1CA427BEA93A9CA9773F
  4. Why not use GPS Technology? by xmas2003 · · Score: 4, Interesting

    While this looks pretty cool, I'm confused by the examples provided in the writeup - "Imagine taking a photo of a street corner to find out where you are, or the photo of a city building to see its history" since GPS technology would probably be a better enabler for those specific applications.

    --
    Hulk SMASH Celiac Disease
    1. Re:Why not use GPS Technology? by neverkevin · · Score: 2, Insightful

      Or why not just look at the street signs to find out where you are? If the street corner is in a database it is probably in an area that is developed enough to have street signs.

    2. Re:Why not use GPS Technology? by nacturation · · Score: 3, Funny

      Imagine you're going through photos of your latest vacation and you find one of a street corner which you snapped while on a drinking binge. Since, in your drunken stupor, you don't remember where it was, you can just submit it to find out the history of the building and perhaps discover other famous people who have similarly vomited in that vicinity.

      --
      Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
    3. Re:Why not use GPS Technology? by drgonzo59 · · Score: 2, Insightful
      I thought of that too. Why not use GPS instead. It's a problem of machine learning and classification. Given a picture of a street corner, what features make that street corner in any light and weather conditions different than other hundreds of thousands of corners. Also, what about the angle at which the image is taken?

      The database to work will have to understand what 3D objects are (at least in the specific domain) and have an idea of what features of the object are important (like signs for example, so it will need a very good OCR system then too). That becomes a knowledge representation issue.

      There have been many projects like this before attempted. But until a computer knows what a "chair" is, or what "statue" or a "tree" is, it will just not work right. To have a computer understand concepts though is a much larger and more interesting accomplishment.

    4. Re:Why not use GPS Technology? by snorklewacker · · Score: 2, Funny

      Or submit a picture of your vomit to discover similar celebrity vomit. Or celebrities that look like your vomit.

      --
      I am no longer wasting my time with slashdot
    5. Re:Why not use GPS Technology? by bug506 · · Score: 2, Insightful

      GPS would be useful in some situations (if you want to know about a general area), but for the example of taking a "photo of a city building to see its history", GPS itself would not be sufficient.

      GPS can provide a location, but it can't pinpoint what you are looking at. This is the case even with compass data indicating which direction you are pointing your device--what if there are two things in your line of site from that perspective? (Do you want information about the building, or do you want information on the kiosk in front of the building?)

      Also, this is more generic than these examples anyway. What if I want information on the building and then on a street performer in front of the building? I could take a picture of the building, read about it, then take a picture of the street performer and read about him/her. GPS wouldn't be sufficient to tell me about the street performer, because he or she might move around the area.

  5. Man on man by sulli · · Score: 2, Insightful

    the pr0n industry is going to love this.

    --

    sulli
    RTFJ.
    1. Re:Man on man by 2advanced.net · · Score: 5, Funny

      "Man on man"? Man oh man, Freud would love this!

      --
      2advanced.net - Business Quality Hosting
  6. Or a photo of... by dmorel · · Score: 2, Funny

    Imagine taking a photo of a street corner to find out where you are, or the photo of a city building to see its history
    or a photo of your wife to see...
    oh never mind.

  7. New Query Language by Tablizer · · Score: 5, Funny

    Enter search criteria: (.)(.)

    --
    Table-ized A.I.
  8. Scary Results by The_Rippa · · Score: 2, Funny

    I'm a beta tester for this product and have gotten some scary results. For instance, I was on vacation in Yellowstone and took a photo of Old Faithful with my camera phone. I submitted it and it gave me back search results for tubgirl!

  9. Good idea but.... by DrinkingIllini · · Score: 2, Insightful

    how much harder is it to just use a regular text search for the restaraunt, movie, building, etc. that you want info on? It's like voice dialing on a cell phone, good idea, but it's about ten times faster and more effective to either dial or scroll to the name you want to call manually.

  10. It's going to be boobs by Nom+du+Keyboard · · Score: 3, Funny
    Everyone is going to try searching on a pair of boobs to see what they get. I can set it now:

    1: Take picture of current date's frontside archtecture.
    2: Submit to search.
    3: Reply: You can do better than that. Try her older sister.

    --
    "It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
  11. And we have a winner... by Momoru · · Score: 3, Funny

    Sounds like we may have a winner for Wired's 2005-2010 Vaporware awards.

  12. iDating by El_Smack · · Score: 4, Insightful

    Or taking a picture of someone and finding out their history.
    click
    "Whoa Dude!, she's been on 4 amature Pr0n sites!"

    --


    There are 01 kinds of cars in the world. The General Lee, and everything else.
  13. Yes, Imagine taking a picture by Nom+du+Keyboard · · Score: 4, Insightful
    Imagine taking a photo of a street corner to find out where you are,

    Yes, imagine that.

    1: Take picture with ultra-modern all-features camera phone of building while lost in city.
    2: Submit to search system.
    3: Search system queries phone's built-in GPS for position information.
    4: Search system sends back retrieved GPS location.
    5: Customer is absolutely blown away and immediately sends back picture of self signing virtual 10-year contract at Early Adopter prices.
    6: Profit!

    --
    "It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
  14. Stalkers... by Chordonblue · · Score: 2, Insightful

    ...are gonna love this too. Take a picture of the girl you like and do a search. This has some scary connotations I'm afraid.

    --
    "...Well, there's egg and bacon; egg sausage and bacon; egg and spam; egg bacon and spam; egg bacon sausage and spam..."
    1. Re:Stalkers... by That's+Unpossible! · · Score: 2, Insightful

      ...are gonna love this too. Take a picture of the girl you like and do a search. This has some scary connotations I'm afraid.

      Straw-man.

      Stalkers already use Google. It's a lot easier to stalk someone with text than with pictures. What are the chances your image search would actually turn up anything for your average Jane Q Public?

      --
      Ironically, the word ironically is often used incorrectly.
    2. Re:Stalkers... by kaustik · · Score: 2, Insightful

      Not in the near future, of course. I can't image that taking a picture of most of anything would produce valid results in the near future. However, this type of photographic facial recognition is already being reseached and developed for things like bank robberies and terorism. I can picture this taking off to the point where it applies to the general public...

  15. Robot potential? by CaptRespect · · Score: 2, Interesting

    Maybe this can be used for robots to recognise stuff or something like that.

  16. Hartmut Neven and hundres of other researchers... by dioscaido · · Score: 2, Insightful

    This seems less like a technology article and more like an advertisement for Hartmut Neven himself. Yes, he's built a 'google for images'... But how does it perform? How exactly is it 'ingenious'? What sets his project apart from the handful of people at almost every University with a Computer Vision research department that is tackling the problem. The problem of matching images is well known, and very difficult to solve. Even in my grad school (BU), which has a small number of computer vision grad students, there are two different research projects on this very topic.

  17. Duh by dfj225 · · Score: 2, Funny

    "Imagine taking a photo of a street corner to find out where you are..."

    Imagine reading that street sign you just took a photo of to find out where you are.

    --
    SIGFAULT
  18. Re:i can't wait for this! by soricine · · Score: 3, Insightful
    Image recognition is immensely more difficult than people seem to think. And yet every few weeks someone is claiming to be just around the corner from a system that can easily identify the contents of an image.

    When your brain 'recognises' what it is looking at, it is doing a lot more than just comparing two images (as in the street-corner example from the article). Your brain simply doesnt operate in terms of bitmaps.

    The fact that he is basing his hyper-vaporous product on facial-recognition software should set of alarm bells. Facial-recognition in a real-world context has consistently failed to be of any use at all, although it may work fine under lab conditions.

    If all the money invested so far hasn't made a computer able to successfully recognise a subset of the visual field (faces), why should I believe in a machine that is able to recognise practically anything?

  19. Imgseek. by Adhemar · · Score: 5, Informative

    Imagine you're a photographer. Professional or hobbyist, I don't care. You have made thousands of pictures; they all are on your hard drive.

    Imagine you're lazy. (Maybe you don't have to imagine that.) You don't want to describe your photos, you don't want to label them. The only metadata associated to your photos is date and time.

    Imagine you're looking for a particular photo. You know where you'va taken it, you know what is on it, you can remember the subject, the color shades, etc. You just can't remember exactly when you took that picture. How do you search for it?

    Well, you quickly make a drawing in which you try to (sort of) replicate colors and shapes. And you let your computer search for "similar" graphics.

    Such software exists already (for quite some time). There's a beta Free software project (GNU licenced) called imgseek . Current version: 0.8.4. I haven't tried it, I don't know how good it is. But this screenshot looks impressive.