Slashdot Mirror


Finding Yourself With Photo Recognition

itchyfish writes "You are lost in a foreign city, you don't speak the language and you are late for your meeting. What do you do? Take out your cellphone, photograph the nearest building and press send. For a small fee, photo recognition software on a remote server works out precisely where you are, and sends back directions that will get you to your destination. Seems a little far fetched, but amazingly cool if it really works."

30 of 336 comments (clear)

  1. GIS technologies by BWJones · · Score: 5, Informative

    The software then looks for useful features, such as the corners of windows and doors, and extracts the colours and intensities of the pixels around them. Next, it searches the image database for matching data, using the base station the cellphone's signal came from as a guide. Finally, it uses the differences between the two images to calculate the photographer's position.

    To me, it would appear that an easier solution might be to use GIS data in combination with the cell phone signal and comparisons of rough morphological features of buildings. The instructions should simply be: Point your camera at a building near you so that you can approximate its outline and then send that image. This would scale much larger than the methods referenced in the article as you would not have to store every detail of the buildings surrounding you including pixel maps of textures and color. This approach could be handled for a large city by a few commodity servers whereas the other approach would require significantly more computational resources.

    Imagine how difficult it would be to capture details like that in a major city such as NYC? I don't really need directions to find my way around Cambridge city center as you could almost throw a rock from the center and hit just about every building around, but London, Washington, Houston etc... are another story and the data required from their approach would require massive computational infrastructure.

    --
    Visit Jonesblog and say hello.
    1. Re:GIS technologies by orthogonal · · Score: 4, Insightful

      Imagine how difficult it would be to capture details like that in a major city such as NYC? I don't really need directions to find my way around Cambridge city center as you could almost throw a rock from the center and hit just about every building around, but London, Washington, Houston etc... are another story and the data required from their approach would require massive computational infrastructure.

      And I fear that there won't be enough "lost tourists" to make this a paying proposition.

      But how much would it be worth to professional historians -- or to Hollywood, or just to you personally -- to be able to "walk" through a virtual representation of New York circa 1890?

      Or London in Holbein's time?

      This is one of those projects -- much like something called ARPANET, which had to rely on government handouts but later made some guy named Steve Case a fortune -- that will never fund itself but will be of literally incalculable value to posterity.

      Let's be realistic -- physics and fanaticism not being mutually incompatible, eventually "freedom fighters" -- whether named Atta, McVeigh, or Patrick Magee -- will make bee-lines for our biggest cities, carrying suitcase filled with two precisely machined hemispheres of plutonium. And everything but the maps of those cities will be lost.

      Hopefully the Cambridge researchers will by then have completed their -- apparently -- quixotic project, and we will at least have, in redundant storage, a rather precise picture of what will be lost to radioactive ruin, a snapshot of urban life in the twenty-first century.

    2. Re:GIS technologies by Broege · · Score: 4, Insightful

      To me, it would appear that an easier solution might be to use GIS data in combination with the cell phone signal and comparisons of rough morphological features of buildings.

      For most people even photos aren't necessary. Using the data from nearest cells a mobile could be pinpointed within 100 m accuracy (in urban terrain, it degrades down do about 1000 m in rural terrain). It's enough if you are sent a map with the neighbourhood - the biggest problem then is the size of your mobile's screen.

      Such services (sending a map of the neighbourhood, with interesting points, like ATMs, marked) already exist in Poland (Europe). I find them quite helpful.

      --
      homepage: www.tls.pl
      signature: not found
  2. So rather than send out... by Throtex · · Score: 4, Insightful

    ... a GPSr reading and request information about that location, we get this?

    Sounds like a solution in search of a problem.

  3. GPS? by ajiva · · Score: 5, Insightful

    Some problems that I can see with it. What if your in a flat area with no buildings, landmarks, etc? Or even worse, a very rocky, natural area (say something similar to the grand canyon). And even barring those problems, wouldn't GPS just plain be easier? Take the same concept, have the phone grab its GPS location, have you enter the address you want to go, and both pieces of information are sent and the phone gets its route to get you where you want to go (with associated fee). Seems cool, though.

    1. Re:GPS? by ciroknight · · Score: 5, Interesting

      Gotta remember one thing; Civilians aren't in control of GPS. GPS's accuracy can be degraded at any time on the US Military's whim. Same with any of the other networks that are currently being built by different government agencies. IF there were a Civilian GPS, then this would /almost/ always be a solution. But what if there's a solar flare? What if there's some other feature about the region blocking satelite traffic, but not wireless traffic (bad weather maybe?)?

      I believe this could actually be really cool if we get it to work, especially in an urban environment, but not so much out in the desert or anywhere; it's not meant for that. Instead, it's for finding that office building in Portugal when you're about to be late for a business appointment, and yet you've never been to Portugal before.

      --
      "Victory means exit strategy, and it's important for the President to explain to us what the exit strategy is." G.W.Bush
    2. Re:GPS? by grahamsz · · Score: 4, Informative

      Even old-style degraded GPS let you get your position to within a few hundred feet. Assuming that a map was returned with a 400 sq ft circle instead of a "you are exactly here" any half intelligent person could figure it out.

      Anyway most cellphone networks can triangulate your position to within a block.

      As for photo recognition being MORE accurate, i cant see how. To get your position to within a few hundred feet you'd need to know the exact parameterization of the lens, the zoom, the angle of the camera... unlikely.

      Getting GPRS to work correctly in a foreign country so that you can make such a request is hard enough to begin with.

    3. Re:GPS? by PacoTaco · · Score: 4, Funny
      What if your in a flat area with no buildings, landmarks, etc? Or even worse, a very rocky, natural area (say something similar to the grand canyon).

      Then I think you're going to be late for that meeting.

    4. Re:GPS? by grahamsz · · Score: 4, Interesting

      But if we build a new global network where each phone has exactly the same hardware... then why not build our own GPS-like network instead.

      The system being open-source is also not a big deal. Sure it'd be nice to have an OS library which could find similar images of buildings but the real value would be in the dataset which almost certainly wouldn't be free.

      Also this makes no consideration for similar buildings. The company i work for has a campus where 5 of the 7 buildings are cookie cutter - how would it deal with that situation.

      Nokia has a street in helsinki with a whole bunch of identical buildings... same problem.

      What about mirror glass buildings?

      Sure it might work great if you are lost outside the transamerica pyramid, or the flatiron building, or maybe the houses of parliament but god help you if you are lost in the latest "homely community for comfortable family living"

  4. try it in my neighborhood by matlantis · · Score: 4, Funny

    Every house looks the same where I live

  5. Problems by Caedar · · Score: 5, Insightful

    If the photo is taken and the data on the server isn't updated very frequently, couldn't items like cars and other movable objects interfere with the location calculation, as it bases it highly off of the details of the location?

    1. Re:Problems by bobbis.u · · Score: 4, Informative
      This is clearly a problem with object recognition and there are certain techniques for doing this that reject the "clutter" in the scene. I'm not sure exactly how this system would work in 3D, but when viewing a planar scene (i.e. objects flat on a table), you can calculate "invariants" associated with the objects in view. Essentially, an invariant is a viewpoint independent representation of the object.

      You would clearly have a library of objects (e.g. buildings) on the servers. When a picture is sent, the service would perform some sort of feature extraction, and calculate the invariants of the objects in the scene. It would then see if these objects nearly matched any in the database. If they did, it would project possible matches onto the image and look for edges around the model. If there was good correlation (accepting the fact that the match would not be perfect because of moveable objects) it would return the name of the building.

      Prof. Cipolla lectures me on (suprise, surprise...) Computer Vision. You can find his lecture handouts here. (the projection handout, page 46 onwards talks about the process I have just described.)

  6. Ideally by phiz187 · · Score: 5, Insightful

    Ideally, the much sought after all in one convergence device (fone/pda/etc.) would have built-in GPS, negating the need for this otherwise functionally sound service.

    I wouldn't invest to much into this technology, as I think it'll be obsoleted before it comes to fruition.

    -PHiZ

    --
    Pretend I said something meaningful or insightful here.
    1. Re:Ideally by ciroknight · · Score: 4, Interesting

      Once again, as we saw in 2002, GPS quality can be degraded at any moment, even taken wholly offline. Not to mention the "Act of God" possibilities to knock it offline *metorites, solar flares, etc etc*.

      Meanwhile, investing in this technology gives us a reason to improve image detection and image processing. It gives us a reason to build the technologies needed to digitially map our world, which could be useful for anything and everything, including finding the best way out of a highrise during a fire, or even police task forces on drug busts... there's really no end to what a Digital Map can do, that GPS just will never have the capability of doing.

      --
      "Victory means exit strategy, and it's important for the President to explain to us what the exit strategy is." G.W.Bush
  7. Won't work by ExCEPTION · · Score: 5, Funny

    I can just image the server's cpu goes up to 100% when I send in a photo of McDonald's.

  8. Oh good lord. by PedsDoc · · Score: 5, Insightful

    Is this really necessary? Not everything needs to be so darned tech, you know... maybe we should just get a map and use that.

  9. GPS Does This by crass751 · · Score: 4, Interesting

    Where I worked this summer, I had an iPaq with a few software packages installed on it to do GIS tasks. One of the packages was ArcPad from ESRI and the other was StreetMap for ArcPad also from ESRI. When connected to a GPS unit, you could tell ArcPad a destination, and it could either use your current position or one that you entered to calculate driving directions. The accuracy of the maps was amazing, we went out and road tested them (read: drove around with the GPS unit on the truck and compared our path to the roads on the maps) and there was little or no discrepancy between our actual path and the street layer on the PDA. This seems much better than taking a picture of a building that looks like thousands of other buildings in the world. Interesting idea, just not very practial.

  10. So, let me get this straight... by Spiffae · · Score: 4, Insightful

    There's a technology, that can take a poorly shot digital photo, and then match it to a database of images of every building in the world, come up with a single match, and then let you know where you are?

    Does such a database exist? Could it possibly work without bringing up false positives? I mean, I don't have figures, but there are millions of buildings in any large urban area, and within those millions, they all have multiple sides, and then they all look radically different at different times of day. We're talking storage space that seems like it would be incredibly dificult to manage, let alone search efficiently and return good results from a cell-phone camera image.

    Count me as a skeptic.

  11. massive computational infrastructure by Black+Mage+Balthazar · · Score: 4, Funny

    Turns out the company owns stock in a number of hard drive manufacturers...

  12. Hand gestures? by mark_space2001 · · Score: 4, Insightful
    You are lost in a foreign city, you don't speak the language and you are late for your meeting.

    Find the nearest native, start talking and gesturing wildly. Point at a map or street sign and say the name of the place you are looking for. They'll figure it out.

    Sorry I just don't see this one.

    1. Re:Hand gestures? by Jardine · · Score: 4, Funny

      Make sure to speak English very slowly too. All foreign people actually understand it as long as it's spoken slowly.

  13. What if... by Bill_Royle · · Score: 4, Funny

    What if you took a picture of a McDonalds?

    Hell, you could be lost for days.

  14. What do you do? by $exyNerdie · · Score: 4, Insightful

    You are lost in a foreign city, you don't speak the language and you are late for your meeting. What do you do?

    You start thinking about what the hell is this that is so important that you go to a foreign country to have a meeting where people don't understand your language and you bet all your chances on the assumption that your cellphone will find the carrier that will allow you data transfer without a subscription plan. If the meeting is so important in the foreign land, I would think that you would do little more homework than to just depend on a cameraphone!!

    1. Re:What do you do? by tftp · · Score: 4, Insightful
      If the meeting is so important in the foreign land, I would think that you would do little more homework than to just depend on a cameraphone!!

      If I am in a country which language I do not understand, my secret plan would be to take a taxi cab from the hotel (and back.) If the meeting is so important, I can not trust an inexperienced traveller (myself) to deliver me to the location and back.

  15. Make sure you take a good picture... by jea6 · · Score: 4, Funny

    Make sure you take a good picture...of the closest street sign.

    --

    sarchasm: The gulf between the author of sarcastic wit and the person who doesn't get it.
  16. It's cheaper than the current solution.... by raehl · · Score: 4, Funny

    By the time I total up dinners, movies, gifts, time and emotional distress, this would be *MUCH* cheaper, convenient, and more portable than having a girlfriend willing to stop and ask for directions.

  17. Why Bother? by windside · · Score: 5, Informative

    This seems like an overly complicated solution. At the moment, my phone in Japan has a feature where I logon to Vodafone's website (from the phone) and click through a couple of links and then it tells me where I am. I assume it gets this information by figuring out which cell the phone is dialing from. From the subsequent menus, there are various options like "find the last train to station X", "find the nearest place to catch a taxi", and so on. A few months ago it was only available in Japanese, but now they've introduced a bilingual version - hoochie mamma.

    Why bother using the fancy-dancy image recognition software when cellular telephony has a built-in system that basically acts like a constantly-updated "user location" variable?

    (Actually, the answer is simple - to make geeks foam at the mouth. Come on now people!! Excess ain't rebellion.)

    --

    --
    ...Whether my Maker is prepared for the great ordeal of meeting me is another matter.
    Churchill
  18. Same Same but different. you can do this today. by VC · · Score: 4, Interesting

    Here in the UK, try this. Get your cell phone (or mobile as we say here) dial 2580.
    Hold the phone up to the radio till it gets disconnected.
    Wait.
    A text message will arrive with the name of the song.

    It costs about 50p. Disclaimer i do not work for or have any involvment in this venture, except friends who built it.

  19. The inventor's 2 cents worth by dpr20 · · Score: 5, Informative

    Dear All,

    Wow! Thank you very much for all your comments on this mobile phone navigation system. I thought I'd throw in my 2 cents worth since I'm one of the people who invented it! Forgive the lack of structure in what follows, but I'm trying to address several different issues raised throughout this discussion...

    Yes, another way of doing this is radio signal triangulation (including GPS). But actually, this method doesn't work too well in cities because of things like multipath effects and satellite visibility (BTW our system isn't designed to work outside urban environments). GPS car navigation systems rely on a combination of GPS and inertial sensors, i.e. they take a sort of average of a large number of inaccurate readings to get a good fix on position. But the simpler positioning strategies are unlikely to give good enough acccuracy to establish on which side of the street you are standing (and in any case, they don't tell you whhat direction you're looking in). GPS is also expensive: most people would not be prepared to pay more for a phone with an in-built GPS receiver - but camera phones are already selling well.

    No, we're not going to build a database of every building in the world! But a good place to start would be large city centres. FYI what motivated us to invent this system was the familiar problem of getting lost outside London tube stations. Obviously I know which tube station I'm at but I don't usually know which exit I took or what direction I'm facing. Of course I can retrieve a local map via my mobile phone. But the problem is I'm missing that critical "you are here" dot that tells me where to start. This is where our system comes in: by providing the dot (well, an arrow actually because it tells you which direction you're looking in too).

    In practice, builing a database is easier than you might think. Probably we could do it with nothing more than a video camera attached to a car. Granted someone will have to drive down the streets of interest but only once (and this shouldn't be too difficult in somewhere like New York).

    Finally, no, movable objects don't cause too many problems. The system uses a feature based strategy that is robust to 'clutter' in the form of things like cars, pedestrians, changing shop window displays, etc. That being said, there will always be ways of confusing it, e.g. by demolishing a building. But supposing that picture messages will one day cost as little as text messages do now, a system that works almost instantaneously and gets it right 99% of the time sounds as if it might have some commercial potential at least. And what if the hypothetical tourist isn't lost but just interested? For example, the system could return information about the history of any building of interest in the middle of Venice.

    Yours,
    Duncan Robertson

  20. Hotels are good for directions by erroneous · · Score: 4, Insightful

    "You are lost in a foreign city, you don't speak the language and you are late for your meeting. What do you do? "

    You go into the nearest hotel and ask the nice English-speaking person behind the reception desk.

    Even on Mars the hotel receptionists speak perfectly-accented English.

    --
    erroneous: look me up in a dictionary