Computer Scientists Scour Your Holiday Photos

Barence writes "Hundreds of thousands of images on Flickr are being used to teach a program to determine the geographic location of an image, simply by looking at it. The program attempts to mimic the way that humans can deduce the location of an image by searching for visual clues, such as similarities to pictures or locations they have seen previously. In its current state it can guess the location of a photo to within 200km, 16% of the time — extremely accurate given the complexity of the problem."

Comments

Within 200km, 16% of the time?

[14erCleaner]

I'll guess...New York City, without even looking at the pictures that should get me in that ballpark.

This is very hard

[mzs]

Look at this set of pictures:

http://htmlhelp.com/~liam/Hawaii/Kauai/WaimeaCanyon/

Would you know simply by looking at the photos without the sign that this was not say the grand canyon? The whole correct to 200 km aspect is troublesome when the state of the art in computer vision cannot yet even answer that this is a picture of a canyon.

Re:This is very hard

[SpinyNorman]

Would you know simply by looking at the photos without the sign that this was not say the grand canyon?

Yes, because the GC doesn't have so much vegetation growing down the sides.

The whole correct to 200 km aspect is troublesome when the state of the art in computer vision cannot yet even answer that this is a picture of a canyon.

The two things arn't related. You don't need to know it's a canyon to be able to locate it - you just find the closest match in your database and give that as the location. You only need to be able to assess degree of similarity. No need to start reading signs or logically analysing the picture for clues.

Getting the location right to 200 km doesn't really say anything about the accuracy of your matching algotithm - it just says that:

a) You have a database with at least one picture per 200Km radius

b) Enough locations (at least in your database) look sufficiently distinct that "distant but similar" matches occur infrequently enough not to drag your success rate below 16%

Missing double blind

[RichMan]

From the looks of the test selecting London all the time would have a
1/6 chance = 16.67% chance.

They need better double blind testing and a more diverse set of geographical locations.

Re:Missing double blind

[zappepcs]

It's a bit worse than that I think. Trying to identify location of a picture by looking at it in the way that humans do requires that you know the location. As an example of why this implies intimate knowledge to be useful, everyone knows of the big statue of liberty. Not just anyone can guess that your holiday picture with the non-descript base in the background was taken at the base of the statue of liberty. The same goes for > 90% of other places in the world.

Another example: The forests on planets on the show Stargate One, are they in Missouri, Montanna, Canada? Just looking at them will not necessarily tell you anything unless you are intimately familiar with the actual location.

A photo in Syntagma Square in Athens may look like it was taken in Central Park in NYC if not enough of the background was included. It will take huge amounts of data and photos to get anywhere close to what a human can do at this job, and even then it is limited to only what it has seen before.

Other knowledge plays a part too. London bridge is now in Arizona (I think) as it was moved brick by brick and re-assembled. Seeing the bridge does not now mean you know where it is .... it's a trick question. The point is that you need additional information as well. A picture that is a beautiful park setting that has a kangaroo in it? is it in Australia, or a zoo? Additional information is required.

Hats off to them for working on it. It's a tough problem.

Re:Missing double blind

[Woundweavr]

Accents used to be much more pronounced pre-radio/mass media. I know in Boston, individual streets had slight variations so that you could tell the neighborhood of a person by their accent. However, now that a large percentage of the words we hear are from movies/TV/radio our accents get washed out.

Random pick is correct ~8% of the time

[Anonymous Coward]

Surface Area of Earth
510,065,600 km2
*.33 = 170021866 \\Estimate 2/3 of earth is ocean
/ 3.14 * 200^2

=8%

Re:Where pictures are taken

[hagnat]

exactly, its like a d6 die... if i have to identify one (and only one) picture i'll have a 1d6 chance of getting it right

but when you have to roll this for 200 cities, also chosen by a 1d6 roll, you have two dies being rolled 200 times, and you want to know how many times both dies have the same value

Problem with flickr geo-tagging

[stoofa]

The problem with geo-tagged flickr photos is that in many places the detail on the maps and aerial shots provided isn't defined enough to allow an accurate placement.

The even bigger issue is that, although some cameras now have GPS, the majority of geo-tagged shots are placed manually by humans who often get it wrong or deliberately place their photos onto a more popular location just to increase their traffic.

Re:Where pictures are taken

[Rich0]

If you picked a random point on the globe, and I picked a random point on the globe, then they would be within 200 miles of each other a few percent of the time. If the only logic used by the software was to determine whether or not any land was visible it could probably increase that probability significantly - the earth doesn't have that much dirt poking out of the oceans. 200 miles is a VERY large area of land.

Actually, he kinda understands....

[raehl]

The dice analogy is right-on.

The problem is he just doesn't seem to realize that the chances of throwing doubles are 16.66%.

Re:Where pictures are taken

[Belial6]

Now add to that, the fact that populations tend to bunch together, and you can massively increase your odds of those two point being within 200 km of each other. This is without any image recognition at all.

With the most basic of image recognition, you could narrow things even farther with things like, "Is there ocean in the picture?", "what is the height of buildings in the background?", or "how many people are in the background". One almost needs to ask how they got their accuracy so low...

Re:Where pictures are taken

[raehl]

Actually, the earth is pretty big - you'd have only a 0.0246% of being within 200km of someone, counting water. Get rid of water and you get to around 0.075%.

Re:It helps..

[gstoddart]

Do you think it helps or hurts that my photos on Flickr have titles like "Tokyo - Ueno park"?

For the researchers, it probably helps. They chose pics that had either GPS or location information -- so they could manually verify where the photos originated.

If they started out with a bunch of pics they didn't have any location information about ... they'd never be able to measure their results. ;-)

Cheers

Re:Where pictures are taken

[1729]

That's if you choose points at random. If you only choose points corresponding to cities with large populations that frequently use internet photo-sharing sites, then your chances of being within 200km of the location become much better.

Re:Lies, Damned Lies, And...

[raddan]

Your post made me think of something-- the Earth is really BIG! But I'm a nerd, so I had to prove it to myself.

The surface area of the Earth, not counting water, is 510,072,000 km^2, according to Wikipedia. So, that's roughly 5.1 x 10^14 m^2. Again, according to Wikipedia, there are currently 6.67 x 10^9 people on Earth. That translates to about 7.65 x 10^4 m^2 for each person! In terms that Americans can understand, that's roughly 14 football fields (or to choose a landmark close to me, a little bit larger than Fenway Park). When you consider the fact that metro areas often have millions of residents, you realize that we're pretty lumped together, distribution-wise.