Flickr Search Hack Powered by Mouse-Made Doodles

Carl Bialik from WSJ writes "Retrievr gives budding artists an impractical but addictive way to find photographs on Flickr: a search engine powered exclusively by mouse-made doodles. From the article: 'Retrievr, Mr. Langreiter says, "doesn't look at specific forms." Art history buffs might like to think of it as photo-search by way of Impressionism. The Retrievr engine dissects a photo like a gallery connoisseur who lost his bifocals: It focuses on regions of colors rather than specific shapes and lines. "It is, actually, a simple scheme," says Mr. Langreiter. Retrievr creates and stores a compact representation of each photo in its database. The system pulls only the most important features — broad shapes, blocks of color and spatial relationships between different colored areas — out of detailed images to create shorthand approximations of every photo. (The storage mechanism extracts the 120 "strongest" features from an image to create something called a "wavelet transform," which contains much less data than the photo itself and facilitates lightning-fast searches.)'"

Comments

Flickr Retrievr

[tonyr1988]

Direct Link

Requires Flash.

Re:Flickr Retrievr

[RuBLed]

It's powered by "mouse-made doodles" and apparently you're not doodling enough. :)

Re:Flickr Retrievr

[HUADPE]

FTA

In its current incarnation, Retrievr runs on a single computer.

Ow. The Slashdotting. It hurts.

Re:Flickr Retrievr

[inKubus]

A single mound of molten slag, you mean.

That was quick

[Centurix]

I think we just made the world record for the most number of boobies sketched out on the internet simultaneously.

Re:That was quick

[kbob88]

either that or, knowing the audience, the most number of cool-looking dual AMD Opteron Linux boxes sketched out!

Re:That was quick

[The Amazing Fish Boy]

I wish you could take a peek at what other people are "searching" for with this tool at the same time; it would no doubt be profoundly entertaining and troubling.

Yeah that would be great, but I guess we'll just have to wait until AOL buys them out.

Combo of Retrievr & Online Dating

[kbob88]

Hmmm... Think about the combination of this and online dating sites! Especially if I could upload a target photo instead of sketching! ... I think I have some old Cindy Crawford JPGs laying around here somewhere (*dream on*).

Re:Combo of Retrievr & Online Dating

[Psycosys]

You can upload an image. Go to the site and click Image rather than sketch.

That was quick

[5of0]

Already partly slashdotted. Very slow and sometimes you don't get in.
But this is an interesting idea, fun if nothing else.
I drew a tree and I got a pineapple with a guy's face in it, a chinese guy standing in front of a gate, and a dragonfly. Maybe I need to brush up on my drawing skills.

*groan*

It's been done before

[pyite]

I read about this a little while ago. Same principle. It uses a Haar transform (for those unfamiliar with multimedia signal processing and wavelets, specifically, the Haar transform is a specific wavelet transform based on the Haar wavelet and the associated orthogonal basis). The idea is that you compare the low frequency component of an image to the low frequency component of a rough drawing (which is pretty low frequency to begin with) and they should be pretty close of the images have anything in common.

Re:It's been done before

[pyite]

Haar wavelet though? While it's easier computationally (since the mother/father wavelets are peicewise linear in the 1D case) I always saw it as being a "lesser" wavelet in the sense of compression/reconstruction quality and ability to discern edges/other dramatic changes in data

I'm just saying what imgSeek uses. It's certainly a very easy wavelet to implement via lifting. I think it's probably used because more complex wavelets wouldn't be of any help since the rough drawing is so rough to begin with. In the end you could probably do the same thing with a DCT. Wish I had time to experiment.

Re:It's been done before

[pyite]

Hooking up with an online collection of photos might be new I guess - but that seems an obvious thing to do.

Yea, I hate to preempt any of the people who have come up with things like this, but I hope no one tries to patent any of these ideas. It's sort of a process that's implicitly defined by the existence multiresolution image decomposition. We shall see.

Applied to museums?

[andphi]

To find Van Goghs, draw a whirlpool.
To find Pollocks, draw a can of paint.
To find Warhols, draw four cans of paint.
To find modern art sculptures, throw the tablet against a wall.

Re:powered by mouse-made doodles

[dch24]

Okay, queue the slashdot mouse jokes.

First they created mice who don't age.
Not realizing what they were doing, they placed live human brain cells in the mouse's brain.
Then they made the mouse always happy, and gave it its own complete genetic map.
Suddenly we find a new mouse species.

And now they can search the web better than humans. Will someone please welcome our new overlords?

Other flickr Mashups

[vijaykiran]

This is very old .. I read about this first on webmonkey in Feb.

Ten Best Flickr Mashups
by Michael Calore 24 Feb 2006

Here's the link: Ten Best Flickr Mashups

Re:powered by mouse-made doodles

[Asm-Coder]

I, for one, would like to welcome our new, genetically perfected, websearching, mouse overlords. On the other hand, they haven't done any of these things on their own yet, so, obviously they don't have AI yet.

Rating the doodles

[inKubus]

Keep in mind that there's a rating system for the doodles also.. there's some pretty cool artwork in there, as well as 50% boobies, dicks and strange V shapes (everyone draws them a little different). Pretty fun, it's under the Art of Retrivr

Feature Vector

[ArikTheRed]

120 features get mapped into a feature vector, effectively pinpointing a position in 120 dimensional space. All of the other images are indexed in the space, and it's a simple nearest-neighbor search to find the best matches. The interesting thing here is that funky things happen to space when you are in very high dimensions, and without creative indexing, it may be just as quick to do a scan and compare against the whole database. Obviously, not optimal. That's what they mean by "simple", since some multimedia search systems deal with indexes of thousands of features - thousands of dimensions.

wavelet first

[mennucc1]

just for the record: the post claims that ``the storage mechanism extracts the 120 "strongest" features from an image to create something called a "wavelet transform"'' but this is quite misworded. Indeed if you look into the original research project, you see that ``the algorithm performs a wavelet transform on every image, and then collects just the few largest coefficients from this transform''.

O-o-old

[remmelt]

http://www.boingboing.net/2006/01/04/drawing_inter face_fo.html Still cool, but it's flopped in the Flickr Community because it's not that good at actually finding the pictures. It's more colour based than shape based.

Re:powered by mouse-made doodles

[umbra_dweller]

Silly human, we ARE the AI. Maybe this means we're close to finding the ultimate question...

Urg

[hyfe]

(The storage mechanism extracts the 120 "strongest" features from an image to create something called a "wavelet transform," which contains much less data than the photo itself and facilitates lightning-fast searches.)'"

If you're going to simplify, atleast get it somewhere near correct. A wavelet transform doesn't extract features. Features is a human-made concept. A wavelet transform is simply a transform (or for this purpose, a very lossy compression algorithm), very similar to the Fourier Transform, except that it has locality which is why it performs soo much better on non-uniform data.

I mean, 'something called a wavelet transform'. A short explanation linking it Fourier might have been apt, but wavelets are hardly voodoo.

'facilitates lightning-fast searches'.. oohh, thanks for telling us. I would never have guessed that after transforming the data down to 12 vectors, searching would be a lot faster. I mean, if they actually had indexed the data in a clever way or something specifically to speed up searches, this sentence would have made sense.. but they just transformed it. It's not voodoo and market-speech is bad!

Seriously though...

[StressGuy]

This can be a problem. For example, I won't be able to check out the actual demo until I get home because I don't know what image it may pull to my work desktop.

For those of us that run xscreensaver, one of the hacks goes out and gets random images from the web. I had to turn that one off because, random or not, it was getting porn everytime it ran. Now, I'm not a prude, I'm not concerned if my 5 year old son sees the image of a naked woman on my computer screen. To have a problem with that is to have a problem with the human body and I'm not going there. However, if said naked woman is in the process of strapping on a rubber penis....well....that's a bit hard to explain. Alternatively, if it's an image of a person (male or female) being abused....you get the idea.

This leads in to why I support the idea of .xxx domains. If you want access to it, fine with me. All I ask is that you make it easy for me to discriminate what my kids are exposed to. I want them to be able to use the internet and me to not have to worry about what they might find.