Slashdot Mirror
Flickr Search Hack Powered by Mouse-Made Doodles
Carl Bialik from WSJ writes "Retrievr gives budding artists an impractical but addictive way to find photographs on Flickr: a search engine powered exclusively by mouse-made doodles. From the article: 'Retrievr, Mr. Langreiter says, "doesn't look at specific forms." Art history buffs might like to think of it as photo-search by way of Impressionism. The Retrievr engine dissects a photo like a gallery connoisseur who lost his bifocals: It focuses on regions of colors rather than specific shapes and lines. "It is, actually, a simple scheme," says Mr. Langreiter. Retrievr creates and stores a compact representation of each photo in its database. The system pulls only the most important features — broad shapes, blocks of color and spatial relationships between different colored areas — out of detailed images to create shorthand approximations of every photo. (The storage mechanism extracts the 120 "strongest" features from an image to create something called a "wavelet transform," which contains much less data than the photo itself and facilitates lightning-fast searches.)'"
Direct Link
Requires Flash.
I think we just made the world record for the most number of boobies sketched out on the internet simultaneously.
Task Mangler
20 minutes of expert artistry
are there no nipples on flickr?
Hmmm... Think about the combination of this and online dating sites! Especially if I could upload a target photo instead of sketching! ... I think I have some old Cindy Crawford JPGs laying around here somewhere (*dream on*).
Already partly slashdotted. Very slow and sometimes you don't get in.
But this is an interesting idea, fun if nothing else.
I drew a tree and I got a pineapple with a guy's face in it, a chinese guy standing in front of a gate, and a dragonfly. Maybe I need to brush up on my drawing skills.
*groan*
You all have Oo.o and Firefox, so get World Wind.
I read about this a little while ago. Same principle. It uses a Haar transform (for those unfamiliar with multimedia signal processing and wavelets, specifically, the Haar transform is a specific wavelet transform based on the Haar wavelet and the associated orthogonal basis). The idea is that you compare the low frequency component of an image to the low frequency component of a rough drawing (which is pretty low frequency to begin with) and they should be pretty close of the images have anything in common.
"Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman
Error in application retrievr
Maximum Thread count reached.
Sounds like a pretty high risk factor for Hanta virus infection :P
To find Van Goghs, draw a whirlpool.
To find Pollocks, draw a can of paint.
To find Warhols, draw four cans of paint.
To find modern art sculptures, throw the tablet against a wall.
No, just kidding...
Oops I made the search xplode
Do not try to read the dupe, thats impossible. Instead, only try to realize the truth
What truth?
There is no dupe
Here's the link: Ten Best Flickr Mashups
Vijay Kiran
I blog, therefore I am.
It should be obvious how widespread the potential uses of a viable version of this technology would be. It's obviously got a ways to go (#1 on the list would be returning something besides a blank right column when Slashdotted), but comparing this to Google is not an exagerration by any means. Image/Video searching is the search engine market of the future, and even detailed meta-data couldn't compare to the flexibility and potential accuracy of search-by-visual-similarity.
Keep in mind that there's a rating system for the doodles also.. there's some pretty cool artwork in there, as well as 50% boobies, dicks and strange V shapes (everyone draws them a little different). Pretty fun, it's under the Art of Retrivr
Cool! Amazing Toys.
I, for one, welcome our new doodling-mice overloads.
...how does this help me find pr0n?
This is my footer. There are many like it, but this one is mine.
For a second there I read "Mouse-Made Doodies"...
Finally, a good use for surplus mouse poop!
120 features get mapped into a feature vector, effectively pinpointing a position in 120 dimensional space. All of the other images are indexed in the space, and it's a simple nearest-neighbor search to find the best matches. The interesting thing here is that funky things happen to space when you are in very high dimensions, and without creative indexing, it may be just as quick to do a scan and compare against the whole database. Obviously, not optimal. That's what they mean by "simple", since some multimedia search systems deal with indexes of thousands of features - thousands of dimensions.
http://www.coderoshi.com/
Some kind of first-pass search system to find rough matches between Flickr/Youtube/etc posts and copyrighted material will be a big win for the MPAA. There are ways to align and compare pictures, but they're computationally expensive and compare two images, they don't do a general search. This thing might be usable as a first search used to find possible matches, which then get a more detailed examination by the expensive algorithm.
I tried to draw boobs and got this instead http://www.flickr.com/photos/19406332@N00/41144732
I think it needs some bugs worked out. It searches as well as a search engine.
Interesting technique. I used things called Active Shape Models in my MSc thesis which do pretty much the same kind of thing if I understand the article correctly. I've since lost touch with the academic research. Can anyone who still works in the area give me a quick run down of the differences and pros/cons?
Bob
Listen to my latest album here
whatever I draw, this picture at http://www.flickr.com/photos/96302514@N00/58916441 / is always in the list of results!
just for the record: the post claims that ``the storage mechanism extracts the 120 "strongest" features from an image to create something called a "wavelet transform"'' but this is quite misworded. Indeed if you look into the original research project, you see that ``the algorithm performs a wavelet transform on every image, and then collects just the few largest coefficients from this transform''.
But don't take my word for it; see for yourself! http://i11.tinypic.com/48bt477.png
http://www.boingboing.net/2006/01/04/drawing_inter face_fo.html
Still cool, but it's flopped in the Flickr Community because it's not that good at actually finding the pictures. It's more colour based than shape based.
I mean, 'something called a wavelet transform'. A short explanation linking it Fourier might have been apt, but wavelets are hardly voodoo.
'facilitates lightning-fast searches'.. oohh, thanks for telling us. I would never have guessed that after transforming the data down to 12 vectors, searching would be a lot faster. I mean, if they actually had indexed the data in a clever way or something specifically to speed up searches, this sentence would have made sense.. but they just transformed it. It's not voodoo and market-speech is bad!
"" How about taking the safety labels off everything, and let the stupidity-problem solve itself? """
I wonder, is Google looking at these alternative search methods? He mentions in the article that to do this to the entire Flicker would take 'some sort of Google-like infrastructure'...
I used to assume that, being the current king of text search, Google would be expanding into more intelligent media searches, like this kind of 'similar image'-search, automatic tagging (like automatically indexing a picture 'nature' and 'winter'), and searching for songs by entering a couple of notes.
They never did though, and instead began diversifying with maps, mail and office packs. They've been talking about using fingerprint technology in Youtube, though, so I guess we could get some fallout from that.
In a fair world, refrigerators would make electricity.
Is it just me, or does this thing always find nothing that actually looks like the picture? I drew a blue sky, green grass background, and then a brown tree trunk with dark green leaves... Just all blobs, basically. Not a single one of the pictures had a tree in it. They didn't look ANYTHING like the 'picture' I drew.
"If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
As usefull as a broken toaster. Only good porn hits will save thee
"What are we going to do tonight, Brain?"
"Same thing we do every night Pinky, try and take over the world! We shall create millions of doodles which will keep the world's intelligence services occupied long enough for us to take over without opposition."
It seems like if I do a search by an uploaded file, and I upload (or point to the URL of) an image that I have posted on Flickr, that image should come up first in the list of results, but it doesn't. How are these other pictures better matches to my picture than that picture's duplicate?
For a 3D model search engine, princeton has provided a 3d model search engine based on sketches of the three axis aligned planes.
http://shape.cs.princeton.edu/search.html
so do i just sketch the kind of porn i want or what?
(My group did a content based image search engine at OSU: http://www.cse.ohio-state.edu/~hakan/CIS772/index. html)
The interesting thing about color is that it is good enough ("Voxel 4" was our best if you find our website) to find images that have been resized, run through *some* effects filters, or rotated.
So if you use the same usericon etc, resized, on unconnected websites, that could be used to correlate your identities once this feature is in a global search engine. Also, the clustering of related images, for efficiency, restricts your results to the same cluster (for our implementation), which may not be apparent at first.
p.s. Thanks #Wikipedia for the gigabytes of images URL.
If you need text styles to communicate then you don't have a message.
This can be a problem. For example, I won't be able to check out the actual demo until I get home because I don't know what image it may pull to my work desktop.
.xxx domains. If you want access to it, fine with me. All I ask is that you make it easy for me to discriminate what my kids are exposed to. I want them to be able to use the internet and me to not have to worry about what they might find.
For those of us that run xscreensaver, one of the hacks goes out and gets random images from the web. I had to turn that one off because, random or not, it was getting porn everytime it ran. Now, I'm not a prude, I'm not concerned if my 5 year old son sees the image of a naked woman on my computer screen. To have a problem with that is to have a problem with the human body and I'm not going there. However, if said naked woman is in the process of strapping on a rubber penis....well....that's a bit hard to explain. Alternatively, if it's an image of a person (male or female) being abused....you get the idea.
This leads in to why I support the idea of
A goal is a dream with a deadline
That theory only works if you legally require that all porn be on .xxx domains. Otherwise you're still going to need to worry. Given that free speech protection has been extended to porn, demanding a subset of free speech to segregate itself is unconstitutional in US.
Say you somehow get this unconstitutional idea made law in the US. What about the rest of the world, which is free to use their domains as they want? Are you going to block every country which doesn't sign on?
Even if you find some way to get such laws passed it, you now face the question, "What is porn?" Snaps of kids running around naked? Nude art photography? Photos of topless women? Photos of swimsuit models? Photos of women showing their ankles? Fetish photography where the models are completely clothed? The higher you set the bar, the more things you probably don't want your kids seeing that will slip out. The lower you set the bar, the more you trample free speech and expression.
I'm for the .xxx top level domain, but for the same reason that .museum was created: to allow people who want to self segregate to do so. Filtering on .xxx would be a valid first step, but you're still going to need to worry about what they'll stumble upon. Demanding that they self segregate is censorship and unreasonable. It's not the rest of the world's job to make your job easier.
Search 2010 Gen Con events
Please tell me I'm not the only one who thought Mouse Doodles referred to excrement at first?
I worked for Pacific Press Service in Tokyo developing photo copyright and library tech until 94. I first saw a photograph search engine developed by Fujitsu around 92-93 I believe. It required the user to draw the type of image composition very roughly with a mouse and paintbox. So you would draw a horizon line, fill the bottom with blue and draw a yellow circle above if you wanted photos of the sea and sun. No wavelets at that time.
I then corresponded briefly with Ingrid Daubechies of AT&T who brought wavelets to the U.S., and was kind enough to send some of her papers. Wavelets are neat because it is like getting a paintbox full of different waveforms, localized as another poster mentions not just a fourier of the entire image. Anyway they are much better known now, so you can find it on the net.
This is not really the same as Barnsley's fractal compression one startup worked on around that time IIRC. They basically had a library of fractals which would be matched to image features, and once you had covered the entire image with them you would be able to zoom into it infinitely, since fractals are self-similar. You wouldn't necessarily get new detail but it would fool you into thinking you were. (I wonder if they liscensed it to anyone). They claimed 400:1 compression, etc. I don't know if they were the basis of LivePicture or if that was wavelet based.
These technologies all have two things in common, which is selecting an algorithmic strategy for talking about images, and storing it so efficiently that the data can be found quickly. The old Fujitsu system ran on a NEWS workstation IIRC, and it was blisteringly fast compared to any system I have ever seen. Only problem is doodles all look pretty much the same unless you are talented and patient.
It seems PNI (Picture Network Interactive)'s natural language recognition text searching for photos was the best, it was just text but used software supposedly developed for the White House. Only thing was they wanted to take over the entire industry with online contracts (this was around 1993) so everyone hated them. Nice tech though.
Anyway, wavelets may not be the entire solution but certainly they are a very useful way to describe data (not just a photo) and undoubtedly have lots of potential applications that just haven't materialized yet. Here's some tidbits Lancaster's links ImgSeek
Perl Haar decomposition and seeking
Blitzwave lib
wvlt
wvlt #2
Wavelet.org
WSQ used for FBI fingerprinting
I don't see where the article mentions exactly how the "mouse-made doodies" are mined for the energy to power the hack. Possibly by way of methane extraction? Oh, "doodles", not "doodies". Never mind.
It is the 21st century and the time for Klax has passed.