Slashdot Mirror
Automatic Image Tagging
bignickel writes "Researchers at Penn State have applied for a patent on software that automatically recognizes objects in photos and tags them accordingly. The 'Automatic Linguistic Indexing of Pictures Real-Time' software (catchy name) trained a database using tens of thousands of images, and new images have 15 tags suggested based on comparisons with objects or concepts in the database. Not sure how you identify a 'concept,' and they're only talking about having one correct tag in the top 15, but still cool."
The vast majority of the images on the internets including The Google include "Pornography" in it's top 15 tags suggested. The accuracy rate is surprisingly high.
Researchers at a publicly funded institution are using their research results for personal (financial gain). Pennsylvania's tax dollars at work? How is this legal?
Always do right. This will gratify some people and astonish the rest. -- Mark Twain
i prefer the 'yes', 'no', 'amirite', 'itsatrap' tags to the slashdot stories.
clearly the way that they were meant to be used!
I think I could categorise most things using less than 15 (admittedly very broad) tags. Animal, person, plant, machine, sports, vehicle, furniture, book, etc.
I've seen lots of systems like this. The problem is in the 50% of the images that don't work, so basically you have to manually tag 50% of your images.
I saw an interesting one about 10 years ago. It took an X-Ray image, did an edge detection, converted all the edges to a slope vs distance 2D plot, and conerted edge curves to a radius and distance plot, then used a kind of statistical correlation algorithm to pick which part of the body the image was from. I could imagine that you could apply something similar to the luminance of an image to pick out objects, and then maybe do some color transforms and stuff to improve results. The article says they do it in 1.4 seconds per image though, which is impressive.
Australian running a company that does C# / C++ / Java / SQL / Python / Mathematica
Not RTFA to be honest, but can I claim prior art?
I mage_Retrieval__A_Words_and_Pictures_Approach.pdf
http://www.relle.co.uk/papers/2003-Content_Based_
We didn't have enough time to train the system properly, but itstarted off well...
We Build Beautiful Websites
You've got to hand it to those cunning linguists at Penn State.
try { do() || do_not(); } catch (JediException err) { yoda(err); }
I'm sure a lot of research is being done in this area, in fact there is lots of interest implement this sort of thing in DSP for robot vision. How much of what this patent covers overlaps with what the others are working on? Is this something completely out from left field or does it fit the trend of where this area research was headed anyway?
Luis Van Ahn did something almost the same, his idea though is to use humans aswell.
View the video on Human Computation
I only read slash. for the articles...
For those who missed retrievr...http://labs.systemone.at/retrievr/ While this is good for hours of entertainment, i hope what theyre promoting is better.
Never attribute to malice that which can be adequately explained by stupidity
at least that's what I told the psychologist. Then on the second look, it looked like splotched ink on a paper that was then folded in half... I hope this software doesn't think like me cause at the end of it all, I saw a segfaulted X server on fvwm
Infiltrated dot Net
I currently work for the group doing this - a very cool new feature will be launched in the next week that I am writing (stay tuned). Yes - this project has been done many times before by many people (to lesser degrees of success than this), but the thing to keep in mind is that this is realtime. It takes less than a second for the tags to be generated. All previous systems required a much larger amount of processing time. Check out www.alipr.com to try it yourself!
How do they get less than a 50% average that you'd get by just guessing?
(yes, assuming a normal distribution of 'concepts' in the pictures, etc)
I want to delete my account but Slashdot doesn't allow it.
Now almost 7% of my pr0n will get tagged correctly!
That's cool, the rest of it will be like opening xmas presents!
*file: 123456.jpeg>open>Aghh! Goatse!*
Hmmm...This may be neat when it gets a LITTLE more accurate, but a cool start none the less.
Kudus to the gang for getting a grip on a hard problem...erm..nevermind.
Down With Slashdot BETA!!! I've been around the corner and seen the oliphant; you can only abuse me from your perspecti
This enables a word search of images. Suppose that I'm doing research on the advertising for Cat's Paw Boot Products. This technology would enable me to find such images which are incidental parts of other images. In other words, it could turn up the fact that Fred's Boot Shop in downtown Moose Jaw had such a sign in its window in 1935. The photograph might be labeled 'Downtown 1935' and there would be no mention of Cat's Paw anywhere. It might even turn up the fact that Al Capone's old limo was parked just down the street. This is actually exciting news for researchers.
Image recognition software is making it even easier for your kids to find porn! More at 6...
Snap
Not sure how far they got, but remember reading that IBM was working on this and had some reasonable success at object recognition in images. I'd love to be able to classify the 10k digital images I've got around. Especially if it can recognize individuals (not that it would know their names initially, but would be trainable).
Awesome furniture, accessories and cabinetry in Santa Rosa, CA: http://humanity-home.com/
Reportedly the researchers showed the system a picture of a Death Star, and it correctly tagged the image with 'thatsnomoon'.
The system has clearly been let crawl the web for far too long.
When the posters fear their moderators, there is tyranny; when the moderators fears the posters, there is liberty.
"itsatrap" is the best... but use sparingly
'...if only "Jumping to a Conclusion" was an event in the Olympics.'
anyone who has balls please repliy 2 this message!
Unless Jupiter Media gets to it first.
Someone like myself would understand the hours of data-entry and database development that goes into indexing imagery. I research photo copyrights for a living.
The fact that there is a feasible, automated system that can do the work will certainly cut down the man-hours for that sort of work; at least by half.
Pity, though. I heard that Google and others had a telecommuting thing that paid people to recognize what's in a photo. Sorry to hear they'll be out of a job soon.
This post © Copyrite Duggeek, all rights reversed.
.........SexSurfer logs in to begin his daily search of the web to find more images to rip in an effort to increase his database of porn images, utilizing this technology, only to find that most of the images consist of naked women with political statements printed on their asses......
Seriously now, I am sure their are people out there that have already got ideas rolling around in their heads about how they can use this technology to hijack images to their advantage. Once somebody understands how the technology works it is only a matter of time before it is used for nefarious purposes, by means of "tricking" the technology. And in the process, invalidating any possible means by which the developers can realize a return on their investments.
Personally, I'd love to use such a technology(if it actually works) to sift through the plethora of "crap" images I have to search through on the web. It can be really frustrating to do a search only to find that a vast amount of the results are TOTALLY out of context simply because of the title tag attached.
Won't this take away many of the pr0n-sorting jobs and slow down the pr0n economy?
I for one will not stand for this!
well that really had to happen, i've just tried it and you can't really go wrong if one of the top 15 tags is 'photo' and another of the tags is 'thing'.
Blazing Spiders
Now all they need to do is come up with a way to recognize spam words in image text without the overhead of OCR and they can make a fortune on that alone.
Automated dupe tagging.
This reminds me of a uni assignment that i did where we matched images based on colour.
If anyone cares, Penn State has a strict policy with patents, detailed here among other places. It all comes from the fact that most of the University's reseach is paid for by grants and industry cooperation. I didn't RTFA yet, but I'd bet that this is going to be immediately licensed to either the federal government or other such body, whomever funded the research. Otherwise, it could very well become public domain.
... usually a pedant... but you don't train a database. It was likely a neural net, but TFA is rather thin on details. Anyone got a link to their paper?
Oh god, that woman is John Romero!
Couldnt something like this be easily accomplished using neural networks.. I mean u will need a good dictinary and some base patterns to teach the network and then split up the image to compare for patterns. I am not an expert of image processing or neural networks but I have dabbled in some matlab based coding for both and it seems like a doable program the only problem I see is how to split up the image and at what point, edge detection maybe
"ponies" "lasers" "sharks" and "fud" are my favourites, "fud" being the most abused tag on the site..
Makes it easier to process all that data generated by all those security cams.
Is there a "Big Brother" category on Slashdot, yet?
How long before Google buys them out?
---
Yes, +5 Insightful me!
The human body is pretty much the same between people, and XRays are generally shot from similar directions person to person - so the kind of check you are describing seems like it would yield high matches for pretty much any part of the body.
In the real world we have an object you might take a picture of from any angle, using a myriad of focal lengths, with variable levels of distorition depending on the lens and camera used. Really nasty for generic object recognition. I think the best we can hope for in terms of accuracy is perhaps some kind of facial recognition autmatically recognizing and tagging people in images.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
This would be cool and all, but why not focus more on letting humans do the hard work-- like if I could take a picture of a tree and then press a button and say aloud; "redwood tree", and have that tag the file.
When I was studying textbooks on how to do this in undergrad comp sci.
Where are we going and why are we in a handbasket?
That's all fine and great that they can tell us, but why the heck couldn't they make a web-interface for it so I could try it out?
This just your standard data mining classification system, simply applied to image data as input, with the tags being possible classifications. This is an obvious application to ANYONE in the field. Software patents suck.
"To lead the people, you must walk behind them"
And that's one of the problems: does an image define the taxonomy or taxonomy defines the image [type]?
I quickly run out of fantasy when it comes to assign tags to my pictures: an automated mass tag finder will save hours of my precious time while uploading photos to Flickr.
"linux" is a very common word and was not included in your search.
How can you take a neural network and train it, then patent that?
That's like patenting training a dog to fetch a stick, it's completely rediculous.
You take software capable of generalizing a neural network algorithm by feeding it pictures and associating each picture with certain tags. It then creates a generalized algorithm model based on what you fed it initially. So that when you give new input it is capable of outputting tags most similar to what you initially trained it.
So yes this software can recognize boxes, shapes, other objects, maybe scenes etc and associate them with tags... but ask them how the algorithm works under the hood =) They have no idea... a neural network is like a black box after it has been trained. You feed it input and it gives you output based on it's initial training. The inner workings are chaotic spaghetti values set on each neuron weighting and can't be deciphered.
How can you patent software that is a black box inside?
"Yes hello patent office? I have this box that manufactures microprocessors. I feed it all the materials and it outputs a shiny new processor. I am not sure of the manufacturing process internally but the output works great. I would like to patent this manufacturing process.
"Okay your patent number is 247286-"BLACK BOX"-9
The whole point of a neural network is it generalizes what you train it and can future predict any input based on that.
It's like having the invention of the first mirror and everytime someone put something different infront of it, that person called up the art gallery because they had a new painting that they wanted in their name (because depending what was in front of it you get a different reflection).
Instead of speculating, why not just read all about the algorithms?
c h/ALIP/ACMMM06/ h /ALIP/PAMI03/ h /SIMPLIcity/TPAMI/
Main publications:
http://infolab.stanford.edu/~wangz/project/imsear
http://www-db.stanford.edu/~wangz/project/imsearc
http://www-db.stanford.edu/~wangz/project/imsearc
There's a few subjects that are so common that it's more or less a given they'll be in a large fraction of the photos. Outputting "people, buildings, nature, animals, plants, city" would probably give atleast 1-2 "correct" tags for 90% of whats in peoples photoalbums.
I had a class on neural networks and their (weak) sort of "ai", one task was to build a program to separate male from female names. The best programs could manage 80% or so, which is sorta decent. Until you realize that checking against static lists of the top 100 male/female names, if it's not in the list guess female if it ends in 'a', otherwise guess randomly will get you aproximately 95%. Furthermore, the latter program runs an order of magnitude faster, is more easily debuggable, can be understood by anyone, and can trivially be "extended" to reach 99% or more, simply by extending the lists of known male/female names.
So where's the download link? How can software matter if I can't get it? ;-)
Assorted stuff I do sometimes: Lemuria.org
I now have someone to play pictionary with.
this is the most important sig ever! In your face 446154!
So what were they doing, throwing a dart at a damn board? That success rate is no better then randomly applying vague words.
Move along to real research.
I wonder how this compares with Riya. At some point, there were plenty of rumours of a possible Google purchase of Riya. Then again they were rumours.
I haven't RTFA and I don't have any experience with Riya either, so consider the above posting a waste of time (if you must).
Corporate Gadfly
Jonathan Archer: the most beaten up Enterprise captain in Star Trek history
The vast majority of the images on the internets including The Google include "Pornography" in it's top 15 tags suggested. The accuracy rate is surprisingly high.
What the hell kind of sentence is that?
I burst into laughter when I came across this comment.
I wish I had mod points.
Does it detect breast size?
-==- Buy a Mac and leave me alone!
Wouldn't it be easier to pay starving children in [???] $0.01 / hour to tag images?
No, I will not work for your startup
Don't forget occlusion!
Yes, the real world has poles and fences.
"There is more worth loving than we have strength to love." - Brian Jay Stanley