Google Patents Software To Identify Real-World Objects In Videos
hypnosec writes "Google has been recently granted a patent that could not only improve online search, but also will possibly give the search engine giant an awful lot of information about the world. Google wants to scan and analyze the content within videos (YouTube videos, most probably) and look for objects in the real world, identify them, and make a catalogue out of those objects. The patent describes Google's technology of scanning a video, picking out landmarks, objects and context; and subsequent tagging and categorization."
Adds reader MojoKid: "The privacy implications of such an automated system are enormous. Facebook's own automatic facial recognition software was highly controversial when it debuted, and what Google has now patented puts Facebook to shame. The larger question, unaddressed in this patent, is whether we want our individual personal data to be tagged, filed, and logged without permission or choice."
The larger question, unaddressed in this patent, is whether we want our individual personal data to be tagged, filed, and logged without permission or choice."
How is a video uploaded to youtube 'individual personal data"? Sometimes it seems like we just want to complain about stuff.
I definitely want my (very few) youtube videos categorized, and most importantly, I want to be able to look for video contents.
By the way, the IO keynote demoed a search by content on pictures uploaded to google drive (the speaker typed 'pyramid' and the search returned 2 pictures with background pyramids), so this seems like an obvious improvement over that.
Are you also a Warrior-Philosopher hovering in a glass castle over California by any chance?
>> unaddressed in this patent, is whether we want our individual personal data to be tagged, filed, and logged without permission or choice
I'm not sure that you have any expectation of privacy in this case. You're putting videos on the internet at your own will or you are in a public setting where you are being filmed. I'm not sure if things like parties are considered to be private affairs or public outings or whatever and if you can expect that your actions will be kept secret from the world or not. Either way, I don't think that you can expect much. But, I'm not a lawyer, so I don't know. However, I wouldn't expect that any part of my life that I freely share with the world in anyway should be kept private unless it was agreed to beforehand.
Also, with regards to Google in general, I think your permission is granted when you use Google services. When this goes live, and you don't want to participate, you can delete your youtube account and host it somewhere else. Maybe a place with more privacy control. For people who are caught in it because their friend posted a video, well, I think that kind of pertains to my babbling in the first paragraph.
After Andrew Ng's experiment with creating an unsupervised neural net of 16,000 computer cores at Google, it comes natural they have an interest in this field of photo/video tagging and start outputting some patents. There is a similar technology at (http://www.msravs.com/audiosearch_demo/) for transcribing speech to text and making archives of videos searchable by word (kind of like Google Books does for print).
Soon we will have all our huge archives of video, print and images analysed and searchable. Imagine the implications!
... you'd better not go outside. Somebody might see you! Better to stay in with the curtains drawn, and keep away from the windows just in case.
What I want is a little webcam that I can put on my shoulder, and which records everything (without uploading it anywhere). Then, when I want to know where my keys are, it'll show me the last recorded images that have keys in them.
What exactly are the new privacy implications of this system? Governments in western democracies are deploying facial recognition systems at street corners and license plate recognition systems to track vehicle movement, what does this bring that makes things significantly worse?
That last link is particularly egregious. It points to Chromebooks as a device that is dependent completely on Google services. How about also mentioning devices that already do the same, except that they have about a 100 times more users - iPhones and iPads. And even Windows devices are going the same way. What on earth does with ave to do with this patent, and how about mentioning that this isn't specific to Google. I am tired of this FUD implication that Google can steal all your data while others can't? What Google could do theoretically all other 'cloud' providers could do too, theoretically. How about comparing their actual records? Has Google turned on face recognition without your consent? Has Google changed your email contacts and personal email address on your profile and wiped your phone contacts? Do you really think Apple can't access the same stuff in theory from your iPhone the way Google can from their devices? Or the Facebook app that installs with just about every permission available?
And again, what does any of it have to do with this patent. If you upload a video to youtube for the whole world to see, is it really a big deal that Google knows that you use a Macbook?
the article says "Google's own vision for the future of computing is a Chromebook/Chromebox that's completely dependent on their own services for everything". Really? What's Apple's vision of computing? Amazon - ever used a Kindle Fire? In fact, Google sells very few Chromebooks, and most Google affiliated devices sold are Google Android devices, which offer far more freedom. You can use it with non Google accounts. You can disable and remove Google services, and Google allows apps that compete with Google services (Apple bans those - reproduce the functionality of an Apple service or app and you get banned). Amazon is equally restricted if you've ever used a Kindle Fire.
And there are several unlocked Android phones and devices where you can install Google free versions of Android like Cyanogen, and do whatever the hell you want without the privacy implications. How about raising awareness of those for people who are really concerned about privacy instead of spreading all this 'Google will steal all ur dataz oh noez' FUD. That ship bolted, the horse has sailed etc. ALL current device makers do that or are moving towards that model. Go to all the trouble of using and informing about Linux or De-Googled Android devices, or shut up and talk about real disasters when they happen like Facebook's several privacy booboos or the Google Wave fiasco. Not this FUD.
n/t
Having have the time to read TFA yet, but just wondering, how does this patent differ from the Face Recognization routine?
Or maybe this patent will enable the software to pick up one particular face, out of the thousands, in the video?
Muchas Gracias, Señor Edward Snowden !
Both?
AD: There is white room with a dining table and a few chairs ...
AD: A blonde haired girl with large breasts is bent over the table
creaking
AD: A well endowed man is stood behind her
moans
AD: the man is thrusting faster
AD: the mans hands grab the girls breasts
moans increase
AD: the girl is moving her head back and forth, flicking her hair.
"ah ah"
AD: She looks at the camera, now we can see that she has a mans face
Hmm. Well Google good to see you moving accessibility technology forward,,,,,
Rad.
OK, my beef is that it sounded to me that Image Recognition on a certain application got its own patent...
My -1 Troll is actually a +1 funny. And my -1 flame is actually a +1 insightfull.
I'd like to see a radical expansion of their Street View, where someone carries a similar camera but smaller than their car uses, but walks, or uses a dune buggy or motorcycle and goes off road to collect even more data through places their cars wouldn't be able to access.
That is application using image recognition..
Image recognition towards a certain applicaiton... ...
bah
sleep is for the weak.
My -1 Troll is actually a +1 funny. And my -1 flame is actually a +1 insightfull.
This provides excellent way for global mind reading. Like noted in this blog, mind reading can be nicely automated with Google's technology. In fact, this is already in use. It uses a combination of brain pattern matching, tiny video feed from your eyes and Google's technology to identify objects and context. It works wonderfully.
You might question how I know this? Because it's already in use. Such device has been planted in me and it is reading my mind. It knows my every thought and gives the device controllers voice notice of every object I put my eyes on. It is exactly like the technology Google is patenting and wants to develop. It's a huge opportunity for mass scale big brothering. I'm not entirely sure how widespread the mind reading technology is, but I do know this Google technology is part of it.
You have to fight against using these technologies. Yeah, it's "Google". That's what actually worries me a lot!
Sit down on this cot, we're going to give you a shot which will make you much more relaxed. And then
we are going to show you to your new room. Sorry, it has a very small window and you won't get out much,
but you will be safe from THEM.
.
Is landmark identification within Google Street View
This is bound to happen. The question is:
Would you prefer Google to patent this, or someone like Apple?
Personally I would trust Google to consider user rights a little bit more, but the exploitation of such a system is damn scary.
I've been going to lectures and seminars on people trying to do this for years. How in hell is it not obvious, or logical progression, or any of the other things that should prevent a patent?
There are some real world military applications for a patent such as this. Say someone takes a video that is less than kind to a particular government. That government can then learn where that video was take, who it was taken by, and where that person lives. Big brother indeed, with a darker twist.
The problem is you don't have a say over this: Even if you're not a iPhone-weilding Youtube-uploading Instagram-snapping Facebook-addicted Gmail-enabled Twitter-junkie, you will have friends that are and upload information about you without thinking about it. I'm Privacy aware, but many non-technical people aren't. Now add to that webcams and surveillance video and there is no escape. No wonder they've been dragging their heals on privacy legislation with real teeth: Corporations will love it for data-mining and government will love it for surveillance.
Take this girl: She had a photo snapped of her at a friend's BBQ. They uploaded it to Flikr without thinking, and next thing she knows she's on advertising billboards: http://www.flickr.com/photos/sesh00/515961023/ http://www.theinquirer.net/inquirer/news/1047772/virgin-mobile-sued-virgin
Here's an update on what happened with that case: Even though she was only 15 years old when the photo was taken *and* it was used in a for-profit advertising without her permission, the courts sided with the phone company (Virgin) who did this and dismissed her case. Virgin was unapologetic
http://blog.internetcases.com/2009/01/22/no-personal-jurisdiction-over-australian-defendant-in-flickr-right-of-publicity-case/
http://info.ee.surrey.ac.uk/Personal/Z.Kalal/tld.html
TLD
TLD is an award-winning, real-time algorithm for tracking of unknown objects in video streams. The object of interest is defined by a bounding box in a single frame. TLD simultaneously Tracks the object, Learns its appearance and Detects it whenever it appears in the video. The result is a real-time tracking that typically improves over time.
Due to its learning abilities, TLD has been advertised under name Predator. The video to the left introduces Predator and proposes several potential applications.
TLD has been developed by Zdenek Kalal during his PhD thesis supervised by Krystian Mikolajczyk and Jiri Matas. The main contributions of TLD have been presented at international computer-vision conferences. For his work on TLD, Zdenek Kalal has been awarded the UK ICT Pioneers 2011.
Key Features
TLD tracks currently only a single object
Input: video stream from single monocular camera, bounding box defining the object
Output: object location in the stream, object detector
Implementation: Matlab + C, single thread, no GPU
No offline training stage
Real-time performance on QVGA video stream
Ported to Windows, Mac OS X and Linux
Free Version
TLD can be downloaded for testing. We provide a precompiled demo (Windows) and a source code that is released under GPL version 3.0. In short, it means that any distributed project that includes or links any portion of TLD source code has to be released with the source code under the GPL version 3.0 license or later.
Commercial Version
A license has to be purchased for using TLD in closed source projects. The licencing is managed the University of Surrey. Please contact the Technology Transfer Office at the University of Surrey for further information.
More Information
High-level description of TLD
Components of TLD
Learning component of TLD
Application of TLD tracker to faces
Detailed description is in the following papers: ICCV'09 (w), CVPR'10, ICIP'10, ICPR'10
Many technical questions (e.g. installation) are being discussed in the following discussion group.
FAQ
How do you manage the number of templates in TLD?
For the moment, we do not put any constraints on the number of templates in memory. The number of templates typically stabilizes around 100, depending on the appearance variability of the object and background.
What kind of hardware is TLD running on?
TLD has been tested using standard hardware: webcam, Intel Core 2 Duo CPU 2.4 GHz, 2 GB RAM, no GPU processing is used and runs in a single thread. The demands of the algorithm depend on required accuracy of the algorithm.
To what degree is TLD invariant to light levels?
We use illumination-invariant features for detection. So as long as the changes in illumination are global, the performance does not change. If there is a strong local illumination change, the system is learning the appearance changes thus created.
The algorithm shouldn't be too hard if it has a 1% chance of error. Technically it's anything that ISN'T porn that it can't scan for.
Did the Gauls worry about their privacy when Caesar wrote his "Commentarii de Bello Gallico" ? Could they have done anything about it ? Did it matter ?
Avoid your fears , or wonder at the past
I'm sure the government will see to it that they get this so Google and provide them information about brown people in return.
People should host their own content or find a company not interested in takin gall their data to sell advertisements and quit handing their life to google.
We all knew it was coming. Not really a new concept, smart move to patent this since everybody is suing everybody.
Look, I really don't care what Google knows about me. I don't care if they know every single website I visit, how many steps it takes me to get to a McDonalds, or what color my toothbrush is.
If their targeted advertising means I'll never have to see another ad for tampons and lipstick, or a bar in Austria (a place where I don't live), then I'm happy.
There's a Google Tech-Talk that got posted a few days ago, which is not specifically about this patent, but may shed some light on what they have in mind: The Distributed Camera: Modeling the World from Online Photos
XML is like violence. If it doesn't solve your problem, you're not using enough of it. --AC
sure google is an out of control monstrosity that wants to spy on you and you and sell your personal information to the highest bidder but hey they used linux in their phone so it's cool!
the Revolution will NOT be televised, or it's never gonna get off the fucking ground....
too easy to identify the participants now....
every day http://en.wikipedia.org/wiki/Special:Random
doesnt BMW's, and Porsches rear parking cameras already do this, and do it in real time too?
portfolio
A patent search will not turn up this patent. This particular patent holder can retroactively exercise their patent rights. They are the only patent hold that can do this.
I am very small, utmostly microscopic.
All the same .. they are fantastic law enforcement tools , FBI , CIA , NSA information hungry organisations
love to track and know everything about you. They dont give a donkey's tail whether or not it suits you.
It suits the needs of their masters. Spying tools for spies.
As long as idiots are jumping in " social networking " and posting their lives and letting themselves be tracked even by GPS
and the governments and law enforcement know every minute what they do and where they are , why should they stop ?
Go ahead , share your data. You feed the beast.
How will this affect my puny little life exactly?
Oh, it won't in any way, shape or form?
Ok, carry on.
that might be considered prior art. Let me just search for that. Hmm.... what could possibly go wrong.
Siggraph has had several submissions doing this same thing. I hope the patent fails.
Sit down on this cot, we're going to give you a shot which will make you much more relaxed. And then
we are going to show you to your new room. Sorry, it has a very small window and you won't get out much,
but you will be safe from THEM.
You must have missed the memo. The drugs are too expensive and we don't have enough space for private rooms.
Here's some tinfoil. Make a nice hat.
Faster! Faster! Faster would be better!
The next story out of Google will be that the software was deleted after it realized that 90+% of the pictures on the Internet are penises and needed therapy.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
Although I'm skeptical of software patents, if there are going to be software patents, then this would be one that deserves it.
I mean, this actually took some work and innovation. It's not a simple DrawRect (or whatever) with a rounded corner radius set, which any high schooler in a programming class could do.
I'm not a lawyer, but I play one on the Internet. Blog
Sounds good for google glass version 2+ too. I'd like to be able to look at something and be shown relevant information without having to describe it in a way a search engine expects. Reminds me of Continuum.
Especially United States Patent 8250019 (System and method for interactive knowledge visualization) claim 9:
9. A method for interactive knowledge visualization, comprising: receiving, at a knowledge visualization server, a request for one or more visualizations; retrieving data associated with the one or more visualizations from one or more databases; converting the data associated with the one or more visualizations into visualization data,...
DNA is a Turing machine. You, however, being dynamic and emergent, are not.
It's about Google Glass.
If you post a video to youtube, then you want it to be public. If you are filmed and put onto youtube, you were in public or gave permission to be filmed in private. If this is not the case, then it will be easier for you to locate the violations and request their removal. This is a benefit to privacy. I like the idea and would like to see this service in action. I support youtube with this invovation and request that those who perfer privacy continue to stay away from youtube and public spaces.