Fujitsu Tech Can Track Heavily Blurred People In Security Videos

itwbennett writes Fujitsu has developed image-processing technology that can be used to track people in security camera footage, even when the images are heavily blurred to protect their privacy. The company says that detecting the movements of people in this way could be useful for retail design, reducing pedestrian congestion in crowded urban areas or improving evacuation routes for emergencies. An indoor test of the system was able to track the paths of 80 percent of test subjects, according to the company.

Comments

LOL@ Use-case

[SeaFox]

Like any retailer would be interested in protecting the privacy of their shoppers identity while still wanting to track them.

Re:LOL@ Use-case

[sound vision]

I don't see where they claimed to be concerned with privacy.

Re:LOL@ Use-case

[rmdingler]

According to TFA, Fujitsu was forced to scale back A large, long-term facial recognition study it was planning to carry out at Osaka Station because of privacy concerns.

They seem to have several plausible shopper-related motives for tracking people, sort of like in Minority Report, but it doesn't take a Nobel-worthy leap of the imagination to see where this technology might be used to further eradicate personal privacy.

Re:LOL@ Use-case

[lkcl]

actually i worked for a company that provided path information (it's really really important) and privacy was absolutely key. they went to a lot of trouble in the design of the software so that, if they were ever compelled, even by a court order, to "identify individual X", they would LITERALLY be unable to comply and, to avoid contempt of court, would need to go to some technical lengths to explain why. they didn't use images (because they don't work) - instead they used GNURadio to do GSM passive decoding and signal-strength detection. and no, you *can't* track the person themselves, nor can you get their telephone number, nor can you decode their phone conversations, nor can you decode their SMS messages (not "and track 1000s of phones on affordable commodity off-the-shelf hardware at the same time"). they also track bluetooth and wifi, but again, the mac addresses are hashed (with salting) *before* being stored on disk. the reason for this kind of paranoia is really really simple: they ABSOLUTELY DO **NOT** wish to be involved in privacy and identification issues. it would destroy their reputation. so they made damn sure it simply could not happen, even if they were compelled by a court order.

anyway - first important thing: the definition of a "path" (and why it's critical). a "path" is, as the word suggests, the places that an individual goes to, and how they got there, how long it took, and how long and where they were stationary. key factors critical for shopping mall owners to be able to provide to their retailers: (1) how many unique shoppers went into *their* store (broken down by time and date is also helpful). (2) how long each unique shopper spent in their store. (3) also useful to know is where they went *before* going to another store. it's therefore necessary to weed out "passers-by", and duplicates (losing the path then picking it up as a *separate* person, repeatedly) is *especially* bad as it completely mucks up this all-important information that the retailers, it turns out, really really like to have once they know it's available.

think about it: this information is really, really important. in attracting retailers, without this equipment (or anything like it), the conversation is "come to our retail park, we have 6 million visitors a year". the retailer isn't interested in that. *with* the equipment (or anything like it), the conversation goes further, "and the unit we would like to interest you in gets 15,000 unique visitors per day if occupied by someone with your type of retail profile, especially because there's a macdonalds / starbucks within 100 / 50 metres and we know that that gets better numbers for you". *that's* powerful stuff, and it allows the shopping mall management to pick (and test, and research) interesting combinations of retailers that will make the whole mall a lively and attractive place to be, instead of being boring, half-empty of both retailers and customers (the other half being tired, stressed and exhausted), and doing a dis-service to everyone who bothers to go there.

so anyway i had to be up on the "competition" so to speak, because we frequently got questions coming in from clients being pitched the "visual tracking" technology.

first flaw in visual tracking technology: balloons, signs, pigeons, dogs, baby strollers - anything that moves in uncontrollable ways that is big enough to block people: you're hosed. pigeons etc. are fun because they randomly block out huge areas directly in front of the camera if they get close enough. even "other people" is enough to block "other people". even identifying "people" from children, babies, animals - this is hard enough as it is and requires enormous CPU resources... the number of people in some of these malls is *enormous* - tens to hundreds of thousands.

second flaw in visual tracking technology: it's intrusive. put a camera in a shopping mall and people automatically get edgy. it changes their "behaviour", which is precisely what you do not want. the last thing you want in

Re:LOL@ Use-case

[radarskiy]

The retailer is not inherently interested in either protecting or violating privacy, so the choice is purely one of net expenditure. You can either raise the costs of violation and hope that you can catch them or can lower the costs of protection.

Re:LOL@ Use-case

[Ben Hutchings]

Path Intelligence (reference)

Re:LOL@ Use-case

[ArsonSmith]

I don't think you understand. Bigfoot is screwed now.

Re:LOL@ Use-case

[CaptainDork]

People who don't want to be tracked should not go out in public.

Re:LOL@ Use-case

[disambiguated]

instead they used GNURadio to do GSM passive decoding and signal-strength detection. and no, you *can't* track the person themselves, nor can you get their telephone number, nor can you decode their phone conversations, nor can you decode their SMS messages (not "and track 1000s of phones on affordable commodity off-the-shelf hardware at the same time"). they also track bluetooth and wifi, but again, the mac addresses are hashed (with salting) *before* being stored on disk.

I think it would still be possible to deanonymize that path data. If you make a credit card purchase, the information about time and place of the credit transaction can be associated with whatever id you use hashed or not. The path data has information that someone was standing at the cash register at that time and place. With the credit card information (or even just loyalty card information) you know who it was and can associate that with the entire path through the mall. Similarly, if they walk past a Starbucks and their smartphone associates with their WIFI, now if you have access to Starbuck's information you can deanonymize it from that. Or it could be deanonymized with the security cameras.

I don't see then how it could be subpoena-proof if you store the actual path, regardless of however you anonymize it. They can subpoena your data together with other data to get what they want.

Re:LOL@ Use-case

[BronsCon]

False. I don't want to be tracked, but I'm willing to accept it as the current status-quo. People not willing to do that shouldn't go out in public. Those of us willing to accept it currently, however, don't necessarily want it; many of us are acting to change it.

Re:LOL@ Use-case

[lkcl]

I think it would still be possible to deanonymize that path data. If you make a credit card purchase, the information about time and place of the credit transaction can be associated with whatever id you use hashed or not.

you're assuming that the data collection rate is of the order of seconds. if you check the GSM spec for cell tower ping times, make some educated guesses on average phone usage including SMS, GPRS and call usage, and so on - bear in mind that this is a *passive* system as it is illegal to interfere with mobile phone operation - and also bear in mind that the positional accuracy is somewhere around a 20 metre radius - and then think about the number of people in any one store i think you'll find that statistically speaking the argument that you present falls completely flat.

the technology that the company i worked for is pretty cutting edge for innovation, but it is definitely pushing the limits of reasonably affordable off-the-shelf equipment that a shopping mall retail centre is prepared to pay, in order to obtain access to the kind of reports that it provides.

sure you could put in equipment that costs $250,000 per monitoring station, you could then have accuracy of 0.5 metres, you could get cell tower operator licenses (or other agreements) and do man-in-the-middle attacks which would get you the information presented in the argument that you give, but aside from the fact that no retail centre would ever pay for such ultra-expensive equipment in the first place the very fact that it *was* capable of getting them involved in court cases would actually *deter* them from buying it!

think, please. _why_ would a shopping centre manager make the *deliberate* decision to spend money that could cost them both retailers and shoppers if there was a disastrously-bad publicity report (either in the news or from another Snowden-style leak) which implicated them in privacy invasions?? it just doesn't add up, does it?

so ironically the inaccuracy of the off-the-shelf (GNURadio-based) equipment is a selling point (insufficient accuracy and data collection rates to be used to violate individual privacy) whilst at the same time being just sufficient to provide the kinds of large-scale statistical reports that the retailers need... and no more.

Re:LOL@ Use-case

[Hussman32]

The monitoring of subway stations is probably a direct result of the sarin attack at Tokyo station. http://en.wikipedia.org/wiki/T.... Of course the usual Orwellian arguments will apply, but try finding garbage cans in the subways...it really freaked out a lot of people.

Re:LOL@ Use-case

[Hussman32]

bottom line is: i really don't see how visual tracking is going to work out any time soon, especially given that face-blurring helps destroy critical information needed to rejoin paths if the tracking is ever lost, and especially given that the CPU usage is so enormous that you would need a supercomputer in the back office and a massively-upgraded power line to run it. no - don't expect visual tracking to be hitting a shopping mall near you in the immediate future.

Excellent post, one question, obviously people are of similar sizes and proportions, but with some spatial features (everyone has a top of a head and shoulders) along with clothing colors, couldn't you do without the face?

Re:LOL@ Use-case

[Archangel_Azazel]

This is an utterly asinine statement. Being tracked constantly and often surreptitiously is a current social AND rights issue. What you're saying is just rolling over and asking to be kicked in the nuts by anyone with the desire to simply do harm. No thank you.

Re:LOL@ Use-case

[disambiguated]

Well, I still think the data can be deanonymized. I don't need to make any assumptions other than what you've told us.

the places that an individual goes to, and how they got there, how long it took, and how long and where they were stationary. key factors critical for shopping mall owners to be able to provide to their retailers: (1) how many unique shoppers went into *their* store (broken down by time and date is also helpful). (2) how long each unique shopper spent in their store. (3) also useful to know is where they went *before* going to another store.

Even if the time resolution is 5 minutes, and the spacial resolution is only enough to identify which stores I visit, that is enough to identify me. If I go to the mall, stop by and get a coffee, wander around for a while, then make another purchase in another store, using my credit card both times, I may very well be the only person who made purchases at those two stores within a 5 minute window at each store. Each purchase makes it more likely to be unique. Now if I put on dark glasses and a baseball cap and stop by Victoria's Secret to buy some lingerie for my mistress, with cash, it's possible to link that to me via your path data.

It isn't the path data per se that is identifying me -- it's a combination of that and other data. It doesn't have to be credit card data, like I said. It could be wifi, loyalty cards, security cameras, even witnesses... anything that can associate me uniquely with one of your paths. And it doesn't even have to be unique, just narrowing it down to a handful of people is useful to law enforcement.

Don't get me wrong: It sounds like you and the company you worked for care about privacy and did everything you could to protect it. That's commendable. And it sounds like you did a good job. (Plus I think it's cool you used GNU Radio.)

It's also commendable that you understand the conflict of interest. The retailers would like to have better spacial and temporal resolution: they'd like to know which aisles people walk down, what displays they stand in front of and for how long, etc. The retailers will ask for that and if you don't provide it someone else will. So there will always be pressure to make it more useful. But the more useful it is to retailers, the more useful it is to anyone else who might try to get access to it, whether it be through hacking or subpoena.

I am skeptical whenever I hear "don't worry, we've anonymized the data." I've seen too many ways that data can be deanonymized, and I'm not a professional data miner or forensic hacker, so I don't know what other devious methods there might be that I've never heard of and would never occur to me. The key point is that as long as you store the path itself then anything that can link me to part of it can link me to all of it. The only way to avoid that would be to obliterate the path data and only store aggregate information (averages, sums, etc.)

Re:LOL@ Use-case

[CaptainDork]

Google "sour casm."

Re:LOL@ Use-case

[CaptainDork]

I was being a smart-ass.

Re:LOL@ Use-case

[BronsCon]

Might want to look into finding some way to indicate that; the tone of voice you used in your head doesn't come through in text. Personally, I find that seems to do the trick.

Re:LOL@ Use-case

[CaptainDork]

OK.

</sarc>

Re:LOL@ Use-case

[lkcl]

Even if the time resolution is 5 minutes, and the spacial resolution is only enough to identify which stores I visit, that is enough to identify me. If I go to the mall, stop by and get a coffee, wander around for a while, then make another purchase in another store, using my credit card both times, I may very well be the only person who made purchases at those two stores within a 5 minute window at each store. Each purchase makes it more likely to be unique. Now if I put on dark glasses and a baseball cap and stop by Victoria's Secret to buy some lingerie for my mistress, with cash, it's possible to link that to me via your path data.

you had me concerned for a minute! but then i thought about it, and i realised that if you take a venn diagram of the set of credit card purchases (assuming a subpoena has obtained full details), and a venn diagram of the set of paths (from WIFI or other method), what you get if you take the AND of those is no more than what was obtained from the credit card details.

in other words, your privacy has already been violated by a subpoena for the *credit card* details in ways that a subpoena for the path details could not possibly hope to match or add any information to that is *not already known* from the credit card subpoena. except for some outliers... discussed below:

what you get if you *remove* the set of location/time-cross-referenced credit card purchases from the set of "paths" is actually much more interesting. scenarios where the two data sets do not match would include where someone borrowed your credit card (with or without permission), or cloned it.

we're beginning to get into quite complex territory here, but let's say that someone stole your credit card. let's assume that the thief also has a mobile phone. let's say that they (rather stupidly) use the credit card in the same store to make multiple purchases. *then* you have a situation where it woud sort-of be possible to narrow down the numbers from *maybe* 10,000 possible candidates down to say 2,000 possible candidates, down to maybe 100, with each extra piece of information (assuming WIFI / GSM not say camera tracking) ... and at the end of that, what you would have would be a set of anonymised pieces of information, all of which you *still* could not identify the thief - based purely on the path information (even if you add the credit card details) - because of the salted hash. (if you actually caught them then it's still dicey but you *might* be able to provide some "statistically-dubious" circumstantial evidence but it would require an additional subpoena to the mobile phone company to get them to provide the TMSIs... it's complicated, but TMSI stands for *TEMPORARY* mobile subscriber identity - it's 32-bit and it changes something like once every 24-72 hours. i do not know if mobile phone operators keep records of the TMSIs allocated to phones, but it would be unlikely that they bother, as it's something that the base station cell towers allocate locally. WIFI on the other hand would be a different matter, as MAC addresses typically do not change).

so about the only thing you _could_ do was to notice that the credit card was no longer "in proximity" with the mobile phone "path" information and perhaps report it to the credit card company. *but*, bear in mind that it's on a 20 metre radius and on a 5-15 minute "ping" and it's pretty touch-and-go as to whether the information would be in time to stop fraudulent purchases... or even if it would be correct (not a false positive).

now, with this visual tracking stuff, you *might* have better luck (assuming it's ok to run a beowulf cluster on-site within the shopping mall premises), but i have serious doubts that it's within reasonable cost for deployment.

the only thing i can think of, if you are genuinely genuinely concerned about privacy:

(a) take the battery out of your phone or better leave it behind entirely
(b) use cash as long as it's not an attention-

Re:LOL@ Use-case

[Archangel_Azazel]

Google "bite me."

Re:LOL@ Use-case

[CaptainDork]

I did and it gave me links to "sour casm" and stuff.

shoplifting

[sound vision]

What this may get used for is things like tracking shoplifters. Wal-Mart and other large retailers will take down your name, driver's license number, SSN, and take your picture if you get caught shoplifting, with a warning that you are not allowed back on the company property, or they will consider it trespassing.

A system like this could be used to automatically track people who have shoplifted to either get tailed by security or kicked out of the store (and possibly charged with trespassing). It also wouldn't surprise me for the companies to share this info, either directly or through a background check / data-broker company, of which many already exist. Imagine being locked out of 90% of retailers in the country for shoplifting some candy when you were 17...

Re:shoplifting

[Joe_Dragon]

and think of the law suits each they kick you for based on a video from when you where a minor

Re:shoplifting

[CaptainDork]

Where would Wal-Mart get such information and how would they verify it, especially the "heavily blurred" SSN?

Re:shoplifting

[sound vision]

Wal-Mart's security team takes shoplifters into a small interrogation room of sorts, asks them for their ID, and makes them sign some kind of agreement that has a blank for their SSN. Most people hand it right over hoping they won't get the police involved. (They may even be able to get this info after the fact from the police themselves, but that I'm not sure of.)

Re:shoplifting

[jklovanc]

A system like this could be used to automatically track people who have shoplifted

That would require matching an image with an identity which this system does not do. All it does is track a person not identify them.

Re:shoplifting

[CaptainDork]

I am sure.

Entities like Wal-Mart are not on the list of valid recipients of Social Security Numbers for shoplifting.

The Wal-Mart story has no credibility.

I'm pretty consistent

[Snotnose]

My path through the grocery store is pretty consistent. Wander through produce looking for what I need. Beeline to the back wall, where the meat is. Head to eggs/dairy, next to the meat. Hang a 180, down the frozen foods aisle, hang a left, where the deli is for my meat and cheese for the week. Do a 180, head to the checkout.

Every other month or so I need something like fish sauce, olive oil, etc, and then they can track me wandering aimlessly down the aisles, if they had microphones they'd hear me muttering "dammit, why don't they label the aisles better" and "dammit, where's the damned kimchi"

Staples like tomato sauce, chicken stock, etc are bought a couple times a year at Costco.

Re:I'm pretty consistent

[CaptainDork]

Actually, we knew that already using our video tracking system.

Tracking identifying

[jklovanc]

All it can do it identify that a person entered at one point followed a certain path and exited at a certain point. It can not identify who that person is. It is useful to see where to place certain items for easier access to popular items. I knew the tinfoil had brigade would get it wrong.

False Argument

[JimSadler]

Privacy is simply not an issue if one is inside a business or on a sidewalk. Those are both public situations in which there is no reasonable expectation of privacy. Anything done within the view of others is by definition a public display. There is an ongoing trend to obscure the meaning of privacy when in reality what many people are doing is wanting to escape accountability.

Not surprising

[borknado]

If you know how the image was blurred, and you know you should be looking at a picture of a face, isn't it straightforward enough to design a video filter algorithm that could come up with a few unique variables values to track? Maybe the trick here is how to do it quickly enough to process live video and track people in realtime with a standard desktop-class system?.