Slashdot Mirror
Image Recognition on Mobile Phones
mysticalgremlin writes "In a recent presentation, Semacode founder Simon Woodside presents his company's bar code scanning technology that is used in mobile phones. Simon also discusses many places where bar code scanning powered phones are being used. Not bad for an 'image recognizer for a 100 MHz mobile phone processor with 1 MB heap, 320x240 image, on a poorly-optimized Java stack'"
And here I thought a bar code was a hand signal you used to let everyone in a large crowd, in a noisy bar, know where you were going next.
Like standing up and holding up five fingers to let everyone know the next bar is the "Five Spot".
Oh well, live and learn.
Skivvy Niner? Email me!
HEY! Look left just ONE MORE TIME!
Surely you mean "phone-powered bar code scanning", ie using the phone to scan bar codes, not powering the phone by scanning bar codes...
It's official. Most of you are morons.
Beleive it or not this is pretty impressive. Computer vision gets quite difficult when you don't have a lot of pixels to work with, as the shapes are all "helpfully" smeared together by the imager. And with the cheap lenses in camera phones, edges can be smeared by more than one pixel. In some of my prior work doing vision systems for Sony Aibos for RoboCup, we had to deal with similar problems (find an orange ball in an image that may be only 3x2 pixels, while ignoring the boundaries between red and yellow objects). So, kudos for the technical achievement, and hopefully they find a better application than the cuecat :)
Some years ago, I read an article about the possibility of printing tiny barcodes in newspaper stories that would code for a website address. You'd use a special reader that interfaces with your PC to visit the referenced site. This was supposed to be easier than typing in a lengthy, complicated URL.
We've got around this, mostly by having nice succinct URLs and tinyurl.com for everything else, and who wants to carry a barcode reader with them when they're reading the paper?
However, I wonder whether this idea may have some re-interest. If your mobile phone can read barcodes, we could print them anywhere - in papers, on billboards, TV adverts - and all you'd need to do is take a photo and your phone automatically loads the webpage in its built-in browser.
That might be useful.
Argh.
I know it is EXACTLY a bar code, but what about shotcodes? They look to bee similar to bar codes, just round. And they work just great with my phone and the getjar.com.
"You're everywhere. You're omnivorous."
This can be very useful. Scan a barcode while at a store for a TV for instance, and have your phone go out and check amazon.com, bestbuy circuit city compusa and price shop through your phone while at the store and decide whether or not you will buy there. Thats an awesome idea. Heh.
-- Josh
"Whoopie! Man, that may have been a small one for Neil, but that's a long one for me!" - Pete Conrad
Once a barcode is read you just get the product code. What good is that?
You need then to lookup that code up in a database for real info.
> Not bad for an 'image recognizer for a 100 MHz mobile phone processor with 1 MB heap, 320x240
> image, on a poorly-optimized Java stack'"
10 or so years ago we had 3d games on 7mhz machines with 512k of ram, pretty much the same screen resolution yadda yadda - this isn't so impressive.
Please hold downloading the mpeg with the talk until I finish. I'll reply to this message when wget returns my prompt. Thanks.
8 of 13 people found this answer helpful. Did you?
i remember those talks about putting stuff in the newspaper, but if i recall correctly, this wasn't limited to URLs, you could put whatever in there; images, ringtones, apps even :) it just depends on the size of the barcode.
My colleague once wrote a prototype doing the same thing (barcode recognition). This is also a nice solution for building tickets. THe main advantage is that you can give the guy at the entrance just one phone and he'll be able to scan entry tickets without the need for a computer or heavy equipment.
We even have a video showing this technology being used for payment. Note that in the video you see the recognition engine in java run on a PC with a webcam, but the same engine runs on many MIDP 2.0 phones (like a nokia 6230) and is also able to find a barcode instantly. In this case the phone is only used as a client for the payment concept.
The LG V phone has a business card scanner built in. If you take a picture of a business card, it can then use OCR to identify the text and add it to your address book. It's not perfect, but it's still pretty impressive (and you can edit out the mistakes). That's got to be harder than merely interpreting a bar code.
I've seen some Japanese phones that have apparently had this ability for quite some time now, I was absolutely amazed when a friend showed me one that even OCR'd english text out of a snapshot!
And there's a company called Grabba that makes commercial bar-code scanning solutions out of PDAs and PDA-phones (among other things). A friend of mine works there... interesting stuff; they also sell a dock thing that a PDA can clip into, which gives it a camera so you don't need to use a mobile phone. Popular with inventory/warehouse type applications, it also does 2D barcodes as well.
100MHz CPUs, 1MB RAM, etc, were distant dreams. Sure they scanned with a scanner serially rather than doing image recognition, and I'm sure using unoptimised Java virtual machines probably makes the 100MHz CPU feel like a 10MHz CPU, but it's not an amazing thing surely? Especially if you make some assumptions, such as: (1) barcode should try and be as large as possible when taking the image. (2) barcode orientation is generally horizontal, rather than angles over, say, 30 degrees.
... this book next to me has a 2" barcode on it - an ISBN with an extension to the right. So we're talking around 150 dpi if the barcode filled the screen. Assuming that there was white area around it, then the barcode area might be, say, 120dpi. Looking at the barcode, it's 'minimum width bar' count is around 60 lines per inch. So given crap image capture I can imagine moire effects if the angle isn't as flat as possible, and even then the thinnest bar would be 2 pixels wide, followed by 2 pixels of white. Add in aliasing and smudging and it seems to be harder, but I'd be interested in trying to code such a thing.
However
Literally. It's a QR code, not a standard linear barcode, but it's the same concept, and these days probably more ads, information posters and the like have them than don't, and virtually all cell phones have cameras that can decode them into URLs. No links handy (who'd need barcodes when you're already on the web?), so you'll have to take my word for it, but they really are everywhere. Even the wrappers on McDonalds burgers have codes that take you to their *cough* nutrition information page.
But I know 320*240 on a cellphone camera image was lame 3 years ago.
Thats great! What a feat!
Especially considering a friend of mine did exactly this as his final year Engineering project 2 years ago. In South Africa... a 3rd world country...
While I didn't RTFA, I'm surprised that this is being discussed like it's a newfangled idea. "See, we just put these parallel lines on things, and the spacing between the lines represents a number...."
We don't need to invent new things to add bar codes to. There are already dozens of ways to use this system if it could interface with existing bar codes. If i'm at Borders I'd love to scan a barcode on a book and bring up its reviews, If I have a UPS label I'd love to be able to scan the barcode to view its shipping info, etc.
I understand that this is much more then a bar code, it's also a database and lookup system. Not easy to build, but neither is a cellphone-based barcode scanner.
do xyz... on a cellphone...
Donald 'Duck' Dunn: We had a band powerful enough to turn goat piss into gasoline.
http://www.beetagg.com./ They got REAL-TIME 2d code detection, on wm5 and symbian (and treo650 seems to be coming soon) phones. Works on my Java phone (non-RT). Shame the Moto-Q is not on the list. Anybody care to test?
Phones here in Japan have had this technology for quite some time, using square bar codes that can encode way more information than the typical barcodes in North America. I can scan them with my phone and get a URL or other information...
My phone ( which I have had for more than half a year ) besides the bar code reader, has OCR of roman and japanese characters. And the most impresive use of this in the telephone is the ability to input some japanese word (yes in Kanji) directly into the dictionary. Really impresive for us non native japanese speakers. My phone is a sanyo w32SA , in the link you can read about in the part OCR kino.
"We all know Linux is great...it does infinite loops in 5 seconds." -- Linus
http://www.beetagg.com/ . On mobile phones, real-time, Symbian, Wm5, beta for treo650, J2ME (non-RT)... Shame the Moto-Q is not on the list. Anybody care to test it anyway?
Can anyone tell me what the typical operating frequency for the chip handling barcode recognition on a typical decade old point of sale system would be? Not 100 mhz, surely?
I'm actually fairly surprised the slashdot editors don't have a treo and follow palmgear or other palm development sites. There already is a bar code scanner you can get for free for your treo:
BarCode
Of course, quality isn't perfect but it does work. Mostly. Reads the barcode and copies the number stream results to your clipboard.
The next site to slashdot will be ready soon, but subscribers can beat the rush and start slashdotting it early!
www.mobot.com
A bar code reading cellphone, well I think our customers would have jumped at them. Even if the bar code reading software cost a grand, it'd still be cheaper than those devices we were selling.
These guys wrote their bar code software in Java. I'd be a bit interested to see how one coded in assembly would stack up against it...
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
Your comment got me thinking about how much information you could squeeze into one of those barcodes.
At most, a 320x240 tag would give you 76,800 bits of information, or slightly less than 11,000 7-bit ASCII characters. That's assuming you could match the pixels of the tag to the camera's sensor exactly.
I assume you probably wouldn't want to use any more than half of the camera's vertical and horizontal resolution though, which leaves you with 160x120 (for 2,700 characters), and I assume you'd need to have a few rows blacked-out on at least two sides to identify the border of the tag (so subtract (160+120)*2 pixels for bordering...that leaves 2662 characters) and you'd probably want to have a hash or checksum (lose 128 bits).
Still, that leaves about 2,643 characters in an image, which is about a page and a half of typed text using the old guideline of approximately 1,500 characters per page.
That's pretty impressive; provided you could make your reader focus on objects near to the lens, so that you could make the tag suitably small (less than an inch or so across), that's a lot more efficent way to convey textual information than actually writing it out. Instead of just embedding a URL link, you could put written information on there; maybe stuff that would clutter up the packaging / display / poster if you wrote it all down. If these things became ubiquitious, I could see whole advertising campaigns in urban areas (e.g. subways) where the "ad" got you interested, and then you could get more information via the tag.
They say a picture's worth a thousand words, and it sounds like it may not be far off from that.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
Could you tell me which approach was used in your project? I mean, I don't need an uber-detailed description, just some key facts; ex: "we used correlation", or maybe you applied some sort of scaling\rotation - invariant techniques, etc.
As a student, I experimented with image processing last year, and I was amazed by all the cool things that could be done with different algorithms, but I never managed to write a tool that could recognize an object on an image. It sort of worked, but I haven't had time to finalize it and release a version that would work for others too, not only for me, only when launched with a debugger, and only at step-by-step execution :-)
How reliable is object detection on a 3x2 sample? Looking for an orange ball on such a small image... Hmm, won't it be just an orange pixel on such a small image?
Another question - was that pattern recognition? i.e. your program was fed with images of orange balls and it attempted to find them on the target images, or did you somehow define an orange ball (ex: "a closed curve, the color of which must be within the specified RGB range") and the program had to figure the rest by itself?
The saddest poem
I saw this presentation. It sucks. Woodside pointed it at his mom, and the damn thing thought it was his aunt.
My turnips listen for the soft cry of your love
This would be very impressive if they could actually process, identify and match non-contrived real world features (i.e. faces, text, people), but this is just barcodes. Barcodes are DESIGNED to be easily recognized and processed by computers and to be highly tolerant of noise and corruption. Sure, this sounds like a nifty app, but to call it image recognition is very misleading.
I think I read in late 2004 or early 2005 about this being available in Japan at or before the time I had read about it.
2 sh/index.html
They even have pay-for-train-ticket and pay-for-groceries-and-other-things by cell.
While not light years ahead in all areas, Japan trounces the USA in some interesting and invisible ways. I had an analog TV phone (with FM radio, 1.2 megapixel camera, tweaking ability of the hundreds of polyphonics tunes inside, removable SD card, JP/Eng interface, memo, cal, many alarms, and more.) there for ONE PENNY (One YEN) because by Dec 04 it was "obsolete", yet where in the USA is an analog (meaning your viewing habits are NOT trackable) phone available for such a humane price?
Hell, many of the cell phones have PS2 console interfaces so up to 4 or more bored friends can hook up to a large flatscreen and play games designed for the phones.
I used to taunt the local (SF/SJ) cell stores with it, but Karma caught up with me and I ended up having to lose it. I was devastated.
See:
http://www.vodafone.jp/english/products/kisyu/v40
Previously: "Linux... Toward the Sunrise..." Now: "Linux... Toward the-- No, now, part of Every Sunrise"
Just in case anyone needed to know....
T.
[% slash_sig_val.text %]
It's not the best, but it's open source and it does work to read UPC and EAN codes: http://sourceforge.net/projects/barcr-reader/
Why couldnt someone put together a distributed computing cluster for image processing/facial recognition/text recogntion processing? Then, you can make an app that sends the images from your cellphone/mobile device to the distributed cluster, and receive the processed data back?
Pretty soon, the bandwidth available to mobile devices and cellphones will be plenty for sending photos and other data across.
When this technology becomes available to Joe Consumer, I'll bet that retailers will start posting signs saying that customers are not allowed to take photos/scans of barcodes and security guards will start bouncing customers that try it and Congress will eventually get enough $$$ from retailers to pass a law making it illegal probably in the name of homeland security. It's straight from the "big corporation hand book" section 3.5 titled "you can prevent the future if you make a rule against it"
See QR codes (2D barcodes) in action and download for your phone here:
f or-your-Smartphone-your-phone-as-QR-Code-scanner-t 243287.html/
;)
http://www.modaco.com/Quickmark-QR-Code-Software-
Fantasically geeky
P
My DoCoMo phone has had barcode scanning available to it for YEARS.
A few years ago, they added the ability to scan those 2D "barcodes", and they're everywhere in Japan. You scan the code, and it forwards your phone to a promotional website.
Get with the times!
How the hell is Java's stack poorly optimized?? And why the hell would he say that? Who is he to say that Java's stack is poorly optimized? All a stack is is a simple LIFO data structure.....
Image Recognition: Cellphone Transaction - Image Xmit - Scan Confirmation A tested, new technology capability enters the North America marketplace. A technology that has had great success in Europe in the facilitation of transactions on-the-fly has arrived in the form of a scanner that can read images off of a cellphone LCD screen. Using a special two-dimensional image/symbology, a selling party is able to confirm/identify who the purchasing party is to allow a completion of any type of transaction or access. Story Here>> http://symblogogy.blogspot.com/2006/06/cellphone-t ransaction-image-xmit-scan.html
Questions?>> maxine_log@yahoo.com