Google Pushes Open Source OCR

SocialWorm writes "Google has just announced work on OCRopus, which it says it hopes will 'advance the state of the art in optical character recognition and related technologies.' OCRopus will be available under the Apache 2.0 License. Obviously, there may be search and image search implications from OCRopus. 'The goal of the project is to advance the state of the art in optical character recognition and related technologies, and to deliver a high quality OCR system suitable for document conversions, electronic libraries, vision impaired users, historical document analysis, and general desktop use. In addition, we are structuring the system in such a way that it will be easy to reuse by other researchers in the field.'"

Comments

Sign of times to come?

[Anonymous Coward]

Now that they will be able to recognize and tag our images, I wonder if Picassa will finally get increased storage. Google will be able to deliver targeted ads based on our pictures.

Re:Sign of times to come?

[Instine]

What about a free service to upload scanned images to and recieve html in return?... Please....

The goal of the project

[user24]

The goal of the project is to stop the damn email image spammers.

among other things, sure, but it's got to be a high priority for google.

Re:The goal of the project

[sammy baby]

And of course, as a side effect they'll probably wind up with a lovely distributed system for solving captcha. ;)

Re:The goal of the project

[ajs]

The goal of the project is to stop the damn email image spammers.

among other things, sure, but it's got to be a high priority for google. I don't buy either one. I think the goal of the project is to get sued.

Google knows darned well that there are tons of patents around OCR, so they're not going to roll their own internally. Instead, they'll open source the project and make as much noise about enhancing the state of the art through collaboration as possible. Then, when they get sued (and they will), they can bring this case front-and-center in the debate surrounding patent reform, citing this as the textbook example of how the promotion of the sciences and useful arts (as specified by the Constitution) is hobbled by current patent law surrounding software.

I could be wrong, but they'd be stupid to think that high-profile, open source OCR software won't be challenged by those who hold the patents....

Re:The goal of the project

[slashbob22]

Ok, I'll bite and play DA for a bit.

Why Google wouldn't want this:
1) Google's own patents on search techniques, distributing advertisement, etc. Yes, I understand that you are talking about the need for certain limits to patent law - but this could as easily hurt Google as help them.
2) They are already challenging the copyright laws with GooTube, I don't see any sense in tackling both at the same time.

IANIGHQ (In Google's HQ) but I don't see the value of getting sued at this point in time. Besides, if Google is doing this under appropriate conditions there shouldn't be concern of suits - but I suppose their Chinese plagiarism case doesn't support this point.

// End DA

Re:The goal of the project

[ajs]

Why Google wouldn't want this:
1) Google's own patents on search techniques, distributing advertisement, etc. Yes, I understand that you are talking about the need for certain limits to patent law - but this could as easily hurt Google as help them. Google takes the same stand on patent reform as IBM, as far as I know: the current law hurts innovation. They're not looking to have all of their patents stripped, just to reform the system so that innovation is encouraged. At the very least, IBM has (and I think Google too) lobbied for open source exemption. Keep in mind that IBM and Google hold tons of patents, but they mostly use them as a "warchest" to dissuade others from filing patent-related suits.

2) They are already challenging the copyright laws with GooTube, I don't see any sense in tackling both at the same time. I don't buy that one. Patent and copyright law are radically different, and in the copyright case Google is just trying to argue for existing interpretation of the law, not a change.

Google is doing this under appropriate conditions there shouldn't be concern of suits That's not how patent law works. If someone holds a patent on looking at the pixel to the left of the the one you're evaluating, and Google's software does that, then the holder could sue. What's more, there are many dozens of such simple patents surrounding OCR. It's probably the second-most over-patented area of CS next to color-space management.[1]

The beginning of the end?

[Iphtashu Fitz]

... for Captchas? If Google is pushing OCR I could see it eventually becoming good enough to parse at least some types of captchas.

Re:The beginning of the end?

[lawpoop]

I doubt it.

Part of what makes OCR work is that it assumes that the text was written to communicate meaning. It has regular characters in alignment, forming common words and abbreviations in more or less grammatical sentences with close-to-proper punctuation.

A good captcha has a non-sense string of characters in various cases, all skewed and distorted, with extra geometric elements obscuring the characters. This renders unavailable somewhere around half of the clues that an OCR uses.

Re:The beginning of the end?

[user24]

Please, please, please, everybody, stop claiming that "what is 2+2?" is a hard AI question. I could code something in a hour to defeat most of this sort of question, and give me a week and a budget and I'll write something to get past 95% of these type of questions.

If the text is parsable, it takes nothing to google it.
I mean, those two examples you give; just slap it into google and screenscrape it. So you're going to need harder questions than that.

So the next generation of crapchas will ask "what color is the sky".
Go and take a glance at ultraHal or another relatively advance NLP AI; a large knowledgebase is not hard to construct. When it doesn't know, it guesses. If it gets it right, then the knowledgebase increases by one fact.

So then, what, you have to ask "Given that all bleeps and blue, and blank is a bleep, is blank blue?"
Not only is that also easily computationally solved, but also a lot of people aren't going to be able to answer (smartass questions about stopping spam and idiots aside)

So *then* I suppose you have to ask "In the first mathematical antimony, does Kant conclusively prove both that there can have been no beginning to time and that there must have been a beginning to time?"
and give the user a 255 character textarea to put their answer in.

So... please, text question based captchas are DOOMED TO FAIL. stop thinking that they could work. They can't.

the presidential papers

[User 956]

The goal of the project is to ... deliver a high quality OCR system suitable for document conversions, electronic libraries, vision impaired users, historical document analysis

So, will it work on documents written in crayon? It would be a tragic loss for Dubya's presidential documents to get lost in the sands of time. On the scale of the library of Alexandria. No, seriously.

Finally...

[Searinox]

An OCR system that runs on Linux. I've been waiting for quite some time for something like this.

Re:Finally...

[Feyr]

have you tried gocr? it's nice as a random number generator, but beside that... it's pretty much garbage

Very cool.

[Kadin2048]

I've been hoping that someone with deep pockets (Google, IBM, Sun) would take on this area for a while.

There is a major need for an OSS OCR package, and right now the field is pretty bare. There's GOCR, and a commercial offering called OCRShop, and at least that I've run across, that's about it. Nothing really on par with Omnipage, or other commercial packages for other platforms.

I think there are some really neat applications for OCR that have never really been investigated, because it's so expensive to build that capability into other products. A free OCR engine that really worked could lead to some very neat book-scanning applications, just for starters. I don't think that there's really any integrated packages around for helping people scan books and manuscripts. (Right now you have to photograph the pages, keep them organized, then OCR them and proofread the text against the images. Bit of a nightmare.) I'd love to see a free application for libraries that let a user batch scan (via a digital camera -- let's not get into what I think of SANE and scanners generally) a book, and then provided a nice interface for proofreading the OCRed text against the original image.

Something like that could have a huge social impact. There are a lot of libraries where I'm sure they'd love to scan some of their out-of-copyright assets and provide them to patrons in a digital form, but it's just too technically complicated. An easy-to-use program that let the proofreading be done by nontechnical users (maybe remotely, as long as we're dreaming) could vastly increase the volume of digital materials available.

Small price if it helps email spam.

[Kadin2048]

And of course, as a side effect they'll probably wind up with a lovely distributed system for solving captcha. ;)

True, but CAPTCHAs always seemed like a bit of an inelegant hack anyway. First, they're horrible from a disabled-access standpoint, and second they're really not all that effective against a concerted enemy when there's a lot of money on the line. Spammers can just pay a few kids in some Third World country to sit there all day and solve CAPTCHAs if they want to.

Since message boards, which are the major users of CAPTCHAs, are practically by design little fiefdoms, I don't think they're nearly as hard to patrol as a common-carrier network like email. The solution to message-board spam is to either institute a moderator-delay (for small blogs and boards), or simply make enough admins with IP-ban powers so that the second someone starts spamming, they get banned and the spam gets deleted. Lameness filters working on the same principles as email spam-filters are probably helpful, too.

Re:Small price if it helps email spam.

[Pxtl]

You've obviously never fought off a bb spammer. They don't use one or two accounts to spam one or two messages - they inundate the board from a long list of IPs. Even without spamming messages, they create hordes of accounts just for the pagerank provided by the links within their personal account pages. Plus, admin-approval-delays degrade quality for the user. It creates a huge headache all around to handle maintaining banlists and cleaning out garbage.

Captchas are by far the better solution.

The problem is that, long term, they will eventually be cracked. I'm imagining the ultimate solution will only be to allow users with email addresses from "approved" major ISPs.

Orcopus?

[voice_of_all_reason]

Orcopus:

Level: 15
Race: Fell Marine
HP: 290/290
EP: 200/200
Water elemental
Drops: Tentacle

Wonderful!

[jshriverWVU]

This'll be a much needed boost for us Linux users who want to help out Project Gutenburg.

captchas

[gEvil (beta)]

All you people who are worried about this breaking captchas seem to be missing something--there have been a number of fairly decent OCR packages out there for a long time. The goal of this Google project is to create an open-sourced one that does a good job deciphering HUMAN-READABLE TEXT. Captchas are far from human-readable (the good ones at least), and I seriously doubt this project will help very much in that arena.

Re:So much for captcha

[cyphercell]

Captcha (warped text) will probably remain for a long time. This OCR has more practical uses when applied to text that is meant to be legible.

searchable pdfs

[radarsat1]

Anyone know of an open source utility that can convert scanned image-based PDF files into searchable PDFs ?
(Extra points if it somehow re-generates the actual file so it looks nice instead of pixelated.)

Perhaps this library could be used to build such an application if none exists...

Language?

[ceeam]

English only I suppose?

Re:Language?

[fireboy1919]

Since the official language of the Googleplex is Googlese, and the original project was developed by the US Census bureau - notorious for their use of no languages except Esperanto, it goes without saying (though I'm saying it anyway), that it will read only Klingon.

Remember kids, there are no stupid questions.
Only people who don't RTFA who ask questions.

Re:captchas

[arrrrg]

Captchas are far from human-readable (the good ones at least), and I seriously doubt this project will help very much in that arena.

Um, the whole point of CAPTCHAS is that they are human-readable (albeit often not easily so) but not machine-readable.

Well...

[Shawn is an Asshole]

If you're sick of image spam, you can do what I did. Add the OpenProtect channel to SpamAssassin and then add these line to your SpamAssassin config:

required_hits 5
score SARE_GIF_ATTACH 5

I don't see image spam any more. I resorted to that after I was getting a hundred or so of them a day.

More build info; Ubuntu Feisty

[drinkypoo]

Here's some other information you might need/want to build this software; note that I am on Ubuntu Feisty.

To build tesseract-ocr you must install autoconf.

If you are smart you will figure out a way to omit ocropus/data-test-pages from your checkout. Do you really want to use their pages to test? No! You want to use your own data.

I built with gcc 4.1.2, YMMV. Some people have reported errors trying to just compile tesseract.

to build ocropus I ended up installing jam, libaspell-dev and libtiff-dev.

Re:captcha's

[cmacb]

captcha's are not restricted to images of letters. For example: you could ask people to solve a regular text question (this would also fix accessibility issues)

You mean as in:

Describe what the following expression does in 30 words or less:
{"ab", "c"}{"d", "ef"} = {"abd", "abef", "cd", "cef"}

Man, I'll never get into forum postings if they do that!

The CAPTCHA solution

[dj245]

Look, any illiterate kid in a third world country can play type-in-the CAPTCHA all day long. I think the solution is to put up an array of 9 or so pictures, and ask the reader to click on the kitten. The other 8 being something other than a kitten, and all the files having random names which rotate with every view. You could also change the item being asked for to defeat simple image recognition, and have several pictures of kittens/what-have-yous.

To defeat *this*, you would need someone with a greater command of the english language than simple recognition of characters, or very advanced image recogniion software. I wouldn't worry about the software anytime soon if you chose images carefully.

Re:captcha's

[JamesTRexx]

Describe what the following expression does in 30 words or less:
{"ab", "c"}{"d", "ef"} = {"abd", "abef", "cd", "cef"}

Answer: Makes my head hurt...

*click* Access to MySpace granted, have a nice day.

Which forum were you taking about again? :-)

Re:Where's the Package?

[drinkypoo]

Do you have a result from scanning Jabberwocky (or other verse in a similar vein) with Google's OCR?

Just for you, I made one, because I'm that fucking cool.

1. Visited http://www.jabberwocky.com/carroll/jabber/jabberwo cky.html.
2. Printed page 1 (all but one link at the bottom of the page) with default settings on a HP LaserJet 2300.
3. Scanned on an Epson 3170 as a 300 dpi grayscale PNG with otherwise default settings. (God DAMN this scanner is fast. But then my scanner at home is a shitty Mustek 1200UB since I broke my Canon LiDe.) 2528x3281 pixels.
4. mespinoza@sec2lpt7-linux:~/ocropus/ocropus-cmd$ ./ocropus ocr ~/Desktop/out.png | tee /home/mespinoza/Desktop/jabberwocky.html (lots of output)

Prepare to be unimpressed, because Results follow:

JABBERWOCKY Lewis Carroll

(from Through the Looking-Glass and What Alice Found There, 1872) `Twas bri11ig,_ andjghe 4s1it_hy toyes Digl gyre amid gimblejn thg wabe: All xiiimsy wei^e thg borogovgs, And theamome raths outigrabe. ''ggwqre thg Jalgbervvpck,_my sqn! The jaw; that bijtel the clayksathat catch! Bgyvaiie the Jubjub bird, anti shun The frumidus Bandersnatch!' I-Ie took his yorpal sword in hand: Long timg tlgewmangome foe he sought So rgSted he by the Tu_mtum tree, And stood awhile in thought. And, as in uffish thought he stood, The Jalgbgjwoclg, with eyes of flame, Cqmgwhjfflixgg through fhe tulgey wood, And burbled as it came! Qne, two! One, two! And through and thIi`Ollgh The jrorpgal b]ade went; snicaker-snack! I-Ie left iifdead, and with its head He went galumphing back. ''And, has thou slain thejabbexfwpck? Cpmg to my a_rxps!_my ljgaxjgishboyl Ojralqjousi dwgy! Qalladhl Callayl' He chortled in his joy. S

\ A S

X A ?`^s :

, ' Was ga. ka%#* mm. -- M 1 1 Q at ) a iv 2. `Ail A it 3*,* `i 2 (V H ;. ````( * 4 ^Nq@ Eu..*s..%im X M is ? lgh ~ ``A? S [ A Fax I /),2*gE it ^`* 4 ~ *: ' X A mg x ix, ,t~;;;..: v' it ix '~ t ~ ^ ,4~ ---= =-^ A A i gv ; * XX, x> . . N S A ft 1 A-`A 3; `> ' ''YY \Jh ^***`(?i* , ~~ x `* at -;v- *<~ ' H ~~~-=.- ; `Twas bri11ig,_ and_the 4s1it_hy toyes Dig gyre arid gimblejn the wabe; All Qiixjnsy wei^e thq borogovgs, And thdmome raths outvgrabe.

dshaw@iabbenNockv.com

Return to Glorious Nonsense Return to Lewis Carroll

Results End.

Beautiful, eh? I also tried a 100 dpi grayscale scan, which came out even more like hash (one big paragraph) and a 300 dpi bitmap (1bpp) which was about the same as the 100 dpi gray scan in quality, though a bit better.

Looks like ocropus has a while to go before it can slay the Jabberwock instead of thejabbexfwpck.