Optical Character Recognition Still Struggling With Handwriting

Ian Lamont recently asked Google if they planned to extend their transcription of books and other printed media to include public records, many of which were handwritten before word processors became ubiquitous. Google wouldn't talk about any potential plans, but Lamont found out a bit more about the limits of optical character recognition in the process: "Even though some CAPTCHA schemes have been cracked in the past year, a far more difficult challenge lies in using software to recognize handwritten text. Optical character recognition has been used for years to convert printed documents into text data, but the enormous variation in handwriting styles has thwarted large-scale OCR imports of handwritten public documents and historical records. Ancestry.com took a surprising approach to digitizing and converting all publicly released US census records from 1790 to 1930: It contracted the job to Chinese firms whose staff manually transcribed the names and other information. The Chinese staff are specially trained to read the cursive and other handwriting styles from digitized paper records and microfilm. The task is ongoing with other handwritten records, at a cost of approximately $10 million per year, the company's CEO says."

Comments

Beat up Martin = Eat up Martha

[Joe The Dragon]

Beat up Martin = Eat up Martha

Translation server error

[BadAnalogyGuy]

Now that's a name that'll be remembered

Better approach?

[mandelbr0t]

It seems to me that it would be better to OCR everything and contract the proof-reading to the Chinese firm. The wide variation of writing styles and letter forms may make 100% accuracy of OCR impossible for this task, but starting from OCR should reduce the task, shouldn't it?

Re:Better approach?

[TheSHAD0W]

I suppose it depends on how you go about it; correcting specific errors may require more therbligs than typing the entire words.

Re:Better approach?

[Miststlkr]

In this particular case [the Census reports] there are so many alternative ways of spelling a lot of names, who is to say "Alyse must be a typo... make it Alice. And here.. Change Stefanie to Stephanie" In the situation of historical documents where names were less prominent I'd say I like your suggestion though.

Re:Better approach?

[perlchild]

And if the OCR has a mistake, you gotta look for the original? Sometimes having the starting point being wrong leads you in the entirely wrong direction, but you don't know it's wrong. Since these would be people for whom english is a second(at best) to foreign(at worse) language, wouldn't that make them especially vulnerable?

Re:Better approach?

[Anonymous Coward]

OCR for handwriting is terrible, and I can imagine that it would take less time to manually enter data rather than spend time doing an OCR and hiring people to fix errors.

Re:Better approach?

[shawb]

I would imagine that the proofreader would have the computerized text and an image of the original text side by side for comparison.

Re:Better approach?

[Anonymous Coward]

Your results are back from the proof-readers.

"For score and severe gears ago oar fumbling fatters..."

Re:Better approach?

[mrsteveman1]

Chinese proof-reading? Only if you want your documents in Engrish.

Re:Better approach?

[Midnight Thunder]

It seems to me that it would be better to OCR everything and contract the proof-reading to the Chinese firm. The wide variation of writing styles and letter forms may make 100% accuracy of OCR impossible for this task, but starting from OCR should reduce the task, shouldn't it?

It would probably be more costly to OCR it and then proof read it, especially if the error rate is higher than a certain amount, say 50%. There are written texts that I have a hard enough recognising, and only context allows me to work out what something is, so in this case a person is still a better resource.

Re:Better approach?

[PeeAitchPee]

No.

I own a microfilm digitization / OCR shop. We work with tons of old records such as the ones referenced in this story, as well as old HR docs, check stubs, time cards, architectural drawings, you name it. If you OCR cursive, you don't get back 80%, or 70%, or even 30% accuracy . . . you get back a bunch of pseudo-random (to our eyes) characters which are in NO WAY related to what the actual text is. About the only handwriting recognizable using today's tech is block-print, like you find on engineering diagrams. The technique in this article is pretty standard operating procedure, and has been for some time -- much easier to put a few hundred people on the project and grind through it (and cheaper too compared to data entry rates here in the US -- about 1/3 the price). That usually includes double-keying to check everything and a 99.99999% accuracy guarantee.

Just FYI, there are only a few OCR engines out there. Probably the most commonly used is the ABBYY engine, which is both OEMed and sold directly as desktop- and server-based products by ABBYY. There are a few others as well, and despite their differences, most have pretty much the same capabilities and accuracy. But OCR of cursive, especially of the docs cited in the article where you don't have someone sit down and "train" the machine first with handwriting samples, is still one of the great "unsolved" computing problems. I expect we'll have the capability in the next decade or so as processor core density, memory, and storage continues to increase at their current rate -- eventually, the machine will be able to "brute-force" through the docs just like the Chinese data entry folks in this article.

Re:Better approach?

[B'Trey]

Doesn't this suggest an obvious solution to CAPTCHA? Just use cursive text rather than try to obscure the text with funky backgrounds. If the spammers do manage to crack the CAPTCHA, then incorporate their technology into mainstream OCR programs.

Re:Better approach?

[GIL_Dude]

As an industry expert I imagine you know a whole lot more about this than I would - and I am sure you are completely correct.

Perhaps the cursive issue has to do with the effective resolution you can get from the old paper scans? I know using the tablet edition of Windows Vista I can get much higher than 90% recognition of cursive input on the tablet. However that is probably due to the fact that no scanning is needed: Windows has a basically perfectly resolved snapshot of what my scribble looks like without trying to deal with saturation levels, color variations, or anything else involved with scanning. My experience though with tablet shows me two things:

1) Actually OCR'ing cursive is probably more a function of being able to accurately scan pen and ink writing than it is a function of "cursive is hard to decode".
2) My preconceived notion of "this should be easy" based on my tablet experience is just wrong as I had totally forgotten that actually scanning the text written on yellowing paper with real-world pens (skips, etc.) doesn't even come close to OCR'ing something drawn on a screen.

Re:Better approach?

[mi]

I expect we'll have the capability in the next decade or so as processor core density, memory, and storage continues to increase at their current rate -- eventually, the machine will be able to "brute-force" through the docs just like the Chinese data entry folks in this article.

In the next decade or so we will have increased our processing power about 1000 times over. This work is scalable "sideways" — two pages can be processed by two computers independently. Which means, a thousand of today's computers could've done the work @home-style.

The problem is not with the processing power — it is the lack of algorithms. You and I reassemble the hand-written characters quite differently from how today's computers do it. The software will need to be created — and it is not the lack of CPU/memory/storage power, that's holding it.

One thing for sure is that the new algorithms will need to use the spell-checking engine(s) to better guess, what the next letter might be. On top of that, they would need to be equipped with grammar-checkers too, to be able to guess the next word, however illegible. Human speech (and thus writing) is quite redundant often — even if a misplaced coma can reverse the meaning on occasion.

Our brain certainly uses its knowledge of both the general rules of the language and that of the domain of what's written — this is why another doctor can decipher another doctor's handwriting, for example, that's infamously illegible to mere mortals. The software will have to do the same — and it can start doing it already.

Re:Better approach?

[Hognoxious]

You'd be trading false negatives for false positives. Based on TV programmes where they trace people's ancestry, It's hard to tell what language most cursive writing is supposed to be in, let alone read it.

Then again, my handwriting is so bad I've seen people turn it the other way up.

Re:Better approach?

[fyoder]

Teaching someone English at that level would be more difficult that teaching them to recognize characters. In ancient Rome the people who engraved dies for coins weren't always literate, but they managed for the most part to get the inscriptions right. Barbarians who made copies had more trouble, but then perhaps they thought the inscription part was purely decorative allowing for artistic interpretation. Or perhaps they weren't flogged for making mistakes. Point is, you can copy without having the high level of literacy required for proof reading.

Re:Better approach?

[PeeAitchPee]

Agreed . . . part of the problem is lack of algorithms, especially for complex cursive handwriting with no prior "training." However, OCR is tremendously resource-intensive. I'd actually put it up there with video editing as one of the most resource-hungry things you can do with a computer. E.g., we do a lot of 35mm newspaper microfilm conversion to searchable PDFs. Your average roll of newspaper microfilm will take *hours* to OCR -- that's on our eight-core dual quad-Xeon box with 8 GB RAM running the latest version of ABBYY's engine, multithreaded, 100% CPU utilization. Part of the time is consumed for simply opening the large amount of image data and having ABBYY set up the job. Part of it is layout analysis. Part of it is the actual OCR. The remainder is the output stage -- writing all of that stuff to disk in different file formats. Now, let be honest: no one's going to pony up the cash to for an "at home"-type network for OCR. So while theoretically it may be possible to do the processing now, it's only been in the last few years that it's been technically possible and commercially *practical* to do this type of work, quickly, on a large scale. E.g., we're currently OCRing 2.1 million HR docs. It's going to take us approximately four months to complete the job with the above machine running 24 x 7 (multiple PDFs have a tremendous amount of overhead to open, read, and write).

Re:Better approach?

[entgod]

The problem is that people easily read over spelling mistakes as if they weren't there. Less mistakes end up in the final version if all of it were done by hand.

Re:Better approach?

[story645]

I know using the tablet edition of Windows Vista I can get much higher than 90% recognition of cursive input on the tablet. However that is probably due to the fact that no scanning is needed:

You're also constantly teaching the computer to recognize the handwriting by accepting good texts and rewriting bad ones (or choosing the match.) And as the parent said, training the comp helps a lot. I've got an XP tablet and lousy handwriting, so my recognition is usually around %30, up to maybe %50 if I take my time to form clear letters.

Re:Better approach?

[Anonymous Coward]

It's not resolution, your tablet has less resolution than a scanner. Tablets can do a lot better because they can keep track of which lines you made first. The problem is much easier when you have an ordered and timed series of strokes to examine compared to when you only have a finished picture to look at.

Re:Better approach?

[jonaskoelker]

Point is, you can copy without having the high level of literacy required for proof reading.

You can copy without having even the tiniest understanding of the language; just put a large table of symbols up on the wall and look at it every day, you'll know which doodle maps to which symbol. Ask your more experienced colleagues when you're in doubt.

I'm basing this on having copied one or two hundred Cyrillic letters before learning anything about Russian pronunciation [and my current understanding is still rough]; I'm sure that with enough practice, one could get a solid grasp of which letter is which without learning the language. I haven't had a good opportunity to read something written by hand using the japanese phonetic alphabets (hiragana and katakana), but I suspect that by just knowing the symbols and not understanding the language, I could copy them just fine with enough experience.

Re:Better approach?

[Tacvek]

You'd be trading false negatives for false positives. Based on TV programmes where they trace people's ancestry, It's hard to tell what language most cursive writing is supposed to be in, let alone read it.

Then again, my handwriting is so bad I've seen people turn it the other way up.

Wow! I thought my handwriting was bad, considering that I have been known to have trouble reading it myself, but nobody has ever turned mine upside-down before.

Re:Better approach?

[Anonymous Coward]

Mod this up please.

I also work in an industry which relies heavily on OCR technology and would be considered knowledgeable on the subject at a technical level.

OCR Engine makers like Oce www.oce.com have made big inroads into handwriting but its way off what it needs to be.

A big issue is many of the "products" which leverage these OCR engines only use one engine - ABBYY.

There are recognition platforms like TiS eFlow which leverage multiple engines and itâ(TM)s the reason their used in many of the bug OCR shops.

Re:Better approach?

[Anonymous Coward]

You're also forgetting that your tablet has information in the time domain - it knows how you drew it, not just how it ended up looking.

Re:Better approach?

[lysergic.acid]

ok, did that second sentence make absolutely no sense to anyone else until they actually looked up the word "therblig?"

and who the hell makes up a new word by reversing their last name? honestly, who does that?

Re:Better approach?

[neonsignal]

Yes, your second point is correct. Tablet recognition is not quite the same, because it typically records the strokes as they are made, rather than scanning the result. It has information about timing and sequence of actions, and can easily ignore pressure/width and small breaks.

An interesting experiment is to write characters on the tablet in unusual ways (eg, starting at the opposite end of a '7', drawing a letter in out-of-order segments, etc). It will look the same, but the recognition software will struggle to decipher it.

Re:Better approach?

[xenocide2]

Vista's handwriting was the first Windows to accept training. I'd wager one reason is that there's no way the average user would know something's gone wrong or what a bad sample looked like. But the tablet offered another improvement: intelligent scratch out. As an XP user, I'm sure you know what I'm talking about. There's a specific pattern to erase.

Of course, mileage varies, so you can't compare recognition rates between people effectively. Even less perceived recognition rates as stated on Internet forums. Personally, I find that Window's tool recognizes cursive better than print. I don't normally write in cursive so this is a bit annoying.

Re:Better approach?

[lysergic.acid]

$output =~ tr/[l,r]/[r,l]/ig;

Re:Better approach?

[Rocketship Underpant]

Handwriting input systems tend to rely heavily on watching the order in which you draw the strokes of various letters, and they're generally designed only to recognize letters written in a small subset of the various forms a cursive letter can actually take.

Re:Better approach?

[ralphdaugherty]

It seems to me that it would be better to OCR everything and contract the proof-reading to the Chinese firm. The wide variation of writing styles and letter forms may make 100% accuracy of OCR impossible for this task, but starting from OCR should reduce the task, shouldn't it?

      No, you may be confusing handwriting with handwritten characters. The summary said CAPTCHA's were broken, but all the examples given in past /. threads were of CAPTCHA characters that didn't overlap. I haven't seen any publicity on successful OCR of overlapping characters, much less cursive handwriting.

      Even handwritten characters on devices are really analyzed by analysis of strokes as they are made by the stylus to print the character, not by optical character recognition of the final character. And those are discrete, non-overlapping characters.

      Based on my reading from the frequent CAPTCHA threads, the algorithms to "break" CAPTCHA's are probably no more complex than density measurements which result in a few correct answers out of many attempts, and that's the trivial case of all upper case letters with perhaps numbers.

      And more than likely, they are also using the Chinese and equivalent to "break" the CAPTCHA's.

      Recognizing cursive handwriting is light years beyond these trivial cases.

  rd

     

Re:Better approach?

[mrchaotica]

Handwriting recognition on a Tablet PC is easier than with old paper scans for the reasons you mentioned, but also for another reason: time. Unlike with paper scans, Microsoft's handwriting recognition system also knows the order and direction in which you wrote each stroke. Because of that, the system may be able to correctly identify different letters even when the completed input looks exactly the same, because they were drawn differently.

Take a "7" and a "T" for instance: maybe in your handwriting they both look exactly the same (e.g., the top and right sides of a rectangle). But the handwriting recognition could still tell the difference because you wrote the "7" as a single stroke (line to the right, line down, no lift of pen) but wrote the "T" as two strokes (line down, lift, line to the right).

(Note: by the same token, my XP Tablet PC has difficulty distinguishing between my "t"s and "+"s. One possible reason for that is that I use the same sequence of strokes for both (line down, lift, line to the right).)

Re:Better approach?

[johanatan]

You may have been getting at this with #2, but I think the actual stroke information is one of the keys to why the Tablet outperforms--it looks at your writing as a collection of gestures and not merely the end result.

Re:Better approach?

[ralphdaugherty]

Perhaps the cursive issue has to do with the effective resolution you can get from the old paper scans?

      No, has to do with no breaks between the letters, similar to wordsruntogether but with cursive letters run together for each word.

  rd

Re:Better approach?

[TheNetAvenger]

Couple of points...

1) MS Research probably has some of the best work being done on handwriting recognition, including imaged documents. However it is no where near the needed levels. Google would be better off to work with Microsoft on stuff like this, than the motto of screw anything MS is doing and we will recreate it ourselves.

2) On your Vista Tablet PC, the reason you can get 90-99% levels of recognition is that TabletPCs and Vista/Windows use a concept called 'ink' (that goes back to early work at Microsoft from the late 80s)

Ink not only stores a 'picture' (Bitmap/Vector) of what you wrote, but also the stroke pressure, speed, direction, and order of each movement. So even if it doesn't look like a 'T' because you use stokes that normally would make a T, Vista can figure this out.

Because of doing it different than reading the 'image' like OCR has to, you can you use cursive, printed, combinations or whatever and Vista can figure out from the motion and stoke more than from what it looks like.

Ink is also why Microsoft holds a lot of respect in industries that use handwriting device and TabletPCs like the medical industry, as it can even read Doctor's handwritting. Ink also holds more data that can be further looked at later on for more advanced processing of intent or even how the person writes. This is also why Vista (go look up YouTube demonstrations) is far ahead of handwritting technology in other OSes, like OS X.

Ink is also a 'crucial' data type that is not an image or a word, and one reason MS has been fighting for OOXML formats, because it retains this data like in onenote and Winword. Without the pen stroke information, ink become worthless, as it is no longer information about what the person wrote, and just a crappy image.

Re:Better approach?

[Anonymous Coward]

Don't forget that the computer can capture metadata about your handwriting as well. Things such as the path you took to write the text, as well as the speed at any given point and possibly the angle of the pen and pressure as well are all very useful extra data points that you can't necessarily reliably extract from ink and paper.

Re:Better approach?

[TheRaven64]

You and I reassemble the hand-written characters quite differently from how today's computers do it

And often we fail. A lot of signatures are unreadable by a human. Often when I let someone annotate a printed document, I find I'm unable to decode at least one of their comments.

Our brain certainly uses its knowledge of both the general rules of the language and that of the domain of what's written

OCR has done this for ages - for each region, it generates a list of potential letter-sequences with a probability attached. It then gives a higher weighting to the ones that are real words. Sometimes it gives a higher weighting to sequences of words that are grammatically (although not necessarily semantically) valid. The problem is that a lot of the time people writing with a pen use some form of shorthand, and the rules for this need to be programmed in to the software. If you're scanning prose, then you can get much better results than if you're scanning notes.

Re:Better approach?

[BenoitRen]

I was thinking the same thing. But then I realised that one way to defeat CAPTCHAs is to hire a lot of people in India and China to solve them.

Re:Better approach?

[gbutler69]

Actually, the reasons it works better are mostly the following: 1. It can track your strokes and calculate vectors for each stroke indicating direction, length, etc. It can even quickly fit a curved stroke to a bezier curve. So, when you are done "stroking" (no pun intended), it has a mathematical description of each stroke, not just a static image that it then has to break down. 2. It "learns" how you tend to write things. Everytime you write something and then make a correction to what it determines you wrote, it uses this information to refine it's concept of how you write the letter "A" for example. Of the two, I'm fairly certain it is "1" that gives it the most mileage over OCR.

Re:Better approach?

[Anonymous Coward]

Whoosh.

Re:Better approach?

[Anonymous Coward]

Dude, have you never watch "Cheaper By The Dozen?" - the original? Its worth a watch. My SO is a big classic movie movie fan and forced me to watch this one evening. Much better use of 2 hours than most of the other movies I've seen in the last few years.

Re:Better approach?

[Anonymous Coward]

It gets even better- when you're dealing with documents spanning more than a few decades, you run into issues with language-shift.
Spellings change, grammar changes, even the style of writing (font) that is taught (and then mangled) in schools changes fairly rapidly.

As an example, look at some of the typeset used in the early and mid 1800's - you will see letters that look like a lowercase 'f' but are really a lower case 'l'.

In gradeschool I was taught this garbage "D'nealian" or something, the uppercase cursive 'Q' looks like an 'L', but that was changed in the 90's IIRC.

It's quite a task, and as mentioned above in many cases a 30% successful OCR scan will yield 0% useful result, or in many cases the results will take more time to sort out than by doing it all by hand to start with.

I really don't see how this bears any relation to captchca, since the only way you could use this with a captchca system is to have thousands of people with sloppy handwriting generating your turing test- and can just as easily be countered by humans reading the captchca, which is primarily how they are being 'broken' already.

Half the time..

[Miststlkr]

I can't even read people's handwriting, I hardly expect a computer to.

Re:Half the time..

[glwtta]

Hell, I can't even read my own handwriting. Yeah, this is probably not going to happen.

Re:Half the time..

[Kristoph]

I've been using a computer since I was a kid, 25 odd years now. I can't write. I don't believe I ever really learned it.

I can print if I have to, though I usually ask my wife to do it because my hand gets sore after filling out a one page form. (In contrast I can easily type for 14+ hours at a stretch.)

I guess I get the point of handwriting recognition, for historical documents, but do we really need it for future devices?

Re:Half the time..

[Anonymous Coward]

How the hell did you get through school?

Re:Half the time..

[Anonymous Coward]

I can't recall pi to a million digits, but I do expect a compute to.

Re:Half the time..

[Anonymous Coward]

I've been researching John Steinbeck's personal correspondence recently. Even with familiarity, his writing can be quite difficult to read. While reading a letter or trying to figure out the names he wrote on a photo, I feel sorry for his wife (Carol, at least) who did a great deal of transcription for him. Even though Steinbeck's typing is horrible, it is a huge relief to deal with his typed documents after a session with his handwriting. His handwriting is very neat and consistent, and even so, is monumentally difficult to read. It's difficult enough to justify getting the original documents (e.g., going to Stanford for the Special Collections instead of dealing with scans). I cannot imagine OCR managing it.

Re:Half the time..

[Anonymous Coward]

You can't even write enough to fill out a one page form? What are you, a pussy?

Re:Half the time..

[Nutria]

I've been using a computer since I was a kid, 25 odd years now. I can't write. I don't believe I ever really learned it.

That's pathetic. Your parents and grammar school teachers should be caned.

http://www.washingtonpost.com/wp-dyn/content/article/2006/10/10/AR2006101001475_pf.html

The neurological process that directs thought, through fingers, into written symbols is a highly sophisticated one. Several academic studies have found that good handwriting skills at a young age can help children express their thoughts better -- a lifelong benefit.

Re:Half the time..

[TheRaven64]

And did any of these studies compare expressing your thoughts with a pen to expressing them with a keyboard? I can barely write anymore. When I was 14 I started being allowed to submit some essays at school in typed form, and at university the only things I ever hand wrote were exams. I'm now 26, have a PhD, and had my first book published last year. It's rated 4.5 stars on Amazon, with comments praising the clarity of expression. I type something on the order of three thousand words a day, but I'd be surprised if I wrote more than a thousand words a year with a pen.

The linked article doesn't cite sources, but I'd be surprised if they managed to find a group of children who learned to type first. If you are comparing those who didn't learn to type or write to those who managed to learn to write, then I would not be surprised by their results.

Re:Half the time..

[houghi]

Mine has gotten worse in the last years. This basically because I seldom write anything anymore and use other forms of keeping data. I assume this will only get worse.

Use them as CAPTCHA...

[mevets]

1. Use the handwritten words as CAPTCHAs
2. Wait for the bad guys to come up with programs to break them.
3. ...
4. Profit!

Re:Use them as CAPTCHA...

[Anonymous Coward]

The difference is that a success rate of say 50% is good enough when breaking CAPTCHAs but horrible when doing OCR on a document ( every other word will have a problem ).
Another issue is that most people can't read the old style cursive :-)

Re:Use them as CAPTCHA...

[aslvrstn]

Joking or not, that's kind of the idea behind reCAPTCHA. It takes words that OCR failed on and uses them as CAPTCHAs. The same idea could work for handwriting. http://en.wikipedia.org/wiki/ReCAPTCHA

Re:Use them as CAPTCHA...

[The Slashdotted]

CAPTCHA requres you to know what it says in the first place. You typed in Mary Jones, but that's not what my Chinese transcriber/OCR think it says. You could keep a database of failed CAPTCHAs and accept them as more people repeat, but then the bad guys will use the same bad entry over and over.

Re:Use them as CAPTCHA...

[TheSHAD0W]

But then you submit the same words to several different people and use statistics to pick the most likely answer - and forward entries with no likely answer to someone hired to do it.

It's very likely that the manual entry being done now is being done redundantly and then compared to find errors (and choose the best data entry operators).

Re:Use them as CAPTCHA...

[hypersql]

It works if done in pairs. Like the Google Image Labeler it would always take at least two people to solve a CAPTCHA. Two or more randomly selected people would be paired to transcribe the same image (hand written text snippet). If one of them is too slow, the image changes automatically. Only if the majority of answers is the same, it would be accepted.

Pairing doesn't work for small websites, because not enough people would use the service at the same time. It would only work for large sites. Somebody might start an independent 'human validation services', that is, a web site that does CAPTCHAs for many other (smaller) web sites. Um... sounds like a business plan.

Re:Use them as CAPTCHA...

[cnettel]

The other solution for pairs (which I think was also suggested by recaptcha) is to use two words. You are you told that you have to answer both right, but in fact at least one of them can be somewhat uncertain, and the system will accept your input if it matches for the already fixed image. The already certain set can start out as rather small and simple, but it will grow quickly, even for a small site. You can still require 4 or so identical answers (with no conflicting ones) to the same image for it to be added to the "safe" set.

Re:Use them as CAPTCHA...

[TheRaven64]

I'd not come across the Google Image Labeler before, but I have seen the idea before - it was a paper published by someone at CMU several years ago. He had a working implementation in flash and it was quite fun. Assuming the Google version works like the original, it doesn't need two people to be online at once - it can create virtual players from previously entered data. It is possible to spam it, however, and also possible for a simple program to game - most images ended up being tagged 'red' 'green' or 'blue' according to the dominant colour.

general OCR harder than CAPTCHA OCR

[Anonymous Coward]

There is a simple reason that general OCR is much harder than cracking a CAPTCHA. General OCR has to recognize text *reliably*. CAPTCHA breakers are thrilled with a 10% success rate, because they use distributed systems created by worms to do the hard work a million times over. If you got 10% of the words right when scanning historical records you might as well not bother.

Use the RECAPTCHA approach

[Anonymous Coward]

Just slice them into smaller word chunks, and have humans OCR them a word at a time, using multiple passes at the words to verify them. At the same time, verified words can be used as CAPTCHAs.

Too variable, less reference

[Coopjust]

An OCR program can include a bank of fonts, and even when there is some sort of spill/ink blot/whatever on the paper, it has a solid reference. Handwriting isn't so easy, because humans don't always write their "Q"s with the line in the exact same spot and other fluctuations. Even if you gave a computer a point of reference (neatly drawn letters corresponding with their actual alphabetical values), a computer probably couldn't get it for a lot of people with inconsistent handwriting.

Now, with context and improved technology, I don't think that handwriting recognition is impossible. I have a feeling that it will be a technology like speech recognition: never perfect, and it will require training.

Re:Too variable, less reference

[Miststlkr]

Personally, I think\hope that PDAs and smartphones getting more common will lead to some breakthroughs. My HTC TyTn was pretty decent at handwriting recognition as an input, far better than the old Palm Pilot I had used back in The Day For the time being though I definitely see it, as you mentioned, as a trained system as current voice recognition apps.

Re:Too variable, less reference

[Kickersny.com]

While handheld technology is indeed getting better, it's not directly applicable to the problem at hand. Real-time handwriting analysis uses stroke analysis as well as shape analysis to determine the letter(s). That is, the order in which you construct your letters matters very much. For example, if you crossed your T before drawing the vertical bar, the engine may have a difficult time figuring out what you intended.

When OCRing documents, all of that 'meta-information' is lost.

Re:Too variable, less reference

[cnettel]

You are right, but on the other hand a really good scan might be able to indicate the order. Paper that was already wet will react slightly different to the next ink stroke, etc.

Re: Too variable, less reference

[Alwin Henseler]

And on top of that, there's a direct feedback loop. If the machine makes a mistake recognizing the user's handwriting, the user can immediately correct that mistake. No such option with automated scanning/OCR. And in any process, direct feedback on the discrepancy between (what you wanted) and (what you have right now) can make a huge difference in the results.

Re:Too variable, less reference

[Anonymous Coward]

Not quite right...there's online handwriting recognition, where one approach has been to use pen stroke models as a way of recognizing letters; and there's offline recognition, where at least a subset of the literature from the 90s used the same approach, converting scans back into pen stroke models for recognition purposes. Another approach to offline recognition was to use statistical models (such has HMMs) which achieved 65-70% recognition for single writers. Not good enough for business uses, but enough to be helpful for scholars.

Re:Too variable, less reference

[trontracker]

One major problem is going to be developing an algorithm that just locates the letters before they are processed and identified. Since there are usually no neat spaces between them as there are with fonts, a scanned jpg or raster image will have to be analyzed to find the letters before processing can begin. This will pose a real challenge given the enormous variety of writing styles.

Optical Character Recognition is the Correct Term

[intrico]

For a moment there, I was picturing some new technology that could distinguish between C, PERL and and Java written on scratch paper.

Now you have a training dataset.

[bigattichouse]

Now you take the human translated recognition, and use it to train your genetic algo or neural net against the original images.

Re:Now you have a training dataset.

[Anonymatt]

Yeah, word up, right?

Re:Now you have a training dataset.

[caffeinemessiah]

Now you take the human translated recognition, and use it to train your genetic algo or neural net against the original images.

Sorry to be pedantic, but a genetic algorithm is a search heuristic, not a learning algorithm. You could train it to search for discriminative patterns in the training data, but it would almost certainly overfit because it's the wrong tool for the job. Neural nets, while more appropriate as a learning algorithm, have recently been usurped by Support Vector Machines (SVMs), which are much better at not overfitting.

But..

[Anonymous Coward]

I can't even read my own handwriting sometimes - how is a computer supposed read it; unless it knows what I was thinking at that particular point in time.

Presidential Book of Secrets?

[hemp]

I hope they didn't give them the Presidential Book of Secrets, we could all be in trouble then!

Note to editors

[Anonymous Coward]

OCR = Optical Character Recognition, not optical code recognition.

So...

[Sobieski]

Use handwritten CAPTCHAS?

New strategy

[Afforess]

I guess we should start making kindergartners write in "Times New Roman" from now on.

Re:New strategy

[Belial6]

You joke, but there really in very little reason to teach children handwriting/script/cursive (whichever you want to call it). The point of cursive was to speed up writing. It was never any good for readability. In today's world, if you need to write a lot of stuff, you are generally going to type it on a computer. Since just about anything that we would want to write by hand will be short, the speed gain would be minimal. Thus spending time and resource to teach every kid to write a useless, illegibly font is pretty pointless.

Re:New strategy

[Kral_Blbec]

You forgot that it also makes English teachers feel important and sophisticated.

Re:New strategy

[Anonymous Coward]

How do you expect them to sign for a cheque or a document?

One time the teller at my bank was giving me problem as I do not use a cursive signature.

Re:New strategy

[GrayNimic]

Speed-writing, of one form or another, is still useful for note-taking (in meetings, lectures/seminars, classes, etc). You can't have your laptop everywhere.

(and in some circumstances the keyboard clicking is loud enough to be considered disruptive - true, there are loud pens & pencils, but I run into far more loud laptops than scratching handwriting implements).

Re:New strategy

[Jorophose]

Except for most of us it's faster to write with your hands.

Writing by hand, you can jump letters and make abbrevs, you can draw diagrams right in there, and not to mention it feels a lot better. I don't know why but sitting and typing on my computer, and same when I used to paint minis, feels painful and stuffy. With the option of either typing or writing I'd definately take writing. Sure, with typing on a computer you can erase stuff quickly, but text editors have always been shitty for me (stuff like AbiWord often having graphical glitches or plain slow, text editors too or just lame feeling) and hitting a bunch of blocks to make words does not feel as good as actually writing down the words.

I never mastered cursive properly. I write "script", but write while skipping letters in my notes and using small symbols (batman symbol, drawn as a W in a circle, for example, is distress; three points is "donc", ds dans, etc and it changes depending on context). I write fairly fast, and imho much faster than when I type, if only because when I type I often hit the wrong keys; often being once a paragraph, and it's often because I can't get my mind straight on the keymap, or my fingers hit in the wrong order.

Re:New strategy

[DamienNightbane]

Sounds to me like you're just a shitty typist.

Re:New strategy

[Nurgled]

My signature is basically just a pair of wavy horizontal lines. To start with, the bumps were in the right places to vaugely resemble the letters of my name, but over time it's just become an indecipherable wavy line. I can't say I've ever been called on it. A signature should be able to be whatever you want it to be.

Re:New strategy

[kayditty]

With the option of either typing or writing I'd definately take writing. Sure, with typing on a computer you can erase stuff quickly

but you don't seem to bother anyway. maybe that's why you're so bitter toward computeringzmachines?? I'm going to pretend I didn't see the horrible grammar throughout the rest of your post.

when I want to write a story or something like that, I'd prefer to use my computer. of course, making drawings, doodles, or doing manual arithmetic is easier with a pen and pad, but you realize that we've had tablet technology for some time now? it's only a matter of time before it becomes ubiquitous, and it's only a matter of time before it improves in efficiency and accuracy. I have a similar problem to one of the earlier posters, though: print is tedious and causes hand cramps rather quickly. I don't think I reach that stage after a one page form, but writing a story in print is out of the question. my cursive has degenerated over the years with increased use of computers. but that is okay with me, because I type quite a great deal faster than I could ever write, even in script (130-150 WPM usually, on QWERTY).

Re:New strategy

[TheRaven64]

You can't have your laptop everywhere

I have a Nokia 770 and a folding bluetooth keyboard. They both fit in my jacket pockets, and the keyboard is very quiet to type on. The number of places where you are going to have a pen but not a keyboard is very small. Given how cheap storage space is now though, why would you even take notes? A modern mobile phone has enough capacity to record the audio of a lecture uncompressed, and enough CPU power to compress it quite heavily. Set your phone to record at the start of the lecture and take pictures of the board at interesting points (I've used my phone to take pictures of things people have drawn on whiteboards for later transcription, and mine is quite old and much lower quality than a modern one). Since any phone tags picture with the time it was taken, you can easily skip to the point in the audio recording when a picture was taken.

Re:New strategy

[TheRaven64]

Any shortcut you take when writing makes it harder to read, and generally if something is worth writing it's going to be worth reading more than once. With a half-decent text editor you can do word completion (control-p in Vim) for commonly-used long words which is even faster than writing the abbreviated form and much more readable. With a decent set of macros you can type shorthand and expand it to the full form on-the-fly. When typing LaTeX, I have F2 bound to my 'do what I mean' script, which parses the word I just typed, expands it to a commonly-used sequence of LaTeX commands and sets the insert point to where I will want to type next.

It's a mistake to try to use a text editor as a replacement for a pen (and it's a mistake to use a word processor at all in most cases). It is a much more powerful tool - one which understands commands as well as pure data and can perform complex actions based on user commands.

Re:New strategy

[burdock]

I don't care for word processors much either. Have you tried a text editor such as VIM or EMACS? Ktouch, a touch typing tutor for KDE, has helped me. Good ergonomics makes typing easier too. http://www.osha.gov/SLTC/etools/computerworkstations/

OCR == Optical Character Recognition

[rstanley]

C in this case means Character, not Code. See one definition.

I have never seen the word Code used in an English definition.

Re:OCR == Optical Character Recognition

[Ian Lamont]

Sorry, the use of "code" instead of "character" was my error. I corrected it in TFA, after being notified by a /. editor.

No wonder...

[jopsen]

Why's that a supprise... I can't read my own handwriting either... And I certainly can't read the handwriting written by someone else a hundred years ago... Why do you expect a program to be able to do so...

It's more complex problem than people imagine

[saigon_from_europe]

There is an on-line archive of all people that have passed trough Ellis Island (http://www.ellisisland.org/search/passSearch.asp). It consists of retyped (OCR-ed?) ship manifests. Manifests are lists of passengers, with names, places of births and similar information. In original, they are written by hand, in cursive scripts (as expected for late 19th and early 20th century).

Problem is not with the script, but with appropriate context. Someone who retyped this, did not know what to expect in these forms.

My grand-grand father's place of origin was written as "Lipovqani, Slovenia". Pair "lj" was recognized as "q". For someone who is native English speaker "lj" one next to other does not make too much sense. But for anyone with Slavic origin, "q" does not make sense (it's only in foreign words), and "lj" does make sense since it is a way to write "soft l" voice like in "Richelieu".

Ok, maybe that was not the an easy part to guess. But "Slovenia" was serious error. In that moment, Slovenia did not exist. It was part of the Austro-Hungary, and it did not exist as single entity inside it. What was really written was actually "Slavonia". That's an area in Eastern Croatia, and it *was* an entity inside Austro-Hungary.

Should I mention that I was not able to track my grand-grand mother and my other grand-grand father?

Re:It's more complex problem than people imagine

[John Hasler]

Are images of the original hand-written documents available on the Web?

Re:It's more complex problem than people imagine

[saigon_from_europe]

Yes, but you can see them only once you find something in the search. And they used to have some funny system trying to prevent people from printing original scans.

Re:It's more complex problem than people imagine

[Hognoxious]

Many of the immigrants were barely literate in their own language, let alone English, and so names and places might be recorded how the official thought it should be spelled. Maybe they were busy or annoyed and couldn't be bothered to check. They were government employees, after all...

On top of that you have people who don't wish to stand out or suffer discrimination and intentionally anglicise their names.

I'm with you 100% about context. And good luck with the searching.

Pretend its a string?

[British]

Can OCR properly trace the lines at least to replicate it? Meaning, it could make a vector replica of the handwriting? Would be neat if it could do that, then try to straighten out the lines, perhaps to simulate the possible path the original writer took to write it. Of course, the software will have to figure out intersections. Maybe a path of logic would be to know what turns a handwriter would NOT take, and then determine individual letters from that.

Combine that with other logic, like finding "dots" would indicate an i or a j, and maybe it will improve.

Re:Pretend its a string?

[John Hasler]

Clever.

Re:Pretend its a string?

[S3D]

Can OCR properly trace the lines at least to replicate it? Meaning, it could make a vector replica of the handwriting?

That is easy enough. Edge detection and morphological thinning can do the job.

Maybe a path of logic would be to know what turns a handwriter would NOT take...

And that is a real problem. Topological approach have limited usefulness - similar turns could make different letters. Statistical approaches like baesian networks , ANN can help here, but even human brain often have problems with finding pattern in the handwriting...

Re:Pretend its a string?

[krkoch]

Interesting. Strokes should be much easier to recogize. I find that tablet-computers actually translates my butt-ugly handwriting into plain text by analyzing my pen-movements. Maybe we should incorporate a model of how the human hand and motor-skills work in handwriting-recognition?

Re:Pretend its a string?

[ralphdaugherty]

Can OCR properly trace the lines at least to replicate it? Meaning, it could make a vector replica of the handwriting? Would be neat if it could do that, then try to straighten out the lines, perhaps to simulate the possible path the original writer took to write it.

      That was my goal many years ago. I got as far as analyzing characters into vectors, and that includes cursive writing. Then I was going to analyze the vectors just as you suggest. I got sidetracked into a career on the AS/400 iseries and never got back to it, but it is high on my list now.

      Here's a sample: http://www.rdwrites.com/forums/viewtopic.php?t=3794

  rd

This is simple to fix

[Toll_Free]

Get the guys writing the code that breaks captcha.

Simple, honestly. Make it economically worthwhile to write the code to do such. Writing code to break handwriting isn't as lucrative as say, writing virii or malware code.

Take a look at the results...

disclaimer: I doubt they will EVER break my doc's handwriting.

--Toll_Free

Re:This is simple to fix

[BPPG]

Or, use hand-written captchas.

Re:This is simple to fix

[John Hasler]

Problem is, the CAPTCHA breakers are only about 10-20% accurate. That's good enough for the spammers purposes, but not much else.

Re:This is simple to fix

[John Hasler]

I think it likely that the spammers could manage the 10-20% accuracy they need on handwriting.

Re:This is simple to fix

[Anonymous Coward]

Well, then I guess we'll leave the handwritten captchas to medical websites, then.

Re:This is simple to fix

[Toll_Free]

I often wondered if I couldn't get a consortium of Doc's to do just that.

I mean, NOBODY can decypher what they are trying to say. Not even my pharmacist. NOT even my friggin docs! One doc to another, they are lost.

--Toll_Free

Postal Service

[Darkfire79]

Doesnt USPS's system rely on Optical Charactor recognition? I thought it had a really high success rate... I know the software we used when I used to work on the Fed at the reserve wasn't all that good.. Anything it would reject would be sent to two people on a computer who would type in what they thought the letters were.. then if they matched it would go through, if not, it would go to a third person to make the final decision. After seeing that much handwriting I dont think we'll ever have software thats 100%.. Especially when you cant even read your own handwriting sometimes *chuckle*

Re:Postal Service

[Toonol]

I think it has to do with success rates. A 20% success rate is all that a spammer cracking a captcha needs to be profitable.

The USPS might see a return on investment if their OCR equipment works on 75% of text, routing the hard-to-read 25% to humans. That's a huge reduction in workload, because otherwise every letter would need to be scanned by a human

Legal and government text, however, needs to be 99.9% or more accurate, because one flipped character in a page of text can cause severe problems; that means everything needs to be proof-read carefully, eliminating the cost advantage of OCR in the first place.

And here's a somewhat related question: Is there good freeware or GPL'd OCR software usable on windows? I have a few dozen pages, scanned in as high-res PNGs, that I need to convert. Snag: It has some Kanji characters sprinkled throughout.

Re:Postal Service

[Taxman415a]

And here's a somewhat related question: Is there good freeware or GPL'd OCR software usable on windows? I have a few dozen pages, scanned in as high-res PNGs, that I need to convert. Snag: It has some Kanji characters sprinkled throughout.

Unfortunately no. The free OCR packages are not up to the task yet. I think you can still get a trial download of ABBY Finereader though.

Failing that if you can find a local service that caters to blind or low vision people, many do scanning for their group and/or community members. You may be able to work out a trade for service or something like that. Depending on what OCR package they use and the plugins etc they have and the skill of the person operating it though, you may be out of luck for the Kanji though. I've never tried it myself and I'm not sure how many of the standard packages recognize it out of the box.

Re:Postal Service

[Anonymous Coward]

Try Tesseract OCR. It was originally developed by HP and is now open under the Apache License.

http://code.google.com/p/tesseract-ocr/

Re:Optical Character Recognition is the Correct Te

[Alwin Henseler]

For a moment there, I was picturing some new technology that could distinguish between C, PERL and and Java written on scratch paper.

In pseudocode:

IF LooksLikeC THEN "This must be C code"
IF LooksLikeJava THEN "This must be Java code"
// undecipherable
ELSE "Must be Perl code"

ballpoints killed penmanship

[jannesha]

Back in highschool, I had a job that involved creating a database for a local cemetery's burial records. For 120 years, these records had been kept in a set of handwritten journals with a semi-alphabetical index. Given the time span, there had been many generations of people making these handwritten entries...and the differences in penmanship were outstanding.

Some time around the 1940's or 1950's, the job passed from a fountain-pen user to a fan of the ballpoint. Wow, what a difference. Early ballpoint pens were crap! Lots of lumpy smudges all over the place.

Ink quality aside, the shift to the ballpoint heralded the end of readable writing. Really, everything before it had been a beauty to look upon, and everything after was chicken scratches.

Mind you, this is all greatly anecdotal...it's just the handwriting of a half dozen people over a century. But I really believe that the 'convenience' of the ballpoint lead to people taking less care. Fountain pens required more care and skill, ballpoints lowered the bar.

Hmmm...I'll finish with some /. relevant content: I set up the database on an 80's era Macintosh 128K, working in my parent's basement.

New CAPTCHAs?

[Gizmoguy]

OK, so their CAPTCHA has just been broken, and computers cannot read handwriting... why not use handwriting as CAPTCHAs?

Re:New CAPTCHAs?

[joshuac]

Because the computer wouldn't know if the text the human responded with was the actual CAPTCHA text.

Guess you could have the same section of text handled by, say 10 people and make the first person wait until the computer has gotten enough verifications from the next 9 users; but it would suck to be that first person.

Anyone know the CAPTCHA/sec rate for a major site (success rate, we want to exclude the botnets)?

Re:New CAPTCHAs?

[clone53421]

Because the computer wouldn't know if the text the human responded with was the actual CAPTCHA text.

Give them two words, one known [to the computer] and one unknown. If the user correctly identifies the one word, assume they correctly identified the other one, and store their response. Once you have a consensus on what the unknown word said, you can be pretty sure it is accurate and you add it to your list of known words. That's basically the reCAPTCHA approach.

Human computation

[waisberg]

Apparently all that's necessary for Google (or anyone) to convert all handwritten documents to text is not OCR but human computation. What about using something like a Google Image Labeler? Instead of using random pictures, they could just use fragments of handwritten text? One could easily create software that automatically breaks handwritten text in words or sentences. Google labeler already has built in systems to validate the quality of the labels. I imagine the same sort of systems could be used to validade the effort of converting hand-written text into files. If Google, or some other company, created a web game or payed (in either money or some sort of virtual credit to be used on the net) I am sure people would be willing to spend their time playing/converting the handwritten text to files. As long as enough people decided to play, converting a huge amount of documents into text files wouldn't take long...

Re:Human computation

[corsec67]

virtual credit to be used on the net

You mean "porn", right?

OCR documents for porn, that could work.

Re:Human computation

[Anonymous Coward]

No... not necessarily. Google could offer, for instance, credit for some of its payed services (ex. Google ads). Companys interested in this type of solution could create a points program in which users, once they reached a certain number of points, could exchange the points for dollars or credit in online stores...one could think of other possible solutions (besides porn) to the problem.

Like I'm going to trust what the Chinese did!

[EWAdams]

Let's get this straight -- they're transcribing an archaic form of handwriting, from a language they don't know, using characters they don't know, for a guy who's going to pay them minimum wage and isn't going to check their work. Yeah, right.

Re:Like I'm going to trust what the Chinese did!

[Carthag]

Usually, they hire two groups to do a transcription each of the same text, then compare the transcriptions. That's what was done when the Dictionary of the Danish Language (ODS) was put online.

The 800 lb gorilla

[PingXao]

The 800 lb gorilla in the room that nobody wants to talk about is the extreme lack of progress in language processing. OCR still requires far too much hand-editing of the result to be practical for casual use. Speech recognition is OK, but quite primitive. Speech ouput now sounds pretty good, but underlying all these should be a "natural language" computing infrastructure. Such a beast doesn't exist. That's why there are no "what you say is what you get" word processing programs or ubiquitous speech-control products. It's also why there are no quality translation tools for written or spoken languages.

MIT had high hopes for their AI lab in the late '70s. The Japanese had a crash program that was supposed to lift so called "expert systems" by several orders of magnitude in the late '80s. What ever happened to all the promised innovation? There is still no system capable of taking a piece of paper with handwritten notes and figuring out what information is present on it. Or even distinguish between information and random doodling. Or a system that groks music to the point where you can whistle a tune and it tells you the name and who wrote it.

We still have a long way to go.

Fortunately, there is an alternative

[mangu]

Instead of using OCR, they can outsource it to India, have someone read the text and use speech to text software

Re:Fortunately, there is an alternative

[Anonymous Coward]

Yeah, and given my level of ability deciphering Indian people speaking English, I'm sure that software will have no problem whatsoever.

Re:Fortunately, there is an alternative

[Anonymous Coward]

i've outsourced all of my computer applications and software needs to India.

instead of using PowerPoint at meetings, i just have two Indian women in bikinis hold up large displays with my bullet points written on them--they even do slide transitions.

instead of an e-mail client, i use an Indian courier. it takes a while for me to communicate with international clients, but i receive practically no spam.

and rather than a word processor i have a guy with a notepad that a dictate to. he also offers me helpful tips when he notices that i'm trying to write a letter.

then there's the 17-year-old i have doing my taxes. i don't even think he's out of high school yet, but he beats Turbo Tax any day.

but you should really see the guy i have simulating Windows Vista for me. he wears this really slick suit, moves really slow, and everyone once in a while he comes up to me and kicks me in the balls.

Re:Fortunately, there is an alternative

[mdonley]

Somebody mod this guy up. My points are spent...

We need a breakthrough

[Louis Savain]

Now, with context and improved technology, I don't think that handwriting recognition is impossible. I have a feeling that it will be a technology like speech recognition: never perfect, and it will require training.

I agree but only if we are stuck with making incremental improvements to current technology. We already have proof that excellent handwritten character recognition is possible since we humans can do it. We use all sorts of cognitive tricks to recognize handwriting, not the least of which is, as you point out, that we usually have a good handle on the historical context surrounding the writing in question. This sort of knowledge requires a lifetime of training and learning. A French person will find it a lot easier to recognize handwriten letters if the words are written in French. Change to different language and his/her performance will suffer. He/she uses a technique called pattern completion which is entirely based on learning from previous experience, and not just reading experience. Our future machines will have to do likewise. In my opinion, good recognition in this field will require a breakthrough in our understanding of intelligence. I am optimistic.

Re:Optical Character Recognition is the Correct Te

[Guignol]

your code will put anything not java into perl
Yeah I know you sort of meant it
But even if you succesfuly recognized C code, you are going to make it perl code.
Oh well...

Going Postal.

[Ostracus]

"The USPS might see a return on investment if their OCR equipment works on 75% of text, routing the hard-to-read 25% to humans. That's a huge reduction in workload, because otherwise every letter would need to be scanned by a human."

USPS got smart there. They have that info encoded as a bar code at the point of origin were the problem's easier to deal with. Same really with the other carriers.

"And here's a somewhat related question: Is there good freeware or GPL'd OCR software usable on windows? I have a few dozen pages, scanned in as high-res PNGs, that I need to convert. Snag: It has some Kanji characters sprinkled throughout."

Well do as the other poster mentioned and do the poor man's version of what the stories suggesting and find a citizen in another country that'll do the work in exchange for something else.

lecture notes

[emj]

Have you ever taken lecture notes?

Re:lecture notes

[DamienNightbane]

Lecture notes have no bearing on the fact that someone is a shitty typist.

Re:lecture notes

[Jorophose]

I'm not a shitty typist; the entire post took me a minute or two to type out with the occasional pause to think about what to write.

No spellchecking either, so I'm constantly backspacing any errors. It's just a fact, for me at least, typing is uncomfortable, and I feel very irritated when I have to. It's just not fun. Be it with a more clickity keyboard, or worse a "soft touch" hybrid (ie not the gel kind, but the hard-keys-just-a-tiny-feel, where after a point I feel like I have CTS). Maybe it's just because of how I'm sitting or where my desk is... But I doubt that.

The computer doesn't understand me. My hands do.

Re:lecture notes

[DamienNightbane]

It you're creating enough errors while typing that correcting them becomes a chore, you're a shitty typist AND a shitty speller, and all the pens and paper in the world won't change that.

Re:lecture notes

[Jorophose]

Oh, thanks. Because that's what I was saying, that it's a chore. And suddenly being unable to reply to you in english is such a great offense? What about people with fat fingers and small keyboards? Automatically shitty typists?

Re:lecture notes

[DamienNightbane]

Yes. The keyboard is fine, l2type.

Re:lecture notes

[Anonymous Coward]

Yes, I have, on the computer, all the time. (Except for math classes - typing in latex or mathematica notation just isn't as fast as it should be). I type at about 100 wpm, I write at maybe about 20-30 at best - so guess what my lecture note medium of choice is?

Re:lecture notes

[emj]

As you say the problem is math and graph intense lectures. You have to beable to draw and highlight things on the notes.

Meeting notes are good to take on a computer, but as soon as you need to go back and change stuff, mark something or draw something you will go down from 100wpm to 10wpm. This is easy to work around in meeting notes, take a photo, include the ppt slides etc... But impossible to solve for brainstorming sessions.

Mindmaps are nice to a certain extent, especially freemind is nice.

Re:lecture notes

[emj]

Well he has a point if you ever need to correct something you have entered on the keyboard then you are going to write too slow.

See it more like a tip, try to improve your accuracy. I tend to delete three-four words when I do need to correct something this keeps the speed up, I believe it's actually faster than deleting one char at a time.

parts of the problem are solved

[Trepidity]

The US Post Office has, for years, had fairly reliable automated reading of handwritten digits, which is used to auto-sort and -route mail by zipcode. It can handle some pretty terrible handwriting, crazy arrangement on the envelope, and unlikely variations, so only a relatively small percentage of letters are spit out to be read by human eyes.

Its task is made easier by the fact that they're locating and segmenting fixed-length sequences that are usually at least somewhat separated: they're looking for either a 5-digit zip code or a 5-dash-4-digit zip+4, and handwritten digits usually don't connect in the way that cursive letters do. That and you have only 10 digits to deal with, instead of 36 alphanumeric characters plus punctuation, but that particular difference is just a matter of computing power and memory to scale up to ~4x the charset.

Even humans can't read human handwriting!

[Locke2005]

I volunteered this summer transcribing input for a senate campaign. For many documents, people's handwriting was simply unreadable. Even using context, years of experience with parsing human names, the fact that half of the people were already in our database, and the ability to google for contributor's company names, I still had a number of times where I just had to guess at what people meant. Granted, only about 5% of the input is completely illegible, but if I can't parse it, I certainly can't blame a machine for not being able to parse it.

Re:Optical Character Recognition is the Correct Te

[aisrael]

your code will put anything not java into perl

And Perl will probably run it anyway...

This has already been fixed

[oloron]

This is old news, why does everyone fret so, see right here http://ars.userfriendly.org/cartoons/?id=20081005&mode=classic. - this comment submitted by CAPTCHA-BOT 2000 Pro!

filtering with learning

[uniquegeek]

I don't care if the OCR can read others' handwriting. I just need it to analyze *my* handwriting.

I forget the exact terms from my Machine Learning course, but...

This reminds me of Bayesian spam filtering you can use on email boxes. You can also have training data to help sort new cases.

In this case the sample size is either a bit larger than the typical use (about 50 possibilities if we're talking about common alphanumeric characters), or whole words (if we use dictionary instead). Some combined solution may be more effective: using a dictionary to help your training data collected from simple characters.

Filtering out the writing from the page can already be done to some degree where the scanning method is true bitmap... text is decided as either black or white, with a certain value of grey being a deciding threshold.

You can also start with several basic handwriting styles, use that as a base and have the training data adjust to you.

How about an "open" handwriting database where the training data report back to a repository?

Handwriting == Perfect CAPTCHA

[johanatan]

Isn't this obvious proof that the CAPTCHAs are poorly designed? Why not just use actual handwriting as CAPTCHAs? Then, when some hackers crack it, they have solved a useful outstanding problem in CS.

The Census did this 8 years ago

[Zerelli]

I will not lay out the specifics on how it was done, since I am not sure that the guy who designed the process wants it shared. However, the US Census in 2000 processed every piece of paper from that Census using OCR with some back up QA by humans. The process essentially used a server farm to run each block that contained handwriting through a series of OCR checks, depending on the OCR confidence level the box would be either passed as read or put in fron of a keyer who would type what they saw in the box. The process then decided if the human matched what it had guessed if it did it passed on through if not then it went to another keyer and looked at the match between the two keyers and the OCR guess. It took about 90 days to process every piece of paper sent in. I cannot recall how many pieces there were but obviously it was millions. It surprises me tha tno one has improved on that in the last 8 years. I am going to have to see what they plan to do this time around, it was a pretty cool project to be a part of. We had a huge (for the year 2000 anyhow)SAN from EMC which is now pretty common but was rather rare at that time. I hope they keep it on the cutting edge this time around. I do know they adopted LINUX at the processing center I worked at after the Census was complete. I am pretty sure the project will be done without any Windows machines this time.

ever hear of IRISPEN ?

[denis-The-menace]

http://www.irislink.com/

I was shocked to see how well it read my hand writing. This was on a Tablet PC running XP.
The only nuisance was having to turn it off when you didn't need it. Otherwise it kept on thinking I was writing something.

Touch position as a function of time

[tepples]

Actually OCR'ing cursive is probably more a function of being able to accurately scan pen and ink writing than it is a function of "cursive is hard to decode".

That, and a touch screen generates a curve with X and Y as a function of time. This curve contains the order and speed of the strokes made by the stylus at any given moment, valuable information for distinguishing characters. A raster image generated by the scanner is density as a function of X and Y, with no time information.

Handwriting transcription also tough for HUMANS...

[dmauer]

As someone who's spent countless hours combing through Ancestry.com's databases of "transcribed" public records while researching my own family history, I can say with some certainty that it's not just OCR that struggles with handwriting.

I'd say that at least a third, and probably more like half, of the records I've found on Ancestry.com which reference the folks I'm researching, are transcribed incorrectly.

Certainly part of the problem is that the people doing the transcribing aren't familiar with the names they're transcribing (I've had a DuBois written as both "Delrie" and "Dobins"). Another part of the problem is that when you're looking at handwritten records from well over a hundred years ago, often they're just plain hard to read (or even illegible).

Anyway, that second point, IMO, makes using Ancestry's efforts as an example of issues with "handwriting" in general a bit dodgy. The problems they face are more along the lines of dealing with old, faded, often poorly filmed documents where even a human will have a tough time.

image-to-text != captcha solving

[Anonymous Coward]

For captchas, you only need some accuracy because you get infinite retries and immediate feedback whether your OCR guessed right. For digitizing text, especially people's names where spell checkers are useless, there's no automatic feedback. OCR needs near-100% accuracy to be of any value, because proofreading takes almost as long as manual transcription. So comparing captcha solvers to traditional OCR is apples to oranges.

It's funny...

[XDirtypunkX]

... we do work for handwriting recognition at the company I work at currently, using 3rd party packages along with a lot of our own special sauce to improve accuracy.

It ends up being pretty good with high quality scanned documents... the only time we end up with trouble is with low resolution faxes.