Slashdot Mirror
Human and Machine Readable Handwritten Language?
darrint writes "In some obscure corner of the Earth, has someone developed a human handwritten language which can be easily read by a machine? Why is the visual divide between what can be written by a human and what can be read by a machine so wide? At one extreme is the bar code, which I certainly cannot hand write. Machines can read it easily. Bank checks have a human readable account and routing numbers printed in special ink running along their bottom margins. These numbers can be read by a machine and are clearly legible to a human, but I doubt I could write them for input to a machine. My old Palm handheld could read something like handwriting in its little box. OCR exists but I've never thought of it as reliable. I would like to dash off little notes on stickies or in a tiny spiral notebook and be able to suck them into vim, a browser text-input box, and so forth. Perhaps I'd have to learn some kind of machine readable 'shorthand.' Has it been done?"
try every PDA on the market in the last 10 years or OCR software newer than 1990.
-You're wasting your time. Alfador only likes me.
I'm old enough to have filled in punch cards with a pencil. Does that count?
134340: I am not a number. I am a free planet!
Have you ever used one of those tablet computers? They do exactly that.
I don't think that a lot of effort has been made to develop a different language for people to communicate with machines. I think most of research time in that area is spent in improving handwriting recognition, ie changing what machines do rather than changing what we do.
I've always pictured the color of OS zealotry as a sort of bright flamingo pinkish hue
Sure, it's called... THE ALPHABET.
Learn to write it neatly and the computer will have no problem reading it. Or humans either, for that matter. Write it poorly and both will have a hard time.
"If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
It's called QWERTY.
Most of the responses seem to be missing the point of the post.
OCR/handwriting recognition folks: what would the ideal handwriting for machine readability look like? Could simple variations on standard English cursive or printing approach 100% recognizability, or would the ideal have to be synthesized, like shorthand, and if so, what characteristics would such a script have?
The problem with a machine readable, human writable language is that humans aren't neat enough. When I write the letter R it looks one way, which is differant than my sister, or my friend, or my butler (okay, i don't have a butler...but a kid can dream!).
If someone were to develop a language that was machine readable, human writable, it would probably consist of a series of straight lines. Letters would have to be larger, but lines are probably the way to go.
|_|__|-__-__-_||_|__
^like that.
NewslilySocial News. No lolcats allowed.
They don't read from paper. They can get extra info:
* pressure
* speed
* stroke order
* stroke direction
* pen-up and pen-down events
* timing
I guess what would be interesting would be to have OCR look at 100 peoples handwriting and see if there are any letters that are typically difficult to recognize, and then come up with a substitute that would be easy for the computer to read. Block capital letters should be fairly unambiguous, but I think many people don't write solely in that. I tend to mix my caps and non-caps within words, and I could see where the comp would mistake my F and P and O and Q U and V.
Does anyone know how Palm came up with their graffitti handwriting? - they must have done some studies.
..........FULL STOP.
morse code would be easy for a computer to translate, after all, its mostly a series of 1's and 0's
Does it go on forever?
Punch cards aren't really easy for human to read, unless you have only handful of parts.
For the original problem, I think the issue between computer recognizing handwriting is that shapes in everyones handwriting alter so much. I can't get my pda to recognize my handwriting even after training for several weeks, I just gave up and scribble notes as pictures instead.
Main issue to remember is that computers process in numbers, not letters, to completely solve this issue, we'd need a language that's completely based on numbers.
Standardizing handwriting in numbers shouldn't be impossible task,
but plain numbers don't tell people anything, we'd need symbolical dictionary to survive, something like:
0 no
1 yes
2 life
3 maybe
4 meaning
Once you memorize it, you can easily build concepts like 42 and 02 but there's problem for humans that we often need to express more than 10 things, or 100 if all 10 basic elements could be combined together.
We'd soon face the problem that exists with asian languages; you'd have symbolic meanings for 52 5322 and 34 3042. Unless you fully comprehend which can be combined with which, or you don't understand why certain terms combine together, or you simply don't have any clue what certain symbol means, you'll end up writing&speaking total gibberish to everyone else.
And as the size of the dictionary grows, long strings of numbers become hard to read, since the symbols don't vary that much and you'd end up having maybe up to 7 or 8 number long strings which together form sentences.
Sure, you could learn such system, but it would take years to master. You'd have to start learning in numbers at childhood in order to become totally fluent with such system.
There are no atheists when recovering from tape backup.
Can't you type faster than you can write? What are you, an executive or something?
The typical account information line printed at the bottom of your typical credit card statement or utility bill is printed in a font known as OCR-A. Equipment for machine reading this type of font has been around for over 25 years, such as some of the old Banctec 4300 series workstations used for processing bill payments and checks. Even these 1970s era machines had better than a 95 percent read rate of the entire account information line, provided that the printing was clear and properly placed. Later machines, such as the NCR 7780 or the OPEX Eagle can have better than a 99 percent read rate of a full line of characters. Again, the usual limitations on reliability of OCR characters are a result of poor or mislocated printing, or stray marks in the OCR field. Here is the obligatory Wikipedia link if you interested in finding out a bit more about the history of Optical Character Recognition.
MICR fonts, which are those funny looking numbers printed in magnetic ink at the bottom of most checks are designed to be human recognizable but machine readable, and have been around since the '60s. OCRA typically beats MICR today, but a good MICR line is still readable over 95 percent of the time.
Handwritten fonts are the most difficult to read, but the technology has been available to read handwritten numbers and letters for over 10 years, but typical read rates for something like a handwritten zip code or the numerical amount written on a check range from 60 to 80 percent, and are slowly getting better. Again, a lot depends on how much care is taken when writing out the text, and what kind of background clutter is present.
As for me, I typed out school reports in 8th grade in 1973, when our family's word processing hardware consisted of a 1940's vintage Underwood typewriter. Even humans had difficulty decoding my handwriting!
Is this joke article or rejected Cowboy Neal poll?
Often wrong but never in doubt.
I am Jack9.
Everyone knows me.
USPS has been using handwriting recognition hardware and software for some time. They do, however, implement relatively state of the art neural nets and other AI algorithms to interpret the handwriting, so it's probably not feasible for most people. More information on the system they use is here.
But low-wattage CPUs were too understrength at the time.
Maybe if someone tried again now, Newton would a better job.
"I don't know, therefore Aliens" Wafflebox1
I wonder if Braille might fit this bill?
Isn't that subsection of computing more advanced than handwriting recognition? Dictation type software? Dragon done B speaking semi naturally stuff?
what"S wranq vith FngIish? 1 vse a handwntting recOqntion Pragrarn alI the tlme ond it vvork5 great,
Graffiti v1 is very easy to learn, easy to read, and easy to decipher on a machine.
Check out this representation of the alphabet in Graffiti.
You can do X as a reverse of a K in that alphabet; U and V were a bit different (V is easier to do right-to-left for the machine to recognize the stroke, but you can make the shape the same as a "real" v). I actually did some of my paper notes in Graffiti (in University) since they tend to be mono-strokes (rather than the polystrokes to make up them more complicated letters when we write out traditional english), making it easier for shorthand note taking (which has a strict stroke form/order dictated by speed and readability!).
There you go: simple strokes, easy to read by eye, easy to read by my 68000-based Palm from 1998 as well as my current StrongARM Palm of 2004!
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
I think you've hit the nail on the head. Handwriting recognition etc. is for the same set of people who prioritize their laptop choice with how cool it looks in a coffee shop. (That, and special contexts or the disabled, but I digress...)
Given the speed differences between typing and handwriting (even in non-computational contexts), I consider attempts to do handwriting recognition as a kludge.
The real solutions will come in the form of portable/projectable/virtual keyboards or an entirely new input method--as far as I am concerned, handwriting has fallen by the wayside for all but the most light-weight tasks.
In some obscure corner of the Earth, has someone developed a human handwritten language which can be easily read by a machine?
Yeah, it's called Mathematics.
Sera
Slashdot, where armchair scientists get shouted down and armchair theologians get modded up.
Quit wasting your time trying to learn how to speak computer and spend it making the computer understand human.
(And yes, the fastest way to communicate with a computer currently is QWERTY)
Machines can tell the difference between 1's and 0's with virtually perfect accuracy. 01001101 01100001 01100011 01101000 01101001 01101110 01100101 01110011 00100000 01100011 01100001 01101110 00100000 01110100 01100101 01101100 01101100 00100000 01110100 01101000 01100101 00100000 01100100 01101001 01100110 01100110 01100101 01110010 01100101 01101110 01100011 01100101 00100000 01100010 01100101 01110100 01110111 01100101 01100101 01101110 00100000 00110001 00100111 01110011 00100000 01100001 01101110 01100100 00100000 00110000 00100111 01110011 00100000 01110111 01101001 01110100 01101000 00100000 01110110 01101001 01110010 01110100 01110101 01100001 01101100 01101100 01111001 00100000 01110000 01100101 01110010 01100110 01100101 01100011 01110100 00100000 01100001 01100011 01100011 01110101 01110010 01100001 01100011 01111001 00101110
or else!
Or so my blind friends tell me. The reason why is that in order to write braille, it is necessary to make little bumps in the paper. Not trivial. Usually a typewriter is used, which might as well be replaced with a computer keyboard. Computers are great for the blind because they avoid braille.
Simon's Rock College
"Main issue to remember is that computers process in numbers, not letters, to completely solve this issue, we'd need a language that's completely based on numbers"
I don't know how you've reached that conclusion, there's actually not that much difference between numbers and letters to a computer - both have binary values. The only reason a computer might be able to recognise digits 0-9 easier than also including A-Z, is that there are less glyphs to recognise in the alphabet. All you'd by doing by writing down numbers instead of letters is changing the set of patterns it's trying to recognise.
Numbers would also be easier to recognise than say, joined hand writing, as it's obvious what's part of each digit, with joined it's harder to tell if a line is part of the first character, the second, or if it's just part of the link between the two.
So, block capitals?
The revolution will not be televised... but it will have a page on Wikipedia
That's a great point. Braille is quite easy for sighted people to learn, and is fairly unambiguous.
v ious2.htm
As far as systems that are a little easier to write, but still machine-readable, a fun alternative might be the Fardraut font from Xevious and Solvalou.
http://hg101.classicgaming.gamespy.com/xevious/xe
I've got it as an actual font (called "CZP_Fardraut"), but I can't seem to find it anymore. I'm sure it's out there somewhere.
As alphabets go the letters aren't as distinctive as most, which would make learning it or reading it from a distance a bit of a hassle, but it's definitely possible. Also, the number system is nicely logical.
Boundless Expansion, Self-Transformation, Dynamic Optimism, Intelligent Technology, Spontaneous Order- BEST DO IT SO!
[For braille entry,] Usually a typewriter is used, which might as well be replaced with a computer keyboard.
Braille can be entered into a handheld device with a six-key chordpad on the back and a space key on the front. Would that be so hard?
Punch cards are not coded with pencil, they are coded with physical holes "punched" out of the paper (becoming... can we say it.. Chads!)
Unless your refering to MarkSense which turned marked cards into punch-cards by a machine that would sense the mark and punch it out.
Thanks to the last US Presidental election, the whole worlds knows the term chads, even if they don't all know what they mean. :)
OK, I got "S" and "W" from python just fine. Having trouble wriggling it around to form "H". It's easier when it's dead but then the python doesn't last as long.
Somewhat off topic, but there was a certain language that functioned like what you described, just not with numbers. It is called aUI (with that capitlaization) and was created in the 50s by Prof. John Weilgart, a (bored) psychologist. The language is composed of 42 very simple ideographic "letters" that each have both a meaning and set pronunciation. The letters combine to form concepts that can be as simple or as complex as you want to make them, and the latest edition of edition of his book (1979) has a dictionary of over 4000 words. It was made so that only the most general concepts (plus the numbers 0-10) would be classified as single letters, and I think this system works very well. I really suggest you check it out if you have any interest in languages or communication, but the information available online is somewhat limited. I was able to get his book, aUI, the Language of Space, through an interlibrary loan, but I am pretty sure it is long out of print. I really think this language has a much greater chance of being useful than anything based on numbers, and since it only uses very basic shapes (e.g. number shapes, a spiral, circle, oval, etc.) it could probably be recognized pretty easily by OCR systems, probably as well as or better than current print-letter recognition.
I made a handwriting system a long time ago with the following goals in mind in designing it:
1. It should NOT be easily readable by a casual observer (for notes I didn't want other people to read).
2. The most commonly used letters should be the simplest to draw, so it should be fairly fast to write, like cursive.
3. Letters should be as umambigious as possible, so even the most scribbled/hurried writing would be distinctly recognizable.
4. Each letter should try to hint to the original latin letter to some degree, whenever possible. Although goal #2 usually would take priority over this one when in conflict.
5. A mid-height clear horizontal marked the beginning/end of a new letter.
6. (just for fun) It should look kinda weird and cool in a sci-fi sort of way, so if someone came across my notes they would be kind of baffled =)
While #2 and #3 might work towards making this an easy-to-OCR handwriting system, #1 and #6 probably makes it moot, at least for the system I made. However, I imagine it wouldn't be too hard to make a less-obfuscated more-practical writing system which try to accomplish similar goals to #2-4 above.
I made a font out of my handwriting system a few years ago. If anyone is curious, here is an image chart of the font. =)
I'm curious what other more "efficient" writing systems may exist out there (other than standard and cursive). Does anyone know of any others?
Please consider making an automatic monthly recurring donation to the EFF
You know? Those dots in convenient, rectangular patterns? I'm sure a braile teacher (with sight) can read what all those men, women bathroom signs say in movie theaters and other random places. Those dots might not be very convenient to draw with a ball point pen, but if you wanted to design a pen for a different style of writing, I'd use a whiteout as a template, filled with magnetic ink.
Probably 100 times more legible once you get used to learning braile, and super machine readable so long as you're "writing" or stippling on 1mm graph paper.
moox. for a new generation.
Hiragana or Katakana has a specific traditional form which should be machine readable. Japanese kids spend ages learning the correct stroke order and style.
Once I was a four stone apology. Now I am two separate gorillas.
The reason for the difficulty with regard to handwriting is the following: Humans can usually read much faster than they can write. Therefore, to mitigate that fact, handwriting is usually optimized for speedy writing than readability. It is assumed that making it harder to read isn't much of a problem since humans can already read fast so a bit of a slowdown due to a somewhat fuzzy handwriting isn't much of a problem. On the other hand, writing neatly takes a lot of time but makes reading easier.
So handwriting is optimized towards fast writing instead of easy reading, hence the difficulty for a computer to decipher it. Using a handwriting style that is easier to read will most likely result in slower writing speed, which is not a design goal for a handwriting style.
http://en.wikipedia.org/wiki/Languages_of_Middle-e arth
No coincidence that a couple Tolkien invented languages, Khuzdul (Khûzdul, dwarven tongue) and cirth runes look very similar to Viking runes.
Of course, Quenya is the only true language to any real Linus fanboy.
Processors can function with other than 0 and 1, think vector processors.
If multithreaded vector processing sounds strange, maybe you're more familiar with the fuzzy logic buzzword.
Yes, I'm oversimplifying things, but I don't have readymade solution here, I'm just trying to explain concept.
"there are less glyphs to recognise" - You got my point, it's far more accurate to recognize 10 different symbols than it is to recognize 34, or more when we have accents. If we have language that's based on 10 symbols only, we could represent those with numbers from 0 to 9. And most teenagers today are used to input information with 0 to 9 already, think SMS.
Personally, I'm still waiting for accurate speech control.
There are no atheists when recovering from tape backup.
This can read them better than I can (check out the crazy examples)!
Sendou Wave Kick!!
You got my point, it's far more accurate to recognize 10 different symbols than it is to recognize 34, or more when we have accents.
Not necessarily. Trying to write a phonologically complex language like English is bad enough when the number of symbols is half the number of sounds, as currently; if the number were 1/5th, as you suggest, then words would have to be much longer, and reading would become more difficult for humans.
English already has to use more than one letter to represent many sounds: "ch", "sh", "th". With only 10 letters, it would be necessary to have a lot more combinations like that. This is not simplicity.
And most teenagers today are used to input information with 0 to 9 already, think SMS.
When I type a text message on my phone, I type "how are you", not "4466691277733199966688".
The input method is irrelevant; the question is how the data is represented. And it's represented with letters, not numbers.
Personally, I'm still waiting for accurate speech control.
That's only possible with perfect AI. I expect you'll be waiting for the rest of your life...
.. fit the description in some sense.
Just sayin'
What I find annoying on my PocketPC, is that as long as you only use US english, it performs reasonably well in recognizing my writing and guessing words, but my native language is Dutch. This gives 2 problems:
- It tries to guess Dutch words using an US English dictionairy, which is so much of a PITA that I switch off the entire dictionairy function.
- Dutch has a few characters that aren't in the standard US character set, this leaves me "international" as the only other option, but this also contains a lot of characters I will never use, and only cause confusion for the OCR system.
- Next to that I don't like that it forces you to learn it's alphabet instead of it learning yours.
In short I am very disappointed about my PocketPC, also because of some other limitations I was unaware of when I bought it. (remove battery and it forgets everything, coupled with an ActiveSync backup that doesn't work; I'm lefthanded, which makes the user interface very akward), I now have a Nokia Series 60 phone and prefer that.
RogerWilco the Adventurous Janitor
According to the Wiki article I referenced, MICR has an error rate of about 1 in 20,000 checks. This does deserve a bit of explanation though. As someone who works as a technician with modern check processing equipment, I can say that an error rate of 1 in 20,000 does not mean that a MICR or OCR system can successfully read 19,999 out of 20,000 checks fed into the machine. This is the rate that the MICR will read one account number and mistake it for another. In reality, the typical Magnetic MICR can read about 96 or 97 percent. If the MICR comes across an unreadable MICR character, it will reject that item. The Account Numbers and Routing Transit numbers on the MICR line of a check are also set up so that a checksum can be performed on the sum of the digits on the MICR line and verify that the information is valid. Inconsistencies in printing can affect MICR as it does OCR, but the fact that the data is printed in magnetic ink and read magnetically mean that stray marks from customers signatures, check decorations etc. do not adversely affect the readability of the information.
I believe that the handwriting technology pioneered for Newton lives on in Inkwell, the handwriting facility built into Mac OSX for use with tablets.
I used to sometimes try to approximate different fonts particularly typewriter-like serif fonts like courier. Even now sometimes I emphasize things by adding serifs.
However probably all caps would be easiest to read I would imagine, that or using grid paper which helped you to keep discipline. Of course this is assuming you don't use something to help capture in advance, like Anoto's system.
I just had a flashback from all those movies where you see ICBM operators get a phone call and they whip out a notebook in which they write in grease pencil the all caps code word using military speak (alpha, romeo, etc.). Now those would probably be readable by OCR I bet. Of course things like the Omnipage have been around for I think 20 years, and I see a bunch of OCR programs in freshmeat.net.
On the other hand you might just try shooting your index cards at a captcha cracker, it might just work.
A purely constructed alphabet that would be easy for humans to write and easy for machines to read would involve a group of connected strokes.
/|
---
|\
| X |
|/ \|
---
From the 6 strokes here you have 64 total possible combinations. Discard the 24 that are disjoint and youve still got plenty for 26 letters and 10 numbers.
As to an english-based alphabet, the problem is that so many letters are far too similar, especially b / h / k, i / j, rn / m, and that handwriting is too fluctuous. Capital letters are an obsolete idea that only further complicates things.
The outdated nature of most written languages is mirrored in spoken alphabets. There is absolutely no reason for 'w' to have a 3 syllable name. I have encounterd a number of people who say "www" as "dub dub dub", and I am considering spending a week or two training myself to permanently replace "double-u" with "dub" in my vocabulary (that is how long it took me to unlearn 20 years of tying my shoelaces wastefully and ingraining a better faster way).
This is what gadgets using c-pen and anoto technologies is for (I have no affiliation with them). I know gadgets wasn't what you asked for - but these are the closest solutions I have seen.
Yes, I am a biological organism. All rumors to the contrary are just that, rumors.
...or alternatively, please unambiguously authorize people to hack up their own TTF of your glyphs and distribute it as OSS.
(handwriting input method). They are impressive, but they are not perfect.
My other first post is car post.
I'm a left-handed person and I ended up selling my PDA because it had no idea what I was writing 90% of the time, I had to use the little onscreen keyboard.
When I gave it to any of my righ-handed friends, no problem. But myself and another leftie just couldn't use it!
Anyone else have this problem?
"damnit, trolley I want in your signature." - Elburrito
"aUI attempts oligosynthesis and is said to have been transmitted to John W. Weilgart by a being from another world."
bored? maybe. insane? yes.
Also your previous post was a bit confusing. You started off seemingly correctly (that handwriting is hard because it's individual) and then veered off into:
Which first of all is incorrect and second is completely irrelevant. The only time binary becomes a "limitation" is if you want to force a decision to become one bit, ie "yes" or "no". But it's trivial to enumerate multiple possibilities by adding more states, eg 2 bits can represent a scale of 4; or "no", "not really", "perhaps" and "yes" (or however you chose to define them).
How the computer defines these symbols is completely irrelevant.
If you want to make a good decoder for something uncertain (like human writing) a good start would be a viterbi decoder. It allows you to define a number of acceptable symbols (letters) and then adds statistics to the data so it will find the most likely thing you are writing. Naturally this can be further extended with a lot of tricks.
When it comes to classifying a symbol in the first place a good place to look is eg support vector machines. Or really anything regarding classification wrt computer vision. Basically there are entire mathematical fields devoted to this stuff.
Now if you just want to play around with handwriting reognition and voice regoc. I recommend you try a Tablet PC. I can typically handwrite on mine and it detects it just fine. Pretty darned impressive really.
So in the end it comes down to two things:
1) Good algorithms.
2) Fast enough processors.
We're pretty much there now. At least for PCs.
Such devices are becoming quite popular because they're faster then handwriting! Hopefully we'll eventually find one in every office and home!
No, I will not work for your startup