Slashdot Mirror
Complex Language Support for PDA's?
Jasin Natael asks: "What PDA's/Smartphones, etc. support complex languages in addition to more 'standard' languages? I'm a student of Japanese and am looking for a new PDA or smartphone that operates in English, but supports complex character sets. Input is a plus, but it's really needed for Contacts, Notes, Websites, and incoming E-Mail at a minimum. Would it be easy to add support to a Linux PDA (Zaurus) or Pocket PC for this? What about right-to-left languages, like Hebrew and Arabic?"
its linux, so I guess if you can cross compile it, it might work
FP rocks
I thought you meant Perl support.
I use CJKOS on my Clie to give me Chinese characters, both for input as well as display in applications. It includes Japanese fonts but I have never used them. It works quite well for me.
I'm taking an Automata and Formal Languages class, and at first I thought this said "Context-Free Language support for PDA's" (a PDA is a push-down automata). And I thought, "aren't they already equivalent?"
Just make the characters upside down and turn the PDA upside down. That will make them work right to left.
Easily Fixed.. Where is my million dollars?
Give me a Job... Resume is at http://www.newberrycollege.net
I'm CERTAIN that a few months back I came across a Japanese package for the Palm. Maybe on Palm Freeware? That's the only Palm site I visit regularly so it could be there...
Buy an imported PDA and get full language support. I have yet to see any computer that didn't have some english support.
Palm OS is officially available in English, French, Spanish, German, Dutch, Portugese, Japanese, and Simplified Chinese, and there are translation modules to support Thai, Vietnamese, Korean, Hebrew, Greek, and more. Usually, a device is only set for one language at a time, but some of the overlay programs allow for an Asian language and Engligh together.
A few choice URLs:
http://www.penreader.com/PalmOS/PiLoc.html
Hebrew Localization
Chinese OS for Palm OS
At least Cliff isn't looking for a PDA that only supports the language of crypto-imperialist neoconservative illegitimate psuedo-democratic petroliocracies hellbent on a 21st century (supposedly secular) crusaide...
Why are you even considering the world editions of the Zaurus? The Japanese models have full support for Japanese chars, plus nice dictionaries. But these models are sold only inside Japan (or through gray market), the world models don't have equivalent features.
(Score:+5, Artistic)
...using Windows XP: Tablet PC Edition.
Resistance is futile.
Those blue screens of death are so cryptic, you can take them to be any language you want.
Another problem solved.
You better learn English or the US will stop selling you Palm products. WE WILL TAKE OUR BALL AND GO HOME!
evil adrian
other languages suck. use english. there is a reason everyone in the world knows english.
Do you mean double byte? Do you mean unicode? Could the question be asked in a meaningful way?
I'm pretty sure you can get J2ME to run on almost anything...and it uses Unicode, so would it qualify as a "complex" language, appropriate for requested uses?
------- "From bored to fanboy in 3.8 asian girls" ----------
That would be Pippy, Python for the Palm OS; Python-CE
I know that's the unintended way to take an ambiguous question (ignoring the fact that python is "versatile" and "powerful" but not really "complex" - although you can certainly build complex tools with it), but this is a serious answer.
uses unicode for all internal character representation, so all apps that use standard functions should be able to use all language features supported by unicode.
Beyond that, I really have no idea.
autopr0n is like, down and stuff.
Sorry to be a bastard about this, but please don't use Ask Slashdot for a simple request that takes two seconds to look up on Google.
The VERY FIRST response on Google is a very complete PowerPoint presentation comparing various plugins for complex language support including Chinese and Japanese, and there were a bunch of useful links from there.
Ask Slashdot should be reserved for important things, like whether Go rulez more than Chess, or endless speculation on who will play the Empire State Building in the new Peter Jackson version of King Kong
------
Believe me, I'm as surprised by my comment as you are.
You can get one of the phone versions of the WinCE devices (my brother has the Siemens one - nice device). I have a Toshiba (WinCE without phone). The OS is unicode based (lots of ascii functions are actually missing in the APIs). There are ports of lots of good "traditionally unix" tools at http://www.rainer-keuchel.de/software.html. You can see some of the I18N stuff done with Tcl/Tk on CE and general Tcl/Tk on CE info. Perl/Tk also exists, along with lots of other goodies, at Rainer's site.
The reason that I purchased a WinCE device over Palm was because of all the more fun hack potential.
I would have said "Complex Character Set Support for PDAs?" rather than "Language," since the language is really completely transparent to the device. I initially thought that the article was about actual language recognition, in human-computer interaction, for example.
Who posts to usenet without checking the FAQ or doing a web search aren't you.
Arabic is available here: http://www.arabicpalm.com/ and Hebrew here: http://www.penticon.com/.
I would be interested to find out if any work has been done to get either of these languages workin on the Zaurus.
Simputer from India has support for the complex Indic languages.
However, for Japanese and Arabic (with bidi support), I guess the best option would be to run gtk2 - whose pango text layout engine supports complex scripts.
I don't know what PDA has gtk2 based apps, you'll have to find out for yourself.
I don't see any reason why pda's or smartphones won't have this kind of language support.
Hm, after a quick search on the palm os site I found this palm os page
A quote:
I suppose it's not that hard to find plugins for other languages.
When I was in Japan, I saw a ton of Japanese based pda's. It's much easier for them to support english than for US to support japanese..!
:)
Palm has one and so does Zaurus (in fact, they have many models).
Oh and BTW, they are much more cooler than the US models
-- Leeeter than leet
Well I have an Agenda VR3(www.softfield.com) and I've recently felt a similar need. Although I haven't tried to make it all work yet, but I don't think it should be that hard - it's running a standard XFree so I'll just need to put the fonts in the right place...I'm not sure about Zaurus, but I don't think that getting cjk fonts to work should be a big problem. As for input - can you use the onscreen keyboard with kinput2 or something like that? :) Oh, but kinput2 needs the locale to be set accordingly too, doesn't it? :(
"chinese language support palm pilot" yields a power point presentation that compares both chinese and japanese plug ins.
"japanese language PDA" the first entry is a press release on a whole japanese language system for symbian os.
Google = not hard.
The only acceptable defense of scientific results is to say that they were the product of the Scientific Method.
The Danger Hiptop / T-Mobile Sidekick looks like it will have Japanese IME support, at least as a developer tool.
Wow. my third test is next Tuesday. heh.
-- Every time you kill a kitten, God masturbates.
Neopad Nihongo Input Romazi
I haven't tried either of the above, though...
Also, the program Dokusha, while also being a good English-Japanese dictionary, comes with some Japanese fonts.
Zaurus must be *THE* only PDA that includes kanji input - as in, written by hand. (Okay so you can write kana into it too, so it's more like "glyph input" but I digress)
You have no idea how that saves your life (or, time - which is really just small chunks of life) when looking for the pronouciation of some kanji characters (and meanings - zaurus in Japan AFAIK comes with dictionaries either direction).
So, yeah - buy a zaurus from Japan and be amazed. I don't think the US models are so trick, buc I might be wrong.
My life in the land of the rising sun.
I bought my PI-6500 6 years ago. It has handwriting recognition, wa-ei, ei-wa, kanji and kokugo dictionaries built-in. The handwriting recognition comes in handy when you want to look-up a kanji you don't know. It is also good practice for handwriting skills. The latest zaurus-es have these, but you used ones are really cheap in akihabara (50 bucks or less). You also might be interested in http://www.jisyo.org if you are serious about japanese study.
Yo, Nokia Communicator has a asian language version available, plus the phone issuper-sweet. You can play MPEG movies on it as well as do powerpoint. (tiny keys though)
Windows CE (PocketPC is a version of Windows CE 3.0) has been fully Unicode since version 2.0.
There is an Ipkg for it somewhere google for zaurus nunome. (works on Sharp and OpenZaurus ROMS ) Also, Kanjinirvana is a very good kanji dictionary/quizzer (supporting on-, kun-, and direct kanji input, with a best guess (good for beginners) (Sharp and OpenZaurus ROMS) I will look at responses to this, so if you need help, post replies.
because as we all know, neither Japanese nor Israelis like smartphones.
Qué nos diste?
you had me going up until (supposedly secular).
Just because a country uses danger to an ally (Israel) as one of, albeit not nearly the most important of, the reasons for attacking an evil regime, doesn't mean that the leaders of said country are Jewish or want to protect Jews specifically. It's all political, it's not at all religious.
Bush wouldn't do this if it weren't for the tons of cash that siezing control of the oil would infuse into the American economy- obviously, american oil companies will be the ones (along with BP, but isn't that Amoco now?) doing the drilling when Bush owns Iraq. He's thinking about furthering his own political agenda, but not necessarily his political career. I assure you, the oil companies will make sure he's well taken care of...
I don't know what to say. We are all evil bastards, and we should live in grass huts and walk everywhere and eat fresh not-packaged foods and not farm destructively and also decrease some of the population, perhaps through legalizing people hunting... People! Yummy!
So, either we all get off our capitalist thrones and stop being hypocrates, or we shut up and watch all the shit we caused pan out. I guess, right now, I'm too lazy to go live in a grass hut and carry my own water and wash my clothes by hand and walk everywhere and grow my own food and all that. So fuck the world, get on with destroying it.
well just the language isn't a bad thing anyway .. and i'm sure most 'terrorists' speak a bit english too ;)
Linux is like a Wigwam. No Windows no Gates but Apache inside
I am a student of Japanese too.
I use my Zaurus SL-5500 (a GNU/Linux based PDA) daily and I think it's a good choice now, as the new 5600 model is out and you can buy it for a very low price (around $250).
Even if it's a EU/US model you can easily add Japanese language/input support, go to:
http://www.killefiz.de/zaurus/
and search for "japanese".
I use Nunome for japanese input, KanjiNirvana to lookup kanji and mioReaderLite as a japanese text reader and dictionary.
What dictionaries are included or available in the Japanese Zaurus?
"Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
But its a shame you Americans cant speak it, you cant even spell things correct....
repeat after me...
colour
armour
catalogue
Aluminium (Al - you - min - i - um)
Simon
dont get on your high horses this is called humour
Kingdom of Loathing (www.kingdomofloathing.com) Addicted is me
Do the latest ones have any new features useful for Japanese language learning, or are the dictionaries and language-related features in the new ones the same as in the old ones?
"Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
Then set the locale to English and modify the fonts as described at this site . This gives you English Menues and English and Japanese input and gets rid of any mojibake in the Japanese applications. The Zaurus has the same handwriting recognition as all its predecessors which is the best I have ever seen for Kana, Kanji and various alphabets all at the same time.
***Quis custodiet ipsos custodes***
www.sofmap.co.jp sells clie and an older sharp zaurus (SL-B500, cheaper and also linux, with a chicklet keyboard). I have some older clies which I dislike due to their being entirely too slow for input using normal input methods. The newer clies are nice-looking too, and at least for the older ones there are apparently ways to localize them. I'd stick with linux and as much RAM as you can get though.. the new zaurus would be perfect with a little faster cpu and an extra hundred megabytes or so of ram. Undoubtedly you can run emacs with any language you like on it.
Sony has been making asian language Clies on the PalmOS for years. And ANY Palm can use CJKOS(Chinese, Japanese, Korean Operating System) which is exactly what you need. Sorry for having you be in the dark for so long. I mean, it's darn near the most popular file at PalmGear.com, the 2nd leading Palm site to Handango.
Next time use google instead of wasting space on the front page.
Display rendering is very RAM intensive. It is particularly costly for these small devices.
For a 32,000 character Japanese or Chinese font, at 14 pixels square (about the smallest readable resolution), un compressed, you are talking about 800K.
On an 8M Palm, that ends up being 1/10th of your available memory.
Hebrew, Arabic, Tamil, Devengari, or other ligatured languages have much smaller fonts, but since the character rendering changes as a result of which characters are adjavent to each other, or the start or end of the line, you have similar memory constraints for the ligature rendering software, which could be considered "part of" the font.
That's just for display, and doesn't include input.
For something like Pinjin (Chineseh input) or Kanjihand (Japanese input), you are talking additional RAM taken up to allow both "chording", and translation of the pseudo "chords" (unless you have a keyboard) into the textual representation.
Storage for data is less of a problem; but most storage uses EUC or UTF or some other multibyte encoding. If it didn't, you couldn't shove it into 8-bit "files" on a PALM; if PALM supported 16-bit "files", this would be much easier.
But since it doesn't, you don't get the average 2.5:1 information density increase you would normally get from an ideogrammatic language (average English word length is 5 8-bit characters), and it drops down to about equal density (~1.2:1), so you don't win back your memory used on input and display processing that way.
So the net result is about the same as the original Macintosh: all the RAM is taken up by system processing, leaving nothing left for data or programs.
So what this boils down to is that the support has to be built into the OS area, instead of into the user area.
About the only PALM-like device I know that can do this is the Sharp Zarus. All the other vendors tend to fill their FLASH up with, well, pieces of PalmOS, not leaving any private-use areas for language add-on vendors.
PS: Yes, I know my font size of ~800K is uncompressed; the alternative is to compress it, and then include decompression code. That sort of works, but is compute intensive enough to make the system unpleasent to use, with the underpowered processors on most PDAs.
-- Terry
For using Japanese on Palm, "J-OS" by Yamada Tatsushi is the way to go. It works on US hardwares of Palm devices and allows you to input and display Japanese characters.
As it is "Swing" based. But unfortunatly this mighty OS is not available for retail but restricted to 3rd part manufacturers only :(
...)
It is a great OS, because designing graphics applications is no headaches, as the regular Java applications works without modification !
(J2SE1.3 compliant OS for StronARM/XScale PocketPC, but the port kit enable you to use it on others platform
If you've never heard of it, try to get back an "old" 1.1 version (that used to exist in public demo but full featured version) !
Regards,
I am the author of PocketKanji and I would like to draw attention to these two projects.
They are aimed at students of japanese who own a PalmOS device. You do not need to have a Japanese device as the applications draw the character themselves.
PocketKanji will recognize a handwritten character and give you the definition of a kanji.
PAdict will give you the translation of japanese words to english. It has recently included the PocketKanji code so that you can now draw characters you don't know and find the translation of japanese words.
I think the two projects will merge eventually. I am looking for volunteers to help me add characters to the character recognisation database.
As an aside, it is not CPU intensive to draw japanese characters. It takes a bit of ram to hold a decent font but that's about it.
The Zaurus can do this. It's running Qtopia which is based on Qt, which uses unicode for all text. You just have to install a unicode font. I'm reading Japanese (well, trying to) on a non-Japanese Zaurus 5500.
Can you hear me, Major Tom? I'm not the man they think I am at home...
No, Timothy McVeigh spoke arabic!
"Is there a website that compiles news from Linux, science, hardware, software, Microsoft, and also gives me the opportunity to ask things that is extremely easy to look up on the web?"
Support unicode by default, so it shouldn't be too difficult to add support for other languages...
This is the kind of quality FP slashdot needs more of.
How well does it handle to right to left writing systems, like Hebrew, Arabic, etc?
Wind under thy Wings.
Amber
Suppose you did.
Suppose you did not.
Unicode is "just" a character encoding and so can't handle the localisation issue for right to left languages. (In fact after years of anglocentricity I'm now having to work on a site design with both internationalisation and localisation and, even with J2EE, it's a pig, folks.)
Hebrew, btw, is a lot easier than Arabic because only a few letters morph at the ends of words, but it's still a complication we could do without. I guess we could save effort by having a form on the site for registered users to be sent a free mirror, but then we'd have to have the typeface reversed...oh weh, goyim
Panurge has posted for the last time. Thanks for the positive moderations.
I don't quite get your question, really. The japanese had PDAs before we did, and they've *allways* had better ones. Especially due to their set of glyphs!
They've also got wristwatch computers and use them in ways useful. Mostly 'cause you get a lot more info on that tiny screen with Kanji and Chinese Symbols than with latin lettering. You can get an entire novel on to something like 100 pages that way.
Go figure.
We suffer more in our imagination than in reality. - Seneca
Even a moron knows they sell ALL of those PDA's in Japan. And even a mildly literate moron would be able to firgure out in 1/2 a second that many of these PDA manufacturers are JAPANESE companies.
Since you're aware of the Zaurus, did you even notice the myriad of country options on the Zaurus.com page? The last one being JAPAN? Or perhaps palmsource.com which CLEARLY lists about a million multibyte languages it supports.
RTFM, jesus.
I'm Japanese. English is just second langugage for me and it's complicated enough for me.
BTW, I'm using Palm, actually IBM's workpad. It supports Japanes language well and came with english/japanese and japanese/english dictionary. But, it's very small and seems not to fit study purpose.
Palm OS machine sold in Japan has some HW enhance
ment to make input Japanese compfatable. Missing them could make experience painful. I have used old Palm machine sold by US robotics w/o such a enhancement. It was still usable but far less than what I have.
My vote will go to Sharp's Linux, I have never seriouly used it though. Linux is Linux, anyway.I have used for almost 8 years. Since the first day, I can input/edit Japanese without any problem.
# Acutually there were small problems, but always
# they exists.
OK... Since no one is actully giving any suggestions. I will. Compaq Ipaq. It's probably the most popular PocketPC. With a TON of expansion options, and you can reflash the memory to have linux on it instead of PPC 200x. I do believe it'll support those languages as well although you'll probably have to download some shiznazz to get it to work. You can program Java (albeit a very cutdown version of java) on it, as well as C++ and VB(ick) Also Comeon! integrated WiFi, bluetooth AND a biometric finger reader? SHWEET!
Is there a pda that supports engrish?
Checking out my form of escapism.
There are language packages for various languages, includong Japanese. It also has unicode input support and a unicode enabled ebook reader.
openzaurus.org
but Japanese is a right-to-left language. 'Course, its top-to-bottom then right-to-left, and thats pretty much only for formality now, but still.
Paizurishitetai desu ka?
Nothing imaginary about Perl, as much as some people would wish it so :-)
Infuriate left and right
yeah, japanese works on the Z, but as far as i know, it is impossible to use right to left characters. QTE supports unicode, so as long as you make some input method and the subset of fonts you needed encoded in unicode, any language should be possible. i personally have made a korean input method and fonts for it.
tkcMail does non-latin1 languages.
While theKompany's server is down, here's a (very old) screenshot at my own site.
Rik
Here is an excerpt from the Discussion over at brighthand on this, I followed it and had no problems with installing japanese support on a Dell Axim, Ipaq 3650, and Ipaq 5450
Here is the page
"It seems that some of you want to see and Input Japanese on their US/EUR PPC
here you are all you need and a Step by step process Enjoy!
Fisrt of all you don't need to flash the rom of your PPC, I was obliged to do that on my Japanese Ipaq just simply coz I was fed up to not be able to run some software properly and to wait for Rom Update.
What you need is very simple:
-The Japanese FONT MSGothic From a Japanese PPC
-A file called wince.nls from a Japanese PPC
-2 reg keys
-TascalRegedit soft in order to import those key.
Please not that you will only be able to read Japanese not Input Japanese
By reading I mean that you can even see the Kanji of a Japanese soft installed on a US/EUR PPC and of course surf the web in Japanese.
By not be able to imput Japanese I mean that you can no write in Japanese for that you need to buy a Japanese Input software and as well import some Key in the registry (can give you more details if you Need)
Finally I am not the guy who created this and all the Info are comming form Pocketgames Japan (Thanks Koji !)
And It has been working perfectly on Compaq/HP (3630 1910 3970 5450) or O2 devices but almost destroyed a Casio E-200 (Hard reset Manipulation was even not successful, was obliged to put away both backup battery and main battery in order to be able to use again the PDA)
so you can Dl the files there:
The reg keys (2)
Reg Key
The Font (2.2Mo zipped and 4Mo unzipped)
Japanese Font
The WinCe.nls
Wince.nls
Thru Active Sync overwrite the wince.nls file which is in your Windows folder, put there as well the Font Don't put neither the wince.nls file nor the font in any other place, folder subfolder than the WINDOWS folder.
Now form TascalRegedit Install the 2 Regkey, soft reset Et voila !!!
to make sure go into my regional settings and you should see that:
Now you can read japanese, so what about inputing Japanese?
This is not the most perfect input methode but this Methode is FREE !
Now you just have to download this soft called POBox, and you will have a New Keyborad available in your PPC. Also in the Zip file you will find a folder called Dic, just put this folder in the C: root of your PPC et voila ! it will works like a charm
(There are some bugs, I mean that the imput panel overlap the dictionary but if you know your Knaji you will recognized them easily)
14/03/03 Update on the Overlaping Problem
Thank you very much for Koji of the Famous and extremly well known Pocketgames JAPAN and Have a look to the link above !
He got the answer of the overlaping problem when using the Soft ComPOBox, you need in the registry to change one single value and you will fix the PB. here you are
in
[HKEY_CURRENT_USERControlPanelSip]
You have
MenuBarHight 0000001a
So now change it to
00000000
Enjoy"
moo.
It's all Noah Webster's fault (no, I'm NOT kidding).
-uso.
Uses a bizarre mixture of American and British spelling.
Dreams, dreams, don't doubt dreams, dreaming children's dreaming dreams. Sailor Moon SS
It doesnt. That support was added into QT in Version 3.0, and the Zaurus's use 2.3.2
At http://www.waterworld.com.hk/ is a software package called Chinese OS that will allow input/reading of simplified/traditional Chinese with GB/BIG5/etc. methods. Although it's not completely native/integrated/embedded, I've used it and it seems to be pretty good. Basically the idea is to use grafitti to input Roman characters similar to inputing chinese on a PC with a keyboard. If you're good at graffiti then it's pretty fast.
I am deeply sorry for the earlier truely ignorant comments, given by real cowards. I would like to point you to the dell axim as I am very happy with it and I believe they sell a international version.
Go to http://www.dynamism.com Browse the site find something that fits you needs and buy it.
Simple.
When my sister was researching PDA's that she could use both for english dictionaries and her Arabic studies, we came across Arabic OS. It's for palm, their website is here. It also comes in Farsi. And based off of that, I'm sure that you could find other companies that have created support for languages such as Hebrew and Japanese.
plan9 supports any and all unicode text
and runs on the odd iPAQ
there is a solid base of japneese users
dunno about the status of right to left...
towelheads are cool, bible jockeys are boring, period.
Just as a data point: I've got pretty good Japanese support on my U.S.-purchased Zaurus SL-5500. Look around on the Downloads section of Zaurus Zone for Japanese packages, including Nunome and Kanji Nirvana. Probably not as slick a solution as a Japanese Zaurus, but not bad if you want a primarily English-language PDA with Japanese support useful to a language student.
- - - -
The real Tetsujin 28 is a giant robot.
Do they use contractions in British English?
I'm glad that someone actually knows how to post to slashdot. No, I'm not being sarcastic. This is why I read this site...My hat off to you, sir.
YOU DID NOT FAIL IT!!!!
If not for people like you with a little humor in the day, all the crap posts of people that simply want to badmouth microsoft, and tell everyone how much the want to be able to hump their latest linux distro would be all we could read.
Thanks Again!
I used to use a Handspring Visor, but since I got my iPod, it just gathers dust in the charger. The beauty of the iPod is (for the Apple version at least) it supports unicode text. You can switch the machine's interface to whatever lanuage just for fun (hence mine is Japanese), but the real feature is the contacts, calendars, and song names will all remain in the language you entered it in. So for me, if I forgot how to "spell" someone's name with the right kanji, I can just wheel through my contact list and find their address card, right next to all the other English address cards. Similarly, my calendar has become a pigdin of English and psuedo-Japanese, and my song list contains the proper Japanese titles for Puffy songs. The only downside is no input support, but honestly, using a pen on your (actual) palm and then typing it in when you get back to your computer is easier than Graffiti anyway...
What the fuck are you losers doing on here debating about this bullshit?
ITS FRIDAY!!! Go out and fucking get drunk and laid.
If you have an ipaq, you might try MELON Linux(http://melon.10art-ni.co.jp/). Its a Japanese OS, so it may not be exactly what your looking for, but it'll install english apps too.
Looks like the conclusion to be drawn here is "everything but OS-native support is unusable." Highly disappointing.
So, what are you going to do?
I'm just wrapping up a stint at working for Olivetree ( http://www.olivetree.com/ ). We do Bible Software for handheld devices. We have Hebrew Bibles that do right-to-left, as well as unicode Bibles in other character sets. I'm mainly doing PocketPC programming, and I can tell you that even on the English devices, ALL strings are stored in Unicode on a PocketPC (that's one of the challenges in moving from desktop to pocket device programming).
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
Who wants to bet that WinCE cannot handle modern Unicode (Unicode 3.1) ?
:(
I don't know, but I know that all of Microsoft's desktop & server operating systems are stuck back in the days of Unicode 3.0, when there were no more than 65,535 code points, and mostly they cannot be fixed...