Slashdot Mirror
Unicode Consortium Releases Unicode 8.0.0
An anonymous reader writes: The newest version of the Unicode standard adds 7,716 new characters to the existing 21,499 – that's more than 35% growth! Most of them are Chinese, Japan and Korean ideographs, but among those changes Unicode adds support for new languages like Ik, used in Uganda.
That slashdot didn't support unicode
CJK in Unicode really kills me. I once had to write an appointment that generated PDF documents with both Japanese and Chinese text. When you do this with, say, English and Russian, you just need to pick a font set that covers both alphabets and basta. Not Chinese/Japanese. There are a number of glyphs that share a common historic root in these languages, and the Unicode folks decided to consecrate this historical relationship by recycling the character codes between the languages. Yet, the glyphs are substantially different when rendered. So you don't know what the glyph really represents until you know what font set is being applied to the string.
What I ended up doing was processing each character individually and using a "look around" algorithm that would try to find clues in the context as to what language the glyph was in and render it with the right font. It never worked very well, but it worked well enough that the client decided not to redactor the controller that was generating the mixed language strings.
But I learned two valuable lessons that day: Unicode isn't that great after all and stay away from CJK contracts.
Is Unicode supposed to separate characters that look the same but are semantically different?
Looks like the answer is yes...
'LATIN CAPITAL LETTER A' (U+0041)
'GREEK CAPITAL LETTER ALPHA' (U+0391)
Looks like the answer is no...
'RIGHT SINGLE QUOTATION MARK' (U+2019) -- this is the preferred character to use for apostrophe.
(An apostrophe and closing a quotation are two very different things.)
Most languages can be written with English characters (ie. plain latin).
Name a language written with Latin characters (other than English) that does not use any special characters or diacritical marks whatsoever.
(Even English requires extensions to the Latin character set. which originally had no "U", "J", or "W". )
Il n'y a pas de Planet B.
Sorry, why do we need multiple languages again?
“Common sense is not so common.” — Voltaire
You mean "ij"? The unified ij character isn't used by anyone. Not sure if it's even recommended by any body.
But Dutch does have accents (één, vóór, ...). News headlines this morning:
"Verstekeling valt boven Londen uit vliegtuig na 11u lange vlucht, één overleeft"
"Grieken demonstreren ook vóór de euro"
Help build the anti-software-patent wiki
There were already 113K characters in Unicode version 7.0. Which is more than 2^16 characters, so remember:
perl -e 'fork||print for split//,"hahahaha"'
Don't let the door hit you in the ass on the way out to the pasture of obsolescence. The rest of us will continue to use Unicode, which, despite some flaws, such as their mess-up with Han Unification, does a pretty good job at solving the problem of language intercommunication. If anyone thinks they can do a better job (not counting reverting back to English-only ASCII), have at it.
So go ahead and use ASCII, don't type in any of those dern foreign charcturs, and pretend you're back in the happy past where we had a mess of incompatible standards, and no way to easily discern which of the many possible encodings was actually used, resulting in the scrambled text we always used to see (notice you *don't* actually see that much anymore?). And most software just ignored the rest of the non-English-speaking world anyhow because of that mess.
Personally, I'm thankful people are willing to take on largely thankless (and mind-numbing to most of us) tasks such as these.
Irony: Agile development has too much intertia to be abandoned now.
No we would not do well with only one language, we would loose a lot of culture. It would be like one standard food for everyone. Furthermore, your proposition is ludicrous, as language changes all the time. That's why new street languages pop up and then evolve in something different. Language is reflection of culture. It is not like a programming language. If you want to communicate with other people you should learn additional languages. And while you are at it, also try to learn something about their culture. Otherwise could would not be able to understand them. So even with only one language, you would not be able to understand them.
BTW: The most problems between societies are not based on religion. Religion is only used as a vehicle to transport the hostility. It is about greed, ignorance, stupidity, and frustrations.
Well, no, it's not, they're still working on that bit meaning that you get to keep upgrading all your programs to use newer and ever bigger libraries supporting more complex rules regularly. It's not stable.
Nonsense. The Unicode encoding formats are stable, and have been for a very long time. New character are added all the time, but the underlying OS and it's fonts are typically upgraded to support these, and so most programs need to do absolutely nothing once their support is in place. The vast, vast majority of applications that support Unicode don't actually explicitly need to use those "official" Unicode libraries (which are monstrously complex), because all modern operating systems provide most of the support they need. For simple conversions, there are a number of excellent free and simple-to-use libraries (many languages have standard libraries available), or you can just use OS-specific versions, or a number of very easy-to-use free and open-source libraries.
If you're concerned about size, just use UTF-8. There's no need to "switch encodings on the fly", because that's what variable-width encodings already do for you. And the vast majority of common encodings, even in Asian languages, are only 16-bits, not 24 or 32. The issue of inefficiency of text size with Asian languages is greatly exaggerated, and becoming less and less relevant anyhow with our machines with gigabytes of RAM and processors efficient enough to compress and decompress text on the fly. BTW, you can do that just fine even in Microsoft and Apple environments. It just means you need to transcode from UTF-8 to UTF-16 or back again at any API boundary that takes text, and this is fairly simple to do. I've written my own cross-platform code this way because UTF-8 is a much easier encoding to work with internally IMO.
I don't think anyone would try to argue that Unicode is a perfect solution, but it's a damn sight better than what we used to have. Your comparison to USB is pretty good, in fact. Ask just about any PC user what they'd prefer - modern USB devices or the old system of parallel, serial, PS/2, and joystick ports. Whatever faults USB has, it's a hell of an improvement over the old system.
Irony: Agile development has too much intertia to be abandoned now.
This is exactly why it took decades and crazy hacks for people to write their own language electronically.
Thank God virtually failed (but won) Plan 9 (UNIX2) came by with idealistic developers who respects other cultures came up with Unicode and companies like IBM/Microsoft/Adobe along with Free software supported it.
Who knows if the software/hardware/network combination you use had a line coded by a person who is from those "computer illiterate" regions?
What are you doing these days? When Doves Cry was part of my childhood.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
Instead we are stuck with UTF16 as the default, and even the larger encodings use modifiers etc.
Who's "we"? Windows and Mac use UTF-16, while Linux and the web use the vastly superior UTF-8. Internally, assuming you're in a language that supports it like C++, you can actually use any encoding you want - it just means you need to transcode strings at API boundaries. You'd have to do this for one or more of your target platforms anyhow if you're writing cross-platform code (all three major PC OSes).
A lot of Windows programmers think "Unicode == UTF-16", which is not the case at all. In my own applications, I use UTF-8 as the native format, even on Windows and Mac. When I need to render glyphs (I write games, so I have my own low-level bitmapped based glyph rendering system), I convert them to UTF-32 code points for simple mapping. If you want to, nothing is stopping you from using UTF-32 internally as you'd seem to prefer, but I've found there's really no need, because you can always convert between formats on the fly as needed.
Irony: Agile development has too much intertia to be abandoned now.