Unicode, WWW, Databases and Japanese Web Sites?

Matthew Branton asks: "I have recently started working on a database driven Japanese language web site, and frankly I am getting lost in a sea of complicated Unicode madness. I was toying with the idea of using Python 2.1, PostgreSQL and the mxODBC interface, has anyone else had experience with this particular setup? Are there more appropriate free solutions available, perhaps JSP/Servlet based?" Unicode received a a lot of coverage this week, on Slashdot, and maybe that's due to the increasing worldwide popularity of the net and the desire to read and work in the character sets for users of a given nationality rather than trying to have it all fit in Latin-1. Do Python, PostgreSQL or mxODBC have any issues with Unicode?

Comments

Re:Ask some Japanese Webmasters

[JabberWokky]

The fetish for Anime here notwithstanding, very, very few of the /. readers can read Japanese, or know anything about the culture. Asking an Anglo crowd won't get you much in the way of results.

Even if you ignore the fact that many Anime fans visit Japanese sites on a regular basis, and /. Anime fans tend to be technical, and thus know what is used on Japanese sites, I am sure that I am not the only person who has been contracted for an Asian site (in my case, Chinese).

I *can* tell you that PHP and MySQL work quite nicely with Chinese character sets, but you will quickly run into a few dozen tiny issues involving things like sorting and oddball string functions. In PHP's case (and since I don't know Chinese), I created a testbed which ran an arbitrary string through every function that could possibly be in use. I then had a Chinese tester go through and make sure everything worked (iirc, only a few wierd, replacable functions like ucword() didn't work). In MySQL's case, using the latest version, setting a few flags and writing the queries slightly differently than I normally do (BINARY flag, etc) was the fix: something that I contacted the developers about, and got (as of then) undocumented, cutting edge answers.

--
Evan

Re:Ask some Japanese Webmasters

[Kingfox]

You could always try asking the same question on Slashdot Japan. You might get more of an answer. I think the /. readers there read a bit more Japanese.