Slashdot Mirror
Open-Source Language Translator Opens For Beta
mind21_98 writes "A new machine-translator designed for language translation has offically opened for public testing. GPLTrans is a translator similiar to Babelfish. Pre-alpha testing has shown that it is the most accurate of the major Web-based machine translators. More information can be found here. "
Hopefully we'll see some better translators, because the current ones suck.
:)
And maybe we'll be able to add on some custom vocabulary, that would be really nice for computer journals (or chemistry, medicine, whatever...)
...at least the article wasn't in German, or something.
---
pb Reply or e-mail rather than vaguely moderate.
pb Reply or e-mail; don't vaguely moderate.
We need a web translator that accurately translates swear words, or that will at least handle "Will you please fondle my buttocks?" correctly. My nipples explode with delight!
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
"My hovercraft is full of eels" in foreign languages? Now I can find out! :)
What did you eat today? http://www.atetoday.com/
While machine translation is very practical, it can also provide entertainment. I remember a story about scientists testing an English-Russian-English translator by translating phrases to Russian and back. Input: "The spirit is willing, but the flesh is weak." Output: "The vodka is good, but the meat is rotten."
-- The Sheep --
I'll give this a test at the office,. because half the time I don't understand half of what the customers are saying.
Perhaps I can use it to translate my words to the customer,. so when I say "Ok,. click on My Computer" they don't hear "restart the computer and click on the first icon you see while hitting the esc key and pulling on the power cord".
My studio - www.graylands.ca
I'm not sure if it has been done yet, but it would be quite helpful if an AI could 'evolve' along with the language (because, as we all know, language changes all the time) based on monitoring of user-editing of the post-process text. For example, if at time 'a' it was programmed to translate 'Cool' to 'Froid' in french, it would (after monitoring the changes made by users) learn to translate 'Cool' to the french equivilent of 'hip'. or something. 'cause, dammit, i can't wait until the AIs take over ;)
It would be nice if someone were to make a CORBA translation service and add this to one or more of the linux desktops. Then it could be used for email, documentation, irc, coding, etc, not just for the occasional web page. It would also be good if the data at gpltrans was snapshotted regularly and pushed around, ideally so that everyone would have their own copy.
It's common to here the pundits opine that "open source may be good at improving 30-year-old operating systems, but the open-source model just doesn't work when it comes to large scale applications." Various reasons are given, for example: "open source programmers only do what is fun and interesting, and applications aren't interesting". But here we see yet another large-scale application falling to the barbarian hordes.
Those pundits are wrong: there is no genre of software that the open-source model will never absorb. Simply because the open-source model results in better software, for reasons that are well-known. And no, there is no no software application that is so uninteresting that no volunteer anywhere in the world will touch it. On the contrary: the more an application area remains untouched, the more interesting it becomes to open-source programmers, simply because it's virgin territory.
This is the "stamp collector" syndrome: when you already have a goodly number of stamps in your collection, adding the missing ones becomes an obsession.
Life's a bitch but somebody's gotta do it.
Will users be able to add/update/correct translations or modify dictionaries ala the APT bot in #debian on irc.openprojects.net?
It seems to me the growth would be incredible if users could modify the dictionary (or atleast add suggestions that could later be added by someone with the appropriate power.
I had an ircii (ok well it was BitchX, but probably would have worked on ircii) that worked with babelfish to translate stuff. I will try to dig it up and post a url (or you could e-mail me) It was called gtrans.bx and I dont know if anyone out there kept it up to date.. anyway it wasn't even that bad in realtime since it made a seperate connection for each translation it did.. There was some latency, sure, but it could do 10 or 20 translations at once. I suppose it could have worked on a queue system with http 1.1 to further expedite things but i really didnt get that deep with it. Anyway, it worked like this:
/en, /es, /pt, /fr, /it
/men, /mes, /mpt, /mfr, /mit
/flag juan es and everything you /msg juan was translated to spanish and everything juan said was translated from spanish. Also if you addressed juan in the channel.. eg. said "juan:" it would print in both spanish and your language.
.so loadable lib) interface for other programs to call?
/mylang
Sets your default language (put in your startup)
/de,
Translates your typing into the language of choice and funnels it to the current dialog. With all of the translation commands if $mylang is set to a non-english, translation to english is done before translating to another language due to babelfish.
/mde,
Sends a message to in the specified language
/flag
Sets autotranslation of a person or a channel. This was really the coolest command. If some spanish-speaking person came in, you can just
/trans
Self explanitory. Output for your eyes only.
Additionally, there were some new functions that people could use to implement their own fun foreign language commands..
I have heard there is a babelfish library out there that provides a standard way to interface a program with babelfish. Plus, only one thing has to be updated for all of your babelfish-ized programs to work. With a client like BitchX this would be very easy to simply load and use!
This GPLTrans thing sounds very exciting and i'd very much like to see about building a new (better) irc script on top of it!
~GoRK
PS. Since the site is slashdotted, could anyone who knows please tell us a little more about it? Can we do the translation on our own hardware or is it central-server based? Can it directly translate between languages where english is neither the source nor the target? Does it provide a standard (e.g.
I would very much like some day to see all of my basic network communications apps (mail client, newsreader, web browser, instant messenger, irc client, etc) have the ability to machine-translate both incoming and outgoing stuff. Everyone seems to be so bent on how "good" the translation is. If a machine can translate something so that I have a basic grasp of what is going on, then the translation has been a monumental success! I would like to machine translation people focus on getting the technology more widespread before they go trying to make their software translate everything perfectly!!
~GoRK (again)
"first wave of the pallet of the beginning"
I wonder if the open-source model for something like this could extend to the program's users as well. The idea would be that, as people used the program, it could learn from their input. Thus, every time someone inputs a new word into their local copy, this information could be replicated at some central repository and made available to other users. In fact, you could even ask the user to categorize, define, and give usage examples for each new thing.
For that matter, you could even have the users refine the system's grammar.
How hard would that be to implement? Is it totally far-fetched?
Can your IM do this?
What most of these language translation programs need is a better understanding of context. I was surprised to find that Altavista's Babelfish utility has very poor analysis of context (possibly none at all). For example, when translating from English to French, "run" always translates to "exécute". For a sentence like you get which is reasonable, but if you translate you get which doesn't make any sense. More incredibly, "store" always translates to "mémoire". You would think that, if they were going to force every word to be interpreted in one sense, they would choose the most common meaning. But this choice leads to insanity where translates to
With knowledge of context, a more advanced system could notice situations in which it was more reasonable for "run" to have a particular meaning. In the last example, "run" is followed by a prepositional phrase indicating a direction, which would imply that the meaning involving physical movement is appropriate, and so on.
Even more revealing is the fact that the confusion of meaning happens differently for different languages. If you translate
into Spanish, you get the hilarious result: For translation software that has multiple language targets, i would have expected it to first resolve the meaning of the English sentence into an internal semantic representation before using it to emit Spanish or French. The above would be evidence that the Systran software has no such representation -- or at least that their representation is too weak to indicate the difference between "store" as in "memory" and "store" as in a warehouse.-- ?!ng
Although the site has been slashdotted, it would be interesting to see what sort of algorithms it uses to perform the translations. Mmm, open source.
:) In addition to this, it's very difficult to write simple, lucid grammar rules that also count for the myriad exceptions found in language.
:) The parsing itself is a hefty (and not terribly exciting) task. I attempted to make a term project of a fairly basic English parser and ended up changing the project.
I would be inclined to say that if it is based on grammar rules, the project won't make much headway - machine translation has been butting its head against this brick wall for forty years. The problem with hard-and-fast grammar rules, e.g.,
S = NP VP
NP = Det (Adj)* N
VP = V (Adv)
is that they don't account for rapid linguistic change, and people have this nasty habit of twisting grammar to express themselves in new and creative ways.
I imagine GPLTrans would probably be using some sort of probability frame of phrases and words occurring together, but one can't be sure without looking at the source. I think the best way to do translation software would be to convert the text into syntax, then into a more abstract semantic form, and from the semantic form, translate back into the target language's syntax, and then into the target language's text. Of course, the trick is to figure out just exactly how to do this.
My 2 cents/Pfennig/lire/pesos,
Y
"There is no culture in computer science, only cults." - M. Felleisen
GPLTrans can be quite good, but imagine it's not (I still can't access). Let's suppose that its translation strategy is not very sophisticated and this system ends up being only marginally better than the others. Now, if somebody comes up with a great idea to improve the design of a machine translation system and wants it to be free, what is (s)he supposed to do?
- post it here and hope for the best ?
- report it as a bug fix ?
- do the coding and contribute a patch ?
- fork ?
- start from scratch ?
- try the first five options, in that order ?
Does the outcome depend on the people running the original project?If they are closed to design improvements contributed by others, is their project truly Open?
Do you mean Phillip K. Dick's novel "Galactic Pot-Healer"? (Stupid title, I know). In it, bored office workers sending a book title or folk saying through multiple translator machines, and challenging their friends to guess the original title.
It's just called "The Game" in the book.
"How perfectly Goddamn delightful it all is, to be sure" Charles Crumb
I hope the word databases and algorithm are easily separable from the implementation. I'm sure they can't have bound it too tightly to PHP and MySQL - the presentation layer should be determined by the user, and use of other databases should be possible.
Bruce
Bruce Perens.
Maybe I can stop send letters to my french relatives that say: "I am ambiguously gay" instead of "I love my brothers" etc...
Movie News - "Entertainment news, bitch!"
I don't think it's tenable under the Open Source paridigm. I'm sure there are other, similar examples. So, there's room for proprietary software, coexisting with free software and running on a free infrastructure. I'd just rather keep the proprietary stuff in the leaf nodes of the software "tree", where nothing else depends on it.
Bruce
Bruce Perens.
I've studied compiler design, and I've wondered about how human languages compare to programming languages. I would think the biggest hurdle is interpreting ambiguous phrases like, 'fruit flies like a banana'. And all the implied words seem like typecasting, but are also ambigous. '(you/I/they) Come here, dammit'. But I wonder if the entire thing is more than just a really complex language description (in BNF or something) with a big database and a few enumerated phrases.
english to french to english on babel. Not as bad as it could have been... :)
I posted this a reply to a comment but then thought maby it should be its own thread.
The problime is that most if not all of
these systomes know nothing about meaning at all.
All that do is try to match one set of strings to
a difrent set of strings.
GPL Trans works by the substuation methoud.
>from: Mooneer Salem
>
> It is a system where words in a phrase that
> can be substituted are
> marked by %phrase%
> For example:
>
> English: My name is %phrase1%.
> Spanish: Me llamo %phrase1%.
>
This genreal systome can be extended in to a
phrase sturcture grammer with pares of rules for
each language. ex:
english: S -> NP1 V NP2
irish: S -> V NP1 NP2
these rules would modal sentences like:
english: the cat chased the dog.
irish: chased the cat the dog.
All this is oversimplifyed but you get the poin.
The real problime is that you need to be trained
as a linguist to understand what the structer of
many seantences are and even linguestes aruge a
LOT. The phrase structal aprouch is probly what
altavista a such do. All thoe I rilly like the
idea to GPL Trans I do not thik there aproch will
get them to far; but it will be fun to see what
thay can do.
While contextual knowledge can increase the qualitiy of a translation; the amount of world knowledge necessary to translate a typical web page is simply astounding. Most users of a translation system simply do not want to wait for hours to translate a simple sentence.
And, there is the problem of linguistic knowledge. Most web pages are not written in "proper" English, but in some Web-speak-lingo. This requires the system to be very robust.
The most successful use of MT in corporations today are situations where a very simple grammar and lexicon is used, and very little world knowledge ois required. For instance, the Xerox corporation has its own translation system that translates component manuals. The technical writers that write the original version of the manual are required to use very simple English only, without any ambiguities and with very simple constructions.
For translation software that has multiple language targets, i would have expected it to first resolve the meaning of the English sentence into an internal semantic representation before using it to emit Spanish or French.
This "internal semantic representation" is called an Interlingua. It has been used in various MT systems, with varied amounts of succes.
The most important advantage of an Interlingua-based MT system is that is does not require a translation engine for each language pair. For instance, if you create a system for English, French, Dutch and German texts, you only need to create four analysis engines:
- English -> interlingua
- French -> interlingua
- German -> interlingua
- Dutch -> interlingua
And four generation engines:- interlingua -> English
- interlingua -> French
- interlingua -> German
- interlingua -> Dutch
With a non-interlingua system (which is called a Transfer system), you'd have to create 3^2=9 engines:- French -> English
- French -> German
- French -> Dutch
etc..Clearly, it is easier to integrate new languages into a interlingua system than into a transfer system.
Now this is what I call a powerful demonstration of the quality of open source software: ;^)
English: "I am a small fish who wants to live in your ear."
German: "Ich bin a small fish who wants to live in your ear."
Astounding. I couldn't have done it better myself, and it was 6 years since I last took a German class... Wow. Also, I find this part of the Note at the bottom of each page particularily qualitative, too:
Note: this computer-automated translation is not guranteed. It'll screw up with some text. If it does in fact screw up, first make sure you spelt everything properely.
My note: I have mucho respect and understanding for alpha releases. It's just that I'm a nitpicking bastard, and this was quite funny.
main(O){10<putchar(4^--O?77-(15&5128 >>4*O):10)&&main(2+O);}
Alas, the website has been /.'ed, so I can't look at the translator, but there are some serious questions to ask.
1 - testing: They claim to be the most accurate of the web-based translators. Based on what corpus and measured in what way? This isn't a trivial question, there are no benchmarks for translation programmes.
2 - parsing. If this program uses American style phrase grammar, it will inevitably break down. Phrase grammar is counterintuitive and for AI purposes pretty unproductive. It is computationally simple - see Charniak's last book for good parsing algorithms - but almost certainly isn't the way humans process language.
All of the most successful natural language translation systems are, in one way or another, dependency grammar based. Dependency based systems are also generally more portable to other languages.
3 - morphology. English is very morphology poor. If morphology is only minimally accounted for (as a lot of poorly thought out, English based NLP systems are), I don't see how it can hope to work in Russian, or Turkish or dozens of other major languages with rich morphology. Furthermore, what kinds of morphological rules can it accept? There are languages that use prefix, postfix and infix morphology. The kinds of simple rules that can account for English will not go vert far with other languages.
I haven't seen this program, and I don't know how seriously these issues have been considered, but they are the kinds of things to keep in mind when looking at machine translation programs.
Bruce
Bruce Perens.
Some respondents have pointed out the difficulty in making translations contextually sensible ... whether 'run' should be translated as 'execute,' rather than 'quick bipedal motion.'
;) -- based on your own self-declaration, perhaps followed by a quiz to establish competency.)
I don't see an easy way to get out of this -- the needed 'world knowledge' that people have pointed out as necessary for this really is huge.
But (and this is why I mention slashdot's metamoderation), there is a certain amount of brute-forcing which could serve as a useful basis for creating improved context interpretation. For instance, let's say you visit this translation engine and choose some text for it to translate ("Mein Hund ist in dein Aktentasche," say). At the same time, there might be a few selections of recent translations requested by others, and the resultant translations, which could be shown to you based on the languages you know. (Not telepathically
The resultant translations could be joined with alternate tranlations / permutations, and each reader could (say), rank-order them, or choose the best one, as far as they can determine by context, etc.
And hopefully, the program can then be taught (wrong word, but I'm being figurative)that (anthropomorphically), something like "OK, if there are several computer-related terms in the translated text, like megabyte and power-supply, 'run' is likely to mean 'execute.' If 'run' however appears in a context which does not indicate computer use, and / or directly before the paired words 'away from,' it should probably be the bipedal-movement one. And if it's in front of a business-type name, like 'bank,' 'lemonade stand' or 'brothel,' then it is likely to mean 'manage' or 'administer."
In my (interested but ignorant layman's) understanding of AI translators, this is the kind of discrimination that they try to make, nothing out of the ordinary. But, because words can fit into so many categories, I think this sort of gradual, piecemiel accumulation holds hope of making it work better over the long haul. It would take too many linguists to account for all the wacky ways that words get used.
Just thoughts,
timothy
jrnl: http://tinyurl.com/c2l8yr / foes: http://tinyurl.com/ckjno5
I'll take a stab at your puzzle: "I toss my cookies down the toilet." Just a guess, highly dependent on humorous context. ;)
Vovida, OS VoIP
Beer recipe: free! #Source
Cold pints: $2 #Product
Yeah, I think this is also a good idea. The problem with it is that search engines themselves can only supply answers based on statistics, not judgement. It would be useful to do a search engine search like you say, but the translator engine would have to have a good idea of what size chunks to divide the original text into.
Anyhow, no conflict here -- I think translation engines are going to have to use a number of strategies on every input text and see which ones make the most sense in the end, then applying the information that for text-chunk X, translation X-prime (or whichever) was the best translation. That way when phrasings similar / identical to ones in text-chunk X appear again, there is at least a reference to check against.
timothy
jrnl: http://tinyurl.com/c2l8yr / foes: http://tinyurl.com/ckjno5
"But, if you say something more obvious like "Molten Lead is cool" it's pretty easy to assume which version of cool you mean."
/not/ antonymous in the context. For example, molten describes the noun. Molten is probably also partially synonymous with "hot". Since "hot" is the antonym of "cool", in the temperature sense, then one would not use "froid" to describe it in French, but instead the appropriate term for "cool" ("cool" itself I guess), which would not be antonymous with "hot".
Couldn't one also use antonyms in this case. I.e. a word/phrase can be a replacement, if it is synonymous, and
It's 10 PM. Do you know if you're un-American?