Google's New Translation Software Powered By Brainlike Artificial Intelligence (sciencemag.org)

sciencehabit quotes a report from Science Magazine: Today, Google rolled out a new translation system that uses massive amounts of data and increased processing power to build more accurate translations. The new system, a deep learning model known as neural machine translation, effectively trains itself -- and reduces translation errors by up to 87%. When compared with Google's previous system, the neural machine translation system scores well with human reviewers. It was 58% more accurate at translating English into Chinese, and 87% more accurate at translating English into Spanish. As a result, the company is planning to slowly replace the system underlying all of its translation work -- one language at a time. The report adds: "The new method, reported today on the preprint server arXiv, uses a total of 16 processors to first transform words into a value known as a vector. What is a vector? 'We don't know exactly,' [Quoc Le, a Google research scientist in Mountain View, California, says.] But it represents how related one word is to every other word in the vast dictionary of training materials (2.5 billion sentence pairs for English and French; 500 million for English and Chinese). For example, 'dog' is more closely related to 'cat' than 'car,' and the name 'Barack Obama' is more closely related to 'Hillary Clinton' than the name for the country 'Vietnam.' The system uses vectors from the input language to come up with a list of possible translations that are ranked based on their probability of occurrence. Other features include a system of cross-checks that further increases accuracy and a special set of computations that speeds up processing time."

Comments

What are vectors?

[Anonymous Coward]

We just don't know

Re:What are vectors?

[Arkh89]

I am just terrified right now.
Wait until Google Research Scientists learn about matrices...

I've fallen and secant get up

[fyngyrz]

You've gone off on a tangent. It's polarizing. Stop it. Right now.

Re:I've fallen and secant get up

[chipschap]

You've gone off on a tangent. It's polarizing. Stop it. Right now.

You're making me tensor and tensor.

Re:I've fallen and secant get up

[fyngyrz]

You're making me tensor and tensor.

Well, if you were more coordinated, you'd feel better about it. Get my point?

Re:I've fallen and secant get up

[skatefriday]

I never have mod points when I actually want them.

Re:I've fallen and secant get up

[chipschap]

I'm in line with your derivative meaning.

Re:I've fallen and secant get up

[fyngyrz]

This conversation does seem to have a pretty high rate of change, doesn't it?

Re:I've fallen and secant get up

[fyngyrz]

You really shouldn't encourage me. :)

Re:I've fallen and secant get up

[jamiesan]

When you're delta prime hand, you should play it.

Re:I've fallen and secant get up

[fyngyrz]

Are you talking about cardesian co-ordinates?

Test it with the following

[Anonymous Coward]

Take a reasonably complex document and translate it back and forth between two languages like 5 times. When/If the resulting document is still readable and preserves the content from the original document, I'll consider it their new system a success. Until then, automated translation is a pipe dream.

The current version of google translate (and all other systems I've tried) fails spectacularly when doing this.

Re:Test it with the following

[Anonymous Coward]

Do think that a chain of human translators would even be able to pass this kind of test?

Re:Test it with the following

[raymorris]

Electric translition of exciting product of our company google writes true word all times.

Re:Test it with the following

[Godwin O'Hitler]

A real professional, conscientious translator will make sure their translation is unambiguous, even if the original isn't. We don't have the right to practise GIGO.
So provided there were only conscientious professional translators in the chain, yes, they'd pass the test easily.
Having said that, I don't believe the bouncy translation method is a good yardstick at all. A good translation isn't judged on its repeatability.

Re: Test it with the following

[mbeckman]

You don't see how it's possible for humans to do a better job translating than machines? You don't see that humans have the ability to understand the context of a document, to grasp its semantics and logical implications, and derive the meaning and intent of the author?

You aren't, by any chance, a machine, are you?

Re:Test it with the following

[Jon Peterson]

Yes, they will. We used to do this for translating medical papers. The paper would go from English to Chinese and then that would be sent to a different translation company to be re-translated into English. The derived English would then be subject to the same level of clinical and copy editor review as the original. If it didn't pass the Chinese translation would be rejected.

It's extremely expensive, but it can be done. We did this with millions of words, and it cost millions of pounds. No automatic translation would have stood a chance, because those machines aren't trained on clinical text. Hell, there isn't even enough translated medical text like that too use as a training corpus yet!

Re: Test it with the following

[brilanon]

With gills?

Re: Test it with the following

[mbeckman]

Humans have a lifetime of cognitive learning to draw upon when translating. For example, consider this classic linguistic conundrum: "Fruit flies like a banana." Does it mean that fruit, in general, has the aerodynamic qualities of one class of fruit, the banana? Or does it mean that fruit flies, being insects that subsist entirely by eating fruit, particularly enjoy eating bananas?

Humans can do this translation correctly every time.

The classic response given by AI Researchers to this class of linguistic challenge has always been "Why, once we have enough facts stored in a computer, in appropriate clever structures massaged by surely simple algorithms, the ability to do this kind of task will just fall out as a natural consequence." It was assumed, from the dawn of AI in the 1950s, that once computers had some more speed and memory this sort of achievement would be easy to brute force by calculation alone.

This turned out to be not true. So in the 1970s AI researchers, who still had no idea how humans do this sort of thing (or any other kind of cognition), said "Well, we don't know how people do it, but we have a dim idea how a brain is structured with neurons and synapses and whatnot, so let us simulate crude mathematical models of these "neural networks", and perhaps the AI corpus will magically start functioning like a human brain."

Sadly, no. Nearly fifty years later and AI researchers are still no closer to replicating human cognition, either in understanding or in blind replication. We can do parlor tricks, yes, such as Google Translate. These tricks can even be useful. But they're still just table-driven automata, without consciousness, without cognition (which we don't understand at all anyway), and certainly without Intelligence, artificial or natural.

I call this "Cargo Cult Science". Like the South Pacific Islanders who built stunningly accurate (but non-functional) bamboo replicas of aircraft, radios, and other technology artifacts left by WW2 soldiers who blipped through their lives, we don't have the foggiest inkling of how intelligence actually works.

AI has failed, so far. No breakthroughs on the horizon, either. The "singularity" is just wishful thinking.

Re:Test it with the following

[Knee Patch]

Okay, but that is just a single pass from English->Chinese->English, and you admit that it is not always successful and might get rejected. The original proposal was FIVE TIMES, without the crib of being able to compare back to the original source at each step. Surely you will agree based on your experience with a "single pass" of this translation that if they went back and forth five times, never being able to reference back to an earlier version, that things would go off the rails.

now i want to translate vector

[turkeydance]

to find out what it is

Don't know what the "vector" is?

[deathcloset]

Sounds an awful lot like the WordNet similarity vector which is commonly used in semantic analysis and is a measure of the 'relatedness' of words - http://search.cpan.org/dist/Wo...

Re:Don't know what the "vector" is?

[phizi0n]

The "we don't know exactly" quote wreaks of bad journalism. It was probably taken way out of context because the author didn't understand what they were being told or they asked a slightly different question than what they wrote.

Re:Don't know what the "vector" is?

[Half-pint HAL]

Given that it's a neural network, no-one's going to know precisely what the vector encodes, or quite what it "means", or what a vector would look like for a given word. I imagine that was what the researcher meant.

Re:Don't know what the "vector" is?

[skatefriday]

Yup. We were doing this at PointCast way back in 1996. The answer to the question, "What is a vector?" Shows an astonishing amount of ignorance on the part of the PR person paid to talk to the reporter.

Re:Don't know what the "vector" is?

[hraponssi]

It sounds like WordNet, Word2Vec and bunch of other stuff in the area. It is actually described in quite detail in the article linked from the page in the summary: http://arxiv.org/pdf/1609.0814...

Re: Don't know what the "vector" is?

[brilanon]

These are fuzzy sensorimotor integrators for food finding

Re:Don't know what the "vector" is?

[SpinyNorman]

The summary is complete gibberish. For anyone interested, Google's own paper describing their NMT architecture is here:

http://arxiv.org/abs/1609.08144

and a Google Reseach blog entry describing it's production rollout (initially for Chinese-English) is here:

https://research.googleblog.com/2016/09/a-neural-network-for-machine.html

The executive summary is that this is a "seq2seq" artificial neural net model using an 8-layer LSTM (variety of recurrent neural network) to encode the source language into a representation vector, and another 8-layer LSTM to decode it into the target language. A lot of the performance improvement is in the details rather than this now-standard seq2seq approach.

The "vector" being discussed doesn't represent words but rather the entire sentence/sequence being translated. This is the amazing thing about these seq2seq architectures - that a variable length sentence can be represented by a fixed length vector!

The representation of words used to feed into this type of seq2seq model is often a wordvec/GloVe embedding (not WordNet), but per the Google paper they are using a sub-word encoding in this case.

Re:Don't know what the "vector" is?

[ole_timer]

technically it's an eigenvector, journalists don't get that kind of training or exposure...

Re:Don't know what the "vector" is?

[ole_timer]

by the way, that's how twitter analysis on different languages, for one example, is done.

tldr

[Anonymous Coward]

They're doing context-aware translation based on massive well organized training sets and clever search algorithms. Woo.

"rolled out" - to translate.google.com?

[ffkom]

Neither the Slashdot summary nor TFA contains a URL to where we can try this now rolled out new translator. Does this imply it's already used by translate.google.com? If so, I didn't notice any improvements, yet.

Re:"rolled out" - to translate.google.com?

[Half-pint HAL]

The first link in the summary is to Goole's blog. The first translation roolout is Chinese to English -- all such translations from now on will be carried out by our new connectionist translation overlords, and I for one welcome them. What I'm curious about is how it will handle languages than had insufficient data for old-style Google Translate's statistical translation engine. Will this do better, or will it be even more sensitive to small datasets?

Re:"rolled out" - to translate.google.com?

[Fwipp]

It's rolled out for Chinese->English, with more on the way.

The Google Translate mobile and web apps are now using GNMT for 100% of machine translations from Chinese to English—about 18 million translations per day. The production deployment of GNMT was made possible by use of our publicly available machine learning toolkit TensorFlow and our Tensor Processing Units (TPUs), which provide sufficient computational power to deploy these powerful GNMT models while meeting the stringent latency requirements of the Google Translate product. Translating from Chinese to English is one of the more than 10,000 language pairs supported by Google Translate, and we will be working to roll out GNMT to many more of these over the coming months.

Re:Bullspit

[JustAnotherOldGuy]

This AI hype has to stop. Neural networks are nothing like how the brain works. We have known that since 1975 at least! The only thing more annoying than a space nutter is an AI nutter.

There you go again.

AI-assisted translation is only going to get better and better and better as time goes on. It won't happen tomorrow or next week or next month, but come back in 5 years and I'd bet that it'll be a whole different ball game.

As someone who saw the idea of a portable phone go from "pipe dream" to "something you can buy for $9.95 at Walmart", I've learned not to scoff or say stuff like this can't be done. It will be done, just not at the breathless pace the press releases would like you to believe.

And that $9.95 phone? It also has a video camera, GPS mapping, accelerometers, a nice color display, and tons of other shit. You can do photo and video editing on it, send texts to the other side of the planet for free, and play all the silly games you could ever want on it. It has more computing power than the entire Department Of Defense had in 1960.

If you had told most people about this in 1960 they'd have had you committed. So yeah, I believe some form of AI will happen. It's inevitable.

Re:Bullspit

[Anonymous Coward]

To be fair "deep learning" is a concept that could potentially spell the doom of traditional AI programs. The first problem for AI is that these learning networks don't use internal representations to do the "thinking", they perform analog computations which are just as mystifying as biological brains, hence the "what is a vector? We don't know" comment. The second problem is that in order to train one of the networks how to do something, you have to create the lessons that teach the subject you want it to learn, which is exactly what we already do for teaching children - something which is itself very hard to do. Symbol processing mechanical intelligence is a dying dream.

Not be evil

[ninthbit]

Hopefully when Google's network becomes sentient, it will follow their "don't be evil" motto a little more closely then the humans running things.

Re:Bullspit

[Half-pint HAL]

To be fair, symbol-processing human intelligence is a dying dream -- just one of the mistakes that Piaget made.

Re:Language translate to Universial Network Langua

[Half-pint HAL]

No human brain could ever hope to process a grammar as "big" as UNL. And no, I've never read Wittgenstein. Although I did take that as my name for a German class once.

But can it pass the ultimate test?

[BoogieChile]

The test of the Lion-Eating Poet of the Stone Den?

Google's New Translation Software

[b783719]

Google Translate the following:

"I ate steak at John's place" -> Chinese -> Russian -> French -> German -> Japanese -> Italian -> English

"I ate the steak instead of John"

Good enough from not getting eaten.

Since we don't know how the brain works...

[mbeckman]

How can this translation software be "brainlike"? Let's see... It doesn't translate the way human brains do...it produces results a small fraction of the quality a human brain produces...and, it can be fooled by trivial procedures like reverse-then-forward translation, where human brains are not fooled.

I know brains, and those ain't no brains.

"Learning" is old translation technology.

[aberglas]

Maybe 25 years ago, the break through in machine translation was to use statistical techniques. The United Nation provided a nice, accessible corpus of texts manually translated to different languages for initial learning.

Statistics is the old word for learning -- it is all about learning patterns from data.

Maybe the the new version of Google is better, and maybe somewhere within it it actually uses an Artificial Neural Network, although tat would seem an odd use of that particular machine learning technology. But nothing fundamentally new that can be seen in the article.

This article demonstrates slash dotter's complete lack of understanding of AI technologies beyond journalistic fluff. For a readable, high level overview have a look at

http://www.computersthink.com/

Nice, now stop spamming me...

[Torp]

... with 'would you like to translate this page?' on every page.

Neural science, so exciting

[brilanon]

Zee Google search for your business and your business is a brain

Re:Bullspit

[mbeckman]

Imagine I started calling a blender an "artificial digestive system" that mimics human digestion. Would you buy that? Not if you're a biologist. Where are the enzymes? Where are the biochemical pathways? Where is the nutrient separation and distribution network? Where, indeed, is the anus?

Yet my blender claim is more accurate, by far, than the claim that Artificial Intelligence mimics biological intelligence. The operative word here is "intelligence." We're talking actual cognition, not pre-programmed reactions. No biologist calls a venus fly trap intelligent, even though it has enough cellular automation to catch and digest flies. An ant has the beginnings of intelligence, although we have very little understanding of how even this primitive life form cognates.

Nobody is saying that computer emulation of various tasks that humans do is useful. It is useful. It just isn't intelligent. Not even as intelligent as an ant.

Stanford AI researcher Andrej Karpathy wrote an excellent essay entitled The state of Computer Vision and AI: we are really, really far away. He summarizes how little we've accomplished in terms of AI's original goals. The piece was published in 2012, and AI hasn't moved a nanometer since.

Re:Bullspit

[JustAnotherOldGuy]

The operative word here is "intelligence."

Yes, defining "intelligence" is a key item here. What does it actually mean, and how can we say whether something is "intelligent" or not? It's a bit of a fuzzy area to say the least. Without a clear definition of what "intelligence" means, we're all just guessing.

A couple of things:

First, I think that a sufficiently sophisticated system could mimic intelligence even though it wouldn't actually be intelligent (whatever that means). No, it wouldn't be truly intelligent or genuine "AI", but it could be good enough to use for a lot of practical applications.

Second, I do think that eventually we will develop some sort of genuine AI, in fact I think it's inevitable. But again, it's going to need to be defined as to what genuine AI is. What's the yardstick for determining whether or not it's really artificial intelligence? I think it would include the ability to learn, to make decisions that aren't pre-programmed or mechanically heuristic in nature, and the ability to potentially be somewhat illogical under the right conditions. Creativity (however you define that) would also be a component. Others would claim that it would have to include empathy and other "emotional" states.

In short, it's a helluva thing just to define what "intelligence" means and how to detect it. But I do believe that some form of "real" AI will eventually happen, even if it's not what we imagine it would be like today.

Again, how would we know whether or not to consider something as "intelligent"? What properties would it have to display to be labeled as such?

Re:Bullspit

[mbeckman]

By "mimic intelligence" I meant to operate in the same way biological intelligence does. Perhaps I should have said "replicate intelligence" to be totally clear. And you're right, we can't replicate something we can't take apart and explain.

Your belief that we will eventually develop genuine AI seems premature, since we don't yet understand intelligence. What if, for example, the brain is just a transceiver that communicates with the true seat of intelligence, which happens to be in another dimension that we can't yet perceive scientifically? Science went millennia, for example, before detecting the phenomenon of radio waves, subatomic particles, and quantum mechanics.

There simply is no proof that intelligence is materialistic, and plenty of evidence that it isn't (such as neuroplasticity, as seen in aphasic brain function reassignment). Yet AI researchers pointlessly bang away at this approach without having done their foundational homework.

I think we should focus on defining intelligence rather than jumping to the end game of creating one.

Re:Bullspit

[JustAnotherOldGuy]

I think we should focus on defining intelligence rather than jumping to the end game of creating one.

I agree.

At the same time, though, by attempting to mimic it or create it we may discover something along the way that helps us define it or understand it. If we needed to completely understand something before we tried to create it we'd be way behind where we are now in all sorts of fields. Sometimes the failures teach things that lead to successes.

Re: Bullspit

[mbeckman]

Precisely. Let's just not call it AI :)

Re: Bullspit

[JustAnotherOldGuy]

Precisely. Let's just not call it AI :)

But what if it is, and we don't realize it or recognize it?

Re: Bullspit

[mbeckman]

Then we'll just have this conversation with it, and ask it. :)