Google Brain Researchers Make Significant Progress On Language Modeling (arxiv.org)

New submitter integralclosure writes: Using neural networks, Google Brain researchers have significantly improved a computer's ability to model English (achieving extremely low perplexity score on a large dataset). Using the model they were able to generate random sentences, such as the following: 'Yuri Zhirkov was in attendance at the Stamford Bridge at the start of the second half but neither Drogba nor Malouda was able to push on through the Barcelona defence.' The sentences are generally coherent and mostly grammatically correct. Advances seem to be a replay of neural networks' dominance in the Imagenet competition.

Comments

Perplexity

[Geoffrey.landis]

"achieving extremely low perplexity score on a large dataset"

That's quite an achievement, since I have a very high perplexity score on that sentence.

Do they check whether it achieves low perplexity only when the sentence is not, in fact, perplexing?

How well would it understand this?

[presidenteloco]

Time flies like an arrow. Fruit flies like a banana.