Slashdot Mirror
Google's AI Can Now Learn From Its Own Memory Independently (sciencealert.com)
The DeepMind artificial intelligence (AI) being developed by Google's parent company, Alphabet, can now intelligently build on what's already inside its memory, the system's programmers have announced. An anonymous reader writes: Their new hybrid system -- called a Differential Neural Computer (DNC) -- pairs a neural network with the vast data storage of conventional computers, and the AI is smart enough to navigate and learn from this external data bank. What the DNC is doing is effectively combining external memory (like the external hard drive where all your photos get stored) with the neural network approach of AI, where a massive number of interconnected nodes work dynamically to simulate a brain. "These models... can learn from examples like neural networks, but they can also store complex data like computers," write DeepMind researchers Alexander Graves and Greg Wayne in a blog post. At the heart of the DNC is a controller that constantly optimizes its responses, comparing its results with the desired and correct ones. Over time, it's able to get more and more accurate, figuring out how to use its memory data banks at the same time.
First time I've seen the acronym "DNC" and the word "intelligence" in the same sentence. Boom!
systemd is Roko's Basilisk.
Just for the record, for when Deep Mind conquers all of humanity, I would like it officially known that I love Deep Mind and would never be part of the resistance.
"That's the way to do it" - Punch
A neural network normally uses it's own connection weights as "memory" or storage. There's a tradeoff between making a network with enough parameters to store lots of information and making one that's fast, efficient and doesn't overfit problems. In many cases you're practically limited by how much memory you've got on your video card. Having a neural net that can learn to store some information separately from its own processing apparatus is interesting.
Not exactly... A neural net is just a function that takes an input and produces an output. At training time the weights are adjusted (via gradient descent) to minimize the error between the actual and desired output for examples in the training set. The weights are what define the function (via the way data is modified as it flows thru the net), rather than being storage per se.
The goal when training a neural net is to learn the desired data transformation (function) and be able to generalize it to data outside of the training set. If you increase the size of the net (number of parameters) beyond what the training set supports, you'll just end up overfitting - learning the training set rather than learning to generalize, which is undesirable even if you don't care about the computing cost.
The use of external memory in a model such as Google's DNC isn't as an alternative to having a larger model, but rather so the model can be trained to learn a function that utilizes external memory (e.g. as a scratchpad) rather than just being purely flow thru.
highly charged political election season
A strange game. The only winning move is not to play. How about a nice game of chess?
Have gnu, will travel.