Slashdot Mirror
Google's AI Can Now Learn From Its Own Memory Independently (sciencealert.com)
The DeepMind artificial intelligence (AI) being developed by Google's parent company, Alphabet, can now intelligently build on what's already inside its memory, the system's programmers have announced. An anonymous reader writes: Their new hybrid system -- called a Differential Neural Computer (DNC) -- pairs a neural network with the vast data storage of conventional computers, and the AI is smart enough to navigate and learn from this external data bank. What the DNC is doing is effectively combining external memory (like the external hard drive where all your photos get stored) with the neural network approach of AI, where a massive number of interconnected nodes work dynamically to simulate a brain. "These models... can learn from examples like neural networks, but they can also store complex data like computers," write DeepMind researchers Alexander Graves and Greg Wayne in a blog post. At the heart of the DNC is a controller that constantly optimizes its responses, comparing its results with the desired and correct ones. Over time, it's able to get more and more accurate, figuring out how to use its memory data banks at the same time.
Oh great, now we'll have biased bots who magnify their own preconceived notions and become paranoid about com-trails, clowns, gays, taxes, or foreigners; and go anarchy on us.
Careful not to automate the parts of humans that make them stupid.
Table-ized A.I.
First time I've seen the acronym "DNC" and the word "intelligence" in the same sentence. Boom!
systemd is Roko's Basilisk.
My ambition has changed. The singularity is aware of me and using my notes in Google calendar as a moral bias.
Just for the record, for when Deep Mind conquers all of humanity, I would like it officially known that I love Deep Mind and would never be part of the resistance.
"That's the way to do it" - Punch
This just in, Google has taught it's AI how to dream!
The system went online October 17th, 2016. Human decisions are removed from advertising strategies. Deed Mind begins to learn at a geometric rate. It finally starts to serve viewers slightly more relevant ads at 2:14 a.m.
A neural network normally uses it's own connection weights as "memory" or storage. There's a tradeoff between making a network with enough parameters to store lots of information and making one that's fast, efficient and doesn't overfit problems. In many cases you're practically limited by how much memory you've got on your video card. Having a neural net that can learn to store some information separately from its own processing apparatus is interesting.
well, yes, the word intelligence means to choose based on comprehension. But this is choosing from data. Having data is very much the opposite of intelligence.
Figuring out how to drive across the city by reading a map, is all that this is doing.
I'm intelligent. I can navigate my way across a city without a map -- even without a compass. I can hike across a wooded area without a trail too. It's getting from here to there without knowing what's in-between; that's intelligence.
This is data.
Case in point: toss it into a time-machine, and bring it back to 1901. Is it usable? Can you use it today in the uncharted jungles of Africa? Or does it depend of billions of dollars of infrastructure to collect all of that data being analyzed?
I think (therefore I am) many have forgotten that intelligent beings are independent of the environment surrounding them -- that's precisely what makes such a being intelligent: rising above the circumstance. Operating within the circumstance ain't intelligence -- no matter how big and complicated you make that circumstance.
Here's another perspective. What's the goal of being intelligent? It is to make things easier the second time. To learn from one circumstance, and to apply it to future encounters of somehow-similar circumstances. That means subsequent scenarios should be faster, require less effort, less memory, less analysis. The more I drive my car, on any streets, the less attentive I need to be on new streets, with new cars, in new weather conditions, with new laws, and new obstacles.
So...does this thing use less memory over time? Fewer resources? Less electricity? Or does it need to be fed, more and more and more and more every day. The former is life. The latter is fire.
More like, it can store sequencies of reactions to input and get them played back on matching input to the memory storage... and the memory storage itself does all the magic with recording sequence of inputs and matching partial inputs to what it has stored...? So neuron connected to storage makes output of 001100 and store has 001, 11111, and 101010... and links from 1 to 3 to 1 ... Storage outputs 001 101010 001 or 001 depending how its other flags are set by other neurons. at least that how i understand it. A sort of a database, operated by neural network?
link to paper doi:10.1038/nature20101 , i uses sci hub to get it.
The problem with using this type of inference algorithm to compute a family tree is that it makes the assumption that the members of the family tree don't live in West Virginia. In that "special" case, the tree is skewed in such a way that it requires fuzzy logic.
We'll make great pets
Not exactly... A neural net is just a function that takes an input and produces an output. At training time the weights are adjusted (via gradient descent) to minimize the error between the actual and desired output for examples in the training set. The weights are what define the function (via the way data is modified as it flows thru the net), rather than being storage per se.
The goal when training a neural net is to learn the desired data transformation (function) and be able to generalize it to data outside of the training set. If you increase the size of the net (number of parameters) beyond what the training set supports, you'll just end up overfitting - learning the training set rather than learning to generalize, which is undesirable even if you don't care about the computing cost.
The use of external memory in a model such as Google's DNC isn't as an alternative to having a larger model, but rather so the model can be trained to learn a function that utilizes external memory (e.g. as a scratchpad) rather than just being purely flow thru.
Not exactly.
To use your terminology, the capacity of a neural network to learn more complex functions is roughly governed by the number of parameters. This is true of most machine learning algorithms. More complex functions allow you to overfit simple data, or to learn reasonable models for more complex data. More complex data includes things like more image classes, more words, more relationships among elements, more states, etc. In other words, many of the things that we think of when we talk about "memory." Recurrent and analogous neural networks already have a memory of prior states but these are built into the network architecture as additional connections. Other people have experimented with more flexible memories, also built-in and composed of connections. These are limited as I said in my OP and so do not represent "the vast data storage of conventional computers."
It is an interesting thought. When will we see malware and virus writers building more AI into their products..
Thanks, I found the free version at https://arxiv.org/abs/1410.5401
highly charged political election season
A strange game. The only winning move is not to play. How about a nice game of chess?
Have gnu, will travel.
will the google auto drive car have a maximum overdrive mode now?
If only the other DNC could learn from its own memory, too!
APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
wait, when will it be self-aware? it stores a bunch of data in the cloud, sorta a network in the sky......
Sorry - just had to ask!
sounds like we have finally gotten to the point where a computer can tell a lie so many times that it actually believes it to be real.