A Neural Network Can Learn To Recognize the World It Sees Into Concepts (technologyreview.com)

An anonymous reader quotes a report from MIT Technology Review: As good as they are at causing mischief, researchers from the MIT-IBM Watson AI Lab realized GANs, or generative adversarial networks, are also a powerful tool: because they paint what they're "thinking," they could give humans insight into how neural networks learn and reason. [T]he researchers began probing a GAN's learning mechanics by feeding it various photos of scenery -- trees, grass, buildings, and sky. They wanted to see whether it would learn to organize the pixels into sensible groups without being explicitly told how. Stunningly, over time, it did. By turning "on" and "off" various "neurons" and asking the GAN to paint what it thought, the researchers found distinct neuron clusters that had learned to represent a tree, for example. Other clusters represented grass, while still others represented walls or doors. In other words, it had managed to group tree pixels with tree pixels and door pixels with door pixels regardless of how these objects changed color from photo to photo in the training set.

Not only that, but the GAN seemed to know what kind of door to paint depending on the type of wall pictured in an image. It would paint a Georgian-style door on a brick building with Georgian architecture, or a stone door on a Gothic building. It also refused to paint any doors on a piece of sky. Without being told, the GAN had somehow grasped certain unspoken truths about the world. Being able to identify which clusters correspond to which concepts makes it possible to control the neural network's output. The team has now released an app called GANpaint that turns this newfound ability into an artistic tool. It allows you to turn on specific neuron clusters to paint scenes of buildings in grassy fields with lots of doors. Beyond its silliness as a playful outlet, it also speaks to the greater potential of this research.

Comments

AI

[ledow]

So it statistically correlated randomly-grouped information over millions of trials?

Still not AI. Still just statistics. Bad statistics. With complete lack of inference. With shady, if not downright dishonest, assertions made about its capabilities.

You did what "AI" has had done to it for decades now... flip a bit to indicate success in some fashion, and throw millions of trials at it until it trains itself to activate "bit" more than a handful of times.

It could be flipping because it's majority green. Because the top-left pixel is green. Because there's watermark on the image. Because some frequency curve (if you have those, it's highly unlikely to form those itself in even a billion attempts / generations / evolutions / trainings) hits on a certain colour.

The fact is: You have NO idea what it's correlating on. It's almost superstition on the part of the AI (if it wasn't completely lacking in any intelligence).

Did you know that if you feed a pigeon in a box at random times it starts to associate feeding time with whatever it happened to be doing, and so repeat that? Whether that's bobbing its head, pecking at the floor, or looking a certain direction.

It then spends most of its time trying to replicate that convinced that it's "just not doing it quite right", like someone with a superstition about their team winning because they were wearing their lucky underpants - no amount of negative correlation will convince them they are wrong and get them to change their ways.

And that's exactly the problem with "AI" / neural networks. Of course you can train them to a statistical correlation - you know why? Because you're eliminating / training out / not breeding from those that don't correlate somewhat. It doesn't mean that what they are working from has anything to do with what you were after. And, most importantly, it does not mean you can trust them further on new data, nor that you can "untrain" them when they get it wrong, nor that you can perpetually improve them by more and more training.

All that happens is that it plateaus before it ever really gets useful (usually within the range of a PhD study - write your thesis up quick!), people release rubbishy apps "to show what it can do" and then it's never touched again because it can't be used for anything else and isn't particularly good or useful at what it does do.

We don't have AI, stop trying to pretend that we do. When you get a machine that can infer, that can actually reason its answer (not just "well it matches shape 22%, colour 17% and overall pattern roughness 19.4%", but "I can see branches here, here and here. They are connected. The connection grow and increase in width. The thickness part, which looks trunk-like, ends in a solid base which resembles soil", etc.)

Until then, this is all just a waste of time, and heuristics (YOU told it when it got the tree right).