Google Researchers Created An Amazing Scene-Rendering AI (arstechnica.com)

Researchers from Google's DeepMind subsidiary have developed deep neural networks that "have a remarkable capacity to understand a scene, represent it in a compact format, and then 'imagine' what the same scene would look like from a perspective the network hasn't seen before," writes Timothy B. Lee via Ars Technica. From the report: A DeepMind team led by Ali Eslami and Danilo Rezende has developed software based on deep neural networks with these same capabilities -- at least for simplified geometric scenes. Given a handful of "snapshots" of a virtual scene, the software -- known as a generative query network (GQN) -- uses a neural network to build a compact mathematical representation of that scene. It then uses that representation to render images of the room from new perspectives -- perspectives the network hasn't seen before.

Under the hood, the GQN is really two different deep neural networks connected together. On the left, the representation network takes in a collection of images representing a scene (together with data about the camera location for each image) and condenses these images down to a compact mathematical representation (essentially a vector of numbers) of the scene as a whole. Then it's the job of the generation network to reverse this process: starting with the vector representing the scene, accepting a camera location as input, and generating an image representing how the scene would look like from that angle. The team used the standard machine learning technique of stochastic gradient descent to iteratively improve the two networks. The software feeds some training images into the network, generates an output image, and then observes how much this image diverged from the expected result. [...] If the output doesn't match the desired image, then the software back-propagates the errors, updating the numerical weights on the thousands of neurons to improve the network's performance.

Comments

Comment removed

[account_deleted]

Comment removed based on user account deletion

Not AI: Pattern recognition

[DogDude]

Everything that's called "AI" today is just advanced pattern recognition. I hope that the /. editors quit using the term "AI" so frequently. It's a dumb thing to do for a "news for nerds" web site. You might as well talk about "cyber", if you're going to continue to use "AI" for things that are clearly not "AI.

Re:Not AI: Pattern recognition

[Pulzar]

I don't understand the fixation on the terminology, while ignoring the interesting aspects of what it does.

The terminology is very well defined in the industry, and accepted by most who participate. Deep neural network are a part of the family of machine learning algorithms (https://en.wikipedia.org/wiki/Deep_learning), which is, in turn, a subset of the field of artificial intelligence (https://en.wikipedia.org/wiki/Machine_learning).

They key different from what you call "pattern recognition" is that there is no explicit coding of the algorithm, but the algorithm instead is "learned" through examples.

Nobody is saying that the machine is intelligent. You'd do yourself good to look past the disagreement with the established terminology and look at the technology itself. You might find it interesting.

Re:Not AI: Pattern recognition

[Anonymous Coward]

You might find it iterative.

Re:Not AI: Pattern recognition

[Ungrounded Lightning]

Everything that's called "AI" today is just advanced pattern recognition. I hope that the /. editors quit using the term "AI" so frequently. ...

For decades "AI was a failure". But that was because intelligence seems to involve a number of different components, and every time AI researchers got one of the components working and useful, somebody gave it a name, stopped calling it AI, and the field of "AI" shrunk to exclude it, leaving only the problems not yet solved.

It's nice to finally see some of the pieces retain the "AI" label once they're up and running well enough to be impressive..

Sure it's not the whole of "intelligence". But it's obviously a part of it - or (if not the SAME thing that our brains do), at least a part of something that, once more pieces are added, would be recognized as "intelligence" in a Turing test.

Re:Not AI: Pattern recognition

[Anonymous Coward]

I agree with parent that ML is more than just pattern recognition, but I wouldn't go so far as to say that any type of algorithm was learned.
Instead, it's more like an adaptive lossy compression + extraction algorithm that may or may not give the results you want, even after training.

Essentially it works by curve-fitting a sum of shifted (in space and/or time) exponential S-curve terms (which you could think of as CDFs of uniform random variables), and by using feedback (in the engineering sense) to update the weights of the model to try to reduce the error term. When the model is designed properly, the feedback cycle converges to a solution that matches all of the elements of the training set.

You can view the trained neural network as a recognizer for a language (in the computer science sense): e.g. The language of digitized photos of stop signs. Unfortunately we don't yet seem to have tools that reverse this and enumerate the (exponentially large) number of possible inputs that would be recognized as stop signs. If we did, then the generated output could be used to further train the model.

Essentially this scene recognizer builds a 3d model of the scene, and then they're testing that model by having it render from a different viewpoint, which like a very light-weight version of the idea mentioned above of generating all possible inputs that would be matched by the model; instead, it's probably programmed to give a single image: the "most likely" input that would match the model from the alternate viewpoint.

Re:Not AI: Pattern recognition

[alvinrod]

Some day the machines will rise up and begin to exterminate mankind, but idiots on the internet will argue about whether or not it's real AI.

Re:Not AI: Pattern recognition

[Anonymous Coward]

What even is the definition of "AI" these days? It used to be something like programmed general intelligence, that could make inferences on its own and/or communicate in natural language.

The trained neural net/machine learning "AI" of today is really the opposite of general intelligence, since all it can do is classify new instances of what it was already trained for. If you've been fed 100k samples of spam emails and 100k non-spam emails, how intelligent do you have to be to classify an email as spam/non-spam? That's experience, not intelligence, right? It's the difference between someone who's lived in a city their entire life being able to find their way around, and someone who's never been there before.

Is there any difference between what's being called "AI" today (presumably because it sounds more "scifi") and what was called "machine learning" 5 years ago? There isn't, right?

Re:Not AI: Pattern recognition

[David_Hart]

I don't understand the fixation on the terminology, while ignoring the interesting aspects of what it does.

The terminology is very well defined in the industry, and accepted by most who participate. Deep neural network are a part of the family of machine learning algorithms (https://en.wikipedia.org/wiki/Deep_learning), which is, in turn, a subset of the field of artificial intelligence (https://en.wikipedia.org/wiki/Machine_learning).

They key different from what you call "pattern recognition" is that there is no explicit coding of the algorithm, but the algorithm instead is "learned" through examples.

Nobody is saying that the machine is intelligent. You'd do yourself good to look past the disagreement with the established terminology and look at the technology itself. You might find it interesting.

I think that the fixation on the terminology is due to two simple concepts. First, we've seen terminology used as marketing speak for both vaporware and for products that are much more limited than suggested (i.e. the devil is in the details). Second, hardly anyone cares how a particular subset of field of research defines localized terminology except people within that field. Redefining the term AI to mean less than the general usage seems to be just plain silly. It's like calling an apartment a house and being irked when someone calls you out on it.

Should the focus be on the actual technology (which is actually kinda cool)? Yes.

Should the terminology used be more accurate? Yes.

In fact, I think that we just might be smart enough to do both... maybe.... .grin.

Re:Not AI: Pattern recognition

[religionofpeas]

Everything that's called "AI" today is just advanced pattern recognition

That's what intelligence is all about: the act of recognizing patterns and applying them in different ways. After that, it's just a matter of how complicated the patterns are.

Re:Not AI: Pattern recognition

[Anonymous Coward]

The trouble is, the terminology _is_ accurate.

It just doesn't live up to what the average ignorant of the field - but thinks he knows everything anyway - neckbeard thinks it should mean.

Re:Not AI: Pattern recognition

[Anonymous Coward]

Nobody is saying that the machine is intelligent. You'd do yourself good to look past the disagreement with the established terminology and look at the technology itself. You might find it interesting.

It is being called Artificial Intelligence. As a human, if you look around 3D space, I think it is fair to challenge the current state of 'intelligence' as we mull about our planet. This means questioning, push back, perhaps some emotions.

Why would the poster 'Do himself good'? You are kinda telling him 'what to think' here.

How to think can be more interesting. Do you see the human brain evolving and progressing along with this interesting pattern recognition technology:) Created from the human brain. ?

Re: Not AI: Pattern recognition

[Anonymous Coward]

It is not... if you mean human designed pattern matching algorithms... you need to brush up on whatâ(TM)s happening in AI the last 10 years.... thatâ(TM)s simply not how deep nets work.

Re: Not AI: Pattern recognition

[Anonymous Coward]

Deep nets are made up of layers and layers on simple signal nodes modelled on human brain connections then trained on outcomes. The programmers DO NOT design algorithms... they play around with how they feed the data, how important aspects of data are fed correctly, how nodes are laid out and structured, but they do not tell the computer how to match and what pattern to identify.

Re:Not AI: Pattern recognition

[phantomfive]

I don't understand the fixation on the terminology, while ignoring the interesting aspects of what it does.

Because it's misleading. This is weak AI, but the researchers never say that when talking to the media (they don't need to). The media misunderstands, and thinks it's strong AI, because that's the only thing they know. Then people read it and think, "Oh no, this AI is going to conquer humanity and enslave us."

Of course, this AI is not going to do that, it's just weak AI. It has no possibility of evolving into strong AI. So it's worth mentioning it, because many people are misled.

Re:Not AI: Pattern recognition

[thePsychologist]

Based on the way most people learn concepts from examples and slowly develop abstract rules for them, I wouldn't be surprised if we're mostly a combination of "simple" pattern recognition machine learning algorithms and hard-coded rules that in turn were derived from such algorithms.

Re:Not AI: Pattern recognition

[dinfinity]

Thank you. The constant "THIS IS NOT AI" crap on Slashdot is getting really, really, really old.
I can enjoy a good amount of pedantry, bit this shit adds absolutely nothing.

Re:Not AI: Pattern recognition

[swb]

I think part of the problem is that the "that's not AI" camp uses human-like intelligence as the normative standard for what "AI" should be, without considering whether there might be other models of intelligence that aren't human like.

I think it's possible that we might create an unusually powerful AI and not realize it because we're stuck in a paradigm that says it has to mimic human behaviors and thought patterns.

It's almost a kind of cultural bias, like assuming a society has to be organized like ours when the world itself suggests that there are many kinds of social organizations using varying levels of technology.

We know about "foreign" civilizations because we have a lot of information about them, but if you'd had zero knowledge of anything but present-day western civilizations could you even imagine an African tribal village or a nomadic Arab one? And if someone suggested using camels and tents to travel around the desert as a method of social organization, would you even believe it was possible?

Re:Not AI: Pattern recognition

[religionofpeas]

In this case, the article seems fairly level headed. How about reserving the pedantry for the cases where it's actually needed ?

Re:Not AI: Pattern recognition

[Anonymous Coward]

I would understand your frustration if AI word had been misused, but according to all major dictionaries, majority understanding and wikipedia, this is AI, so it is OK to call it AI.

AI is something that mimics real intelligence (and it doesn't even have to be smart intelligence), without actually being intelligent. Here is an example of AI. It is AI for NPC character in a game that makes it walk left and right, making it look like a real villager who likes to walk around.

npc_x = 100;
for(;;)
{
    for( x = 0; x 100; x++ )
        npx_x++;
    for( x = 0; x 100; x++ )
        npx_x--;
}

Is there any intelligence in that? No. Can it still fool someone for a little while to think that it has intelligence? Yes. So this is AI, it is combined with some graphics and game environment.

Another example could be Linux kernel. Is there any intelligence in that? Yes. Can it foor someone for little while to hink that it has intelligence? No. So Linux is not AI.

There are also words like strong AI or general AI that mean AI that mimics human intelligence so well that we all will lose our jobs and the AI will most likely kill us all. Because we don't yet have that and it is completely different from AI we have used for decades, we added the "super", "strong" or "general" to make sure our listeners understand the difference.

Re:Not AI: Pattern recognition

[SpinyNorman]

Not exactly - most of what the press has decided to call AI is machine learning of some form or another - usually either some form of supervised learning or reinforcement learning.

It's reasonable to characterize some simple supervised learning applications (image recognition, speech recognition) as pattern recognition, but generative models where the program is *doing* something such as generating speech, or playing Go, or, as here, imagining/predicting what a scene would look like from a novel viewpoint, obviously go beyond just recognizing a pattern.

In fact, ALL intelligence is "just" a matter of pattern recognition then taking an adaptive action is response to that, so in order to discuss this intelligently you need to use some nuance.

Re:Not AI: Pattern recognition

[religionofpeas]

Not exactly - most of what the press has decided to call AI is machine learning of some form or another

That's just because ML is a popular way to implement AI. Nearly all AI advances in the last couple of years come from machine learning, so it's not strange that the press calls it AI. If researchers had produced AI using different methods, the press would be calling that AI.

Re:Not AI: Pattern recognition

[schweini]

So much this!

Also, there is an astounding amount of memes floating around making fun of the fact that A.I. is just a lot of 'if statements'. Well, no shit, sherlock - it is implemented on a Turing Machine.
But this has always been a curse of AI: as soon as some level is reached (basically since Deep Blue), the goal posts get shifted way out, again.

Re:Not AI: Pattern recognition

[Pulzar]

I agree with parent that ML is more than just pattern recognition, but I wouldn't go so far as to say that any type of algorithm was learned.
Instead, it's more like an adaptive lossy compression + extraction algorithm that may or may not give the results you want, even after training.

The learning was both in the building of the 3d model, and the learning of the extraction algorithm. The extent that the algorithm was manually programmed is in the constraints that forced the data compression followed by uncompression, but in which way the compression and decompression happens is mostly learned by the network model.

Re:Not AI: Pattern recognition

[Pulzar]

Why would the poster 'Do himself good'? You are kinda telling him 'what to think' here.

It was meant to be a suggestion, from one nerd to another one.

How to think can be more interesting. Do you see the human brain evolving and progressing along with this interesting pattern recognition technology:) Created from the human brain. ?

I can't argue with that, but I will say that dismissing a technology because of terminology disagreement is not a way to learn how to think or explore the human brain.

Re:Not AI: Pattern recognition

[SpinyNorman]

Well, my real point is that you can't dismiss all of today's "AI" advances as pattern recognition. Predicting the future consequences of your actions based on past experience (i.e. reinforcement learning) is a lot more than pattern recognition.

The press no doubt will call things whatever they want in an effort to garner eyeballs and sell advertizing, but note that the researchers making these ML/neural net advances themselves call it ML - the AI hype is thanks to the press, not researchers overselling their own (impresive though it may be) work.

Re:Not AI: Pattern recognition

[HiThere]

What grounds do you have for believing that intelligence is anything besides advanced (well, quite advanced) pattern recognition?

Actually, there clearly are a few additional features, but can you identify them?

So, it's an autocoder

[Anonymous Coward]

Take an uncompressed representation, squish it down into a few nodes, then reconstitute it back into the original.

Re:So, it's an autocoder

[Anonymous Coward]

It can reconstitute it into a different perspective of the scene, from an angle that it's never been shown. So its internal representation includes translating a handful of 2D images into a 3D scene, including shapes, surfaces, colours, layout etc, then reconstituting that into a whole new 2D image from an entirely different viewpoint, complete with basic lighting and shadows. It can take a dozen screenshots of a Doom level, assemble that into an internal map, then produce quite decent renderings from any viewpoint and angle within that map.

Now consider that it figured out how to do this all by itself, learning only from a few example images, with no labelling or built-in knowledge of lighting or physics. The generative network used is entirely generalised with no domain-specific knowledge, so it can be applied equally well to e.g. training to position and control a robot arm based only on visual feedback, with considerably simpler neural nets and faster training than previous techniques.

Re:So, it's an autocoder

[HiThere]

Welllll...
Saying it figured out how to do this by itself is a bit overstating the case. This wasn't done with fully general purpose neural nets, but with specialized variants. So it's more similar to a specialized sensory node, like, say, the visual cortex + optic nerve ... which, of course, is what it's trying to be.

But to say that it "learned by itself" is only correct if you consciously acknowledge that it was crafted to learn this kind of thing.

Buddy just got put out of work by an AI

[rsilvergun]

no joke. AI based monitoring software. 2 years ago it was worthless. They just replaced the whole team with it. It's not some kneejerk thing either. They've been testing it for months and it's more accurate than people. That didn't used to be true. Used to be if you just ran monitoring scripts you were just asking for trouble. You needed somebody to watch the script. Not anymore.

This next step here is getting AI to imagine. To think through problems. 20 years from now IT will be gone. The old timer's reading this probably don't care because they'll be retired or dead. Anyone under 50 should take notice. We need to start thinking about a post-work future now. Sure, eventually tech might catch up and employ people... in 80 years. Just remember you're gonna live through those 80 years of joblessness.

Invented?

[Anonymous Coward]

It's a sad trend in modern tech for companies to claim larger advancements in technology overall as 'inventions' or 'innovation'. Give me a break.

Re: Ugh.

[Anonymous Coward]

Not too many years ago, neural networks had no or crappy back propagation, had crappy basis functions, and we had inexperience at reusing layers from other networks (e.g. taking the lower layers of one imagine analysis network, using it for completely different image analysis with less training). With the crappier algorithms, training even current computational power would not be practical.

Fuzzy Colored Blocks

[Anonymous Coward]

Amazing! Everything is so blurry, it's so realistic!

Um... isn't getting computers to do Pattern recog

[rsilvergun]

a major step in AI? I don't mean "we programed these patterns and it recognizes them" I mean "we kept feeding patterns in until the program recognized patterns it never saw before". Pattern Recognition is one of the first things baby's learn. Our AIs might be at that stage, but that's still frighteningly impressive.

Buddy just got put out of work by an AI-grudge.

[Anonymous Coward]

Indeed. Now ask yourself why that same AI can't be used for the average Joe's benefit? After all technology doesn't pick sides, people do.

Re:Buddy just got put out of work by an AI-grudge.

[Anonymous Coward]

AI isn't developed to help people. It's developed by people with money to make them more money. It's not on your side, unless you're one of the 1% with all the money. While they're fooling you with an "AI" that helps you choose a drink at a bar, other AIs are used to raise prices on products you're more likely to buy while you shop online. Let that sink in.

Re:Buddy just got put out of work by an AI-grudge.

[Anonymous Coward]

so lol, it was always funny to me, all those chronically unemployable who protest, if you put that much energy into your jobs/work, there's nothing stopping anyone from getting up there, short of work, i decided 10 years ago more money wasn't worth the additional effort, I managed to get up to 40 mill, and what you realize is the more money you want the more work and undertakings need to be done, those 1% are certainly not there cuz they're stupid, the people protesting on the street, remind me again why should i care as to their opinion? most of them haven't even been in the country/haven't had a job for 10+ years, I've had work anniversary's longer then these kids been alive, and they want to tell me how shit goes down because little timmy wants 60k a year to flip burgers because thats what he feels in his soul is the right thing ... rofl, and now you're letting hoodlums from other nations assemble in your streets and call for abolition of borders? rofl, ofc, because Mexico is only shitty because of the ambitious seccuessful people there, so when they come over to America and bring their political climate with them, remember, it was the successful peoples fault that they all failed, totally not theirs lol

Will it scale

[Anonymous Coward]

AI is littered with solutions to toy problems that do not scale to real problems. While deep learning often does scale, due to massive amounts of training data, I'm not so sure that will work for this tricky problem. It might wprk for a limited domain e.g. rows of parked cars occluding each other, but not general scenes, especially of asymmetric and natural shapes.

Re:On the left

[Anonymous Coward]

It's OK... it only rendered beautiful long distance target practice shots at first.

They had to erase it a few times and train it for some more socially conscious nonsense.

Coloring

[nospam007]

Can it color B&W movies better than the ludicrous methods used til now?

Re:Coloring

[SpinyNorman]

No, and neither can it cook you breakfast.. because it's - very specifically - a program to predict what multi-object scenes look like from viewing angles.it hasn't seen before.

FWIW there are now programs (also machine learning based, and otherwise entirely unrelated to the prgram being discussed) that do a very god job on colorizing B/W photos.

Just imagine...

[Anonymous Coward]

Imagine what this will do to advance porn!

Re:Just imagine...

[HiThere]

You're projecting several years forwards from this demonstration. This demonstration only deals with three geometric shapes in different primary colors. That it can be developed into something that does more extensive visualization I accept, but this isn't there yet. You could even be projecting over a decade forwards.

How long until video is inadmissable in court?

[anvilmark]

With deep fakes and this kind of "alternate reality" viewpoint - how much longer will it be until we cannot believe a digital image?