Game Developers' Quest To Cross the Uncanny Valley

Nerval's Lobster writes "Nearly 30 years after Super Mario Bros., video game graphics have advanced to heights that once seemed impossible. Modern sports games are fueled by motion capture of actual athletes, and narrative-driven adventures can seem more like interactive movies than games. But gaming's increasing realism brings a side effect — a game can now fall into the 'uncanny valley,' a term coined by robotics professor Masahiro Mori of the Tokyo Institute of Technology in 1970. Jon Brodkin talked to game developers, engineers, motion scientists and a variety of other folks about the 'uncanny valley problem,' in which (some) people feel revolted when confronted by a robot or digital character that doesn't quite look real. In games where human-like characters are necessary, the uncanny valley can be an even bigger problem than in animated movies; gamers control characters rather than just watching them, creating more opportunities for the illusion of realism to falter. New and better tools can help developers and animators deal with some of these issues, but crossing the 'valley' successfully still remains a challenge. Or is crossing it even possible at all?"

Comments

Aki Ross

[TWX]

I still have a bunch of posters from when Aki Ross made the Hot 100, in Maxim all those years ago.

Some day I'll be able to sell them for tens of dollars!

Fool's Quest

[binarylarry]

Games are not meant to be merely a simulation of reality.

Is music an attempt to accurately recreate the sounds we hear in nature? No, that would be moronic.

And then there's this guy.

Re:Fool's Quest

[Nemyst]

Games are not meant to be merely a simulation of reality.

By whose definition, exactly? If the game designer/developer desires to make something that looks closer to reality, they're more than welcome to. That's the thing about art: you do what you want, not what some random Slashdot commenter says you should be doing.

Is music an attempt to accurately recreate the sounds we hear in nature? No, that would be moronic.

Wrong comparison. A better comparison would be synthesizers, which have evolved ever closer to reproducing actual instruments. Many people have lauded them as being heralds of a new age, where people could produce music without requiring expensive recording and mixing equipment. Are you saying that they're also wrong for not following your extremely narrow point of view?

Even better: striving for more realism isn't mutually exclusive with striving for more stylized renditions! Just look at Pixar's movies: they've kept their own style and their art is very far from realistic, but they've most certainly leveraged the latest advances in light transport, materials, filtering, particle effects and much more.

realism doesnt improve gameplay

[johnrpenner]

in the hyper quest for realism — we forget that all those perfect pixels doesnt improve the story nor the gameplay.

pac man was a hit without all the fancy graphics

Re:realism doesnt improve gameplay

[QuasiSteve]

pac man was a hit without all the fancy graphics

So is flappy bird. But to compare it to, say, The Last of Us would be bordering on the ridiculous; they are completely different types of games.

we forget that all those perfect pixels doesnt improve the story nor the gameplay

While they might not improve story or gameplay, they may improve the experience. Take any reasonably modern game with 'perfect pixels', and force it to be completely lighting and texture-less - I doubt it would be quite as good. That isn't to say that those 'perfect pixels' are the game's only saving grace - or that there aren't titles produced that are complete turds despite all the graphical polish they receive - but the fact that those 'perfect pixels' can be used does allow game designers to make these games in ways they otherwise could not. They would have to make 'the next pac man' instead.

Re:realism doesnt improve gameplay

[SleazyRidr]

Well, Flappy Bird was pretty popular, and it had even less story than Pac Man.

Re:Citation Needed

[emj]

'and narrative-driven adventures can seem more like interactive movies than games'

Citation Needed

You can watch The last of us as a movie instead of playing it. It's 4-7 hours depending on how much gameplay there is I guess. But live streaming or editing your game sessions is big business now, so I guess it might just be a shift in what is considerd entertainment.

"I make videos of me doing stuff, so you don't have to!" - Washington.

Cole Phelps gently slides down the Stairs

[decipher_saint]

So imagine you have super-realistic characters, then you have them do something impossible like ride a dragon or glitch out on the physics engine... no matter how many hours you put in making them look really Real all it takes is one fuckup and you find yourself staring up from the Uncanny Valley wondering what happened.

Re:Cole Phelps gently slides down the Stairs

[akozakie]

Ride a dragon? That's not at all what uncanny valley is about. This is strictly about things almost perfectly resembling humans. Riding a dragon will not cause this problem. Glitches in the physics engine... Maybe, depends. Something like a not-quite-anatomical pose. Or maybe timing glitches in movement sequence (Crispin Glover's character in Alice in Wonderland - intentional application of this).

In other words, this is a very strong but purely emotional reaction. It gets stronger as you get closer to reality. "Humans" from Shrek? No problem. Aki Ross, at least in motion? Definitely a problem. When it's at its strongest, you might actually have problems pointing out the imperfections that cause it. That's because they are not spotted by conscious reason.

Why is this distinction important? Most deviations from reality in entertainment are spotted by reason and easily covered by willing suspension of disbelief. If the entertainment is good, we will tolerate almost anything, if not, the deviations from reality will add to the list of critical comments. In short: "Yeah, it's BS, but it's fun!"

However, uncanny valley is a subconscious emotional reaction and willing suspension of disbelief does not make it subside. You may consider the movie/game/whatever really fun, but you still simply feel bad looking at it.

That's why it's a big problem for creators of "realistic" games. With simple models this feeling was not there. As models get better, consciously they seem more realistic, but "the body" starts telling us that something's wrong. So, only three solutions - stay away (keep human models imperfect enough), get it perfectly right (is it possible?) or... find a way to eliminate this problem.

Civ5 and strategy games in general

[Anonymous Coward]

I'm not a big pc gamer myself, but when I do play on my computers, usually I play 4X strategy games. My biggest complaint about the genre is by far the difficulty. Civ5, for example is either too easy(anything bellow immortal) or just artificially difficulty(immortal and deity difficulties). When I play at those difficulties, I don't really play a game, I just follow an algorithm(build order, research focus, etc), and if I don't do that I will lose. And it's not only me. Let's Play videos on youtube are pretty much all the same as well. For me it's simply not fun anymore. Better AI is simply mandatory. I don't need photorealism, I don't need 3D, I don't need 4K, I don't need VR. Immersion comes from the gameplay. If the gameplay is flawed, no amount of eyecandy is gonna fix it.

Re:Civ5 and strategy games in general

[ildon]

When I play at those difficulties, I don't really play a game, I just follow an algorithm(build order, research focus, etc), and if I don't do that I will lose.

That is a game. It might not be the game you're looking for, but it's still a game. I'm not going to, e.g. tell professional StarCraft 2 players or speed runners for various games that they are not playing a game. They're just not playing a game I want to play, even if they're playing it in the exact same game engine with the same tools I am. Optimization, memorization, and execution are all "game" skills. For some people, a game isn't fun until they're maximizing those specific attributes, for others, the more important those attributes are to success the less fun the game is. But it's still a game.

As for "better AI", I think what you're really looking for is a strategy game with a human opponent.

This will take a long, long time

[Andrio]

CGI humans in movies--pre-rendered by giant server farms for as long as it takes--still fall into the uncanny valley.

It'll be a long, long time before graphics can be rendered in real time with no uncanny valley. Although, with that said, humans still look fake enough to me in games that there is no uncanny valley. So I don't think it's a problem yet.

I don't think graphics really matter anymore, though. They're far from perfect, but 3D graphics have been "good enough" for a while now. There was a time that 3D graphics meant that hands had to be mittens with no individual fingers, and faces were just drawn on textures. Not anymore.

Re: This will take a long, long time

[Anonymous Coward]

And Neo in The Matrix. Looked so lifelike in stills, but the lack of any facial movement killed all belief that he wasn't CG.

Re:Citation Needed

[The Grim Reefer]

Perhaps more importantly, why would the category 'interactive movie' even get to exist when the category 'game' already does?

I don't know if "interactive movie" is the correct term. But I think there is a difference between that and a game. I don't have time to play games like I did in my youth. But I did enjoy the Mass Effect series. I was surprised with the third Mass Effect game in that it had not only the typical easy, medium, and difficult settings. But it also had one in which you could go through all of the dialog but not have to shoot or do any of the things that typically make up a game. I think it's a significant difference from what I would consider a game.

In the past the story was usually a bunch of cine images that you wanted to skip through as quickly as possible so you could get down to shooting stuff. For me, this actually changed when I played Dues Ex. The game play was fun in that game. But all of the character interactions really made the game a lot of fun. Plus there was a ton of dialog that I found really interesting that didn't have any effect on the game play at all, but was just really in line with the story.

Re:Citation Needed

[noh8rz10]

Bioshock infinite. It was more compelling and entertaining than most movies on screen

Need more Uncanny Valleys

[Nyder]

Being a male gamer, I can not get enough of uncanny valleys. The deeper the better, lots of bounce doesn't hurt. I remember my first glimpse of uncanny valleys in Custer's Revenge, but now with realistic graphics, I can finally enjoy uncanny valleys how they were meant to.

Not such an issue for games

[Dan East]

In games where human-like characters are necessary, the uncanny valley can be an even bigger problem

I disagree. The very fact that you have control over a character that you are watching is unnatural, and for me disconnects from "reality". That pretty much goes for anything else interactive as well. We know already know the actor in the game cannot be human because it behaves arbitrarily as commanded by the controls we are operating with our hand. Our brain can't be fooled by pure visuals, because we already have a far deeper realization of the truth (that it is not a real human) because it is interactive.

When it comes to movies we are total observers, and the uncanny valley kicks in when we recognize that something is intended to look perfectly human, but our incredibly acute perception in identifying humans isn't fooled.

We have now become so used to seeing CGI humans that it's more of a boolean flag when they are recognized as such - I simply have an awareness that what I'm seeing isn't an actual human. When that happens it is a distraction and reduces how immersed I am in the movie. I don't think of it as "spooky" or that I want to kill the fake human or something, but it is simply a realization - I get a glimpse of the man hiding behind the curtain pulling the strings. A perfect example: The big Matrix Reloaded fight scene. Some little switch in my brain kept going: Real. CGI. Real. CGI. Real. CGI. Kind of makes it hard to enjoy a movie.

Re: Citation Needed

[AudioEfex]

You are looking a little too deeply into this in hopes of finding something offensive.

"Interactive Movie" was actually traditionally used by the games industry back in the early days of digital video when they would incorporate it into a "game" but there wasn't enough game to actually call it a game, like Night Trap.

In modern context, it simply means a game that is so realistic that it would be indistinguishable from a motion picture visually, if one could choose character actions during a motion picture. It's an aspirational goal of the game industry, not the film industry trying to hone in on the games industry.

That said, the real issue with realism in games is that game developers keep pushing the envelope in the wrong direction. Even on the next gen systems (well, since they are out I suppose they are now current gen), they keep focusing on textures and increasing numbers of polygons on the screen instead of making what is there more realistic. I am always stunned when I see a brand new game and they STILL cannot get lip sync right. It doesn't matter how detailed the hairs on a characters head are if their lips don't move in sync with their voice.

not about people being "revolted" because they sense something "wrong" on an unconscious level, it's that they spend so much time trying to increase resolutions and textures that they don't focus on what makes characters alive - how they move and how they react. It's not about making single frames look more realistic, it's how they work in motion which really hasn't improved in step with the "how many hairs or pores can we texture on to this character".

I wouldn't forgive me

[Miletos]

From the article:
Ira isn’t an actual human being—he’s just a computer model—but you’d be forgiven for not being able to tell the difference.

Well...I wouldn't forgive me. You can tell by:
- The crazy amount of unnatural (colored) lighting used to hide low detail and/or too-uniform shading. Show me the same head model in a field on a cloudy day at 2pm in March
- The limited polygon count; look at the edges of his ear (which is a bit weird looking in itself btw)

Much more impressed by these, but they are pre-rendered:
http://www.cgtrader.com/blog/w...
http://www.cgtrader.com/blog/w...
http://www.3dtotal.com/index_g...

Instead of unnatural lighting they have a lot of added skin detail (wrinkles, dirt) to hide too-uniform shading. There's a lot of detail / noise / subtle imperfections in real life you don't normally think about, but when it's not there you instantly notice it on a subconscious level.

No need for a bridge.

[westlake]

In games where human-like characters are necessary, the uncanny valley can be an even bigger problem than in animated movies

In 3.38 seconds watch Disney bring a character to life. Disney's Frozen "Let It Go" Sequence Performed by Idina Menzel

This is how it's done and you don't need photo realism to do it.

Re:No need for a bridge.

[QilessQi]

This. In Scott McCloud's "Understanding Comics" and "Making Comics" books, he talks about how human beings tend to identify more with a character when that character is less realistic in appearance. I can't remember the specifics, but IIRC this may be because the figure we see is overtly representational, so our unconscious is freed up from dealing with fine details and we can project ourselves into the character more easily.

The ultimate examples of this are stick figures and emoticons. We're fine-tuned to read faces, so all we really need to convey a human emotion is this: :-)

Many cartoonists also rely on techniques like caricature and exaggeration of important or distinguishing details. For example, the oversized eyes/mouths/heads in manga don't look wrong, because for human beings these are the most important features to attend to (take a look at many comics, and you'll see that heads and eyes especially are abnormally large when compared to the bodies).

If catering to Western tastes, attractive women are drawn lithe and curvy whereas men are large, stocky and angular, because these are simple exaggerations of the body differences between the sexes.

Which brings us back to your example of Disney and "Frozen". Elsa is beautiful (by Western standards), but a real woman with those facial proportions would look frighteningly wrong walking down the street. She's beautiful in part because (at some level) our brains understand that she's a caricature. But within that context, her face and movements are far more natural than the best CGI simulations of any "realistic" woman I've seen so far.

Re: Citation Needed

[Number42]

But it had a lot of gamy-ness. "Interactive Movie" is most often used to refer to games that involve little gameplay.

Re:Any actual research?

[akozakie]

Wow... Just wow.

It doesn't exist because the Wikipedia page lists little research and that research is not statistically significant.

Is it just me, or is using scientific big words like "statistical significance" in an argument based solely on the contents of a Wikipedia pega is so wrong it's just funny?

Not implying that this is not true - I have no idea how much research was done on the subject. Google scholar seems to know about thousands of articles about this (about the same number as e.g. "Hawking radiation") - are any of them really good? No idea. Maybe it really is overrated.

But criticizing something as not supported by research using Wikipedia as a source? Ridiculous. Wikipedia does have its uses, but this certainly isn't one of them...