Slashdot Mirror


Nvidia Uses AI To Render Virtual Worlds In Real Time (tomshardware.com)

Nvidia is using artificial intelligence to draw new worlds without using traditional modeling techniques or graphics rendering engines. "This new technology uses an AI deep neural network to analyze existing videos and then apply the visual elements to new 3D environments," reports Tom's Hardware. From the report: Nvidia claims this new technology could provide a revolutionary step forward in creating 3D worlds because the AI models are trained from video to automatically render buildings, trees, vehicles, and objects into new 3D worlds, instead of requiring the normal painstaking process of modeling the scene elements. But the project is still a work in progress. As we can see from [this image], which was generated in real time on a Nvidia Titan V graphics card using its Tensor cores, the rendered scene isn't as crisp as we would expect in real life, and it isn't as clear as we would expect with a normal modeled scene in a 3D environment. However, the result is much more impressive when we see the real-time output in [this YouTube video]. The key here is speed: The AI generates these scenes in real time.

44 comments

  1. Experience needed by Anonymous Coward · · Score: 0

    It seems that this system needs lots of guided experience with the real world to accomplish its goals. How will we teach it to render big butt cheeks?

    1. Re:Experience needed by Anonymous Coward · · Score: 0

      Show it videos of creimer?

    2. Re:Experience needed by Anonymous Coward · · Score: 0
  2. Your first glimpse of matrix glitches by Anonymous Coward · · Score: 0

    "oooh, aaahhh, wake me when it doesn't look like a shitty overlay of VHS video"

    1. Re: Your first glimpse of matrix glitches by tysonedwards · · Score: 1

      They use AI *and* Unreal Engine 4 to render virtual worlds.

      --
      Thirty four characters live here.
    2. Re: Your first glimpse of matrix glitches by Anonymous Coward · · Score: 0

      More specifically, they use AI for filling polygons rendered by UE4.
      It's a pretty clever method but I wonder how it knows when to do a yellow brick wall instead of a red one.
      Does it match the specific building and its surrounding to know what color to use.
      Also, it doesn't seem like they feed it the color of cars so I wonder how those are selected.

      Too bad that this is probably too niche to get dedicated hardware. It would have been nice seeing it at usable framerates.

    3. Re: Your first glimpse of matrix glitches by Somebody+Is+Using+My · · Score: 1

      It seems more like the AI is procedurally generating the world, while the actual rendering is still using the Unreal 4 engine pipeline. It is impressive, but the title /is/ misleading.

      Right now they apparently feed the AI video and it creates models and animations based off that video, which it then recombines into new worlds. Imagine this tech combined with GoogleEarth's library of images: it could create a fully-3D view of every street. It will be sort of like jumping from the old step-based Bards Tale games to Skyrim.

    4. Re: Your first glimpse of matrix glitches by Type44Q · · Score: 1

      More specifically, they use AI for filling polygons rendered by UE4.

      What the fuck? Not even close.

    5. Re: Your first glimpse of matrix glitches by Anonymous Coward · · Score: 0

      "oooh, aaahhh, wake me when it doesn't look like a shitty overlay of VHS video"
      Same mistake lee sodol made before being beaten by AlphGo, thinking the next step up is really hard for the AI.
      Graphics modelers start exploring alternative job options now.

    6. Re:Your first glimpse of matrix glitches by Anonymous Coward · · Score: 0

      It looks like PS2 level graphics recorded on an old VHS.

  3. Thanks for making me feel stupid!! by MarkTina · · Score: 3, Insightful

    Whenever I see someone creating things like this it makes me realise how dumb I am in comparison! :-)

    1. Re:Thanks for making me feel stupid!! by Anonymous Coward · · Score: 0

      It's a very specific knowledge base. Neither one of them can change a tire or patch a roof. Heck, just getting a decent haircut or understanding the need to run a comb through it before a TV spot is beyond their reach.

      Watching either one of them install a water heater would be a much more entertaining video IMO.

    2. Re:Thanks for making me feel stupid!! by Anonymous Coward · · Score: 0

      I realized how dumb you were from your posts alone. I didn't need this video.

    3. Re:Thanks for making me feel stupid!! by Anonymous Coward · · Score: 0

      How do you know that they can't do those things? I'm not saying that they can, but you seem to assume that if they are smart in one area they must be dumb at everything else. Isn't that just as silly to assume that they are smart at everything?

    4. Re:Thanks for making me feel stupid!! by 110010001000 · · Score: 3, Funny

      We all were thinking that too. Man, Mark Tina is so dumb compared to this.

    5. Re:Thanks for making me feel stupid!! by Anonymous Coward · · Score: 0

      Soon we will have all the news rendered in real time, or maybe they are already rendered we just don't know it?

    6. Re:Thanks for making me feel stupid!! by Anonymous Coward · · Score: 0

      Are you kidding lol? LOOK AT HIS HAIRCUT. The kid can hardly speak. The idea of him successfully swinging a hammer or swinging a torque wrench is laughable. $20 says he can't open a jar.

    7. Re:Thanks for making me feel stupid!! by Anonymous Coward · · Score: 0

      It sounds more like you are jealous of their ability and are trying to convince yourself that they can't do something that you can in a pathetic effort to assuage your insecurities.

  4. Like in real Life by wolfheart111 · · Score: 1

    You folks dont exist until I go on slashdot... whos cat was that :)

    --
    [($)]
  5. Roko's Basilisk? by Anonymous Coward · · Score: 0

    But is the AI punishing those in the virtual world who aren't helping bring about an AI that can simulate virtual worlds?

  6. Re:Machine Learning != Artificial Intelligence by Anonymous Coward · · Score: 0

    And where are the blockchain and cloud aspects from this announcement? A PR release without full line in buzzword bingo is not really worth reading.

  7. Comment removed by account_deleted · · Score: 0

    Comment removed based on user account deletion

  8. The AI generated video looks like The Matrix by Anonymous Coward · · Score: 0

    One step closer to The Matrix. Once Nvidia figures out how to interface their renders with the visual cortex, we can start putting humans into their pods. The Age of Bio-Electricity is nearly upon us!

  9. Youtube video ending is scary. by Anonymous Coward · · Score: 1

    Its like the deepfakes. Make a movie of anyone doing anything.

  10. rendering? by Anonymous Coward · · Score: 0

    I don't see any rendering here. The static picture just shows an ugly colored copy of the original, nothing new. What have they created? real-time Photoshop filters?
    Also, the Gangnam Style video is explained as "we used AI to synchronize the movements".
    Impressive... not.

  11. My AI does it too by nospam007 · · Score: 1

    Real time.
    For a 100 year old oak, it only needs a hundred years, real time.

  12. Fuck Nvidia by Anonymous Coward · · Score: 0

    Bullshit company. Bullshit products. Bullshit drivers.

  13. Needs more blockchain by Anonymous Coward · · Score: 0

    upload my HODLgasms in realtime.

  14. The Simulation Hypothesis... by Anonymous Coward · · Score: 0

    ... is alive and well. Nick Bostrom wins the day.

  15. Nvidis uses algorithms . . . by Anonymous Coward · · Score: 0

    . . . to do what algorithms have always been used for. Fixed this one, too. This is not news. It might as well say, 'Nvidia Uses Computers For Computing'. Millennials are just sad, man.

  16. So that... by MaryannG · · Score: 1

    ...was pretty damned impressive. Training sims will be a lot more effective anyway.

    --
    Social Media Handywoman at Texas Boys Balloo
  17. Announced a while ago by Anonymous Coward · · Score: 0

    Particular example, I haven't looked at. But the tech was announced with the 2080 line. It let's them realistically predict pixrls they don't have the time to compute, even when dealing with global illumination/multiple bounce's of photons

  18. just texture map generation? by Anonymous Coward · · Score: 0

    It sounds like they're just using the ai to generate texture maps. In the video they segment the real time video into discrete objects which they color differently. Then they feed that into the ai to get the final results. If I'm understanding that right then they're basically just using AI to generate a texture map which in of itself isn't trivial but still... not like the machines are on the verge of taking over.

  19. As Useful as FXAA by BrendaEM · · Score: 1

    If successful it would do little more rob people of a creative outlet and a living, for a lot of that stuff you saw in computer games was art, as well as bread-and-butter. What problem does it solve?

    --
    https://www.youtube.com/c/BrendaEM
    1. Re:As Useful as FXAA by Anonymous Coward · · Score: 0

      What problem does it solve?

      It cuts costs of game development. Instead of paying a team to model a full city, you simply set up the "AI" to generate a city, which will no doubt appeal to executives at game development studios.

      Beyond just cost cutting, it would also allow smaller indie developers to create games with complex environments that they might not have the budget to produce themselves.

      Ultimately, it's another labour saving innovation. You might say it robs people of a living, but if we took that attitude to every labour saving innovation we'd still be farming fields by hand.

  20. AI by nasch · · Score: 1

    Waiting for someone to comment "that's not AI", as always happens with any story about AI.

    1. Re:AI by q4Fry · · Score: 1

      Waiting for someone to comment "that's not AI", as always happens with any story about AI.

      Found the AI!