Re:State of the art?
on
Shrek 2 How-To
·
· Score: 4, Insightful
To be fair I find these sort of comments amusing. It's almost like saying Tex Avery cartoons are dull because they look dated and crude compared to todays more complex cartoons and anime. After all 3D animated movies are also stylized visions, just like their 2D counterparts of yesteryear.
Am I alone in wanting a completely computer-generated movie that looks real instead of cartoony and actually has a good plot?
Well if that were the case why not film it as live action instead (and with VFX where need be). The only reason to do it that way is to prove it can actually be done.
ILM uses mostly commercial renderers: mostly Photoreaistic RenderMan (PRMan) and now quite a bit of Mental Ray. But being on the leading edge they sometimes needed to render stuff that couldn't be done in any commercial app, like hair, they had their own renderer when first used in Jumanji, or their own particle renderer (pRender) which was used on Twister.
They mostly use commercial renderers but for certain specific things they might still mix and match. You want to use the best tool for the job at hand.
They re-wrote an entire renderer? Granted, Shrek is still behind some of Pixar's work but i've got to ask... Why not use some of the other renderer tools out there?
Because PDI is a mostly propietary place. They wrote their own renderer years before there was anything commercially available. As such they have an R&D team continually updating their infrastructure. Interestingly enough I saw a couple of PDI guys at the SIGGRAPH photon mapping course by Henrik Wann Jensen a few years ago in San Antonio.
The upside is you don't have to wait for a commercial vendor to get those new features. They control their own destiny rendering wise. Witness for example how long it took Pixar to make Depp Shadow maps available in PRMan (something like 2 years) even though they had published a SIGGRAPH paper and were using it internally (for Monsters Inc.). Some clients were a bit upset about that.
Dan Wexler used to write their renderer (he is now at Nvidia with Larry Gritz and those crazy Entropy guys). He has some interesting statistics on the first film:
Though it's better at this point to check out the Universities web pages now. Back then there wasn't much info and since it was fairly nwe I had no idea what it was about. At least the SIGGRAPH websites can serve as a starting point for new references.
Doesn't seem to be much for this year, although George Borshukov of ESC will talk a little about their work on the Matrix sequels on a SIGGRAPH course. Though I think their implementation was more image based tricks as opposed to the SSS implementations from most other places but it might be of interest to you.
Yes I remember that paper, although I didn't hear it at SIGGRAPH. Though there was some discussion of it a SIGGRAPH course in 2001 about VFX R&D which I did attend (though it was lighter on details). Even some of the tricks posted in subsequent RenderMan courses (there was one by Matt Pharr I remember) for simulating skin without SSS are pretty neat and do look very good.
Well I wouldn't worry much about recognition, you all will get plenty at VES and SIGGRAPH. It's luck of the draw. Like the fluid dynamics that you guys got. The release dates do screw up things. Imageworks potentially could have showcased their fluid dynamics for Cast Away early in that year but they had to shut down production for 6 months to let Tom Hanks thin for the island shots. Consequently Perfect Storm was seen before it.
Actually I think I met you dude. Weren't you also playing soccer with us with a voleyball at the Reception at that Disney sports place in Orlando during SIGGRAPH 98. Boy were my feet burning up the next day.
Even more intriguing is that Mark Levoy, the fourth coauthor, did not receive a Scitech. I wonder if the Academy has a rule that only a maximum of 3 people can be put up for the award (though I'm sure I've seen more than 3 recipients in the past).
Maybe it's because Jensen was more directly involved in the "dirty work" (and also being the poster boy for photon mapping and SSS) since Hanrahan being the head of Graphics at Stanford has to direct all those students and research projects. He isi usally listed in several SIGGRAPH papers at a time (I think 2 for this year).
It was a published paper so yes the math is put in the open. Specific implmentations might be closed. mental ray 3.3 just added a few more SSS capabilities.
But you could study some implementations in the open. Christophe Hery's Stupid RAT trick slides are aailable, and his course notes from last SIGGRAPH are accesible as well:
No he didn't make Gollum. Jensen was the main researcher of this new subsurface scattering technique when he was at Stanford (he is now in San Diego). As was mentioned earlier this was published in a SIGGRAPH 2001 paper, so at least the research might date as far back as early 2001, late 2000. Jensen along with Steve Marschner and Pat Hanrahan got a SciTech Academy Award earlier this year for it (though Marc Levoy was omitted).
The original implementation used raytrcing to achieve the effect, to slow for actual production work. Some people from ILM spotted the paper and decided to implement in a way more friendly to production. Originally it was going to be used for Ep. 2, but the research wasn't completed on time. The first time it was applied was for Dobby in Harry Potter and the Chamber of Secrets. Christophe Hery even presented a Stupid RenderMan Trick using shadow buffers that could be applied to SSS. Around that time Ken McGaugh and Joe Letteri left ILM (tough they were involved with this research at ILM) and joined Weta Digital to work on the Two Towers. Consecuently, Hery, McGaugh and Letteri also received a SciTech Academy award this year for finding a way to implement Jensen's SSS in a production environment.
Actually most of the work for "The Lion, the Witch and the Wardrobe" was awarded to Rhythm and Hues. They have donde several things since then like building miniatures for Master and Commander and Peter Pan.
More recently Weta Digital seems to have done some work for Van Helsing (where ILM is the main provider) and some for I, Robot (where Digital Domain is the main provider).
Well I don't think it would be too difficult to guess. When they started on the first film of LOTR they used a lot of SGI machines, so conceivably they could have been using SGI fileservers which would mean XFS. There aren't that many options to guess from. ILM uses SUN fileservers for NFS.
It wasn't really an independent animatronic, but a costumed performer. The practical gorilla was made by Rick Baker, though the face was animatronic (radio controlled). Lots of those shots were done either with perspective tricks or just composited in.
Both Dream Quest Images and ILM did CG version of Joe with digital hair. ILM did several shots when Joe is running evading capture, when he crosses the freeway at night with the heli on top and the final shot of Joe running, among others. DQI's Cg Joes was featured in the ferris wheel sequence at the end, when it's in Hollywood (the fun mirror and sitting on the car) and several others.
There are even previous examples than that. One of the first movie CG hair examples is from Jumanji for which they won a Sci-Tech Academy Award later. Other early examples were some commercials by PDI (I believe at least a few months earlier than Jumanji), and Island of Dr. Moreau by Digital Domain.
ILM had about 2500 processors on the renderfarm as recent as last year, could ne more now with the work for Ep. 3. Pixar also recently upgraded. At Rhythm and Hues (and many other places) they incorporte the workstations at night for the render jobs so it also depends how you count "render processors".
The work was more or less evenly split between Asylum and ILM. Most of the work was actually compositing, there wasn't that much CG used. ILM did generate digital oceans I I think built a few models as well, but Asylum stuck to compositing and rotoing.
Well it wasn't that obscure. An article on CGW from last year about matte paintings specifically mentioned that. The Rebel Unit at ILM is mostly PC based and they do use Max, Brazil and AE if I remember right. More or less the same with Imageworks (the work on Bad Boys 2 was discussed).
Couldn't go to NAB but hopefully I'll see it at SIGGRAPH.
Well who knows. The press release does mention ILM, certainly not a small fish.
big developers write their own usually... think Pixar
Yes though that's a special case since they sell it as well as is fairly open (API wise). What you probably mean is those with propietary solutions, in particular Blue Sky Studios, PDI and Rhythm and Hues. But even the big houses who use PRMan and Mental Ray, write customized renderers for specific problems, even if they use a commercial solution like PRMan and MR, like Digital Domain's VoxelBitch, Weta Digital's had a special renderer for Massive (though Massive spits out RIBs), ILM had (has?) propietary hair and particle renderers, etc.
The first common digital nonlinear editor was the casablanca, and with an 8 gig scsi drive ran close to a grand when it was released.
Though that was nothing compared to EditDroid that was the first film nonlinear editor, in which film was scanned and transferred to laser discs. And it certainly cost more than a grand;-).
The press release does mention Frantic Films as well which is a smaller house. I guess they could use it directly for final renders. No idea how it would fit on a pipeline like ILM's though.
Although in a certain way they are the exceptions that prove the rule, or better yet they signal the shift to hybrid solutions (use the best tool for the job). If shots or elements can get away with using regular PRMan features (scanline REYES) with no raytracing I'm sure it'll get used.
But indeed more and more stuff is hybrid where for example PRMan and Mental Ray are combined (either in the same movie, same shot or same element). Some examples of this that spring to mind are the exosekletons in T3, Hulk, Dobby in Harry Potter 2, etc. The biggest counterexample is the Matrix sequels, since ESC uses Mental Ray exclusively. Not to mention the boutique shops that use stuff like Brazil.
This year we'll get even more exmaples, one that springs to mind is I Robot. In any case it's all a matter of context but I guess we should refer to it as hybrid approaches now.
It depends on the context. Most major and high end VFX houses heavily rely on PRMan, like ILM, Weta Digital, Digital Domain, Imageworks, Tippett Studio and more. Mental Ray is also being used quite a bit, ESC in particular is an MR only place. On the other hand the small and boutique houses tend to use stuff like Brazil, Lightwave, etc. Places like Cafe FX, Blur Studios, etc. For example the movie The Core was done mostly by small houses and had quite a bit of 3DMax use coupled with something (Brazil I think). But most major VFX movies have PRMan rendered in them simply because it's what the major use and they work on the big projects. A handful of places have propietary.
Just to witness some of the biggest VFX films this year will probably rely quite a bit on PRMan, like Van Helsing, Day After Tomorrow, Harry Potter 3, Troy, Spider-Man 2 and I Robot. Actually Pixar has a partial lists of where PRMan has ben used:
As far as price, well PRMan is a product for professional of a very niche market. As far as limitations, well almost every other renderer might be a bit more limited IMHO. PRMan is geared towards technical users.
1) Yes they did have a Linux infrastructure but that was after purchasing Shake. Originally Weta used a lot of SGI gear (although at least some of it also ran Linux not Irix) when worked started on Fellowship of the Ring. But by Two Towers they switched quite a bit to Linux, which was around the time Apple came out with Shake for Mac after purchasing Nothing real.
As far as cost in many situations in high end VFX the cost of the software is not that important considering the cost of the artists and technicians. And even at the end they didn't have to worry too much about infrastucture costs as New Line Cinema ponied up the money to upgrade the renderfarm for return of the King if I remember right.
Rendering is just a generic term. It just means transforming some scene description into an image. There are countless rendering methods: raytracing, radiosity, scanline, REYES, Metropolis, hybrids and stuff like photon mapping, etc.
RenderMan, strictly speaking is just a standard (for scene description). Pixar's implementation, Photorealistic RenderMan or PRMan for the most part has used the REYES architecture, hich doesn't use raytracing.
Yes, although the use of Macs has been somehwat dephased. The Rebel Unit was called the Mac rebel Unit, but now a lot of the work is done on PCs with 3D max, Brazil, etc. The matte painting department, closely tied, also used some Macs though I don't know the current status:
Most of the Mac syuff in Episode 1 was actually the Naboo ship and several other shots. The Pod Race was mainly done in Maya plus a whole bunch of propietary software (Viewpaint, the soft body dynamics engine, their terrain generation code, etc.)
In most/all of those cases it was Shake under SGI or Linux. Apple only recently acquired Nothing Real, and even though it's hardware was not used in those productions (there wasn't even an OS X version then) it didn't prevent Apple from capitalizing on the Lord of the Rings hoopla.
Actually Pixar is the exception to the rule. Most big companies (ILM, Weta, DD, Imageworks, R+H, Tippett, etc.) are a combination of Linux, SGI and PCs. Small studios are mainly PC/Win based. A few other exceptions, I think Tweak Films is also OS X, and ESC mainly using Win2000 in the Matrix sequels. Other companies have been pushing multimedia Linux, from ILM's OpenEXR, Rhythm and Hues contributions to CinePaint, DD's Nuke, even Pixar with PRMan and the tools. Most big CG software vendors haqve Linux versions (Maya, Softimage, Houdini, mental ray, PRMan, etc.).
Most big VFX movies have credits at the very end to key software used like Alias' Maya, Softimage, PRMan, Shake and the like. In the good old days even SGI was also listed on some. It's not that uncommon.
Slashdot Mirror
To be fair I find these sort of comments amusing. It's almost like saying Tex Avery cartoons are dull because they look dated and crude compared to todays more complex cartoons and anime. After all 3D animated movies are also stylized visions, just like their 2D counterparts of yesteryear.
Am I alone in wanting a completely computer-generated movie that looks real instead of cartoony and actually has a good plot?
Well if that were the case why not film it as live action instead (and with VFX where need be). The only reason to do it that way is to prove it can actually be done.
ILM uses mostly commercial renderers: mostly Photoreaistic RenderMan (PRMan) and now quite a bit of Mental Ray. But being on the leading edge they sometimes needed to render stuff that couldn't be done in any commercial app, like hair, they had their own renderer when first used in Jumanji, or their own particle renderer (pRender) which was used on Twister.
They mostly use commercial renderers but for certain specific things they might still mix and match. You want to use the best tool for the job at hand.
They re-wrote an entire renderer? Granted, Shrek is still behind some of Pixar's work but i've got to ask... Why not use some of the other renderer tools out there?
Because PDI is a mostly propietary place. They wrote their own renderer years before there was anything commercially available. As such they have an R&D team continually updating their infrastructure. Interestingly enough I saw a couple of PDI guys at the SIGGRAPH photon mapping course by Henrik Wann Jensen a few years ago in San Antonio.
The upside is you don't have to wait for a commercial vendor to get those new features. They control their own destiny rendering wise. Witness for example how long it took Pixar to make Depp Shadow maps available in PRMan (something like 2 years) even though they had published a SIGGRAPH paper and were using it internally (for Monsters Inc.). Some clients were a bit upset about that.
Dan Wexler used to write their renderer (he is now at Nvidia with Larry Gritz and those crazy Entropy guys). He has some interesting statistics on the first film:
Renderfarm Statistics
Shrek Rendering Statistics
Though it's better at this point to check out the Universities web pages now. Back then there wasn't much info and since it was fairly nwe I had no idea what it was about. At least the SIGGRAPH websites can serve as a starting point for new references.
Doesn't seem to be much for this year, although George Borshukov of ESC will talk a little about their work on the Matrix sequels on a SIGGRAPH course. Though I think their implementation was more image based tricks as opposed to the SSS implementations from most other places but it might be of interest to you.
Yes I remember that paper, although I didn't hear it at SIGGRAPH. Though there was some discussion of it a SIGGRAPH course in 2001 about VFX R&D which I did attend (though it was lighter on details). Even some of the tricks posted in subsequent RenderMan courses (there was one by Matt Pharr I remember) for simulating skin without SSS are pretty neat and do look very good.
Well I wouldn't worry much about recognition, you all will get plenty at VES and SIGGRAPH. It's luck of the draw. Like the fluid dynamics that you guys got. The release dates do screw up things. Imageworks potentially could have showcased their fluid dynamics for Cast Away early in that year but they had to shut down production for 6 months to let Tom Hanks thin for the island shots. Consequently Perfect Storm was seen before it.
Actually I think I met you dude. Weren't you also playing soccer with us with a voleyball at the Reception at that Disney sports place in Orlando during SIGGRAPH 98. Boy were my feet burning up the next day.
Even more intriguing is that Mark Levoy, the fourth coauthor, did not receive a Scitech. I wonder if the Academy has a rule that only a maximum of 3 people can be put up for the award (though I'm sure I've seen more than 3 recipients in the past).
Maybe it's because Jensen was more directly involved in the "dirty work" (and also being the poster boy for photon mapping and SSS) since Hanrahan being the head of Graphics at Stanford has to direct all those students and research projects. He isi usally listed in several SIGGRAPH papers at a time (I think 2 for this year).
It's weird nonetheless.
It was a published paper so yes the math is put in the open. Specific implmentations might be closed. mental ray 3.3 just added a few more SSS capabilities.
But you could study some implementations in the open. Christophe Hery's Stupid RAT trick slides are aailable, and his course notes from last SIGGRAPH are accesible as well:
Stupid Rat Trick (Shadow Buffers)
RenderMan SIGGRAPH course notes (check the ones from 2003)
ZJ's Slim template at Deathfall
Potentially the open implementations like Aqsis, Pixie and Toxic are getting capablle enough that you might have free implementations soon enough.
Me thinks someone is impersonating me ;-).
Saw Van Helsing last night as well. The switchovers between the digital doubkles and creatures to the real performers was pretty stunning.
No he didn't make Gollum. Jensen was the main researcher of this new subsurface scattering technique when he was at Stanford (he is now in San Diego). As was mentioned earlier this was published in a SIGGRAPH 2001 paper, so at least the research might date as far back as early 2001, late 2000. Jensen along with Steve Marschner and Pat Hanrahan got a SciTech Academy Award earlier this year for it (though Marc Levoy was omitted).
The original implementation used raytrcing to achieve the effect, to slow for actual production work. Some people from ILM spotted the paper and decided to implement in a way more friendly to production. Originally it was going to be used for Ep. 2, but the research wasn't completed on time. The first time it was applied was for Dobby in Harry Potter and the Chamber of Secrets. Christophe Hery even presented a Stupid RenderMan Trick using shadow buffers that could be applied to SSS. Around that time Ken McGaugh and Joe Letteri left ILM (tough they were involved with this research at ILM) and joined Weta Digital to work on the Two Towers. Consecuently, Hery, McGaugh and Letteri also received a SciTech Academy award this year for finding a way to implement Jensen's SSS in a production environment.
Actually most of the work for "The Lion, the Witch and the Wardrobe" was awarded to Rhythm and Hues. They have donde several things since then like building miniatures for Master and Commander and Peter Pan.
More recently Weta Digital seems to have done some work for Van Helsing (where ILM is the main provider) and some for I, Robot (where Digital Domain is the main provider).
Well I don't think it would be too difficult to guess. When they started on the first film of LOTR they used a lot of SGI machines, so conceivably they could have been using SGI fileservers which would mean XFS. There aren't that many options to guess from. ILM uses SUN fileservers for NFS.
It wasn't really an independent animatronic, but a costumed performer. The practical gorilla was made by Rick Baker, though the face was animatronic (radio controlled). Lots of those shots were done either with perspective tricks or just composited in.
Both Dream Quest Images and ILM did CG version of Joe with digital hair. ILM did several shots when Joe is running evading capture, when he crosses the freeway at night with the heli on top and the final shot of Joe running, among others. DQI's Cg Joes was featured in the ferris wheel sequence at the end, when it's in Hollywood (the fun mirror and sitting on the car) and several others.
There are even previous examples than that. One of the first movie CG hair examples is from Jumanji for which they won a Sci-Tech Academy Award later. Other early examples were some commercials by PDI (I believe at least a few months earlier than Jumanji), and Island of Dr. Moreau by Digital Domain.
ILM had about 2500 processors on the renderfarm as recent as last year, could ne more now with the work for Ep. 3. Pixar also recently upgraded. At Rhythm and Hues (and many other places) they incorporte the workstations at night for the render jobs so it also depends how you count "render processors".
The work was more or less evenly split between Asylum and ILM. Most of the work was actually compositing, there wasn't that much CG used. ILM did generate digital oceans I I think built a few models as well, but Asylum stuck to compositing and rotoing.
Well it wasn't that obscure. An article on CGW from last year about matte paintings specifically mentioned that. The Rebel Unit at ILM is mostly PC based and they do use Max, Brazil and AE if I remember right. More or less the same with Imageworks (the work on Bad Boys 2 was discussed).
Couldn't go to NAB but hopefully I'll see it at SIGGRAPH.
This is more likely for smaller developers
Well who knows. The press release does mention ILM, certainly not a small fish.
big developers write their own usually... think Pixar
Yes though that's a special case since they sell it as well as is fairly open (API wise). What you probably mean is those with propietary solutions, in particular Blue Sky Studios, PDI and Rhythm and Hues. But even the big houses who use PRMan and Mental Ray, write customized renderers for specific problems, even if they use a commercial solution like PRMan and MR, like Digital Domain's VoxelBitch, Weta Digital's had a special renderer for Massive (though Massive spits out RIBs), ILM had (has?) propietary hair and particle renderers, etc.
The first common digital nonlinear editor was the casablanca, and with an 8 gig scsi drive ran close to a grand when it was released.
Though that was nothing compared to EditDroid that was the first film nonlinear editor, in which film was scanned and transferred to laser discs. And it certainly cost more than a grand ;-).
The press release does mention Frantic Films as well which is a smaller house. I guess they could use it directly for final renders. No idea how it would fit on a pipeline like ILM's though.
Although in a certain way they are the exceptions that prove the rule, or better yet they signal the shift to hybrid solutions (use the best tool for the job). If shots or elements can get away with using regular PRMan features (scanline REYES) with no raytracing I'm sure it'll get used.
But indeed more and more stuff is hybrid where for example PRMan and Mental Ray are combined (either in the same movie, same shot or same element). Some examples of this that spring to mind are the exosekletons in T3, Hulk, Dobby in Harry Potter 2, etc. The biggest counterexample is the Matrix sequels, since ESC uses Mental Ray exclusively. Not to mention the boutique shops that use stuff like Brazil.
This year we'll get even more exmaples, one that springs to mind is I Robot. In any case it's all a matter of context but I guess we should refer to it as hybrid approaches now.
Almost every FX house? I don't think so.
It depends on the context. Most major and high end VFX houses heavily rely on PRMan, like ILM, Weta Digital, Digital Domain, Imageworks, Tippett Studio and more. Mental Ray is also being used quite a bit, ESC in particular is an MR only place. On the other hand the small and boutique houses tend to use stuff like Brazil, Lightwave, etc. Places like Cafe FX, Blur Studios, etc. For example the movie The Core was done mostly by small houses and had quite a bit of 3DMax use coupled with something (Brazil I think). But most major VFX movies have PRMan rendered in them simply because it's what the major use and they work on the big projects. A handful of places have propietary.
Just to witness some of the biggest VFX films this year will probably rely quite a bit on PRMan, like Van Helsing, Day After Tomorrow, Harry Potter 3, Troy, Spider-Man 2 and I Robot. Actually Pixar has a partial lists of where PRMan has ben used:
PRMan Movies
As far as price, well PRMan is a product for professional of a very niche market. As far as limitations, well almost every other renderer might be a bit more limited IMHO. PRMan is geared towards technical users.
1) Yes they did have a Linux infrastructure but that was after purchasing Shake. Originally Weta used a lot of SGI gear (although at least some of it also ran Linux not Irix) when worked started on Fellowship of the Ring. But by Two Towers they switched quite a bit to Linux, which was around the time Apple came out with Shake for Mac after purchasing Nothing real.
As far as cost in many situations in high end VFX the cost of the software is not that important considering the cost of the artists and technicians. And even at the end they didn't have to worry too much about infrastucture costs as New Line Cinema ponied up the money to upgrade the renderfarm for return of the King if I remember right.
Rendering is just a generic term. It just means transforming some scene description into an image. There are countless rendering methods: raytracing, radiosity, scanline, REYES, Metropolis, hybrids and stuff like photon mapping, etc.
RenderMan, strictly speaking is just a standard (for scene description). Pixar's implementation, Photorealistic RenderMan or PRMan for the most part has used the REYES architecture, hich doesn't use raytracing.
Yes, although the use of Macs has been somehwat dephased. The Rebel Unit was called the Mac rebel Unit, but now a lot of the work is done on PCs with 3D max, Brazil, etc. The matte painting department, closely tied, also used some Macs though I don't know the current status:
Painting the town
Most of the Mac syuff in Episode 1 was actually the Naboo ship and several other shots. The Pod Race was mainly done in Maya plus a whole bunch of propietary software (Viewpaint, the soft body dynamics engine, their terrain generation code, etc.)
In most/all of those cases it was Shake under SGI or Linux. Apple only recently acquired Nothing Real, and even though it's hardware was not used in those productions (there wasn't even an OS X version then) it didn't prevent Apple from capitalizing on the Lord of the Rings hoopla.
Actually Pixar is the exception to the rule. Most big companies (ILM, Weta, DD, Imageworks, R+H, Tippett, etc.) are a combination of Linux, SGI and PCs. Small studios are mainly PC/Win based. A few other exceptions, I think Tweak Films is also OS X, and ESC mainly using Win2000 in the Matrix sequels. Other companies have been pushing multimedia Linux, from ILM's OpenEXR, Rhythm and Hues contributions to CinePaint, DD's Nuke, even Pixar with PRMan and the tools. Most big CG software vendors haqve Linux versions (Maya, Softimage, Houdini, mental ray, PRMan, etc.).
Most big VFX movies have credits at the very end to key software used like Alias' Maya, Softimage, PRMan, Shake and the like. In the good old days even SGI was also listed on some. It's not that uncommon.