I fail to see what is impressive about this.
It looks like the wheel reinvented several times.
For cluster installs on several machines, use system imager .
For using and controlling a cluster of machines for various taskes, use LSF .
The number of machines is pathetic too... 225 @ 733 mhz? That makes it to #325?
How sad. I need to bench mark our render farm (200+ boxes, 120 are dual 1ghz) and see what we can come up with. I know it is higher than that... and we have a smaller install for the industry.
I looked for info to spec our machines but I couldn't find any info.... any help?
Agreed.
This would make a killer render system, assuming the renderer can handle that many threads.
This is why beowulf rendering is bad. Network performance for shared memory sucks.
With renders hitting the 2GB + mark for memory useage, do you really want a network passing that data arround.
What could happen with systems like this is that the render time vs. load time would get extremely lopsided. 30 minute loads and under a minute a frame. It would force a rethink of how the render jobs get distributed and ran.Best case would be a few of these, for each different departments render needs. But then we are talking 20+ million for rendering. That buys a lot of intel boxes.
If I was given one, I would try to use it. But I don't think I could ever seriously suggest buying one. But that is me and my particular application.
As much fun as rendering seems over a large network, if you are really doing any major rendering and you don't have Cray-linked SGI's.. you are better off treating it as a NOW (network of workstations).
The POVRay rendering dropped off at 8 procs or so, if i can remember correctly.
We render a hand full of frames per box and do this on several boxes. This keeps the network overhead between machines low.
Sometimes when people talk about beowulf, what they really mean or really want is just a bunch of managed computers running tasks. If the task can be broken down at submission time, like most renders, then take advantage of a NOW.
Thos are some of the studios that i am refering too. They are the ones that have spent the time converting the code.
People are now making a conscious effort to remove dependance on the SGI Irix desktops.
I can see legacy apps for old shows not being converted... and keeping a few machines in storage, just in case, but that is about it.
Agreed... but even that is switching. Most of the inhouse tools and software packages are being switchted to Linux. The SGI Irix desktop will soon dissapear for most of the major studios.
I think SGI likes it.
SGI sells linux boxes that can work as a renderfarm just as much as any other rackmount linux solution.
But this is where they should really like it. Hollywood has trusted SGI for years. SGI has major name recognition based on hardware quality and support.
Linux has been in Hollywood for a while now, chances are that the 3D that you see in current titles has had some Linux involvement along the way.
I know we are heading that direction.
All the studios I have talked with are heading that way, if they haven't all ready.
In my opinion, this is a place where VA could have made a name for themselves. Now, I think that the big Linux battle will be between HP, SGI, and the next person to have a killer 3D desktop. If I had to place money on it, I would be pulling for HP.
This just in...
Cell phone and cigarett manufacturers are claiming ownership of any cancer cells that may potientialy be caused by use of their products. A plan for an EULA for use of the products now voids the individuals right to any cancerous cells in the body. It is unclear if the EULA is covering all cells created, both past and presant, and if they are specific to the product being used.
When questioned, spokesmen from the companies admit that this is being used as a way to ofset research costs into creating stronger, longer living cancer cells. Although they appear to want ownership of cancerous cells, they claim no responsibility for the creation of those cells.
Other industries are expected to jump on this compensation bandwagon soon.
People always mention this.
It would take a lot of development time and coopereation from the software companies to support low bandwith render systems.
The scene files that we are working with are 10s of MB, and they reference other files that are of similar sizes. You may pull accross 100MB of proprietary scene files (which means encrypted to the users) and then the system determines what to render. It may take 30+ minutes a frame, while either creating a scad of misc files or eating up memory (such as shadow map files, motion blur files...) and then assemble all of them to make a 2-3 mb image to upload.
The average user's home machine would only be a waste to studios. The bandwidth would kill us. Legal would kill us for letting proprietary data out. Your system would be smoked while rendering... or it would take a long long time.
All the transfer time of the scene files and the textures would take longer than the render.
We keep a nice fat backbone to the renderfarm for a reason. No sense in having 200+ procs waiting on data.
We do use software that allows us to use the users desktop, but this is over a LAN and not a WAN... and that makes a big difference.
Why would somebody do that. The implication is that the card is doing render quality at real time. If this is the case, no farm is needed. Just run the stuff through and hit record. Or transfer in what ever method you want.
Some software does use the video hardware to render. It is grabing the frame off the buffer and saving to disk. Slower than real time, faster than a full blown software render.
Not if you were using depthmap shadows and decent textures with high enough anti-aliasing.
For the image quality of a render, even from Maya, you will have to wait a while... or you are doing some really simple stuff.
The lighting that the GF3 does does not compare to what Maya renders. There are a couple levels of complexity in difference.
I agree.
I was typing faster than thinking... not rare for me.
Everything you state is correct.
What I was intending to say is that what the graphics card is doing is different than what renderfarm rendering is doing. I don't consider it really rendering unless its from some render package such as Maya, PRrenderman, Mental Ray or the like.
I have had to do some raytraced rendering before. I know the difference. My fingers must not of.
Thanks for pointing this out so others are not as confused as my fingers.
You are correct... In my haste of work and other things... I didn't really go through what I was meaning to say.
Most of what people think of for CG work is raytraced... this isn't always the case, nor is it what I was really going to talk about.
Real time raytrace isn't going to happen for a long time...
But... for animatioin, under Maya, it takes longer to go through all the math with the deformers and the skeleton solvers and all that than it does to display. So sometimes, you can only get 1 fps.
Games use a much different approach, where everything is approximated to some extent.
Its true that they have IK solvers and are more realistic than in the past, but the dependency is not like what you would have in some animated scene files. What happens for secondary effects, such as hair moving is all based upon the characters movements. This is not really forseen... and only few animators would want that level of control on a character under most situations.
I also agree with you on the direction for 3D cards and movie CG. It will get better and better. And rendertimes will still be the same, because artists always want to add more realism or quality. As speed goes up, so does complexity, but render times stays about the same. Thats what I call job security.
Most of what we see with "realistic rendering" on desktop boxes is OpenGL / direct3d based. This isn't realy rendering, well... its not raytraced.
Its true that they are getting close and blurring the line between rendering and desktop 3D for all practical purposes there is a difference.
I just hope rendering never goes away... I need this job!
Another difference is that game movement is not near as complex as cinematic animation. Most game movement is pre-definded movements trigered by something. A lot of secondary animation and even some primary animation is done by a complicated set of equations. It all depends on the package, but sometimes with these solvers on, you might get 1 fps when viewing the animation. Until issues like that are fixed, you will not be able to generate stuff like that on the fly.
They did it for toy story 2, if my memory is correct. I thought they were crazy, but they had a monster render farm.
I don't know if we are doing it either.
The nice thing about rendering for NTSC vs. Theatre is it is smaller. So, rendering for video lets you preview what it will look like for the big screen.
But, our last sillysong, which was 16:9, was renered once, except for one shot where the cameraa did a pull/slide move. We rendered the 4:3 version with just the pull.
But rendering is the cost of re-doing it.
If my boss told me we were re-rendering a video instead of dubing it, I would tell him not on my render farm.
A lot of production time and money goes into the rendering side of the production.
The amount of extra money a specific version for a language would produce would not be why anyone would do it.
If somebody was crazy enough to do this, it would be for arts sake.
Why the zx10?
If going with SGI, look at the 330 series.
or if you need more gusto, the 550 series.
They both run linux and are fast.
Its faster than an Octane in tests that I have run.
The HP systems are also pretty sharp. I haven't benched them against the 330 or 550 but I know that they seem to keep up.
A lot of studios are looking at this.
We are looking to replace all our desktops with Linux boxes over the next year as people roll out of our current production. We all ready have some in house to start using now.
We switched fron IRIX ro Linux for final renders during the last year and except in a few places, all of our last three videos were rendered Linux only. This will continue on in to the production of our movie, Jonah.
So, Linux is starting to take over, it just is so new to the animation industry that it took some time.
Re:Maya is out for Linux now too
on
Linux and Shrek
·
· Score: 2
Good luck getting good support for your debian or mandrake installs.
A|W is a great company, but part of them moving on the Linux scene was to use a single distro.
The notes say to install on RedHat 6.2, not 6.1 or 7.0 as it can cause problems.
This is understandable. They can only have so many flavors to test on... one of the reasons that software vendors love a "real" *nix, like IRIX. One flavor... you only have patches to watch out for.
I agree it is a win/win situation for rendering and desktops... we are moving that direction ourselves.
The only thing that is skewed is the price of the software vs. the OS. When the NT boxes came out, the price of animation software dropped. One of the good things that came out of the temporary Microsoft and SoftImage marriage.
Now with the box price dropping even more, the software will mostlikely follow, which helps everyone but the vendors.
Also, not as many render studios render on Maya (or at least not exclusively) I imagine the PR RenderMan port made more of an impact with the majors.
I also would not consider production houses a corporate environment... mostly artists and geeks... not too many shirts... but... i agree, a step in the right direction.
This mirrors (but not to scale) our setup.
We are curently VA only on the Linux side, but SGI Octanes (and Octane2's) on the desktop side.
We have no propriatary modeling/animating/rendering software... pure Maya, which has worked well for us. Our scale has been smaller than Shrek, so we have been able to keep our heads above water.
Our feature, Jonah,(due out next year) will be done on Linux render boxes with Maya with SGI desktops (most likely IRIX... don't want to change mid production) with the strong possibility of rolling to Linux on the desktop for the next project.
We are using LSF (from Platform) to do the load balancing and custom MEL and perl to interact with the database, submit to LSF and keep the renders in order.
Our group of programmers also do production support. I, overseeing the render process, get to play wrangler, architect, sysadmin and programmer. Fun Fun.
We use photoshop and a few other tools, but most of the paintfixes we do is with the GIMP.
Good to see that we aren't far off of the big boys.
Back to my misbehaving render boxes....
Tim Toll
Re:Don't extrapolate from PDI/Dreamworks
on
Linux and Shrek
·
· Score: 2
I disagree...
We are looking at it (and in some stages doing it). We have coders in house doing some custom stuff, but we have made 90% of our code cross platform from the get go. C and MEL (Maya's scripting language). My custom code is in perl, so it is just as happy in IRIX world and Linux world. LSF, the other key to our render solution is also multi-platform, so the migration has been painless.
As long as the software is ported by the vendors, there is no reason to stay on NT.
Most companies being happy with Windows may not be accurate... I think they are more scared of change than pleased to use Windows.
Pulling for HP...;)
We have demoed both boxes in house.
I am impressed with both HP and SGI Linux Graphics workstations. I do know that my home grown box has close enough the performance for me (but being a coder, that is not saying it would be good for an animator).
IMHO, I like the SGI systems a bit better because of the NVidia chips, but that is me being a bit biased. As always, it boils down to price vs. performance.
With us looking at going to Linux desktops and render farms (not near the order of PDI or Dreamworks) it is good to keep all of the major parties in mind.
This person is confused.
The industry is not moving away from SGI.
They aren't even moving away from IRIX.
A lot of places are getting SGI Linux boxes in house. With an Origin 3000 server and using an Onyx for video streaming, you can have a nice setup using all SGI stuff.
And A|W is an SGI company, but they support intel boxes with RH 6.2.
I know I have been waiting for these AMD beauties to come out the door. When MHZ and CPUS are a must and you are running on a tight budget, dual AMDs are a dream.
Now I need guranteed software support for them...
(for some reason some vendors will not list AMD as a supported proc... )
IMHO there wasn't a good character fit for him in the story, so why force him in.
We made sure that he was back in Lyle.
You should check penguins out... we are working on video #2 right now, in the finishing stages... it should be out this summer.
Also, for the geeky side, all our final renders since King George have been almost exclusively linux. And it will be this way for Jonah, our feature length movie that comes out next year.
Slashdot Mirror
I fail to see what is impressive about this.
... 225 @ 733 mhz? That makes it to #325?
It looks like the wheel reinvented several times.
For cluster installs on several machines, use system imager .
For using and controlling a cluster of machines for various taskes, use LSF .
The number of machines is pathetic too
How sad. I need to bench mark our render farm (200+ boxes, 120 are dual 1ghz) and see what we can come up with. I know it is higher than that... and we have a smaller install for the industry.
I looked for info to spec our machines but I couldn't find any info.... any help?
Agreed.
This would make a killer render system, assuming the renderer can handle that many threads.
This is why beowulf rendering is bad. Network performance for shared memory sucks.
With renders hitting the 2GB + mark for memory useage, do you really want a network passing that data arround.
What could happen with systems like this is that the render time vs. load time would get extremely lopsided. 30 minute loads and under a minute a frame. It would force a rethink of how the render jobs get distributed and ran.Best case would be a few of these, for each different departments render needs. But then we are talking 20+ million for rendering. That buys a lot of intel boxes.
If I was given one, I would try to use it. But I don't think I could ever seriously suggest buying one. But that is me and my particular application.
As much fun as rendering seems over a large network, if you are really doing any major rendering and you don't have Cray-linked SGI's .. you are better off treating it as a NOW (network of workstations).
The POVRay rendering dropped off at 8 procs or so, if i can remember correctly.
We render a hand full of frames per box and do this on several boxes. This keeps the network overhead between machines low.
Sometimes when people talk about beowulf, what they really mean or really want is just a bunch of managed computers running tasks. If the task can be broken down at submission time, like most renders, then take advantage of a NOW.
Thos are some of the studios that i am refering too. They are the ones that have spent the time converting the code.
People are now making a conscious effort to remove dependance on the SGI Irix desktops.
I can see legacy apps for old shows not being converted... and keeping a few machines in storage, just in case, but that is about it.
Agreed... but even that is switching. Most of the inhouse tools and software packages are being switchted to Linux. The SGI Irix desktop will soon dissapear for most of the major studios.
I think SGI likes it.
SGI sells linux boxes that can work as a renderfarm just as much as any other rackmount linux solution.
But this is where they should really like it. Hollywood has trusted SGI for years. SGI has major name recognition based on hardware quality and support.
Linux has been in Hollywood for a while now, chances are that the 3D that you see in current titles has had some Linux involvement along the way.
I know we are heading that direction.
All the studios I have talked with are heading that way, if they haven't all ready.
In my opinion, this is a place where VA could have made a name for themselves. Now, I think that the big Linux battle will be between HP, SGI, and the next person to have a killer 3D desktop. If I had to place money on it, I would be pulling for HP.
This just in...
Cell phone and cigarett manufacturers are claiming ownership of any cancer cells that may potientialy be caused by use of their products. A plan for an EULA for use of the products now voids the individuals right to any cancerous cells in the body. It is unclear if the EULA is covering all cells created, both past and presant, and if they are specific to the product being used.
When questioned, spokesmen from the companies admit that this is being used as a way to ofset research costs into creating stronger, longer living cancer cells. Although they appear to want ownership of cancerous cells, they claim no responsibility for the creation of those cells.
Other industries are expected to jump on this compensation bandwagon soon.
People always mention this.
It would take a lot of development time and coopereation from the software companies to support low bandwith render systems.
The scene files that we are working with are 10s of MB, and they reference other files that are of similar sizes. You may pull accross 100MB of proprietary scene files (which means encrypted to the users) and then the system determines what to render. It may take 30+ minutes a frame, while either creating a scad of misc files or eating up memory (such as shadow map files, motion blur files...) and then assemble all of them to make a 2-3 mb image to upload.
The average user's home machine would only be a waste to studios. The bandwidth would kill us. Legal would kill us for letting proprietary data out. Your system would be smoked while rendering... or it would take a long long time.
All the transfer time of the scene files and the textures would take longer than the render.
We keep a nice fat backbone to the renderfarm for a reason. No sense in having 200+ procs waiting on data.
We do use software that allows us to use the users desktop, but this is over a LAN and not a WAN... and that makes a big difference.
Why would somebody do that. The implication is that the card is doing render quality at real time. If this is the case, no farm is needed. Just run the stuff through and hit record. Or transfer in what ever method you want.
Some software does use the video hardware to render. It is grabing the frame off the buffer and saving to disk. Slower than real time, faster than a full blown software render.
Not if you were using depthmap shadows and decent textures with high enough anti-aliasing.
For the image quality of a render, even from Maya, you will have to wait a while... or you are doing some really simple stuff.
The lighting that the GF3 does does not compare to what Maya renders. There are a couple levels of complexity in difference.
I agree.
I was typing faster than thinking... not rare for me.
Everything you state is correct.
What I was intending to say is that what the graphics card is doing is different than what renderfarm rendering is doing. I don't consider it really rendering unless its from some render package such as Maya, PRrenderman, Mental Ray or the like.
I have had to do some raytraced rendering before. I know the difference. My fingers must not of.
Thanks for pointing this out so others are not as confused as my fingers.
You are correct... In my haste of work and other things... I didn't really go through what I was meaning to say.
Most of what people think of for CG work is raytraced... this isn't always the case, nor is it what I was really going to talk about.
Real time raytrace isn't going to happen for a long time...
But... for animatioin, under Maya, it takes longer to go through all the math with the deformers and the skeleton solvers and all that than it does to display. So sometimes, you can only get 1 fps.
Games use a much different approach, where everything is approximated to some extent.
Its true that they have IK solvers and are more realistic than in the past, but the dependency is not like what you would have in some animated scene files. What happens for secondary effects, such as hair moving is all based upon the characters movements. This is not really forseen... and only few animators would want that level of control on a character under most situations.
I also agree with you on the direction for 3D cards and movie CG. It will get better and better. And rendertimes will still be the same, because artists always want to add more realism or quality. As speed goes up, so does complexity, but render times stays about the same. Thats what I call job security.
Most of what we see with "realistic rendering" on desktop boxes is OpenGL / direct3d based. This isn't realy rendering, well... its not raytraced.
Its true that they are getting close and blurring the line between rendering and desktop 3D for all practical purposes there is a difference.
I just hope rendering never goes away... I need this job!
Another difference is that game movement is not near as complex as cinematic animation. Most game movement is pre-definded movements trigered by something. A lot of secondary animation and even some primary animation is done by a complicated set of equations. It all depends on the package, but sometimes with these solvers on, you might get 1 fps when viewing the animation. Until issues like that are fixed, you will not be able to generate stuff like that on the fly.
They did it for toy story 2, if my memory is correct. I thought they were crazy, but they had a monster render farm.
I don't know if we are doing it either.
The nice thing about rendering for NTSC vs. Theatre is it is smaller. So, rendering for video lets you preview what it will look like for the big screen.
But, our last sillysong, which was 16:9, was renered once, except for one shot where the cameraa did a pull/slide move. We rendered the 4:3 version with just the pull.
But rendering is the cost of re-doing it.
If my boss told me we were re-rendering a video instead of dubing it, I would tell him not on my render farm.
A lot of production time and money goes into the rendering side of the production.
The amount of extra money a specific version for a language would produce would not be why anyone would do it.
If somebody was crazy enough to do this, it would be for arts sake.
Why the zx10?
If going with SGI, look at the 330 series.
or if you need more gusto, the 550 series.
They both run linux and are fast.
Its faster than an Octane in tests that I have run.
The HP systems are also pretty sharp. I haven't benched them against the 330 or 550 but I know that they seem to keep up.
A lot of studios are looking at this.
We are looking to replace all our desktops with Linux boxes over the next year as people roll out of our current production. We all ready have some in house to start using now.
We switched fron IRIX ro Linux for final renders during the last year and except in a few places, all of our last three videos were rendered Linux only. This will continue on in to the production of our movie, Jonah.
So, Linux is starting to take over, it just is so new to the animation industry that it took some time.
Good luck getting good support for your debian or mandrake installs.
A|W is a great company, but part of them moving on the Linux scene was to use a single distro.
The notes say to install on RedHat 6.2, not 6.1 or 7.0 as it can cause problems.
This is understandable. They can only have so many flavors to test on... one of the reasons that software vendors love a "real" *nix, like IRIX. One flavor... you only have patches to watch out for.
I agree it is a win/win situation for rendering and desktops... we are moving that direction ourselves.
The only thing that is skewed is the price of the software vs. the OS. When the NT boxes came out, the price of animation software dropped. One of the good things that came out of the temporary Microsoft and SoftImage marriage.
Now with the box price dropping even more, the software will mostlikely follow, which helps everyone but the vendors.
Also, not as many render studios render on Maya (or at least not exclusively) I imagine the PR RenderMan port made more of an impact with the majors.
I also would not consider production houses a corporate environment... mostly artists and geeks... not too many shirts... but... i agree, a step in the right direction.
This mirrors (but not to scale) our setup.
We are curently VA only on the Linux side, but SGI Octanes (and Octane2's) on the desktop side.
We have no propriatary modeling/animating/rendering software... pure Maya, which has worked well for us. Our scale has been smaller than Shrek, so we have been able to keep our heads above water.
Our feature, Jonah,(due out next year) will be done on Linux render boxes with Maya with SGI desktops (most likely IRIX... don't want to change mid production) with the strong possibility of rolling to Linux on the desktop for the next project.
We are using LSF (from Platform) to do the load balancing and custom MEL and perl to interact with the database, submit to LSF and keep the renders in order.
Our group of programmers also do production support. I, overseeing the render process, get to play wrangler, architect, sysadmin and programmer. Fun Fun.
We use photoshop and a few other tools, but most of the paintfixes we do is with the GIMP.
Good to see that we aren't far off of the big boys.
Back to my misbehaving render boxes....
Tim Toll
I disagree...
We are looking at it (and in some stages doing it). We have coders in house doing some custom stuff, but we have made 90% of our code cross platform from the get go. C and MEL (Maya's scripting language). My custom code is in perl, so it is just as happy in IRIX world and Linux world. LSF, the other key to our render solution is also multi-platform, so the migration has been painless.
As long as the software is ported by the vendors, there is no reason to stay on NT.
Most companies being happy with Windows may not be accurate... I think they are more scared of change than pleased to use Windows.
Pulling for HP... ;)
:)
We have demoed both boxes in house.
I am impressed with both HP and SGI Linux Graphics workstations. I do know that my home grown box has close enough the performance for me (but being a coder, that is not saying it would be good for an animator).
IMHO, I like the SGI systems a bit better because of the NVidia chips, but that is me being a bit biased. As always, it boils down to price vs. performance.
With us looking at going to Linux desktops and render farms (not near the order of PDI or Dreamworks) it is good to keep all of the major parties in mind.
Thanks for the reminder, Bruce...
This person is confused.
The industry is not moving away from SGI.
They aren't even moving away from IRIX.
A lot of places are getting SGI Linux boxes in house. With an Origin 3000 server and using an Onyx for video streaming, you can have a nice setup using all SGI stuff.
And A|W is an SGI company, but they support intel boxes with RH 6.2.
I know I have been waiting for these AMD beauties to come out the door. When MHZ and CPUS are a must and you are running on a tight budget, dual AMDs are a dream.
Now I need guranteed software support for them...
(for some reason some vendors will not list AMD as a supported proc... )
I would die for your American prices...
Several of our soldiers have.
IMHO there wasn't a good character fit for him in the story, so why force him in.
We made sure that he was back in Lyle.
You should check penguins out... we are working on video #2 right now, in the finishing stages... it should be out this summer.
Also, for the geeky side, all our final renders since King George have been almost exclusively linux. And it will be this way for Jonah, our feature length movie that comes out next year.