Slashdot Mirror
Weta Prepares to Render LOTR: ROTK
Dee Arsmith writes "Peter Jackson's special-effects company Weta Digital has just taken delivery of 588 IBM blade servers, each with two 2.8 gigahertz Intel Xeon processors. Seven racks of IBM blade servers have been added to Weta's existing 15-rack server cluster to make up the largest Intel-based high- performance computer site in the world with more than 2000 linked processors. The cluster will be used to render the frames drawn by the animators to complete the final installment of The Lord of the Rings trilogy, The Return of the King."
Actually, Beowulf was not a beast.
If you look at top500.org, you see that the current top Intel-based cluster is #5, the one with 2304 procs in LLNL.
The article says their cluster has 'more than 2000 processors'. So presumably they mean 'more than 2304'?
but its not a Beowulf cluster...
Rendering and beowulf do not play nice together... its a distributed system... with queueing... much more like Sun's Grid, I am sure.
-Tim
-I just work here... how am I supposed to know?
See for instance http://www.wired.com/news/digiwood/0,1412,56778,0
Q: What platforms does Massive run on? A: Massive runs under Linux and Irix.
Many interesting details at http://www.massivesoftware.com/
Here are some nerdy technical details to (hopefully) satisfy the rest of us :-)
RLX (formerly known as Rocket Logic) was the first company to introduce blade servers. Headquartered in Woodlands, just north of Houston and the old Compaq. I think they intially got a lot of smart engineers from Compaq, but they're probably laid off by now. They had the misfortune of starting their business amidst the tech slump. The other big companies (Dell, IBM, HP) have been quick to spoof RLX and steal some of its thunder. Guess it helps that you have services and other technology to sell to a customer. After all, buying things from one company can be simpler than having multiple suppliers with different contracts, etc.
How unfortunate it is that the first-to-market is sometimes never the market leader.
ok, my "render math" isn't the greatest, but I can NOT imagine that the system he had before was THAT bad? What do you really gain by adding that much MORE horsepower? Is that the difference between a frame being rendered in 45sec vs 50sec? I understand that every little bit counts, but a LOT of these movies was done live action. Unless that little Gollum thing is in every scene, why does he need more? (ok, I know, I always want faster, better too....I'm just saying)
Well, when you're talking about a 2.5 minute CGI shot, you have 24 frames/second (minimum) X 60 seconds/minute X 2.5 minutes = 3600 frames to render. 3600 frames X 5 minutes/frame savings = 18000 minutes or 300 hours in total saved by reducing a frame render from 50 minutes to 45.
That's just in 2.5 minutes of on-screen CGI, too - when the lions share of the film requires complex digital effects, the rest is easy to justify to the bean counters. In fact, I'd be suprised if they don't end up with even more horsepower by the time ROTK is in theatres - saving that much time provides big returns on investment.
Soko
"Depression is merely anger without enthusiasm." - Anonymous
45 seconds for a frame??? At film resolution?
I don't know about LOTR, but the Pixar movies take an average of 8 hours of processing time (single processor) per frame.
Of course these are all averages, and the folks at Pixar don't have to deal with integrating live action and digital at all! In LOTR, you have tons of compositing with all of that live action footage, and that eats up plenty of CPU cycles in addition to the 3D work they're doing.
So this isn't really about making things a bit faster, this is about getting this thing released on time with all the sfx the director (and the audience) are looking for.
So How fast is fast? Twice the processors is twice as fast.
Rendering scales beatifully across multiple processors: doubling the processors pretty much cuts your rendering time in half... this isn't like some Office application that'll run 5% faster on a multiprocessor machine...
You can select using a button on the front of each blade which of the 14 blades in the BladeCenter chassis has ownership of the 'MediaTray'.
Of course this switching can also be done remotely over Ethernet using the management interface (which also provides power, reset, remote video and much, much more).
From the OS viewpoint the Floppy and CD-ROM drive are USB devices, so switching the MediaTray to another blade server actually causes a USB disconnect/connect.
If you want to see what they look like: http://ibm.com/servers/eserver/bladecenter/
Well if anything you can't fault VFX with the story. But yes each prequel had over 2000 VFX shots. You have to read the article though, last film Weta did about 800 VFX shots and for this they are doing upwards of 1200. As their technology matures (Massive, muscle dynamics, subsurface scattering) you can even throw more things at to the VFX.
It certainly is a big setup, they are adding 1,176 new processors to what they already had (which was stated in an article some time ago). Probably ILM and Imageworks have a bit more though. The article says that they have the largets Intel deployment, but places like ILM and Imageworks, besides their Intel/Linux machine still have quite bit of SGI hardware around. An article on the SGI websitye a couple years back stated ILM had an 800 CPU Origin 2000 machine, and around 500 O2s. Since then a lot of the TDs, animators and compositors have gotten Dell Linux workstations and several of them keep the 2 machines side by side (the O2 and the Dell). ILM and Pixar also recently added to their renderfarm via RackSaver:
Pixar switches from Sun to Intel
Racksaver testimonials
AMD debuts server processor, readies 'Barton'
SGI Powers 5 Summer films
It certainly is nice that New Line is paying for this though. I'm sure other studios are envious ;-).
In terms of number of processors, ASCI Red at Sandia has had > 9000 Intel pentium pro (and them pentium II Xeon) procesors since the late 1990s.
It's still # 15 on the top 500 list
Bah, last post was apparently in HTML format and managed to make a single unreadable block of text. That will teach me not to preview...Anyway, reposted in plain text:
A long, long ways.
Computer games can run at 60+ frames per second because they are barely doing any work when compared to top of the line rendering engines.
Raytracing, dozens of texture passes, multiple realistic lightsources; and these are just for a two dimensional surface. Making realistic looking skin requires multiple translucent layers to simulate the complicated appearance of skin.
Also, there is the size factor. Video games generally run at 1024x768 to 1600x1200. Movie quality shots are rendered at many times that resolution, which greatly increases the number of pixels that have to be rendered. Gollum may only be 800 pixels tall on your monitor, but he's probably rendered at least ten times as large; we'll say 10,000x 10,000 for calcualtion simplification.
That's 10E7 pixels, so to display it at 24 frames per second you would need to be pushing 24E8 pixels a second. 24,000,000,000.
Even if every pixel only took a single cycle (which it might, with the right hardware pipeline in the future), you would need 240 terahertz of power (plus overhead) to display it in real time, along with enough RAM to hold the model and texture data for everything that's going to be onscreen within the next minute or so.
Considering that they have around 2000 x 2.0 X 2 = 8 terahertz available to them, and it still takes ages to render each frame of the complicated battle scenes, I'd say we are going to hit the limit of Moore's law before we could reasonably get hte power to render cinematic scenes in real time. Perhaps with quantum processing we will be able to within the next 20 years or so.
nah...it wasn't an imax of the movie. It was an imax about the titanic itself.
http://us.imdb.com/Title?0297144
I seem to remember PJ being interviewed during the lead-up to FOTR, in which he explained that pretty much every shot in the movies would be altered anyway to ensure that the colour saturation is even throughout the three movies, mostly because the colours of the backgrounds had been altered to make Middle Earth feel different to our world by making the colours a fair bit darker and richer than those you can see around you. I wouldn't be suprised if no scene remained untouched by the colour adjustments. Just like how they used technicolor to add colour to old movies, but on steroids.
Never fight naked, unless you're in prison...
They screwed up the number of blades though....
It mightn't be in the "home video production" realm but Shake, now brought to you by your favourite fruit company, can distribute rendering tasks. iMovie can't be far behind...
When they came for the communists, I said "He's next door. Take him away. Goddam commies."
I would just like to know why 588 computers?
IBM Blade Center that holds the blade server is 7U. Each Blade Center holds 14 blade servers. IBM's racks are 42U.
42U Rack / 7U Blade Center = 6 Blade Centers/rack
14 servers X 6 Blade Centers = 84 servers/rack
7 Racks X 84 servers = 588 Servers
Also, there is the size factor. Video games generally run at 1024x768 to 1600x1200. Movie quality shots are rendered at many times that resolution, which greatly increases the number of pixels that have to be rendered. Gollum may only be 800 pixels tall on your monitor, but he's probably rendered at least ten times as large; we'll say 10,000x 10,000 for calcualtion simplification.
This is actually not true. Film resolution is around 2048x1556 and everything is rendered the size that it is needed. For the most part, the difference in rendering speed is because hardware is very fast and very efficient, and takes lots of shortcuts. There aren't many textures, they aren't very high resolution, there isn't any raytracing, there are very few lights, no global illumination, no hair rendering, no volumetric rendering, not nearly as many polygons, no particles or cloth simulations, very few deformations, and lighting calculation is done on vertexes and then interpolated instead of on every pixel (this will change with Doom 3 and Half Life 2 which is the real reason they look so much better). Renderman also subdivides everything down to one polygon per pixel to get perfectly smooth sufaces and good displacement. There is also the issue of motion blur, depth of field, and rendering of composites, which also takes a very long time. Anti-aliasing in every step is crucial for any kind of non-realtime CG, but it not as important for games, and that by itself makes a huge huge difference.
This Wiki Feeds You TV and Anime - vidwiki.org
You are wrong. Standard film rez is 4096x3072, usually rendered at least 4x for oversampling antialiasing. The original remarks were correct.
Combustion (a compositing package similar to shake) already does this. Most 3D rendering applications support network rendering now as well. Weta is rendering to an individual file for each frame. Because of the nature of multithreaded processing and the inherent problems of multiple computers writing to the same file simultaneously, encoding to mpeg and to dvd is still time consuming, and typically only involve one machine. Even the pro-level applications (cleaner for encoding to MPEG, and Apple's DVD Studio Pro or Sonic ReelDVD for authoring DVDs) still only use one machine. Rendering 3D scenes is a completely different process than encoding video.
I'm out of my mind right now, but feel free to leave a message.....
Probably. Supoposdely they work on King Kong for PJ next. Apparently they might work on the film versions of the Chronicles of Narnia and rumor has it a movie based on Neon Evangelion. A lot of people were only meant to be thre until the end of the year since they didn't have any other projects lined up afterwards. A chunk of them are also foreigners, rom the US, Europe and elsewhere so they might want to go back home regardless. They will certainly go on, it just depends on the timing on the nest projects as to how much they will downsize before ramping up again.
As far as ILM that's why they need to bid on a lot of projects. Pipeline has to be busy to accomodate the huge number of people and overhead. That's why they try to work on several projects concurrently, at the moment on Peter Pan, Van Helsing, Harry Potter 3, Ep. 3 and Master and Commander. They just wrapped The Hulk and if T3 and Pirates of the Caribbean haven't wrapped up they should do so in a week or 2 at the most.
It's the big dilema of finding the balance between size and capacity with workload and potential work.
588 blades
x 2CPUs each
== 1176 physical CPU's
x 2cpus/cpu (hyperthreading on the xenons)
== 2352 hyperthreaded cpu's
x 2.8GHz
== 6585.6GHz
~6.6THz
well... thats a just a bit of rendering power, wonder whats gona happen once they are done with them. Which also makes me wonder, what happended to that somewhat famous renderfarm for toystory? Seems whenever a movie requiring horsepower like this comes out, they just buy new equipment since the stuff used on the last movie is probably obsolete already... ohwell
Tm
Support TBI Research: http://www.raisinhope.org
Code analysis is notoriously difficult to do, and some problems encountered along the way are probably impossible to solve, at least from an algorithmic perspective. You will never see lines of code analyzed and rid of bugs completely due to that analysis. If you could do that, the vast majority of programmers would be out of jobs.
-- Fighting mediocrity one bad post at a time.
I've worked with Academy Award winning animators and effects people, and their #1 continual complaint is that their clients have no imagination. They whine that they get asked to do the same effects over and over, because the director saw some effect somewhere else and wants to copy it.
I think the same problem presented itself 30 years ago when analog 'sythesizers' such as the moog modulars became popular. Early work was highly experimental (especially so with Buchla boxes), but once synths were used in pop recordings, other people would want to use the same particular sound as well (often as a gimmick to sell more records, cashing in on the synth craze).
Because of this trend of using similar sounds, rather than trying to be unique and develop your own, synth makers began to make smaller, less versatile synths that satisfied most peoples' needs, while limiting the creativity of the true artists. This trend continued into the 80's when synthesizers tried more and more to sound like instruments that already existed. You've got to remember that in the beginning, there were people against having synths be keyboard controlled, because that would influence people into playing one and treating it like other keyboard instruments that already existed (organs, piano, etc). I believe that they had a point, but I also don't think there was a better controller device available.
(for a good book on the subject, check out "Analog Days" by Trevor Pinch and Frank Trocco)
Think of it in this perspective: If there were only two or three computers in the world capable of rendering VFX for movies (substitute in synthesizers for creating music), only the most talented artists would be allowed to have time on the machines. Most of the work coming out of this would be top-notch. Now that the barriers to entry in the field are much lower, there aren't enough brilliant minds to go around, so you end up with much higher percentage of crap, but in absolute numbers, you should also get more great works.
There are truly brilliant, creative people out there. They are, however, a tiny minority of the populace at large. Maybe I have a bad outlook on things, but it's very demoralizing when you realize that most people are boring, uninteresting, uncreative blobs of matter.
I thank God that once in a while, I find something that gives me hope for humanity.
man tunefs | grep fish
The plastic mail was only for the extras and for battle shots. Still shots on primary characters often used real chain mail. They found some outfit that was selling whole sheets of thin chain link, and made armor out of it. Don't ask me how I know this.
If you do not like to read the printed page I would recommend getting an unabridged audio tape set of the LOTR and listening to it. You could borrow such a set from a library without too much searching. www.recordedbooks.com has an unabridged reading of the complete LOTR broken into the three books.
I second that recommendation. The performance by Rob Ingles was excellent; listening to his voices, I could visualize almost every character from the movies, because his voice matched the voice of the actors in the movie very well, and this was recorded in 1990. Notable exceptions included Elrond, who was completely different; I don't think Hugo Weaving was a particularly good choice for that role.
$x='S24;r)>63/* h@<5+oZ)32"5cz';$me='phroggy'x$];
$x=~y+ -xz+\0-Tx+;print$_^chop$me for split'',$x;