Lord of the Terabytes

Dracos writes "TheOneRing.net has an article from OnLine (a New Zealand film mag), in which Jon LaBrie, CTO of WETA Digital, discusses the hardware and software WETA is using to produce the sfx in Peter Jackson's Lord of the Rings trilogy."

Comments

Re:4 hour Hamlet

[ShinerMan]

I dont know so much that they had longer attention spans, but maybe that there was less to do back then. A visit to the globe theater in it's time, was basically a day long affair. It wasnt like movies are today when you can slip one in when you have a couple of hours to kill while your downloading the latest release of your favorite distro :p

Newspaper article or SGI ad?

[aiken_d]

Good god, I'd want a couple of Octane boxes for my house if I gave an interview like that...

-b

Do the math

[crisco]

3 films X 2 hours each = 6 hours
6 hours X 60 miniutes per hour x 60 seconds per miniute X 24 frames per second = 518400 frames

3000 pixels X 2000 pixels X 3 Bytes per pixel (ignoring very possible alpha channel) = 18000000 Bytes per frame (18 MB - ignoring the 1,000,00 doesn't equal 1 MB conversion)

18000000 Bytes X 518400 Frames = 9,331,200,000,000 Bytes

Add all the extra takes, the takes done for special effects, the 3D models, the textures, the stuff done for compositing, the database to keep track of all these resources, my math errors and all the stuff I'm forgetting. 100 TB is a reasonable spec for storage.

Now of course that is uncompressed. But seriously, are you going to be compressing images that are getting processed many times? You might be able to encode the final cut, but you can't be encoding, decoding and then re-encoding stuff every time you want to process something.

Here's what they /.'ed image says

[tweder]

Lords of the Terabytes

Figuring out how much data the The Lord of the Rings project would create was the starting point for Jon Labrie. "The three films are being shot simultaneously," he says, "and we're creating the effects for all three films in a single block."

"We expect to create and manage 100 Terabytes of data before this show is over. It's my job to plan for that growth and to meet the infrastructure challenges associated with it."

"There are 200 people in the Weta production facility, every one of them involved heavily in the creation of a lot of digital data. I think of it as a kind of blizzard or storm of data flying through the facility," Labrie says. "What we're about is the ability to move large amount of information around the facility all day, every day, and we rely on SGI to help us do that. Ninety percent of our equipment is SGI."

Weta uses two SGI Origin 2000s as its primary file servers. One Origin 2000 is for near-line and offline tape-based storage. The other is for central online disk storage.

"Given the 100 Terabytes, we're demanding a lot from those file servers. We currently have four Terabytes of data on hard drives, which will eventually grow to 10. For nearline / offline storage we're using DMF, SGI's hierarchial storage management system. It's greatly simplified our management of the thousands of tapes needed to store the bulk of the data."

Weta's primary rendering resource is based on SGI I200 Linux servers. "Rendering hinges on the ability to efficiently use processor cycles," Labrie says." We have 32 dedicated processors today, and expect to extend that to well over 200 by the time we're finished."

"We also have 90 Octane Irix workstations for the principal artists, and another 25 or so Linux-based workstations that we're using primarily for paint, rotoscoping, and compositing. Tese systems also contribute to the rendering pool when available."

The Linux workstations run Nothing Real's Shake, which is the primary compositing application at Weta. An eight-processor Onyx II also runs Inferno, Discrete Logic's high-speed, single seat compositing system.

"We have a large number of seats running Maya, Alias/Wavefront's modelling, rendering and animation system. Maya is the core 3D application for the facility," Labrie says.

"We also have a number of other applications for niche requirements: Houdini for effect and particle animation; 3D Equalizer and Softimage for camera match-moving. There's a sprinkling of other things, and a lot of proprietary technologies that we've been working on specifically for The Lord of the Rings.

"There are unique graphical applications that PEter Jackson has asked us to create. We've been in research and development for three years, planning and working on standalone proprietary systems or extensions to off-the-shelf applications. We're moving into the actual production of shots now."

"We've been writing custom extensions to Maya for the past two years to improve the look and performance of our computer-generated characters."

"Over the past four years we've used SGI wrokstations to custom-build a new crowd animation system, called Massive. We're using Massive for battle animation scenes with hundreds of thousands of fighting, screaming, and dying Orcs, Elves and all the other magical and fantastical creates that appear in The Lord of the Rings."

"For those sorts of graphical challenges, we prefer to work in the Irix/Unix world. The graphics available to us on the SGI platforms make our jobs easier. Of the 140 special-effects artists that will work on this project, nine out of 10 will work on SGI workstations."

"There are both classic and unique IT challenges," says Labrie. "Classic challenges are the kinds of IT issues any company would expect to face: What sort of networking technology you use, how do you manage backups and disaster recover? What kind of workstation is typical for a user? This typically breaks down to 'what's the fastest machine I can use to achieve what I need in a reasonable time for a reasonable cost'?"

"The unique challenges are things that are specific to digital visual effects for film. Every director wants to deliver a unique viewing experience, to show people things they've never seen before. We break down those expectations and requirements into specific etchnical challenges and get to work on them. The results could be as mundane as a new way of describing the internal skeleton of a creature or as fantastic as Massive."

The three movies are The Fellowship of the Ring (to be release worldwide on 14 December 2001), The Two Towers (December 2002), and The Return of the King (December 2003). The trilogy shoot should wrap up early next year.

Should We Be Excited?

[danmil]

Am I the only person who finds themselves wildly swinging back and forth between sort of fevered anticpation of this series and totally cynical distrust? I keep on having conversations with myself along the lines of "This could be so cool!... It it doesn't totally suck".

Plus side: Peter Jackson is pretty cool. Dead Alive is an incredibly funny, incredibly gross movie. Heavenly Creatures was good. Some of the casting sounds good. The One Ring preview was good.

Minus Side: Is it even conceivable that they won't fsck it up? I mean, how can it compare with the depth, the atmosphere, the brilliance of what is unquestionably the greatest fantasy series of all time?

Anyone else?

-Dan

Re:Should We Be Excited?

[gevauden]

It should be good as long as people remember one very simple fact.
It's an adaptation of the trilogy. So no, it won't have all the scenes in the books, the characters won't all be exactly the same as in the books. Things may even happen in a different order. That's what happens when you adapt a book for cinema.
As long as people don't go expecting to see a scene for scene copy of what their imagination built during the course of reading the trilogy I don't think they'll be disappointed. Peter Jackson is a great director who really knows how to put his vision on film, the cast is really solid, generally being well respected actors (not the latest hollywood 'cool' people) and the location is perfect.
From what I've seen online and the reports I've heard from cast members it will be a fantastic cinema experience.

Gev.

In summary ....

[Anonymous Coward]

for those who couldn't be arsed reading the article. The New Zealanders are employing revolutionary "SHEEP"(tm) technology. The sets are entirely made of sheep and gaffer tape. Support cast ... sheep and the protagonist is Bilbo Baaaaaaaggins. Check it out for yourself if you don't believe me. Baaaaaaaa.

Re:What about D&D ?

[mwalker]

:: Ghost in the Shell : Blade Runner

oops.

You were doing great until you took one of the best movies ever made and tried to compare it to that movie where harrison ford tries to act like a robot, but succeeds only in imitating himself. I'm not sure how it ends, i fell alseep, but hopefully mr. ford dies. with luck he dies folded up like a paper crane, but beggars can't be choosers.

I think a more accurate analogy would have been:

Cool World : Ghost in the Shell

But then that's just me, and I don't snort as much coke as george bush, so i could be wrong.

Re:What about D&D ?

[Speare]

But which will be better... Lord of the Rings or Dungeons and Dragons?

Standard analogy format reads (A : B :: a : b), or 'A' is to 'B', as 'a' is to 'b'.

My guess, just from trailers (and I'm sure people will disagree)...

• D&D : LotR
• :: Tremors : Dune
• :: Johnny Mnemonic : The Matrix
• :: Ghost in the Shell : Blade Runner
• :: Spaceballs : Star Wars IV New Hope