Slashdot Mirror
More on LoTR Special Effects
sushi writes: "Another LoTR article: this one focusing on the technology used at Weta Digital (the CG shop). Interesting that they are undertaking "major" R&D into running more Linux, and that Linux "delivers about two times the price performance compared to systems running proprietary operating systems". I've been lucky enough to have seen inside this place, and it's cool to see a render-wall of linux boxen. Full story
from a New Zealand newspaper." We linked to another good article about WETA a month ago.
I'll tell you why -- good old fashioned ego. Whereas the low end (kernel developers, compiler writers, etc.) and high end (clustering software, 3D modelling and rendering, etc.) of development is led by strong, well-organised teams of well-trained developers with vision and understanding, the middle ground of the Linux is polluted with warring egos that suffer too much from the problematic NIH (Not Invented Here) syndrome. There are a myriad of competing, mutually incompatible yet separately inadequate office suites (Star Office, KOffice, Applix,...), desktop environments (KDE, Gnome, XFCE, CDE, UDE, ROX,...), and X servers (XFree86, MetroX, XiG). We can't even decide on a printing system! I realize that, according to Eric S. Raymond's famous "Cathedral and Bazaar" text, that open-source software is primarily written to scratch an itch and get peer recognition, but this is taking it too far. If all the man-hours poured into KDE and GNOME were combined into a common vision, we would have one perfect end-user desktop, instead of two poor imitations of Windows.
Don't give me the old "competition" argument either. There is only one Linux kernel, which seems to progress just fine without another competing project nipping at its feet and instigating flamewars. The endless KDE vs. GNOME, Applix vs. StarOffice, and other feuds have wasted more productivity than would be gained by and competitive drive.
I, for one, am somewhat miffed that while my operating system powers Hollywood blockbusters and NASA supercomputers, it still can't fully replace Windows on my office desktop. Linux is growing up; its users need to grow up with it, shed their egos and work towards the common goal of creating an excellent working environment.
With SGI's announcement that they are supporting Linux on the new Visual
PC does anybody know if Alias is going to port MAYA onto the Visual PC
but running Linux ? The MAYA renderer should be easy to port as it
requires no graphics capability.
I am thinking of setting up a MAYA render farm and my preferred
platforms would be the Visual PC running Linux. I am VERY wary of using
NT which has an appalling reputation for unstability, requires far more
support than Linux, is subject to multiple upgrades/service packs and
has VERY poor performance under load. Linux would provide superior
through-put, superb stability/reliability and also integrate very well
and easily into my otherwise SGI dominated setup.
Anybody else interested in a MAYA renderer port to Linux ?
Please do not reply if you are trying to tell me how good NT is - the
growth of Linux in comparision to NT tells me what I need to know - even
with Microsoft spending millions of dollars advertising NT its sales are
only comparable with Linux sales - virtually unadvertised compared to
NT.....
The two articles didn't say. Are they using PRMan, or something else? Or are they using their own proprietary renderer, a la PDI/Shrek?
There's 10 types of people in this world, those who understand binary and those who don't.
Aside from the impressive technological feat, imagine looking forward to the day when effects like these are availble for Gaming Engines.
Imagine Quake IX out in an open plane of battle with literally hundreds of thousands of soldiers and other things out there all at once.
I am reminded of something similar to the weekend dogfights/lanparties at the Airforce Academy, but with a much larger field of action.
[smile]
"It is a greater offense to steal men's labor, than their clothes"
For ordinary home users, in the main centres there is a choice between ADSL or cable modem, depending where you are.
Then there is the cost of labour, they are probably paying about half what they would have to pay in the US or in Europe. There are some countervailing advantages of course, depending on your interests. For instance there is a good surfing beach about ten minutes drive from Weta's site. Then check out the movie itself for some samples of the local scenery.
Incidently the Weta is a large fearsome looking NZ insect that you would rather not find crawling up your trouser leg, although it is mostly harmless.
Given that the server is being given a good hammering at the moment (and not handling it very well), here's the contents of the article (after trying to retrieve it about 10 times)
Lord of the special effects
03 December 2001
By AMANDA WELLS
Weta Digital chief technical officer Jon Labrie was looking forward to a lull after delivering the special effects for The Fellowship of the Ring, the first in Peter Jackson's Lord of the Rings film trilogy.
The facility, based in the Wellington suburb of Miramar, delivered the effects shots in early October, with the film's Australasian premiere scheduled for December 19.
But, Mr Labrie says, things are not proving quite as quiet as expected, and Weta's biggest workload is yet to come.
Work has started on key shots for the second film, The Two Towers, and a detailed plan is being drawn up for tackling the sophisticated effects needed in this film and its successor.
Weta will deliver The Two Towers effects by October 1, 2002.
Mr Labrie says the facility has yet to get a clear picture of the workload for the film, which involves animating several key characters.
Gollum, an evil creature bent on retrieving the One ring he once possessed, emerges in the second film, and Weta's graphic artists have begun bringing him to life. Gollum is shown close-up and must appear believable.
Treebeard, an ent or talking tree, also appears in the second film, along with other creatures for which Weta has developed digital fur.
Mr Labrie says Weta will probably have to grow by another 10 per cent to complete The Two Towers, up to about 250 staff.
The trilogy's second and third instalments contain some impressive special effects set pieces, he says.
The third film features "an extraordinary number of battle sequences".
Weta crowd supervisor Stephen Regelous has created software, dubbed Massive, that creates realistic crowds. Every individual in the crowd moves in response to stimulus such as terrain, and to the actions of others.
The battles in The Return of the King will see hundreds of thousands of these intelligent agents in frame at the same time, Mr Labrie says, stretching the software to its limits.
Massive was developed on SGI operating system Irix and has since been ported to open source operating system Linux.
The deadline for delivering the third film's effects has not yet finalised, but could be October 2003.
In terms of the facility's workload, creating effects for The Return of the King will be equivalent to the first two films combined, he says.
Mr Jackson has kept tabs on filming and effects while offshore through extensive use of videoconferencing. He could view the output of a camera remotely, and artists could transfer frames via an FTP connection.
Mr Labrie says that at the start of the project in 1998, it was hard to conceive how much work it would be to complete simultaneously all three films based on the 1200-page epic.
JRR Tolkien's world is hugely detailed, with a vast variety of landscapes and diverse array of creatures.
Elves, dwarfs, hobbits, trolls, orcs, ents, wraiths and balrogs populate Middle Earth, and purists will be watching keenly for a literal rendition.
Mr Labrie says Weta's naivety was probably a good thing.
"Nobody would want to tackle three films again, not at the same time."
Weta grows seven-fold
Mr Labrie joined Weta Digital in 1995, when the company was set up to provide effects for Mr Jackson's films.
Weta is privately owned, with Mr Jackson's 34 per cent the largest individual stake.
Mr Labrie came to Weta from the US, where he had mostly recently worked on effects for science fiction blockbuster Independence Day.
He has overseen the facility's expansion from 30 to 230 staff.
Weta has created effects for previous Jackson films Heavenly Creatures and The Frighteners, but the Lord of the Rings trilogy far outstrips these in complexity and volume of effects.
About 1500 effects shots will be created in total.
Mr Labrie says if he was doing it again, he would spend more time up-front on planning for growth and "less time reacting".
Planning of this kind is underway for films two and three at the moment.
In terms of technology, "there's not a lot I'd do differently," though getting digital asset management up and running was problematic.
"There are still issues to be addressed, but for the most part we have done it right."
The growth of the facility has been "far greater than we anticipated". Originally, Mr Labrie thought Weta would need between 80 and 90 graphic artists to complete the trilogy's special effects.
But artist numbers hit 167 in August, at the peak of The Fellowship of the Ring production, with 225 staff in total.
At the moment, the facility has about 205 staff, with some people leaving to work on other projects or heading home to other countries after the first film was completed.
Numbers will ramp up again in the New Year to hit between 230 and 235 in April or May.
Finding graphic artists for the project is not a problem, Mr Labrie says.
"Everybody wants to work on Lord of the Rings."
Just under 40 per cent of the artists are Kiwis, 31 per cent from the US, 11 per cent from Australia, 5 per cent from Britain and the rest from countries as diverse as Japan, Egypt, China, Germany, Korea, Russia and France.
Weta has amassed substantial world-class talent, he says.
"In terms of pure technical infrastructure, we are one of the three largest facilities in the world.
"We consider ourselves, at the moment, to be one of the top five visual-effects facilities on the planet."
He says Weta staff are focused on the work remaining during the next two years, with the future of the facility after that yet to be decided.
Weta will not be able to support existing staff numbers when Lord of the Rings work is completed without securing another project of the same magnitude, which seems a tall order.
Effects artists tend to be nomadic, Mr Labrie says, and will head off to the next project in which they are interested.
Mr Labrie says the business could become more broad-based, tackling interactive gaming or commercials. "It's hard to make a profit out of visual effects."
This time next year, serious consideration of Weta's future will start.
The Hardware
of the Rings
The facility's technological infrastructure has mostly coped well with its exponential growth, Mr Labrie says.
Adjustments are being made at the moment to network switches and data distribution systems to ensure Weta's technology will scale up again when facility growth peaks for The Two Towers work.
The machine room, housing the processing power at the heart of the facility, will probably be enlarged, and work will be done to increase electricity flow and the Uninterruptible Power Supply service into the premises.
Mr Labrie says Weta will probably look at buying more hardware in February or March to meet The Two Towers' requirements.
Between $20 million and $30 million has been spent on Weta's IT infrastructure so far.
The total is "a little more than we expected", because of some unanticipated costs near the end of the first film's work.
A "rendering crunch" of last-minute work meant more processors were needed to complete the final six weeks of visual effects production, partly because of some late additions to the cut.
Mr Labrie says he has probably exceeded budget estimates made three years ago by about $1 million.
Weta does not have formal hardware agreements in place, but has developed strong relationships with vendors SGI, Auckland-based DVT, and Infinity Solutions, and Mr Labrie says he would go to them first to see if they could supply the hardware he needs.
About 90 per cent of the company's systems are from SGI.
The machine room has about 12 terabytes of storage, with about 20 terabytes in total at the facility.
Mr Labrie says working storage needs will probably hit 30 terabytes for the second and third films.
But adding storage is not as difficult as it used to be, and prices have come down.
By the time the effects for the third film are finished, between 70 and 80 per cent of the hardware will be out of date. These systems will be written down.
Some PCs will be able to be used for the company's next project, along with a StorageTek tape robot which has a long lifespan.
Mr Labrie says Linux is gradually replacing Irix as the operating system of choice in the effects world.
Weta has a "major" research and development effort under way at the moment into running more Linux-based workstations.
Mr Labrie says the facility is running a substantial amount of Linux at the moment, on processors in the machine room that are "the core of the rendering wall".
Between 40 and 50 workstations run Alias/Wavefront's Maya character animation software or Nothing Real's Shake compositing system on Linux.
He says he is looking at "making a more determined move" into Linux for the second film and will probably at least double the facility's number of installed Linux systems.
Linux delivers about two times the price performance compared to systems running proprietary operating systems, he says. Unlike several years ago, sophisticated animation applications are increasingly able to run on the free operating system.
The project has brought with it huge public and media interest.
Mr Labrie says he is receiving a couple of interview requests a day at the moment, from New Zealand and offshore media.
"I thought maybe we'd be able to keep a low profile for a bit longer."
Fans will be keen to check that technology has brought Middle Earth to life correctly.
Mr Labrie says visual effects in The Fellowship of the Ring are "all over the film", with audiences not going for long without seeing footage that has been manipulated in some way.
"There's always some kind of trick going on."
But these are seamlessly integrated into the film's background.
Weta is gathering material from its archives for use in The Fellowship of the Ring DVD release at the moment.
The DVD, which is being produced by New Line, will contain information about the making of the film.
American software company Electronic Arts is creating a Lord of the Rings game, for which Weta is supplying images and models.
The facility has a fulltime staff member dedicated to finding material for the game, though has no involvement in its production.
Mr Labrie is setting up a games company at the moment, with details under wraps for another couple of months.
He has written film scripts in the past, and says he will write storylines for the interactive games the company will produce.
The company will probably launch early next year, and Mr Labrie will continue in his Weta role.
As the facility's infrastructure becomes complete, being chief technical officer is more administrative and takes up less time, he says.
But his focus will remain on the enormous project till it is completed, sometime towards the end of 2003.
-- No, no gems to be found in this sig.
I've seen some screenshots and they don't look that hot, have a look for yourself
I vote the guy who made that the least likely to ever have sex...
Wax-Museum Fire Results In Hundreds Of New Danny DeVito Statues
Apparently it's a Tolkien Ring network - yuk yuk.
The main reason not to run a Microsoft OS for a farm of anything is that it's getting harder and harder to turn the resource-wasting crap off.
I just found my own NT application loading Internet Exploder whenever I bring up a file open dialog. As soon as you call for a common dialog, an incredible amount of crap gets loaded and three more threads start up within your app, doing who knows what. It looks like the file browser in the open box uses IE.
You could have all the rendering boxes in the world. If you have high-school hacks, you're going to end up doing hack-level work.
Directors at Pixar once said that they hire artists that can use computers, not computer people who claim they are artists. This movie would have been just as impressive if it was made without the technology.
And let's not forget the artistry involved in writing the book, which required millenia-old technology.
Not at all, they are running PRMan from the looks of it. One guy from WETA digital is a frequent poster at the renderman newgroup and has even posted job openings for shader writers. besides Maya and Shake they are also running things like Houdini which has a great RenderMan connection. Weta does indeed have propietary software like Masive, but as most other places they have a mix of in house built and of the shelf stuff.
Most old timers in the CG arena have in house tools because there wasn't any off the shelf stuff to buy. Besides Rhythm and Hues, you have places like PDI, Blue Sky Studios, Pixar (though they decided to sell their renderer most of their other tools are propietary) and others. Most FX studios run on a mix off commercial and in built software. Most places run mostly SGI but many are switching or at least experimenting with Linux. Rhythm and hues is even helping develop the Film version of the Gimp (for 16 bit work, tha's 16 bit per component).
In my experience, and based on what I've read on the mailing lists and newsgroups, Linux has by far the widest hardware support. I've personally installed it on a huge range of machines, only finding one machine I could never get it to install on (and that was a BIOS problem). Conversely, the few times I've tried BSDs have been unmitigated disasters.
On the other hand, a Ford Escort can't haul half a ton of widgets across town to a client's emergency, and even an SUV would have trouble with an emergency sheet of plywood. Fortunately when it comes to operating systems you can, in fact, "drive a cargo truck to work". Sticker price is not the only cost. Adaptation and maintenance dominate, and Linux has significant benefits over many other operating systems. Need Beowulf-style clustering to do your rendering? Linux has excellent tools. Have software written for a 32-processor machine? Use the Mosix patches for Linux to tie together 32 ordinary PCs. Want to be able to rapidly upgrade and reinstall all 1000 machines in a cluster? Network boot Linux using the well-documented tools, and watch the entire cluster reboot in less than 10 minutes. (Try that with Windows sometime.) Big jobs are different in kind from small jobs, not just different in size. A 10% mistake in a small job means you stay late that day. A 10% mistake in a one month, 1000 CPU job means you threw away the company. IMHO, the key to keeping large projects running smoothly is having flexible, adaptible tools, and the Unices come standard with awesome tools of all kinds. Windows, on the other hand, is worthless out of the box and remains inflexible even when you shell out the $$$ for the official tools.-- ;-)
Kuro5hin.org: where the good times never end.