Slashdot Mirror
Lord Of The Rings Being Rendered Under Linux
Along with an adventuring band of others, tmatysik writes: "Came across this article in the New Zealand Herald the other day. Seems that Weta Digital is now moving over to Linux for the rendering work on Lord of the Rings. Two quotes from the article especially caught my eye: [1)]
'We were able to get the SGI 1200 servers for about $15,000 each or $7500 a processor, and they run more than twice as fast as the [$40000] Octanes for pure rendering.' [and 2)]
'Just by putting in a Linux processor, the price to do a frame is up to a tenth of the cost as on an SGI workstation [running SGI's Irix operating system] so the things we can attempt are more complex.'" Update: 08/27 09:35 PM by CT : Rebecca from WetaFX sent us pictures of the team, and the mighty stack that shall render lord of the rings.
A lot of comments here equate LoTR with the MPAA. While the film is being produced for and funded by a MPAA member, I think it bears pointing out that the LoTR people have negotiated a fair amount of freedom from MPAA influence, and that the production companies involved (WetaFX for example) are not subsidiaries of MPAA companies (even if most of their more recent income has come from MPAA members).
:-))
(And just to rub it in, the MPAA's region coding system is an illegal trade barrier in New Zealand
Unfortunately, if LoTR is as cool as it looks so far, the MPAA will reap the financial reward. On the bright side, it might convince them to break the mould again rather pump out more purile cringe-comedy films and the other garbage they won't leave alone.
Another pain for the boycott idea - how to support people like the LOTR teams while not supporting the MPAA. Sigh.
Fair comment, but [while I don't go shopping for this type of equipment myself] the prices on those SGI 1200's looked a steal to me.
:-)
Buy something at half the price - buy twice as many - more than make up for the speed loss. I think that Titanic was rendered on x86's. Go figure.
Though.. for a farm... yeah. You fire out tons of little linux boxes for the price of a single sparc...
Peter Jackson has already done that sort of thing.
---
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
Free Software: Like love, it grows best when given away.
Ahh I remeber it now... The arguments on comp.sys.amiga.advocacy:
:)
"The Amiga is slow, old and obsolete. It's dead, time to move on."
"Oh yeah! They use the Amiga to render the scenes in Babylon 5. Yeah! And I heard they used an Amiga 2000 to prop the doors open when rendering Jurassic Park!"
Years from now in comp.os.linux.advocacy when people point out how pointless Linux is in the modern world, the advocates will now be able to say "Oh yeah! Well they used Linux machines to render Lord of the Rings!"
Kind of touching, really.
It's interesting that Linux is being used, but that's not what makes it cheap. What makes it cheap is the Intel hardware. Even if they had used SCO Unix, it would have been far cheaper than that equivalent SGI platform.
And just to be controversial [:)], I'll say that you really have to give the credit to both Microsoft and Intel. They are really the ones who brought affordable computing to the masses, and through that economy of scale we have the incredibly cheap power that we have. [God knows Apple didn't bring cheap computers to the masses, despite their "computer for the rest of us" claims. Their tagline should have been "the computer for the arrogant elite"]
--
Sometimes it's best to just let stupid people be stupid.
One of the big reasons is that the rendering software is expensive, and it's priced 'per processor', so you really want to have the processors be as fast as they can be. Right now, that means Intel architecture (here at Hammerhead we use Athlons); and Linux is by far the nicest way to use the IA32 machines.
Interestingly, the only company I can think of that doesn't use primarily IA32 boxes for rendering is Pixar (who write and sell RenderMan, the most popular rendering package). They use mainly Sparcs. One reason, I suppose, is that they don't have to pay for RenderMan :) The other is that on a speed/cubic-foot (as opposed to speed/dollar) metric; I'm told that the Sparcs are a little better. I don't believe that, though.
I interviewed people at every effects house a couple of months ago for an slashdot article I never finished :( and every single one was building Linux render farms.
thad
I love Mondays. On a Monday, anything is possible.
Well, they are working on better networking. Clustering isn't forthcoming, but that's a user-space issue anyway. Linux SMP is better, but nowhere near as good as BeOS's yet. (actually it never will be. BeOS apps are inherently multi-threaded while Linux apps are not.)
A deep unwavering belief is a sure sign you're missing something...
If more people would seriously consider using linux for their work, they might be surprised at what they could do - and how much money they could save. I'm glad that the people making LoTR decided to use Linux because now we will most likely get better effects. Now I just hope that other companies would look at Linux seriously... It would spread linux much more, AND the companies would be saving money so they could produce better/cheaper products.
Big Idea made the same decision not too long ago.
/. about rendering and Linux vs. SGI. It is great that there is now a story dedicated to it.
I never bothered to figure out what the cost/performance ratio was between the two platforms. I just knew that Linux was a lot cheaper.
The thing that suprises me is that they only have 16 boxes... and that this is newsworthy. We have 42 (with almost exactly the same config) and plan on ordering more in the near future.
One other difference is that they are using RenderMan and we are using Maya's renderer, which has recently been ported to Linux. For the type of work that we do, Maya is more than enough for us. Also, we hardly have any frame times of an hour... if we do, I yell and scream to get it cut down. The difference is that we are only rendering to 724x486. The use a higher resolution for the big screen, we only have to wory about NTSC for now.
The prices quoted in the article seem to be inflated a bit too. Unless they are quoted in the NZ dollar and that is about a 2-1 with the American. Octanes should be about $20k in the States. Linux boxes similarly configured, from VA, which we use, instead of SGI, are in the $5-6k range.
The performance boost that we have got from adding the Linux boxes is amazing. We went from being able to render on 50 MIPS CPU's 14 hours a day to rendering on an aditional 84 Intel CPU's 24 hours a day. The comparison in render speed is about 1-1, slightly infavor of the MIPS chip.
When it is all said and done, a move like this should be a no brainer to any studio. Rendering needs horsepower and system memory, both of which are cheaper in the Intel world. SysAdmins need Unix for ease of administrating and lack of down time... every studio I know considers the render system to be H.A. (high availablity). When things go down, projects get delayed... and that is a big no no in the production world (think of all the billboards you see with release dates on them... months in advance). Linux is a great choice for this.
The only drawback is that the SGI boxes have the ccNuma interface, which is great for single frame renders.
I know that I have posted many times on
FWIW: Our next two videos are the first to use our Linux render farm. Esther is at least 3/4 rendered on Linux boxes and Penguins will be mostly, if not all, rendered on Linux boxes.
-Tim Toll
Render Architect
Big Idea Productions
-I just work here... how am I supposed to know?
A lot of people have touted IRIX as being so vastly superior in performance to Linux under these circumstances. It's good to see that Linux has caught up quite a bit. Granted, I'm sure that Irix still has a bit of an edge, since it's well customized for the task, but the race is close enough that Linux is more cost-effective. I've always been a bit leery myself of the concept of one operating system powering everything from PDAs to render farms, but I'm thrilled to see a Free Software product that can scale in both directions so phenomenally.
WARNING: there is a trojan on your
They'd have to take their head out of their ass before they could take their foot out of their mouth...
;)
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
He's talking about the 3D model for the image. I'm still not sure if this is several GB large, but it has nothing to do with the resulting pixel image.
Using desktop machines for render farm is very good. And yes, currently, the desktop machines have to be NT to get the GUI software. At where I work we are using both Linux renderfarms and NT desktop machines for running batch processing with no problems. Most rendering software is available for both. We use tcsh on the NT machines so the batch processing looks the same.
I couldn't agree with you more.
We looked at NT boxes. We are a Maya only house and we were unsure when the Maya for Linux release would be. We actually thought of buying the hardware, throwing NT on it, rendering with it and then switching to Linux when it was available.
That was a shortly lived idea. None of us wanted to be responible for porting our code to work on the NT's. Even with a perl back bone, it looked like a night mare.
Thankfully, the Linux renderer was announced and it came out, all in good time for us to use it on our current productions.
-I just work here... how am I supposed to know?
That's maybe 75% true. There are several requirements on the OS, though:
I don't care if it's 90,000 hectares. That lake was not my doing.
whine ();
}"
May I suggest.
while (!world->perfect) {
act ();
}
A Dick and a Bush .. You know somebody's gonna get screwed.
War is necrophilia.
...unless Natalie Portman is playing one of the trolls who gets petrified, and who insists that the best way of cooking the dwarves is by pouring hot grits down their pants.
Also, Frodo would have to be escorted to the Cracks of Doom by a whole team lead by Beowulf (IOW, a Beowulf cluster).
Finally, this would have to be distinguished from the novels and the cartoons, not as the "live action movie", but as the "post-Columbine version".
I'm not sure if a penis-bird would have to be somehow involved, but it would help.
---
Despite rumors to the contrary, I am not a turnip.
Frodo: Well, we made it to Mount Doom, the seat of Sauron's power.
Samwise: Yes.
Both look pensively at Mount Doom.
Samwise: What does 'Microsoft' mean?
The "real work" is always done by some other program. That doesn't mean that the OS is irrelevant.
---
Despite rumors to the contrary, I am not a turnip.
YES! Rendering under DOS!
Wanna make a movie?
(two weeks later...)
What do you mean DOS is (at best) 16 bit and not running in protected mode?
---
pb Reply or e-mail; don't vaguely moderate.
pb Reply or e-mail; don't vaguely moderate.
>Hate to break it to you guys, but when you're
>doing "real work" like rendering (or anything >else), the OS is 100% totally and completely
>IRRELIVANT.
In theory, yes. Rendering is just math.
.... BUT. The os does come into play - context switching speed, VM system, general efficiency, etc.
Originally production houses were switching from SGI to NT boxes to save $$$$ - hard to justify the price performance of an Octane when a cheaper dual PIII would whip it in number crunching.
Now people are switching from NT to Linux. Personally I don't give a crap about what OS you use (i use both), but from my own personal benchmarks and graphics companies that i've contracted for, Linux can give 10-30% greater throughput on the same hardware.
Also - most large production houses WERE based on SGI's - so it is easier to move their custom code from IRIX to Linux.
(have you tried to administer NT boxes remotely? )
j
Rendering also needs oodles of memory.
:-)
The 1meg limit could be a bit of a problem
Then apparently DOS is the ultimate render farm OS.
A deep unwavering belief is a sure sign you're missing something...
Sure, but you've missed the point. I'd agree the OS itself is irrelevant where rendering is concerned. I'm sure it could be done on MS-DOS if you wanted to. But that's still not the point.
The point is that Linux is becoming something that is useful to everyone. Not just for UNIX geeks. Not just for trying out UNIX at home, because you can't afford a Sparc.
The shared development over the Internet towards Linux has produced something useful for entirely new groups of people. This is the point. This is the wonderful thing.
Linux doesn't need 100% marketshare to win. Linux wins everytime someone new downloads Linux and says "I like this".
Tom Duff wrote:
:-)
>
> Daniel McFarland wrote:
>
> > What size (mb ) is considered a large scene file.
>
> Around here, 2000 MB is large, 500 MB is small.
> Not counting texture maps, which normally
> add up to larger than RIB files.
note that 2GB would probably be the size after a catrib -ascii, i.e.
the gzipped binary RIB file would probably be
smaller. Having said that, I've routinely seen files in the 1GB+ range (after ASCIIfication)
that were only a portion of the scene (i.e. they were to be composited
over some other elements).
Most RIB files of a scene would occupy only a few
hundred megabytes on disk, assuming (as most are) that they are
compressed binary RIB files.
Also note that these days, many of your RIB files
have Procedurals in them that are trivially compact in the RIB
file, but expand out into a fair amount of data when the renderer
actually evaluates them.
2 steps forward, one step back
--
--> Michael B. Johnson, Ph.D. -- wave@pixar.com
--> Studio Tools, Pixar Animation Studios
--> http://xenia.media.mit.edu/~wave
I am what you would probably call a Linux evangelist, but in this case I have to point out that rendering is a CPU bound process, and probably the only OS charasteristic that could significantly affect the performance of such jobs is stability or in other words *uptime*.
I also have to say (even though I am an SGI stock holder) that lowend SGI servers are kind of expensive, and probably the best price/performance ratio under Linux is on Alpha or Athlon boxes. And if you use 1U or 2U rack units you can fit *a lot* of CPU power just in a single rack. VA's boxes are not that bad either.
__________________________________
Stop privacy invasion!
Hello, this is Frudu Baagins, and I pronounce Elbeereth as Elbeereth.
"The best we can hope for concerning the people at large is that they be properly armed." - Alexander Hamilton
bad form to follow up on myself
>Linux can give 10-30% greater throughput on the
>same hardware.
I'm speaking about *pure* rendering - number crunching.
Interactive 3D graphics on Linux is still kinda sad and unstable. Getting better, but it ain't there yet.
j
Linux Helps Bring Titanic to Life
- |Daryll
"I will take the Ring," he said, "though I do not know the way."
Three Tux's for the Elven Kings under the sky.
:)
Seven for the Dwarf-Lords in their halls of stone.
Nine for Mortal Men doomed to die,
One for the Dark Lord on his dark throne
In the Land of New Zealand where the shadows lie.
One Tux to rule them all, One Tux to find them,
One Tux to bring them all and in the cluster bind them.
In the Land of New Zealand where the shadows lie.
Sorry, I couldn't resist
"The words of the prophets are written on the Slashdot walls."
Lord of the Rings Being Rendered With Linux. Well, that it: the ultimate geek story. No point in hanging around trying to come up with something else. Let's all pack up and go home.
-
-
Give me liberty or give me something of equal or lesser value from your glossy 32-page catalog.
After all, it is more secure, just what you need to deal with Cracks of Doom...
No? I'll get my coat...
-- Bah weep grah nah weep nini bong
First, there's the issue of studios protecting their intellectual property: have to be sure that those 3D Jar Jar models don't make their way onto the internet, or that images from the films don't leak out early.
More importantly, the amount of data required to describe a frame in these kinds of movies is immense. We're talking gigabytes for a single image. So you're talking about gigabytes of communication and then a few hours of processing. It just doesn't make sense.
linux has been used before for CGI rendering- a lot was brought up about Digital Domain turning to Linux (on DEC alphas) as a cheaper solution to do a lot of the water rendering effects in Titanic
DVDs in New Zealand will be region coded, just like any other region of the world (although there is one 'whole world' region code that is very rarely used.)
Some Chinese manufacturers are now selling DVD players with 'hidden' region hacks in their firmware - of course the codes to unlock these hacks are magically made known via the Internet. So there are probably ways of getting a region-free DVD player in NZ, quite apart from the whole PC and software hack approach.
For those interested, there was a BOF meeting at SIGGRAPH 2000 dealing with Linux and 3D. The notes by Brian Paul are already available: SIGGRAPH 2000 Linux / OpenGL / 3D Birds of a Feather Meeting
It's only twice as fast for rendering which is mainly a function of processor speed, system bandwidth and memory (and to a lesser extent network bandwidth). Intel and AMD processors are way faster than MIPS processors, but SGI has high througput systems, and also the best OpenGL implementation. They probably use the machines for interactive work during the day (and I don't doubt the new Octanes are better than any), but during the night they join the rendering farm, where they could really shine. Your claim about the software is incorrect, Photorealistc RenderMan runs mainly on Unix based OSes (Solaris, IRIX and Linux) and Windows NT. So they for sure have measured how "good" PRMan runs under de 1200 and standard Octanes, and the CPU performance would make a differnce.
After all, it is more secure, just what you need to deal with Cracks of Doom...
Why would I want cracks of Doom? It's already had the source released. Better would be cracks of Quake III, I think.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
Dare I say it... BeOS?
A deep unwavering belief is a sure sign you're missing something...
Moderators: when the guy SHOUTS 'IRRELIVANT' he is doing it to piss people off. This is ill-informed flamebait.
A render farm *does* require these things of it's OS:
-
Reliability. if you spend half your time crashing, rebooting, and redoing work that has been lost, you need twice as much hardware, at twice the cost, to get the same amount of work done.
-
Low memory footprint. Rendering is very memory intensive. The less space that is wasted by the OS, the faster you go.
-
Runs on the hardware that you have decided it approprite to your requirements.
-
A nice, friendly, productive operating environment. Remember, many of these animators will be happiest telling the render machines what to do in a UNIX environment, coming from an IRIX background.
-
An OS that keeps out of your way. Eg how much time does your OS spend sitting in the scheduler?
Sorry if I get a bit flamey, but this post is just *SO* overrated.This just in!
DVD-CCA has is suing the makers of "Lord of the Rings" movie for using Linux. One of the lawyers for the organization says that "Anyone who uses that illegal and immoral open source software diserves to get sued". Moments later the lawyer was seen setting old ladies on fire and pushing them into oncoming traffic while chanting satanic messages.
Buying a Dell computer is equivalent to dropping the soap in a prison shower.
Frodo: "Aww, crap! Not another Nazgul!"
Samwise: "Umm... Hey, Fro, that doesn't look much like a Nazgul to me!"
Gollum: "Then what the hell issss it?"
Samwise: (squinting) "Uh, I think it's a... it's a penguin!"
Frodo: "WTF!!?!?! There are no frickin penguins in Middle Earth!"
Gollum: "It'sss thosssse damn foolsss in ssspecial effectsss! They've ssscrewed up our precioussss movie, they did, they did!"
Frodo: "Hey, penguin! Get the hell out of here! We're trying to make a movie, dammit!"
Gollum: "Curssse thossse Open Ssssource bassstardssss! Thisss really pisssesss me off!"
Samwise: (Squints again) "Oh, never mind... it looks like it is a Nazgul after all! My bad!"
Frodo: (Smacks Samwise) "Dumbass!! Hobbits never, ever say 'My bad!'"
(Looks over at Gollum) "And what are you looking at, you shriveley little gimp?"
Gollum: "That'sss it! Ssscrew you guyssss; I'm going home!"
THE END
--
while ( !universe->perfect() ) {
hack (reality);
--
while ( !universe->perfect() ) {
hack (reality);
}
Maybe 50% or more of DVD players sold in the UK have region hacks as well. Mine does.
Pfft, CGI is for losers. They should be using Jim Henson's Muppets for LotR!
Right. What we have here is a truly heterogeneous network, with latencies you cannot calculate, as well as complete unreliability as to the availability and processing power of a node. The best use of such processing network is when you have something like SETI@home: Massive local computations on very little transfer of data across the network, and the ability to slice the processing needs into minuscule chunks so you can easily duplicate processing of those chunks in case a node goes down. Distributed internet processing is excellent when you want to solve problems with lots of computations for simple answers, much harder when you want to produce large output.
"Linux is not bleeding edge stuff", which I guess means that Linux must be mainstream, and also:
"We can't afford for the system to go down" - Why didn't they use Windows 2000 then - it's the "most stable windows yet".
Oh, and I submitted this several days ago, but no-one was interested then.
2000-08-22 09:28:12 Lord of the Rings to save money using GNU/Linux (articles,movies) (rejected)