Slashdot Mirror
ILM's Datacenter
kylegordon writes "CGW has inside scoop on Industrial Light and Magic's facilities after they moved from San Rafeal to San Franciscos Presidio. With 3000 disks, it can shift 170Tb to 5000 rendernodes over 10GbE and 1GbE network links. It's an impressive system, for impressive films."
I bet I could make a graph that represents how the quality of movies is characteristically inversely proportionate to the amount of CGI effects in them. Oftentimes, eye candy is used to shroud the plot and mask the bad acting/directing. American audiences especially just go looking for explosion sequences and CGI in the annual summer action flick hunt. We often fear a movie that might prove to be too cerebral and that pretty much disgusts me. Way to reinforce bad movies that are only good for one viewing with volume set to 'loud' and TV set to 'huge.'
ILM is responsible for making movies like The Mask (of which there are seven films) and characters like Jar-Jar Binks possible. Be sure to thank them for that.
My work here is dung.
Now they can destory our childhoods with just computers.
They seem to have a really nice set up there, I would be curious about how their 'hybrid' NAS/SAN benchmarks, and see some comparisions against some of the big boy equipment like IBM sharks.
GeekServ Unix Consulting Services (http://www.geekserv.com)
Plus, it can hold a decent percentage of my pr0n collection... well, the JPGs anyway.
My name is Wootzor von Leetenhaxor
Way to go, guys! Who would have known that a small startup from Pittsburgh with some killer engineers, could make it into ILM's datacenter. Hi, Gus!
Zhrodague.net - I do projects and stuff too.
According to Mapquest a trip from San Rafeal to San Francisco would take about 35 minutes (Est. Distance: 21.06 miles). Therefore, if I loaded up all 170TB on a truck my effective bandwidth would be about 3.06e28 bps (or roughly 3e16 Tbps). Once again for huge data repositories there is no substitute for shipping physical media.
The more you regulate a company, the worse its products become.
zzzzzzz
-------- In Soviet Russia, "Soviet Russia" sigs hate Slashdot.
LucasFilm hasn't made an impressive film since Episode IV the original.
Had to say it.
Okay... not everything they do is shit. That, and CG doesn't make movies worse... only if it sucks. You can go watch claymation if you would like.
http://www.ilm.com/ilm_services.html
Look at all they have done. While some of the stuff on there may have sucked... there is some really fucking good stuff on there.
Also, if I remember correctly, they were some of the first to experiment with particle renders for CG (they used it in the Mask to create some of the storm/tornado transformations). Anyways... thats all aside from the point
Hey... more power to em. They get cooler stuff, they make more realistic CGs. And when all you nay-sayers are watching a movie, and don't notice a good CG... it has worked, and they have won. Don't fight CG now, soon it will just look like everything else.
...
The new ILM DataCenter looks alot like the one at Trinity College in Dublin, Ireland.
He who knows best knows how little he knows. - Thomas Jefferson
Nice setup you have there ILM. Its a shame if something should happen to it ;)
"Python plays a key role in our production pipeline. Without it a project the size of Star Wars: Episode II would have been very difficult to pull off. From crowd rendering to batch processing to compositing, Python binds all things together," said Tommy Burnette, Senior Technical Director, Industrial Light & Magic.
"Python is everywhere at ILM. It's used to extend the capabilities of our applications, as well as providing the glue between them. Every CG image we create has involved Python somewhere in the process," said Philip Peterson, Principal Engineer, Research & Development, Industrial Light & Magic.
Python Quotes
Jar Jar Binks wassa born here, meesa know!
Or put another way, it's not size that's impressive, but what you do with it.
It isn't that fucking good yet, so get off your high horse.
Blar.
It's an impressive system, for impressive films.
If it's for impressive films, why are they using it for soulless dreck? Some sort of beta testing period maybe?
It's an impressive system, for impressive films.
Thank you, I will decide what's impressive. This is like when vendors tell me that their product is "really cool" or "great".... tell me what it DOES, SHOW ME, then I will decide if its "cool" or of value to me.
I want to delete my account but Slashdot doesn't allow it.
I bet I could make a graph that represents how the quality of movies is characteristically inversely proportionate to the amount of CGI effects in them. Oftentimes, eye candy is used to shroud the plot and mask the bad acting/directing. American audiences especially just go looking for explosion sequences and CGI in the annual summer action flick hunt. We often fear a movie that might prove to be too cerebral and that pretty much disgusts me.
OK, here's the thing. Movies that are "cerebral" and thought-provoking don't draw people to theaters. I see previews for a movie like "Sixth Sense" or "Se7en," and I say to myself, "renter." Studios make more money by drawing us to the theater, and the way they do that is by making movies that benefit most from the big-screen, big-sound environment of your local megaplex. Usually, that means "action flick."
And it turns out they're right. Look at the biggest moneymakers. They're not the "Good Will Huntings" and the "Brokeback Mountains." The summer blockbusters are the "Spider-mans" and "X-Men" and "Independence Day" and "Star Wars."
Regarding your other point, about using CGI to mask bad directing, etc., I can only half-agree. While movies like "Star Wars: The Phantom Menace" certainly used CGI as a crutch, don't you agree that at least the CGI was good? Don't you agree it would have been an even worse movie if the CGI had been terrible? Look at "Air Force One." Decent movie, spoiled by absolutely terrible special effects at the end. Or how about "King Kong." Great movie, that benefitted from good CGI. Same with "Titanic."
Good CGI can make a terrible film bearable, or a good film great.
Like woodworking? Build your own picture frames.
and that movie did not suck...most of the acting was quite good.
impressive. most impressive.
I ALWAYS notice CGI.
No you don't. You think you do, but you don't. When you do notice it, you point it out and say to yourself, "that was so obvious, CGI sucks." But when you don't notice it, you don't realize that what you're looking at is CGI. You think it's real. You think the man really has had his legs amputated ("Forrest Gump") or Arnie really did jump his motorcycle off a 15 foot ramp ("Terminator 2"). CGI is used all over the place in movies now, not just for the big explosions that still may not look 100% convincing (however, it's much better than stop-motion animation).
Like woodworking? Build your own picture frames.
Still, there was Jurassic Park, which had that wow effect, but only in a suburban, sterilized kind of way. Maybe it was just the seventies with their more "adult", less megaplexy, disney-fashioned attitudes. The one company that embodies that spirit of combining new tech with a fresh attitude is obviously Pixar. They still have their mojo intact. We also have tons of great political films, clever films and documentaries coming out nowadays, so there is no reason to despair in the times per se, I was just thinking of ILM specifically. Maybe George will lose that weird suburban Disneyfied taste of his at some point and get back to some Thx1138 type goodness.
I think that those people fucking leave almost instantly. There are a couple of small companies and startups, but I think it is more difficult to make it as a tech business here in Pittsburgh, than in places like San Francisco, or Boston -- even though there's a whole hell of a lot more competition. There are a few standouts, and it is easier to stand out here, but people are generally less receptive to new ideas.
I met lots of CMU people in Boston, though.
Zhrodague.net - I do projects and stuff too.
am i the only person who thinks its funny that there is quite a few machinima (game engine rendered movies) that are of better film quality than anything ILM makes on their billion dollar systems?
"And when all you nay-sayers are watching a movie, and don't notice a good CG... it has worked, and they have won. Don't fight CG now, soon it will just look like everything else."
It doesn't matter because according to slashdot ALL movies suck, and we've boycotted the MPAA, because...you know, we have principles.
Nice to see that ILM had the sense to use Ethernet rather than InfiniBand. IB has some great features in hardware. Too bad the software to drive it is less than wonderful.
-- Sometimes you have to turn the lights off in order to see.
More like an impressive system, for disappointing films.
Massively parallelized Markov chains!
Why have 5000 render nodes when you could virtualise with 250 physical processors running 20 apiece?
now Lucas can rape my childhood memories faster than ever.
While I'm by no means and engineer and my evidence is strictly anecdotal, I know of at least a few hundred bright people from that area (I hail from about 40 miles east) that headed to either coast in search of better/more abundant jobs.
More specifically, the Johnstown area, has suffered greatly from brain drain as there just isn't anything there (aside from family, which, thanks to technology, is much easier to keep in touch with these days) to keep young people from getting out at the first chance they get.
Believe me, I keep my eyes peeled, because it is such a nice area to live in with relatively low cost of living, but there just aren't as many opportunities these days as there were when our parents were entering the job market. I'd gladly move back there, and take up to a 10% pay cut, just to get away from the rat race and all the other crap associated with big market living.
Bashing Lucas is soooooooooooo tired and 1999. "ZOMG!!1! Teh LucASS r4pZ0r3d my k1dd13h00d!!! H4XX!!!11!ELEVEN!1!" I guess people need to do something when they aren't dry-humping their Tux dolls.
I suspect your inverse relationship theory will be pushed to the limit, as they push this new datacenter to its limit in Stealth II.
quality ~ 1/cgi
Anybody that saw the first installment of Stealth should understand where I'm coming from.
Skycaptain is another one in this category...
Way to go ILM! I want my money back!!!
Comic Book Guy's Voice: "Worst. Movies. Ever."
-@
Move all sig!
Does it run Linux?
That's San Rafael, not San Rafeal. A sidenote on place names: Skywalker Ranch was located in Lucas Valley, but the valley had its name long before George located there.
ZERO
ILM does not make movies. They providce special effects for movies made by completely separate creative entities. And they're the best in the business.
Perhaps ILM's greatest strength is that they try to minimize the amount ot CGI they use. When a director says, "Then the spaceship crashes, the hero gets out, morphs into HairyBeast(TM), and fights the Mole People(TM)," probably the only pure CGI will be the morphing. The spaceship will be a model; the explosion will be blended from a real explosion; HairyBeast(TM) and the Mole People(TM) will be actors in well-designed, partially animatronic suits.
Look at everyone's favourite target: Star Wars III. How many pure CGI characters were there? Yoda. Obi-Wan's ride. That's about it. Everything else was primarily actors in suits or animatronics; whatever looked the best, not whatever was cheapest. Even Jar-Jar (who was created by George Lucas, not by ILM) was an actor in a suit, enhanced with a CGI head
You want to blame someone for the generally poor quality of movies nowadays, don't blame ILM; blame Disney. Why? Why not? They're as good a blame target as anyone.
Find environmentally and socially responsible products on http://buy-right.net
Imagine a Beowulf cluster of these things.
Against stupidity the Gods themselves contend in vain.
I wonder if they are using NICs that have offloading capabilities in the compute nodes. InfiniBand, Myrinet, and iWarp NICs are all designed to get rid of TCP/IP processing. One would think with relativly large data sets, TCP could be a big CPU consumer. Also, standard NICs using TCP have horrible latency compared to InfinBand, Myrinet, etc. That latency really eats up cluster performance when the nodes all wait for something (like new data, results, etc.) In lots of high performance computing applications latency matters more than almost anything else.
On the spinnaker side, I wonder if this type of solution (clustered NAS) competes well with clustered file systems where the compute nodes are the storage nodes as well. It eats up CPU for running the clustered file system but it puts the compute nodes right on the SAN without having to go through a NAS head.
-Ack
-- soldack
Infiniband is great till you try to find drivers or use up the onboard memory
Cliff Claven
K.E.G. Party Chairman
Founding Leader of: Koncerned for Egalitarin Governance
Summary is off by a factor of 8.
"The crows seemed to be calling his name, thought Caw."
The Mask is one of my favourite movies. The acting and the plot may be poor (though I like the good/bad girl twist), but I thought it was very funny. I like funny movies. I didn't like the new Star Wars stuff at all. Maybe ILM is best at slapstick humour movies.
I believe the NetApps do, in fact, run Linux.
Zhrodague.net - I do projects and stuff too.
I wonder how many pirated movies they have lying around on all those disks...
ILM doesn't make movies. They only do special effects shots. Movie studios hire ILM to add effects shots into the movies. What they do with them, and how they cast/write/act/script the movies is really out of ILM's hands.
ILM used to have a digital movie group. Steve Jobs bought it, and it's now called pixar, err... Disney.
It's nice but can it run "The Force"??
Hey, all three of the IB HCAs have OpenIB drivers now.
I can't imagine a Beowulf cluster of ILM's facilities, sadly.
There you are, staring at me again.
I tend to think that the problem with CGI from a creative standpoint is that it's a relatively new tool - there's a certain novelty factor attached to it (filmmakers can create effects that would be difficult or impossible before for various reasons) and in the creative sense it hasn't quite matured, I think.
The problem IMO is that it's overused. It's opened the floodgates, so that all kinds of movies can have all kinds of effects that might have been too costly to justify before - the problem is the decisions on when to use and when not to use these effects haven't reached a point where they're made for the right reasons. If an effect doesn't look good enough to be in the forefront of a film, it shouldn't be. That doesn't mean the effect has no value as a background piece. It's the same way with a costume or prop - if it doesn't look good enough when filmed up close, don't film it up close. I think in particular the "rag-doll" effects when a character gets thrown around tend to fall into this category.
I have a bit of a sentimental stake in this, myself - as someone who spends a great deal of time building models, I mourn the fact that the role of physical models in films has been diminshed. But even in the "good old days" the situation wasn't perfect... just look at all the abuse the Starship Enterprise model went through in the movies - having that beautiful paintjob, which was promptly ruined for the second film (I'm referring to the dullcoat, though the botched battle damage certainly played a part, too), abused in the fifth film, etc...
---GEC
I'm but the humble pupil, seeking to snatch the scratchbuilt pebble from the master's fully articulated hand
What use is InfiniBand /without/ drivers? At least for Linux, the current driver release is scary, and the 2nd generation, isn't 'officially' released yet-- after how many YEARS of IB hardware being sold?
And it's not like the story is any rosier for any of the other OSen; there just isn't a decent set of IB drivers.
I put in 4 years working on IB for SilverStorm so I have been doing IB since before there were any ASICs or even a 1.0 spec. Can you be more specific? What kind of problems do you have with IB software and which IB software are you talking about? OpenIB? VendorX's proprietary stuff? Which HCA (Mellanox, Pathscale, IBM, etc)? Which switching (Voltaire, SilverStorm, Cisco, etc.)? What kind of setup were you doing (big HPC cluster or little database)? I not looking for flame war, I am genuinely curious about what you didn't like about the software. I promise, no angry words even if you knock the stuff I used to work on!
-Ack
-- soldack
From TFA:
... oh wait
"Data access over fiber between San Rafael and San Francisco was very fast, but when you're shooting packets to Singapore and introducing millisecond delays, the computers start bogging down," says Thompson. "It's not the throughput; it's the round-trip time. We're looking at Network Appliance, Hewlett-Packard, and a lot of start-up companies that deal with these WAN issues for a solution."
A solution for round trip time to the far side of the planet? So he's looking for packets that travel faster then the speed of light? What is this, a Sci-Fi reality?
Even by mimimizing the hops (less router jumps) you are still going to hit the hard limit of "the speed of light"... maybe these guys haven't heard of this... they could be thinking about maybe piggybacking their packets onto subspace radio waves...or reverse polarized tachyon pulses... or maybe 50,000 CD's in a freighter which can make the trip in less than 2 parsecs...
wbs.
Huh?
Alexa is still in the Presidio, though the data center recently moved. They pull 1TB a day from the web and manage over 300TB of data stores.