Slashdot Mirror
Inside the Lucasfilm datacenter
passthecrackpipe writes "Where can you find a (rhetorical) 11.38 petabits per second bandwidth? It appears to be inside the Lucasfilm Datacenter. At least, that is the headline figure mentioned in this report on a tour of the datacenter. The story is a bit light on the down-and-dirty details, but mentions a 10 gig ethernet backbone (adding up the bandwidth of a load of network connections seems to be how they derived the 11.38 petabits p/s figure. In that case, I have a 45 gig network at home.) Power utilization is a key differentiator when buying hardware, a "legacy" cycle of a couple of months, and 300TB of storage in a 10.000 square foot datacenter. To me, the story comes across as somewhat hyped up — "look at us, we have a large datacenter" kind of thing, "look how cool we are". Over the last couple of years, I have been in many datacenters, for banks, pharma and large enterprise to name a few, that have somewhat larger and more complex setups."
Only a few boxen are used rendering and effects. The rest is to track and calculate sales of Star Wars merchandise.
There are many corporate data centers larger and more powerful than that, it is much more impressive if the entire thing can run one giant application. Still, I'm pretty sure that Google's new datacenter wipes its ass with a datacenter the size of this one.
stuff |
San Diego Super Computing Center (SDSC) has 2 Petabytes of online Storage with 400TB for researchers. They have 18PB of archival tape storage.
a tures/print.php/3634881
Still....I like datacenters. The hum of equipment. 65 degree temps and lower. I once had my cube re-located to a tape library. Quiet...peaceful place
http://www.enterprisestorageforum.com/hardware/fe
Is that the speed you can talk at?
...would post this as a news item. Front page, too.
Let's break this down submission down..
"Hi. I found this article on the web that totally didn't impress me, I think they fiddled with the numbers to make themselves look better than they are, and overall I really couldn't give a shite."
Yes. Obvious front page material for a Sunday!
Nope. They run LucasOS. It's perfect for their needs, since it's constantly being updated to suit his vision.
This guy's the limit!
Well passthecrackpipe, if you and your vast knowledge of large scale datacenters are not impressed with the story, why the hell did you submit it?
10.000 square feet for a datacenter is not very impressive. The datacenter that I work in did a relatively modest 100,000 square foot EXPANSION which was the result of absorbing an adjoining atrium. I suspect that the power equipment and air handlers may take up 10,000 square feet.
and format it?
No folly is more costly than the folly of intolerant idealism. - Winston Churchill
So why submit this if you don't like it? Why not at least title it "Lucasfilm thinks it's soooo great."? I'm sure you've seen bigger data centers, and you can type 500 lines of code a minute, and maybe you defeated a ninja in hand-to-hand combat, but for the rest of us "normal" nerds it's still neat to read about the machines that get the work done in a business. Of course it's hyped up, it's a press release disguised as news. Take it for what it is, relax, and try to imagine those 2,000 servers in a secret cave under your house, manipulating the stock market in your favor. That's what I do.
Why all the negativity toward Lucas? Jar Jar's dead man, let it go. George said he was sorry already. I think it's a good story. It's absolutely fascinating to me to see how they make movies today, how much data gets pushed around, and how they make sure that the creative people have access to what they need, when they need it. And they do all this to support incredible time schedules, with boatloads of cash riding on every second. I don't know how anyone can say that this isn't an impressive operation. As for Lucas thinking they are so great... well, they pretty much are. I'd say that being organization that created the special effects for tons of blockbuster movies and being nominated for several major movie industry awards pretty much gives them some bragging rights.
300TB storage and 11 petabits/s bandwidth.
This means
A) they can push their entire storage through the network in 300*8Tb/(11Pb/s)=200ms.
or
B) the article author does not have a clue.
I think an anlogy would be: I drive back and forth to work everyday, or 400 times a year. My speed on each trip is 60mph, so in a year my speed is 60x400 or 24000mph.
don't cut it off www.mgmbill.org
As in reference to THX 1138?
Of course, it could just be a coincidence.
I've got a fever and the only prescription is more COBOL.
I have to wonder how many systems they have? They accomplish a great deal with what is a fairly small area. I would guess that they each computer has major ram and is simply NFSed back to a central server.
What I have found funny is the number of ppl who are speaking of how big their centers. Offhand, I tend to suspect that those centers could go on a MAJOR f%^&ing diet and need to have their budgets cut to a fifth. And finally, it is time to fire a bunch of the incompetents who can not run a tight center.
I prefer the "u" in honour as it seems to be missing these days.
...running FC6 x64.
Why? Because my rig has never so much as contained - much less rendered - an image of Jar Jar Binks.
Pwned.
I am very small, utmostly microscopic.
The datacenter at one of my employer's satellite sites has four CLARiiONs, at 2 racks each, a 5-bay DMX-3, and a 4-bay XP1024, for 380TB raw, in 3,200 sqft, along with thirty racks of servers, a P595 mainframe, and several multi-rack computing clusters. There's plenty of cooling and it's really not THAT crowded. Managing to pack 10-12 racks of storage into a 10,000 sqft data center is not anything noteworthy.
This is the Lucasfilm datacenter. That number finds its way into all sorts of Lucas-related material.
They wanted me to move across the continent from a place with average cost of living and a 10 minute commute to work in San Francisco (right in the city, not even an outlying area) for about a 15% increase in pay. The only way I could afford that would be to take on a 2-3 hour commute and even then I'd have to run an even tighter ship, financially speaking, than I do now.
I suppose they were counting on the "cool factor". The job was cool, but not so cool I was willing to stick a stake through the heart of my family. Right after this, I read that Lucas donates 170 million to his alma mater. Hey George, why not donate 10% less and actually pay your people something more since you're insisting on setting up shop right in the freaking Presidio?
600 Tbyte of disk in total can't be right. I wrote an application a couple years ago that has 6 terabytes of disk allocated to it to cache its work. This was for a single app. Admittedly, we worked with fairly big data files where I was working, but I've got to think Lucasfilm's files are way larger than my 1-2 gig files.
As somebody who (ab)uses that particular rig daily, the article misses the point about what's so awesome about the system.
It's a good sized datacenter, but what it's able to support in processing ability is the impressive part, and that the fat bandwidth runs at capacity almost all of the time by the demands of processing jobs. Proprietary software doles out jobs 24/7 to thousands of procs all over campus-- including artists' desktop machines-- for heavy duty computation: rendering and simulation and whatever it takes.
I can't imagine a facility where so many people are creating and pumping so much data around.
I toured their new facility in San Francisco. They have over 300 10Gbps ports and all PCs are connected via gigabit. Their datacenter was 2/3 full of dual-Opteron servers running SuSE Linux (though they were considering switching). Their server room was spotless. No cables were visible anywhere, but I did see a Roomba moving about the floor. The fellow who ran it said that since they're ILM, they have to have droids.
The facility was absolutely beautiful. When going between two buildings on an overhead walkway I saw the Golden Gate bridge with a nice orange sunset behind it. I wish I had my camera with me.
They said that they have many dedicated OC-48 pipes to various studios and can handle just about any format, since every studio uses their own format. They convert it to their own internal format, which I believe they open sourced.
When they moved from Skywalker Ranch, it was completely seamless. They had an OC-192 (10gbps) link running between the old and new facility as more and more equipment was migrated to the new facility but people continued to work at the old one.
-Aaron
This post is encrypted twice with ROT-13. Documenting or attempting to crack this encryption is illegal.
They're certainly living a long, long time ago.
Nobody else has this sig.
TFA talks about 2000 servers equipped with 10 Gbps network cards.
11.38 Pbps is 11380 Tbps or 11380000 Gbps. This means that each
server has 569 network interfaces !! This is total bullshit. If
they had said they had 10*2000*2 = 40 Tbps, it would have been
based on more real (though irrelevant) data.
I hate it when ignorant journalists post meaningless data for public
consumption.
Willy
Hopefully its security is better than that of the Death Star...
Insert something witty here...