Shell and the World's largest Linux Supercomputer

thefullmonty writes "Yahoo! News is reporting that Royal Dutch/Shell is going to install the world's largest Linux supercomputer. Shell's Exploration & Production unit will use the supercomputer, consisting of 1,024 IBM X-Series servers, to run seismic and other geophysical applications in its search for more oil and gas. The article goes on to talk about how larger companies are moving towards Linux and some of the advantages of making such a move. ."

Comments

Re:TCO

[Ed Avis]

It's not cost, it's *hassle*. If you want to plug in an extra node, you just do it. No need to phone up (insert software vendor) to buy another 'Flexible Licensing Extension Pack' or to sign another contract. And no need to worry about random audits and threats from anti-piracy organizations.

Neither machine is the largest Linux Cluster

[Anonymous Coward]

There are other clusters that have similar numbers of nodes. Down at Sandia National Labs, they have Cplant Antarctic. It is a Linux cluster of > 1024 nodes and growing. The nodes are Compaq DS10s, which are faster than the Intel nodes the article above talks about. See http://www.sandia.gov/LabNews/LN08-11-00/cplant_st ory.html

Farting in a tin can

[Graymalkin]

Is this REALLY Shell deciding Linux is the end all be all of high powered computing? Or is it just cost effectiveness? Think about it a fucking second, this bad boy is being provided by IBM, IBM also has a little product they like to call AIX. AIX as opposed to Linux has quite the pricetag. IBM has a pretty cheap charge attached to sticking Linux on their big iron. If you want a new big iron box do you really want to shell (teehee) out the big bucks for hardware and high priced OS licenses? No fucking way. I really doubt this has anything to really do with Linux being the best thing since sliced bread and has more to do with it being cheap and working well enough to get the job done.

Re:Another solid gain for linux

[GypC]

but you're just spouting off inaccurate Linux propaganda here

I don't even use Linux. And I was talking about Windows 9x which, believe it or not, is still a very popular workstation.

"Free your mind and your ass will follow"

Re:Penguins Trapped in OIL

[Stephen Samuel]

It's one thing to find oil to run (and make) computers. It's another thing, entirely, to kill people who complain that your methods of exploration and extraction are destroying their environment.

From their activities in Nigeria (and, I'm sure, elsewhere) Shell is a terrorist organizaion. The question isn't whether you want oil, it's whether you want your oil supply controlled by a terrorist organization.

(just because they make a profit doesn't mean that they're not a terrorist org. Drug dealers and hitmen make a profit too.)
`ø,,ø`ø,,ø!

Re:cray os

[john@iastate.edu]

UNICOS

I wonder how these news are written...

[Ektanoor]

1024 computers - Interesting number. a new unit, the Penguin. One Penguin, two Penguins. Anyway M$ will never reach half Penguin...

According to the text we know know the place of Linus Torvalds in the development of Linux. It occurs he is just a "Fin" of the Penguin... And a group of volunteers on the Web helped him... Yeah Alan Cox upgrading a new patch through a web form...

Interesting licensed OS they found - UNIX. And it seems Linux is an alternative to it. I wonder how interesting is to be an alternative to itself...

By what math is this the largest Linux cluster?

[Troy Baer]

AFAIK, CPlant at Sandia is ~1600 Alpha nodes and counting. I think one of the genetic research companies has a 1000+ Intel node cluster (although it's used as a job farm rather than for parallel applications, IIRC).

--Troy

Re:By what math is this the largest Linux cluster?

[Greg Lindahl]

CPlant has 2400 nodes in New Mexico and something like 500 nodes in California. These are Alphas, so for floating point computations, they're 2x as fast as x86 cpus.

Incyte has 3,000 cpus in their genomics cluster. It runs embarrassingly parallel computations, but they're still parallel.

Re:By what math is this the largest Linux cluster?

[Paul Komarek]

By what math are Alphas 2x faster than Intel on the FP side, per clock? Last I checked, SpecFP was 3x faster, and my code moves about 5x faster. And if you can fit everything into the mammoth caches, you'll really move flat-out. (Yes, I'm an Alpha evangelist!)

I tested povray once on my AMD K6-2 450 versus an Alpha 21264 500 I have access to, on a scene that took my old 386 SX-20 a week to render (ah, the good old days). The Alpha finished *20* times faster than the AMD. I think the times were about 12 seconds versus less than 1/2 second, but I really don't remember.

Even if Intel's chips were twice as fast as the K6-2 chips, that's still 10x worse than the Alpha 21264. It is difficult to explain a 20x speedup. That's why we think it was a matter of cache. Also, the compilers we used, some version of egcc on the AMD and DEC's cc on the Alpha, would of course affect things. But egcs on the AMD is pretty good...

-Paul Komarek

Re:Have we created a monster?

[Anonymous Coward]

After Mr. AC posted this comment from his mostly-plastic encased PC, which runs on electricity from the (probably) coal-fired power plant, he put on his rubber-soled shoes and hopped in his gas burning, plastic-part-filled auto to hop down to the 7-11 to buy some more Jolt in the plastic bottles. He then returns to his (probably) gas-heated vinyl-clad home to take a hot (probably gas-heated) shower, the water of which probably passes through PVC to get to him.

Dude, unless you're willing to strike a deal with the Amish to let you into their fold, don't even sit there and preach bullsh*t about ethics.

Big Oil Companies(tm) are one of the places that Linux is *needed* as they are so very ingrained in the 'old ways' of doing things. Including software, and spending exhorbitant amounts of cash on damn near anything.

I for one applaud Shell for taking this route.

(posted anonymously since I'm a part of one of these Big Oil Companies(tm))

Penguins Trapped in OIL

[thinthief]

Okay, so the oil co.s can use The Penguin to find ways to destroy the habitats of penguins.

Already their company is more profitable

[bluelip]

Look how much they saved in licensing fees. That alone is reason enough to warrant the project. Talk about immediate ROI.

thanks for the boring article.

[garcia]

great news for the "movement" but an overall boring read.

basic information that we always hear, used here -- used there, developed by, seeing more use, coming of age, the end.

Re:Can you...

[cyber-vandal]

Rio Tinto Zinc, British Aerospace & McDonnell-Douglas for aiding and abetting the mass-murder in East Timor; Total/Unocal for (allegedly) taking advantage of forced labour provided by the Myanmar government; various international clothing companies for using companies that employ children in appalling conditions in sweatshop factories. The international corporates have a pretty disgraceful record when it comes to human rights abuses.

Yea, but will it...

[the_tsi]

...run Linux?

Oh, wait, yeah, I guess it will.

Doh.

Well, picture a Beow... err... nevermind.

-Chris
...More Powerful than Otto Preminger...

Re:Another solid gain for linux

[GypC]

Well, apparently what corporations look for in a desktop OS is something that is:

1. A buggy descendant of an 8 bit CP/M clone.
2. Superficially user-friendly but actually quite opaque.
3. Almost completely bereft of built-in functionality. No text filters, no remote backup, no shell to speak of, nada.
4. Unstable.
5. Heir to a huge collection of bloated and expensive software.
6. Perpetually increasing system requirements, requiring a hardware upgrade on the client side every 2 or 3 years.
7. The main supported OS for a buggy and again only superficially user-friendly office suite.

   So, no... I don't think we can convince a fortune 500 that Linux is a viable OS for use on the desktop.

"Free your mind and your ass will follow"

I know!

[talks_to_birds]

Let's digress for a moment and talk about how *crappy* the article on Yahoo! is..

I mean:

"Linux..., the free computer operating system..."

Free? Open-source, maybe, but without cost?

"Data collected in Shell exploration surveys will be fed into the computer, which will then analyze it."

And this is remarkable how? (I hope noone gets bitten when the computer gets fed...)

"Linux, developed by the Fin Linus Torvalds and a group of volunteers on the Web..."

Of course, there is *only* "the Web". The Internet is "the Web" -- I keep forgetting.

(hmm.. I thought most of the work on Linux was done through maillists, but I could be wrong..)

Anyway..

"...as a flexible alternative to licensed software systems such as Microsoft's Windows or the Unix platforms..."

uh.. Well. If you say so. It's on "the Web" so it must be true.

"...indicate that Linux usage is becoming more versatile, with the operating system moving into many different applications..."

An operating system "moving into applications" -- isn't this what got Micro$oft into trouble?

Or was that moving applications into the operating system? I forget..

Anyway, good job Yahoo!

t_t_b
--
I think not; therefore I ain't®

Re:TCO

[alienautopsy]

This is *not* about linux vs MS.

MS was never in the business of providing OS's for supercomputers.
This is linux vs commercial unices.

Forget TCO, think $/flop.
99% of seismic applications are "embarassingly parallel" - a natural fit for beowolf-type machines. You dont gain much from fancy architectures, so there's no point paying for it.

Does it say they're gonna use java?

[AntiPasto]

I heard of some gas stations watering-down their gasoline, but I didn't know they were going start putting coffee in my tank...

----

TCO

[bdavenport]

the one part that stood out to me was this line:

With Linux companies can quickly add or remove computers without worrying about licenses for the operating software.

i know it has been debated that some companies think MS is better for lower TCO b/c of their huge programming staff and vast CS minions, but it appears that Royal Dutch/Shell is leaning towards proving Linux is more cost effective. a step in the right direction for sure.

Re:odd

[dmatos]

Not a particularly hard claim by Avis. Given the amount of hot grits that get poured down her pants, there is bound to be some residue on the sheets.

Re:cray os

[fgodfrey]

The Cray vector systems back to the X-MP, I think, run something called Unicos. Before Unicos, there was something called COS that I know very little about. On the scalar machines, the T3D,based on Alpha ran almost no OS at all and required a C-90 as a front-end (that's one heck of a front end!). The T3E (also Alpha) ran Unicos/mk, a micro-kernel based version of Unicos (more like a complete re-write, but...). Basically, every processor ran a microkernel and then certain processors were responsible for taking care of the I/O and interactive jobs. The application processors used RPC's to the I/O processors to get data. The big difference between this and a cluster is that all memory on the system is visible to all the processors. I believe that jobs can migrate between CPU's on the T3E. The T3D and T3E are certainly not "clusters" in the sense of the cluster of individual boxes.

Re:What software?

[fgodfrey]

Well, the cluster is being installed by IBM, so I suspect they will supply the software shell wants, which is basically (probably) a compiler (probably Fortran) and an MPI or MPICH library. As for the exploration code, I suspect that Shell writes their own. Most really large supercomputer customers run their own code rather than canned software packages.