IBM & CERN openlab for DataGrid Applications

Jules V.D. writes "CERN and IBM today announced that IBM is joining the CERN openlab for DataGrid applications to collaborate in creating a massive data-management system built on Grid computing.IBM's innovative storage virtualization and file management technology, will play a pivotal role in this collaboration, which aims to create a data file system far larger than exists today to help scientists at CERN understand some of the most fundamental questions about the nature of matter and the Universe."

Comments

Distributed networking

[CrazyJim0]

Excellent.

Much like the appeal of Seti at home was searching for AI... People now have a choice which distributed net they want to support.

Its a system, similar to voting, that will have every distributed net in the future trying to please.

I forsee distributed nets of the future attempting to produce results, in order to keep people interested and donating their computer cycles.

Its an interesting system, that works a bit like deomocracy.

Re:Distributed networking

[k-0s]

I was going to say this is nothing like democracy but then I noticed key word in some of your phrases in your reply such as:

"in the future trying to please"

or

"I forsee distributed nets of the future attempting to produce results"

and

"to keep people interested and donating"

then it all made sense.

Re:Distributed networking

[PSC]

I forsee distributed nets of the future attempting to produce results, in order to keep people interested and donating their computer cycles.

Ain't gonna happen. Unlike seti@home, the data per CPU minute throughput in high energy physics (aka particle physics) is much higher.

As an order of magnitude, simulating a single collision of the upcoming LHC's proton-proton beam (Large Hadron Collider is CERN's upcoming accelerator, supposed to start 2007/2008), takes about 1 CPU minute and generates ca 1 MB of data... that's 16 kB/s to send from your PC to CERN. Easy going for ethernet, but no fun with Modem, ISDN, or even DSL.

And consider that this is a high-CPU job. Most jobs in particle physics require way more bandwidth. Especially searches for (new or known) particles requires you to loop over huge data sets (some Terabytes for sure).

The Grid computing ansatz in high energy physics is intended for relatively few local computing centers (about one per country; Germany's center is here at Karlsruhe), interconnected with multi-Mbit links. (Our institute has a nice GigE link to the German Grid center :-)

after Deep Blue comes Deep Thought

[ElJosho]

Finally, IBM is building a computer powerful enought to answer the ultimate question of Life, the Universe, and Everything.

The ultimate question

[manseman]

some of the most fundamental questions about the nature of matter and the Universe.

I'll save them the effort.

42.

Re:The ultimate question

[cperciva]

No, you've got is wrong: They're trying to understand the questions, not the answers.

Re:The ultimate question

[Fjornir]

...I thought the question and the answer couldn't exist in the same universe... Or maybe that understanding came from an alternate me in an alternate universe where "what do you get if you multiply six by nine" is forty-two.

More on-topic though, I am utterly boggled by the amount of data they're looking at holding. I can see a stack of CDs and know they're a terrabyte worth of data -- but I barely grok what a terrabyte can really hold -- maybe I don't and just think I do.

But a petabyte. Wow. 1.5 million CDs. That's just... Just... *shrug*

Re:The ultimate question

[cperciva]

But a petabyte. Wow. 1.5 million CDs. That's just... Just... *shrug*

Think of it as being about a quarter of a Google.

(I don't know exactly how big Google is now, but they were at 1.5PB a couple years ago, so they're probably somewhere around 4PB now.)

One of the coolest places

[Anonymous Coward]

CERN is definitely one of the coolest places on Earth. For a bid a couple of years ago, I had to do some reasearch regarding their storage requirements and data management facilities. These people produce 10 *Peta*bytes* of data per year. For us mortals to understand that number, it's got to be converted to gigabytes per second: 300+ GB/s. On the basement we've got a 2 TB RAID. The people at CERN fill that baby in less than 7 seconds. No, scratch that. Our RAID (dual fibrechannel 10000 rpm SCSI discs) tops at 120 MB/s. See what I mean? Just try to grasp the kind of SAN these people have in order to move 300 GB/s arround.

Filling application right away.

Re:One of the coolest places

[Anonymous Coward]

here. Seems they move 300 Gb/s not 300 GB/s arround. Still impressive.

Re:One of the coolest places

[cperciva]

10 PB/year = 10^16 B/year = 2.74x10^13 B/day = 1.14x10^12 B/hour = 317 MB/s.

You're off by a factor of 1000.

Re:One of the coolest places

[Lindril]

Are you the same guy that's trying to sue those college kids for $97 trillion dollars?

Re:Isn't IBM's Grid PS3 technology?

[SirDaShadow]

I goofed. I was thinking of Cell technology...damn sundays :)

Specifically. . .

[Fritz Benwalla]

This system stores, crunches, and distributes data generated by the Large Hadron Collider. They generate a million gig a year in data, and need to make it available in some functional way to physicists. Manditory groovy collider pic here.

A major collaborator on this stuff is Globus which provides an API for grid applications. Same people who are partners with IBM in the butterfly.net game grid.

Maybe MTU can use it to store their students' Kazaa archives.

------

Re:Specifically. . .

[Dan Yocum]

You really meant to say it *will* store, crunch, and distribute LHC data, since the LHC ain't done, yet.

Re:Hmm - looks like luck CERN is not U. S. based..

[Blaskowicz]

Ha! everything might be used to infringe copyright laws : floppies, CD-R, paper, analog tapes...

RIAA&MPAA can sue my ass. It's able to infringe a good deal of the music industry.

Great!

[stephenry]

1 Petabyte! Thats a lotta porn!

Re:Hmm - looks like luck CERN is not U. S. based..

[paja]

...and they are used to bring them more money - CD-R, CD Writers, xerox machines - they sue them all. Remember the hysteria around videotapes?

I think someone will create post-p2p device/protocol/software, which will use some of the grid computing and grid storage technology to share not only data, but to sell Your processing power, storage or data via any kind of internetwork. And as the GSM providers are able to crate CUGs. there will be some kind of closed user groups, which will trade data, they do not own.

The question is, if these organisations like MPAA, RIAA will have enough power to terminate any research before it proves its benefits.

Here are some links..

[abhisarda]

More about IBM and Cern- Gridcomputingplanet

Cern and Java- Vnunet

More about Cern-Hepwww
The Large Electron Positron Collider at Cern-Hepwww

Re:Grids suck

[Anonymous Coward]

Grids are intended for cpu and data-storage sharing, which means that all institutions contributing in a grid can, according to the permissions set, use the systems in the grid. Not all scientists working on the data at Cern can aford to be there, to have all the data on local storate, always, and to convincingly get Cern's computing farms dedicated to their calculations.

Grids make more power and more storate awailable to more people.

What does suck about grids is complexity, lack of good software supporting grids and, perhaps, latency.

Re:!!!BOYCOTT CERN!!! !!!BOYCOTT CERN!!! !!!BOYCOT

[Spellbinder]

do your homework
20 member states running it form all over Europe