disCERNing Data Analysis

technodummy writes: "Wired is reporting how CERN is driving the Linux-based, EU funded, DataGRID project. And no, they say, it's nothing like Seti@Home. The description on the site of the project is: ' The objective is to enable next generation scientific exploration which requires intensive computation and analysis of shared large-scale databases, from hundreds of TeraBytes to PetaBytes, across widely distributed scientific communities.'" If you're interested in this, check out the Fermi Lab work with LinuxNetworkX data as well as the all-powerful Google search on the Fermi Collider Linux project. As jamie points out, "Colliders produce *amazing* amounts of data in *amazingly* short time periods... on the order of "here's a gigabyte, you have 10 milliseconds to pull whatever's valuable out of it before the next gigabyte arrives".

Comments

Storage to the rescue

[KarmaBlackballed]

"here's a gigabyte, you have 10 milliseconds to pull whatever's valuable out of it before the next gigabyte arrives".

...or just write it all as it comes in and analyze it later. That's how most other science takes place. Since when is scientific analysis "real-time?"

In general, the scientific process does not require conclusions during an experiment. I think CERN should cite a different reason for this project, there are many valid ones.

Re:Storage to the rescue

[tr1n0]

40 million _events_ per sec, actually; multiplied by about 20 collisions per event gives almost a billion collisions per sec. Which results in a primary data rate ( before low level triggering ) of about 100 Terabyte - 1 Petabyte per sec.

Grid computing?

[Exmet Paff Daxx]

Taking a look at Gridcomputing, it's pretty clear to see that Grid Computing is actually... Distributed Computing. There's no new concept here - so why the new name? It doesn't make sense until you read the sound bite: "I believe grid computing will revolutionize the way we compute".

Yes, if you can't invent an idea, rename it, and maybe you'll get some credit. What the hell, it's worked before.

Oh well. More power to them. It looks like a great opportunity for the world to learn that Linux is a powerful tool.