Supercruncher Applications

starheight writes "Bill McColl has written an article contrasting traditional massively parallel supercomputing with a whole new generation of compute-intensive apps that require massively scalable architectures and can deliver both incredible throughput and real-time responsivenes when processing millions or billions of tasks."

Comments

Wow, can you imagine a Beowulf cluster of these?

[$RANDOMLUSER]

Looking at his examples (Search, Ecommerce, Software-as-a-Service, Infrastructure-as-a-Service, Fraud Detection) I have to think "wow, single point of failure". Lots and lots of fault-tolerance needed to put all your eggs in one basket like that.

Re:Massively parallel?!

[guaigean]

In most cases, researchers request a specific number of cores, based on experience of how well their code scales. Some codes to auto-scale, depending on available cores, but these are rarer. The way it works is in a batch queue system... Users submit a job required 2000 cores, and wait until that many are available. Then, when the cores become available, their job runs for 6-48hrs or more, depending on the job. In most cases, a large number of researchers are often in contention for computing time, and wait their turn in line. The good ones tend to understand the system better, and will submit workloads that reflect the current available resources, thus limiting the time their work spends sitting in the queue.