Slashdot Mirror
Building the World's 4th Fastest Supercomputer
ngkabra writes "In November 2007, a previously unheard of supercomputer called EKA, built by CRL, India came out of nowhere to become the 4th fastest supercomputer in the world. It is also the only supercomputer in the top 10 that hasn't taken any government funding — which means it has no strings attached against commercial exploitation. That is one of the reasons why Yahoo! chose EKA for the cloud computing research that they announced at the Hadoop Summit earlier this week. Yesterday, I attended a presentation by the team that built EKA, and they touched upon a lot of the technical details of EKA, and the challenges faced in designing and building it, which makes for interesting reading."
Well, interesting enough for me to read to the end of the article. The article, however, does seem to lack substance and details. It seemed to spend a lot of time apologising for inadequacies and seemed to aimed at a less technical audience. That is fine, writing to meet the target audiences needs or wants is good (I guess I just wasn't in the target audience.) The article raised more questions for me than it answered though (which, again, might be good--it will at least make me google for more information.)
India is now on track to earning the respect it deserves. Good for them! But I wonder what countries like Russia have hidden from the public. I am sure they have respectable hardware. Those very deadly [nuclear] weapons and missile systems must have had their design done on some pretty cool hardware.
Will tech support for this Indian computer be outsourced to Dell reps in Texas?
Airplane Photos, Airline News, Planespotting Guides
"Those very deadly [nuclear] weapons and missile systems must have had their _design_ done on some pretty cool hardware."
The quote at the bottom of the page-- "design, v.: What you regret not doing later on."
...to save you reading the article, it actually IS the worlds fastest super computer, but it's running Vista so now it only comes in 4th.
A Vista spec machine in non Governmental hands!
Imagine a beowulf cluster of those things!
I have known about EKA for years now. Being an Indian, I try to be aware of all the developments in IT in and around India, so I knew about this project when it was at it's earliest stages.
What most people don't know is that this project is funded by the Tata Group, and is directly influenced by TCS (Tata Consultancy Services), which I am strongly related to. The initial goal of this project was to surpass IBM's Blue Gene in the field of supercomputing, but then, things went wrong (as is usually the case with Indian projects), and it landed up in the 4th position, which I still feel, is quite a big feat for a projcet funded privately.
RutSum.com
Probably means no...
It's only paranoia if your wrong...
Its surprising that there is hardly any tech discussion on this topic. Best of all...not much India bashing.....wonder what happened to slashdotters...dumbfounded is it???
Why didn't this hit Slashdot sooner? Anyway, it seems to be very "green" as well. It looks as though there is only one computer more powerful than it that beats it on the Green500 complimentary list:
http://www.green500.org/lists/2008/02/ranks1-100.php
Insert self-referential sig here.
Let's start with the switches. You can parallelize network traffic - fragment the packets, stick full headers on each fragment, switch the fragments in parallel, reassemble, have the queue re-order inbound traffic by sequence number. (That last step isn't too hard, you have some fraction of the sequence number and the full fragment number map to a unique address in a permanently allocated ring buffer. Copy the payload to that address and the packets are in sequence order, not delivery order.) So, instead of having individual switches that are fast, bank the switches up and have the combined virtual switch run very fast. You can do that on exportable commodity hardware.
Or, you could sneak through Homeland Security (who have much more interest in nipple rings than dangerous weapons) a bunch of 24-channel 5Gb/channel InfiniBand switches. You wouldn't need many of those to get a decent Quake LAN party, and not many more before you could run weapons design software. It is unclear how many are required to run Vista, once service pack 1 is installed.
Interconnects. Obviously, InfiniBand is hellishly fast. So is SCI. 10 Gb ethernet, ideally with iWarp extensions, would be much slower but still perfectly good for a commodity cluster. If you scrap the idea of having machine-to-machine communication and do memory-to-memory communication, you could actually use PCI-e 2.0 as one gigantic interconnect. Ideally, you'd have the memory appear as two separate devices - slave and master - so that direct memory-to-memory RDMA could be initiated. A lot of very similar work has already been done by US supercomputer giants, and given how many have either been bought, gone bankrupt, or otherwise vanished, it's reasonable to suppose large quantities of such RAM could have "migrated" overseas.
CPUs - well, there are some respectable 16-bit pile-of-pc clusters. One was reviewed on Slashdot some time back. Even a cluster of Cell processors, if large enough and well-enough programmed, could be very effective. A hostile nation wouldn't need high-end 4x4 multi-threaded multi-core SMP systems, although again given how juvenile airport security is, I can't imagine it would be hard for someone to export, say, a couple of hundred motherboards at that spec.
Ok, what about OS? Who needs one? Anyone with a copy of OSKit or something similar can work at almost bare metal levels as if they had a full OS. If they did want a full OS, then NetBSD or MOSIX would be quite sufficient. Or they could take an OS project like Exopc and add high-performance networking to it.
Software? If you've a decent copy of BLAS, LAPACK and some solvers, tightly optimized for the platform, you're set. Those core maths functions are critical. Since the functions and API are fixed, it would not be impossible for someone wanting raw power to have put them into an FPGA, SoG or ASIC. Collective operations are also nasty, but they too can be done entirely in hardware, giving you orders of magnitude speedup over conventional software solutions. Synchronizing is the third killer, but there are meta schedulers to handle that and you could again place those on dedicated hardware.
In short, although I couldn't afford to build a top 500 machine, it is only the affording of it that is a problem, and foreign countries are quite well aware of that. Especially after China built its first (publicly-announced) Government-funded Beowulf. Supercomputing is easy, it's the price tag that isn't.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
If you can read this... 01110101 01110010 00100000 01100001 00100000 01100111 01100101 01100101 01101011
... oh, never mind.....
I'll just have be content with a beowolf cluster of these things imagining me in Russian.
Just because you've built a big cluster, doesn't make you a bunch of supercomputer architects, especially when that cluster has *no* innovative features whatsoever. Please don't try to b.s. us with arguments about how y'all had to solve problems of scale. Been there, done that.
btw, 1800 blades/nodes and 7200 cores isn't particularly big. TACC's Ranger is 3936 nodes and 62,976 cores. Architecturally, the machine looks almost exactly like the systems LLNL and the other National Labs have been building over the last 10 years.
just sayin', ya know ?
# 1800 blades,
# 80TB storage. Simple SATA disks. 5.2Gbps throughput.
So using the 80TB * 1024GB/TB / 1800blades gives us about 45GB/blade. If they're using "simple SATA disks" this would imply internal disks and 1800 internal disks would have an aggregate throughput much higher than 5.2Gb/s (5.2Gb / 1800 = 2.95Mb/s per disk). Now typically you'd boot the nodes from the network (so you can change the identity of the node easily by booting it from a different image) from some sort of FC array accessed via an IB to FC gateway. However, 5.2Gbps is an odd number to get to since FC comes in 2 and 4Gb formats (1Gb fibrechannel is outdated and 2Gb is on the way out).
While I always see all the CPU details in these articles, I really wish they'd publish more about the storage requirements and methods rather than just staying (we have xTB of disk...). How do they back the finished datasets all up? Tape? MAID? VTL?
Not enough memory to install Service pack yet.
Hi
Your comments contradictory
First You say we Indians live in huts, which means we are in stone age but then
you say USA shall bomb us back to stone age!!
You know soon USA Gov. will pay to run simulation on this Cow dung powered computer !!
This project started a couple of years back when Narendra Karmarkar (yes, him ) got a grant from the Tata group to try out his ideas in this 1991 paper. Prior to that he had tried getting funds from Tata Institute of Fundamental Research ( TIFR) while he was still employed with them (he had joined TIFR after leaving the US, though was still with Bell Labs in some way). TIFR was in no position to fund his project as their overall budget for all their activities was less than Karmarkar's requirements. The project started well enough and about 40 people joining CRL. From the grapevine it is heard that Karmarkar never gave any details (that what is contained in the paper referred above) to even people working closely with him. Basically he seemed interested in working all alone to meet his own targets. With not much tangible seen by the Tatas (a business house) and their need to have some clear road-map the relationship grew cold and at a certain stage Karmarkar left the organization. The remaining team put together what is called EKA. So, it is just another parallel machine built with enough money to buy all the components. Nothing innovative there.
The posting title reminds me of the Simpsons gag of Homer imagining what it would be like to be a moderately rich man. "Tee hee! I could rent anything I want!"
I couldn't help but think of the episode where the computer on Red Dwarf became dramatically smarter at the expense of lifespan. This computer is tremendously fast, but it has to be backed up and rebuilt all the time. It seems to be the reliance on thousands of off the shelf parts that creates this problem. I wonder if it is always the case with supercomputers or if this problem can be overcome with improvements like flash storage?
My understanding is that the list of Fastest Supercomputers is for supercomputers that are publicly funded - which is why we don't see any mention of google or microsoft clusters. But yet the summary claims that this one has not taken any government funding.
So then how can this cluster claim to be fourth fastest, when they really are not competing with the publicly funded clusters of the main list?
Obviously, we cannot expect the big corporate supercomputers to spill their beans to contest this, but it doesn't really look like it would be fair to compare this non-publicly-funded supercomputer to those that are.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
Twitter, is that you?
yeah but can it play Crysis?
These words make no sense. This machine uses a Clos topology, without source-routing and using rather small 24-port crossbars I might add. Nothing new there, has been done for 20 years. It's full bisection on paper, but Head of Line (HoL) blocking statistically reduces it to ~60% efficiency. And without adaptive routing, no way around it.
This machine has no new concepts or new hardware, boring.
One thing's for sure, the Tata company will play a huge role in my international pun toolkit.
"You know soon USA Gov. will pay to run simulation on this Cow dung powered computer !!"
d00d - lay off the bong
Build the 4th fastest supercomputer:
Want to drop off the end of the Top 500list? Load SP1.
Have gnu, will travel.
BlueGene/P systems cost roughly $1.4 million per rack. The second fastest computer on the top 500 list is a sixteen rack BlueGene/P system.