Slashdot Mirror
Cray's New Solid State Storage
Sivar writes: "Cray, a well known vendor of extremely fast supercomputing hardware, has introduced a storage system with a 224 GB capacity. The large size seems impressive, but the device can also transfer an unprecedented 80GB(!!) every second. That's more bandwidth than the main memory of most servers, and it's just for storage. For comparison's sake, a typical dual channel DDR motherboard has a bandwidth capacity of barely 4.2GB/sec." Yow.
... they were running off of one.
SEATTLE (April 8, 2002) - Cray Inc. today announced the availability of the world's highest-capability expansion memory and data transfer system. The new system, compatible with Cray SV1e(TM) and Cray SV1ex(TM) supercomputers, includes a 224-gigabyte Solid State Disk (SSD) with a data transfer rate of 80 gigabytes per second-800 or more times faster than the 10- to 100-megabytes/second speeds typical with today's disk servers.
The field-upgradeable SSD system can hold 27 copies of the Human Genome and transfer data at a rate equivalent to 100 Human Genomes per second. With their 32-gigabyte central memories and the new SSD system, Cray SV1(TM) series supercomputers now provide up to a quarter terabyte of ultrafast memory. They can also be linked to a virtually unlimited number of standard disk servers for additional capacity, and to other computer systems via high-speed networking.
"With the new SSD system, Cray SV1 series supercomputers can handle extremely large, data-intensive problems with unprecedented speed, convenience and cost-effectiveness," said Jerry Loe, Cray vice president of worldwide sales and service. "This will be particularly useful in bioinformatics, and for complex automotive and aerospace applications." The Cray SV1 series, named "Best Supercomputer" in 2001 by the readers of Scientific Computing & Instrumentation magazine, includes special hardware features for bioinformatics.
"With the new SSD, bioinformaticists will be able to work with several copies of the Human Genome at a time, or perform whole genome comparisons, or pursue drug design and discovery, without wasting valuable compute time waiting for standard disk data transfers," said Jef Dawson, Cray's manager of bioinformatics development and marketing. "The SSD can keep up with the Cray SV1 parallel supercomputers' processors, which perform up to 12 operations per clock cycle."
Dawson said the SSD will benefit virtually any application requiring large data sets. "The popular automotive application MSC/Nastran ran 2.5 times faster using the new SSD capability. Applications that run 'out of core,' including the popular GAUSSIAN chemistry codes, are also well suited to the SSD. You can think of the SSD as the world's biggest cache memory, or the world's biggest I/O buffer. Either way, it offers the world a new capability."
It might use a technology known as MEMS which is probe based storage. Probe-based storage system supports probe-based reading and writing of bits, is based on non-rotating media and initially
expected to support storage densities on the order of 100 to 300 Gbit/inch2. The storage
devices are envisioned as two rectangular sleds, one with storage media and the other
with a sparse array of very small read-write heads, in the range of thousands to millions.
Seeks will require x and y motion of one of the sleds relative to the other. These devices
are intrinsically highly parallel because some or all of the heads will be able to operate
simultaneously. [MEMS Modeling]
So, is the 80GB/s aggregate access for the (n) processors in the box? It's a Solid State Disk -- In other words, it's memory. And, it's not _that_ fast for a shared memory architecture system. see: STREAM Memory benchmarks
The CPU gets stuff from the cache.
The cache gets stuff from the RAM.
The RAM gets stuff from the hard drive.
The solid state machine won't act like faster memory, making cache misses cost less. It will act like a faster hard drive, making page faults cost less. Using this stuff as a substitute for RAM will slow down your computer unless you have it hard-wired into your system's bus in place of RAM.
Actually, if I recall the details correctly, this only exists because the SV1 is a 32-bit system.
"But it's a 64-bit processor!" I hear you exclaim. Yes, but PVP Crays (that includes EL, C90, J90, and SV1, but not T90 or T3D or T3E or SV2) have only (up to) 32 bits of physical memory address.
"But it supports 64 GB of memory, 32 bits is only 4 GB!" True, if you count in bytes! The Cray PVP line addresses memory in 8-byte words, so 2**32 words is 64 GB.
Thus, the engineers had a dilemma. They could fit 256 GB into the box (due to denser SDRAM parts), but the CPUs could only address 64 GB of it directly!
So they stuck a smart memory controller in the box, and did some OS hackery, and voila! the extra memory became available as a (very fast) disk instead. Additionally, the memory controller itself has a builtin memcpy engine, so if you're just copying data from "disk" to "ram" it doesn't touch the CPU at all, the CPU just tells the controller "hey, copy me 500 pages from address A to address B" and the controller does it.
I don't know offhand if the 80 GB/sec is the CPU-to-SSD bandwidth, or if it's the peak bandwidth of the copy engine...
A whole bunch. One of the difficulties of sequencing the genome is that it's somewhat error prone, which requires that the same region be sequenced several times to make sure that you've gotten it right. The chunks that can be sequenced conveniently are also very small compared to the whole thing, so it's necessary to sequence in overlapping chunks and put it together like a puzzle. The combination of those things means that each base in the genome must be sequenced something like 10 times to get a reliable result. That gets you up to 10 genomes off the bat. Add in the fact that each bit of sequencing information will have meta-data (i.e. where that snippet came from, which machine generated it and when, etc.) associated with it and things fluff up even more. When you start piecing the data together it will require a lot of processing power, so it may wind up being a good idea to use algorithms that trade storage space for processing time, and that can inflate your storage needs even more.
Even once you have the genome as a finished product, you may very well want to have more than one genome available. An important, but less well publicised, part of the genome project was the decision to sequence the complete genomes of several other organisms at the same time. Those include four species particularly popular among biologists: the mouse, the fruit fly, a round worm used in a lot of research, and baker's yeast. Doing genome to genome comparisons is a very good way of finding the areas that are biologically important because they'll remain similar across organisms. As you can imagine, doing a complete chunk by chunk comparison between two 3 GB data sets can chew up a lot of resources, and having fast access to a huge memory space like that is going to make it a lot easier and faster.
There's no point in questioning authority if you aren't going to listen to the answers.
It seems to me that many people here have no idea what a true supercomputer (or more specifically, a Cray) is or what makes it different from a simple cluster. Here's a few things to think about:
- Crays do not have monitors. They do not have keyboards, or mice.
- Crays do not run Windows. Crays do not run Linux. Crays usually run UNICOS, a special *nix designed specifically for Crays.
- Crays communicate with the outside world through a host terminal, like a SGI workstation, or something similar to that. Crays DON'T HAVE CD-ROM DRIVES!
- Nobody but those with 8-9 figure incomes get to buy a Cray. They cost MILLIONS, and the higher end ones can cost many many tens of millions.
- Pretty much the type of people that WOULD buy a Cray would be the government, and very very large corporations. Sorry, guys.
- Simply connecting 30 PCs together in a cluster will result in a nice, fast supercluster, but it won't come close to a Cray, because Crays are designed from the beginning to be as parallel as possible. Face it: beowulf clusters really can't make the best use of the contained hardware because the hardware wasn't designed to be so distributed.
- Be impressed with Crays. Be very impressed.
-James
>What exactly is meant by "Solid State Disk." Are there spinning platters?
:).
No moving parts, you can look at this like a "big ram disk" exept it has it's interface like another storage device. Look at this like A compactflash for example (it's not "SSD" but it's a good comparison.
There are a lot of interfaces (PCI, ATA, SCSI, proprietary (80GB/sec is either a big aggregated pile of raids or something similar) for these "drives" at various price points. The advantage of a SSD drive on a PC is that you have instant access, and it moves the stuff at a lightning speed limited only by your bus. Let's say you run a SSD drive on a Ultra160 interface, what you'd probably see with a disk benchmarking tool is 100nS access time (versus ~10ms for a standard drive) and you could see the real-world numbers of your scsi bus, probably around 140MB/sec (didn't try one on a U160 bus). The application for these babies are numerous: instant access to data on boards that don't handle 100GB of ram to cache everything or you wanting the machine to preload 1 hour at every reboot, bandwidth hungry application (although a raid could do the same here, but I saw some specific application needed both the bandwidth and under 1ms access time needed so..), for heavy swapping of numbers without using a buttload of ram again, etc.. probably some other people could think of something other. Usually when you break a certain amount of GB, the drives becomes cheaper than a motherboard that could handle a load of ram and the ram modules themselves, so it makes more sense if $$ is a factor (but still it's very expensive, we're talking 10K+ easily for a few GB).
There's also plenty of product on the net (search google), like I said, some are PCI cards that you add to your system with PC100 ram on it, some are IDE/SCSI, etc.. But for home people, you'd be better off with a cheap IDE raid card and a few drives, it's way cheaper
--- Metamoderating abusive downgraders since my 300th post.
Not really - consider SGI's servers, for instance. The Origin 3800 can handle 1 TB of RAM -- but it's a CC-NUMA machine, meaning you have to go through an intermediate router (don't think Internet; much faster) to get to the memory. SGI machines have a limit of 8GB per processor "brick", and their bricks interconnect at 1.6 or 1.2GB/s.
Then consider the SunFire 15K - it's an SMP machine; processors fit on boards that can contain up to 32GB of RAM; after that, you have to go off-board through a switch to get to other memory. Each system board has about 9.6GB/s of offboard memory access speed.
In short, Cray isn't tooting needlessly - this is impressive bandwidth to the memory. Latency is probably fairly high on it, but for streaming vast quantities of data in and out of local storage, it's probably amazingly nice.