Slashdot Mirror
Supercomputer Built With 8 GPUs
FnH writes "Researchers at the University of Antwerp in Belgium have created a new supercomputer with standard gaming hardware. The system uses four NVIDIA GeForce 9800 GX2 graphics cards, costs less than €4,000 to build, and delivers roughly the same performance as a supercomputer cluster consisting of hundreds of PCs. This new system is used by the ASTRA research group, part of the Vision Lab of the University of Antwerp, to develop new computational methods for tomography. The guys explain the eight NVIDIA GPUs deliver the same performance for their work as more than 300 Intel Core 2 Duo 2.4GHz processors. On a normal desktop PC their tomography tasks would take several weeks but on this NVIDIA-based supercomputer it only takes a couple of hours. The NVIDIA graphics cards do the job very efficiently and consume a lot less power than a supercomputer cluster."
They didn't have enough dough for 9.
Am I the only one seeing those alternative uses of GPUs as some kind of re-birth of the Amiga design?
I am guessing it has something to do with floating point calculations vs. integer calculations, but if I read the article, this wouldn't be Slashdot, would it? Think about it. We have GPUs to perform vector maths, flops, etc. because the CPU is not all that great at that sort of thing typically. A general purpose CPU is not necessarily going to be the fastest if your problem domain is more suited to an "inferior" chip; general purpose CPUs are not designed to be the fastest chip in every situation.
This article makes it seem like it is possible to use the GPUs as general purpose CPUs. Is that the case? If so, why doesn't NVIDIA or especially AMD\ATI start putting its GPUs on motherboards? At a ratio of 8:300, a single high-end GPU seems to be able to do the work of dozens of high-end CPUs. They'd utterly wipe out the competition. Why haven't they put something like this out yet?
By the benchmark that they solve the particular problem of this specific application in 1/300th of the time?
Help stamp out iliturcy.
Ok, probably a paid NVIDIA ad placement, but check TFA anyway (and even if you don't read, you gotta love the case). It looks like heat generation is one of the biggest problems--sweet.
...and at 4000EUR, that comes to what (rolls dice, consults sundial) about $20000 American?
I like this too:
The medical researchers ran some benchmarks and found that in some cases their 4000EUR desktop superPC outperforms CalcUA, a 256-node supercomputer with dual AMD Opteron 250 2.4GHz chips that cost the University of Antwerp 3.5 million euro in March 2005...
"Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth
In other news Graphics cards are good at . . . graphics.
I can't imagine that it is a coincidence that this comes along just as Nvidia are crowing about CUDA, or that the resulting machine looks like a gamer's dream rig.
While there is ample crossover between hardware enthusiasts and academia, anyone soley with the computation interest in mind probabyl wouldn't be selecting neon fans, aftermarket coolers or spend that much time on presentable wiring.
They are useful for applications that can be massively parallelized. Your average program can't break off into 128 threads, that takes a little bit of extra skill on the coder's part. If, for example, someone could port gcc to run on the GPU, think of how happy those Gentoo folks would be :) (make -j128)!
Obligatory blog plug: http://www.caseybanner.ca/
As far as my understanding goes, comparing a GPU's performance to a CPU's performance is very very task dependent and the comparison with 300 CPUs should not be taken to mean that a 8GPU system is more powerful than a 300 core duo system in general.
If the application requires solving a small task many times over and over and all of these tasks can be done in parallel then using a GPU works great because a GPU has many cores each of which can handle a simple routine. Also the GPU is designed to spend very little time on the way code is hadled (load, switch etc) and spend more time actually running the code (hence the requirement of only very simple functions).
Such problems frequently arise in tomography, physics, astronomy etc and I hear GPUs are a great success in these areas. But don't hold your breath for running your favorite distro blazingly fast using GPUs.
... 3D Realms announced this as the minimum platform requirements to run Duke Nuke'em Forever.
__ Someday, but not this morning, I'll finally learn to use the preview button.
Something that can play Crysis!
this is an example of acceleration architecture. Anyone who have used FPGAs knows that. Ofcourse, making sensational news is a too common thing on /.
They called me mad, and I called them mad, and damn them, they outvoted me. -Nathaniel Lee
It is possible to solve non-graphics problems on graphics cards nowadays, but the hardware is still very specialized. You don't want the GPU to run your OS or your web browser or any of that; when a SIMD (single instruction, multiple data) problem arises, a decent computer scientist should recognize it and use the tools he has available.
Also, this stuff isn't as mature as normal C programming, so issues that don't always exist in software that's distributed to the general public will crop up because not everyone's video card will support everything that's going on in the program.
It is also not difficuult to find other tasks where, e.g., FPGAs peform vastly better than general-purpose CPUs. That does not make an FPGA a "Supercomputer". Stop the BS, please.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
The GPU's are better at floating point than integer; if I remember correctly it takes 4 cycles on current GPU's to do a float operation, but it takes 16 to do an int. No, I don't understand why.
Also, the "multiply" and "add" instructions exist in a "madd" opcode which essentially doubles the theoretical floating point performance, even if you don't use "madd" very often.
I love this picture: http://fastra.ua.ac.be/en/images.html
Between the massive brick of GPUs and the massive CPU heatsink/fan, you can't see the mobo at all.
Wave of the Future? Yes*. Revolution in computing? Not quite.
The GPGPU scheme is, after all, a re-invention of the vector processing of old. Vector processors died out, however, because there were too few users to support. Now that there's a commercially viable reason to make these processors (PS3 and video games), they are interesting again.
The researchers took a specialized piece of hardware, rewrote their code for it, and found it was faster than their original code on generic hardware. The problems here are that you have to rewrite your code (High Energy Physics codebases are about a GB, compiled... other sciences are similar) and you have to have a problem which will run well on this scheme. Have a discrete problem? Too bad. Have a gigantic, tightly coupled problem which requires lots of inter-GPU communication? Too bad.
Have a tomography problem which requires only 1GB of RAM? Here you go...
The standard supercomputer isn't going away for a long, long time. Now, as before, a one-size-fits-all approach is silly. You'll start to see sites complement their clusters and large-SMP machines with GPU power as scientists start to understand and take advantage of them. Just remember, there are 10-20 years of legacy code which will need to be ported... it's going to be a slow process.
Fortunately, Nvidia provides a CUDA version of the basic linear algebra subprograms, so even if your software is hard to port, you can speed it up considerably if it does some big matrix operations, which can easily take a long time on a CPU.
They are comparing their system against normal computers, I'd be interesting to see a benchmark against a vector computer, like, eg. NEC SX9
I think the GP (and myself) were objecting to the use of the fairly general word "power" and the use of this one problem as a "power benchmark". While it is obviously true that 8GPUs is as fast as 300 C2Ds for this problem, this system isn't as fast as a supercomputer for most problems. All this does is point out that the recent trend of building supercomputers out of inexpensive general purpose CPUs may not be a good idea for all applications.
When you get into inverting matricies, or doing matrix vector multiplication the algo is very easily in parallel, but I always wonder where the full matrices live. i.e. they could easily be tens of GBs of matrix, so the CPU would seem to have to be heavily involved as well.
Because for 95%+ of the problems a general purpose computer tackles GPU's would suck. It's only in very special cases that GPU's outperform CPU's. Thus, your idea is a poor one.
As far as I know, GPUs are amazingly fast at matrix operations and other things allowing vectorized evaluation. I guess these tomography applications must make massive use of these. After all, tomography is in essence image processing..
The state you are in while your HEAD is detached... - wait, what?
And... a screwdriver is not always a prybar. A tool's a tool - they have preferred usage but if your requirement is specific and you're creative enough, you can do some fine work outside of the tool's intended purpose. Like this guy. Kudos to him.
Perhaps some more creative people finding this information will now discover if their specific requirements can be met by this interesting configuration. That will save them large quantities of cash or possibly enable some facility that was not previously available because supercomputers cost a grip-o-cash.
Of course for general purpose supercomputing you would want to use modified PS3s.
Help stamp out iliturcy.
Precisely. But that happens to be one of the areas where more performance is still needed.
You don't need a super-duper CPU for text editing, that's for sure. For most of the tasks people do on computers, we have had CPU enough for the last 15 years or more. But where we still need more CPU happens to be mostly in tasks that ARE massively parallel, for instance, physics simulations, of which you will find several examples in the nVidia site.
I'm following this technology with much interest, and I think I will have a major upgrade in my home computer soon. My old FX-5200 card has been more than enough for my gaming needs, but now I have a new reason for upgrading.
The system uses four NVIDIA GeForce 9800 GX2 graphics cards, costs less than 4,000 EUR to build
What's more crazy: calling something this inexpensive a supercomputer, or 4 video cards costing a freaking 4,000 EUR.
Because floating point operation goes on a dedicated path, while the integer operations does not have a dedicated integer-only path.
Also, it's possible that loading floating points operands and storing results in actual code can be pipelined, while integer operations are not pipelined.
(and yes, I don't know what I'm talking about)
Even more: if you don't optimize the code specifically for the GPU-based supercomputer, your performance goes down the drain. I wouldn't be surprised if they obtained a speedup of an order of magnitude or more from the aggressive code optimisation.
The idea is: the original code would run faster on a 8 Core2Duo machine than on the 8 GPUs. Even more optimising of the code will do little for the Core2Duos, due to limited memory bandwidth, FSB bandwidth, and so on.
Meanwhile, optimising a pipelining sistem (load, compute, store) in the GPU would be greatly improved by huge bandwidth (50GB/s on current systems), huge number of computation units (128 or more) and so on.
Too bad this isn't really news. I guess it is news if you consider that someone else has had their application accelerated by NVIDIA GPUs. I guess the only other reason that this could be news is by virtue of having 8 GPU cores.
Unfortunately, this setup won't work ideally for a lot of other CUDA based applications. For the past 6 months, I had a system with 6 GPUs (actual physical GPUs). This is the system that I showed at CES. We are easily able to do 8 physical GPUs, and now I've been solely focused on utilizing Tesla.
Given that NVIDIA released the GX2 series, I was not surprised that someone would announce an 8GPU system. I'm surprised it took this long for someone to do it, and almost equally surprised that slashdot took this long to publish any news that is decent in the realm of GPU super computing. I've been cranking out close to 228 billion atom evals. per second in VMD for months now, versus about 4 billion on dual quad core 3.0GHz Xeons.
or 100's of FPGA's can do what was previously considered a task that even with super computing resources was considered so time consuming to be only worthwhile for groups like the nsa: http://www.copacobana.org/index.html (the EFF had a simlar custom chip device several years before but that cost >$250K)
The 9800 GX2's GPUs have 128 1.5GHz "shader processors". 8 of these is like having 1024 vector-processing-specialised processor cores at your command.
I could easily believe that it performed comparably to 300 2.4GHz Core 2 Duos (aka 600 "over 1.5x faster but not vector-specialised" cores).
Theoretical performance is 576 GFLOPS per 9800 GX2 GPU (4.608 TFLOPS total) vs 19.2 GFLOPS per Core 2 CPU (5.760 TFLOPS total). However in tests the Core 2 gets as low as 6 GFLOPS instead of it's 19 theoretical, and the 9800 GPU gets a lot closer to it's full power.
Just to expand on this stuff: Different tools are (obviously) designed for different workloads. I have a project I was contemplating porting to the Cell. Unfortunately only 40% of my performance bottleneck could take advantage of SIMD, but that 40% could have taken advantage of an enormous number of SIMD instructions just like the workload from TFA.
The other critical 40% of my project would have gained absolutely nothing from SIMD and on the Cell would have lost time due to branches. In this case 300 c2d's would far exceed the throughput of 8 GPU's.
Please, please, please do the math.
8 GPUs are being compared to 300 CPUs. So the single GPU for this pupose isn't 300 times as powerful as the CPU.
It is doing the operation in 1/37th the time approximately. This isn't news or unbelievable. GPUs are dedicated to performing certainly types of tasks far better than a CPU.
http://blindscribblings.com - Tasty pop-culture in conceptual fashion.
I'm extremely curious to know where the performance bottleneck is in this system. Is it memory bandwidth? PCIe bandwidth? Raw GPU power? Depending on which it is, it may or may not be very easy to improve upon the price/performance ratio of this setup. Given that the work parallelizes very easily, if you could build two machines that are each 2/3 as powerful and each cost 1/2 as much, that's a huge win for other people trying to build similar systems.
There's no failure quite as dissatisfying as a complete and total solution to the wrong problem.
Since when have "vector processors died out"? The "Earth Simulator" for example used the NEC SX-6 CPU, currently the SX-9 is sold. Vector processors never died out and were in use for what they are best at. The GPU and the Cell are no match for either processor, first they both are only fast in single precission mode and much slower when they have to do double precission (the second generation of Cell is better at double precission) and they both have a weak memory subsystem when compared to a true VPU. It is slow and they can only use small memories. As far as I know the Cell can't even chain it's VPUs, something which was standard since the Cray-2 on VPUs.
It takes 4 cycles to do a floating point operation, and 4 cycles to do an integer add/subtract. It takes 16 cycles to do an integer multiply because it only has a 24-bit hardware multiplier (needed to achieve the 4-cycle flops, so it has to do long multiplies as four madds, This was for the first generation CUDA CPUs; the second generation, which should be out by now, was going to have double length floating and would be able to do 32 bit multiplies in the same four cycles.
While they can do integer, these machines are not very happy with it, and I found it much easier to do everything in floating point, even if you are talking about 8-bit colour data. It goes no slower, and everything is much better adapted to floating point. Then there are special instructions to get back to integer at the output.
While each operation takes 4 cycles, they are fully pipelined, so that it launched a new instruction per cycle, times 32 pipes per unit, times 8 units per GPU.
And madd is very useful for the sort of tasks for which supercomputers are traditionally used.
Consciousness is an illusion caused by an excess of self consciousness.
Sure - but at 4000 euros, you can afford to do a one-off purchase and write custom software for a limited application. The point of this is that if your application suits it, this is a very cheap way to get supercomputer performance without paying for your own supercomputer (cluster) or time on an existing one.
The point of this is that if your application suits it, this is a very cheap way to get supercomputer performance without paying for your own supercomputer (cluster) or time on an existing one.
No doubt about it. In spite of my admittedly negative criticism, I applaud these guys because I think this shows the amazing potential of multicore parallel computing to bringing supercomputing power to the desktop and even to the laptop and the cellphone. However, this potential will not arrive unless we can find a way to design a universal multicore processor architecture that is at home in all possible parallel environments, not just vector parallel systems. IOW, we need a parallel processor that can handle anything we can throw at it with equal ease. Unfortunately, both the industry and academia are pushing the field toward so-called heterogeneous processors, hideous monsters that will be a nightmare to write code for. Check out Nightmare on Core Street for good explanation of the multicore crisis and how it can be solved.
Which is faster? A Lamborghini or a 5-ton flatbed truck?
Depends on what you're after! If you are trying to get yourself from point A to point B, the Lamborghini is the obvious choice. But if you need to move 4.5 tons of stuff from point A to point B, the Lamborghini would suck ass when compared to the flatbed truck.
It's just a question of what you are trying to accomplish. There is no absolute framework for "power" to solve problems, even if you define it fairly narrowly. For example, let's talk about 'pattern matching': A free database (like PostgreSQL) on cheap hardware can search through millions of records to deliver a query result in a tenth of a second. In that respect, Postgres is WAY faster than, say, the human brain. But the human brain will KICK ASS over just about any other technology out there in deciding whether or not a particular image contains a cat.
Use the right tool for the job, and you'll be amazed at the results. That 8 GPUs handily outperform 512 CPU cores at a specific task is not surprising - the GPUs are designed from the beginning to solve the kind of problem that's needed!
Personally, I'm surprised as to why there hasn't been more development behind the FPGA: are they just expensive?
I have no problem with your religion until you decide it's reason to deprive others of the truth.
The guys in Antwerp have probably got themselves the greater number crunching power, but reconstruction of tomographic images has been done using similar multi-core hardware. See the following (pdf alert) from the University of Erlangen, which uses a cluster of PS3s for a great use of commodity consumer hardware http://www.google.co.uk/url?sa=t&ct=res&cd=1&url=http%3A%2F%2Fwww.imp.uni-erlangen.de%2FIEEE%2520MIC2007%2FKnaup_Poster_M19-291.pdf&ei=t_FBSKnZKoie1gbh2Y23Bg&usg=AFQjCNG7vNGmMM2hBrYdVKbwZAJZL0oS3Q&sig2=sEdlnPROC77CZ_KJ5OOgrg .
Because a floating point number really consists of two value (a large 23-bit mantissa, a smaller (8-bit exponent and a single sign bit), performing a single arithmetic operation on two floating point numbers requires:
1. Aligning the two mantissas so the exponents match
2. Performing the operation
3. Renormalizing the mantissa of new value so that it is in the range 1.0 to less than 2.0
4. Saving the result to the destination register
Each of these stages would probably take one read/write cycle.
Performing an integer operation requires a shift-and-add sequence for multiplication, and a shift-compare-and-conditional-subtract for division.
Previous GPU's just stored the integer as the mantissa of floating point registers. But as integers are now represented separately as 32-bit values, they will be processed by a different hardware unit. Maybe they have two barrel shift registers working in parallel so that only 16 cycles are required.
Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
It's not really that surprising. CPUs are supposed to control the computer's resources and keep some kind of sanity and synchronization.. do one thing at a time and they do it fast. Multiple cores are nice but they just let you do 2 things at once. Yeah they're fancy and pipelined and there's all sort of asynchronous optimization with arithmetic and logic, but as a whole it's executing one instruction after another.
GPUs on the other hand are far more parallel. The thousands of individual subprocessors can be independently controlled in software and given different tasks.
You're being overly simplistic.
In order to utilize this "super computer", your problem has to be refactored in such a way that it can utilize the hardware efficiently. This can be either be fairly easy or incredibly difficult depending on the problem, tool-set available, etc. .
Their benchmark is good for them, but it is most likely meaningless to the general super-computing community. Porting something like LINPACK over and running that as a benchmark however would give a whole lot more insight into what kind of performance boost a typical scientific app might gain from said hardware.
Nice to see someone utilizing this functionality though.
~X~
~X~
I haven't done much of the development of the software myself. We have a developer we hired to work with CUDA. From what I've found, the documentation that is available on NVIDIAs site for CUDA is excellent and their developers are active on their forums.
For VMD, it was necessary to have 1 CPU core per GPU. We tested 6 GPUs with 4 cores and we could only spawn 4 threads for GPU processing. The guys at Evolved Machines told me they can use multi GPU off of a single core. If so, I have no idea how. NVIDIA even tells me 1 core per GPU, so that is the gospel I'm following by. Acceleware for some of their stuff even use 2 cores per GPU, but they have their own libraries outside CUDA for GPU stuff, so who knows.
I haven't come across any books on CUDA other than the support manuals, but since it isn't a very mature API, it is only a matter of time.
You might be a little off, but I'd say you're pretty close, and I just had a couple of architecture and super computer systems classes.