PS3 Folding@Home Begins with Impressive Numbers

hansamurai writes "As we've previously discussed, the Folding@Home client is now available on the PS3, and already some early results are in. The total number of teraflops generated by PS3s has already exceeded all other OS contributions combined and the entire project is heading towards one petaflop of distributed computing power. Stanford notes that their teraflops calculation is conservatively calculated so the total power could be under-appreciated. With the PS3 European release complete and the Folding client already available to them, the number of users will continue to grow for the time being, let's hope that the project does not run out of work units to pass out. Kotaku has some numbers that are a few hours old since the Stanford server is getting hit pretty hard with the renewed interest in the project."

Comments

Very old numbers

[cxreg]

Gizmodo has more current numbers (which are also a little behind). Currently they're showing 346 TFLOPS for PS3s.

Re:Very old numbers

[OddThinking]

I believe the numbers are being taken from this web site.

What about global warming?

[stratjakt]

Remember all that energy we aren't supposed to be wasting?

Last I heard, F@H was a feel-good novelty that is doubtful to ever produce any meaningful results.

Re:What about global warming?

[Anonymous Coward]

As opposed to SETI@Home?

Re:What about global warming?

[Sciros]

I predict that the average global temperature will go up by 3000+ degrees by the year 5,000,000,000 if we maintain the current rate of PS3 usage for Folding@Home.

Re:What about global warming?

[drinkypoo]

Last I heard, F@H was a feel-good novelty that is doubtful to ever produce any meaningful results.

Where did you hear that? I don't know any details, but it's easy to find a voice of dissent from your view:

""For the most part, it's not that we're looking for a needle in a haystack, but we're looking for broad properties that require good statistics," said Vijay Pande, associate professor of chemistry at Stanford University. As one of the scientists behind the project, Pande is proud to say that Folding@home has actually provided useful information to the scientific community. SETI@home, however, has yet to discover a single alien transmission."

""These successes are documented in peer review journals. Over 50 papers have resulted from Folding@home," said Pande. He and his students collaborated with developers from Sony Computer Entertainment of America to build a Folding@home client for the PlayStation 3, but that wasn't really Pande's idea."

(In-Depth: Sony, Stanford Experts Talk PS3 Folding@home)

"Now, for the first time, a distributed computing experiment has produced significant results that have been published in a scientific journal. Writing in the advanced online edition of Nature magazine, Stanford University scientists Christopher D. Snow and Vijay S. Pande describe how they with the help of 30,000 personal computers successfully simulated part of the complex folding process that a typical protein molecule undergoes to achieve its unique, three-dimensional shape. Their findings were confirmed in the laboratory of Houbi Nguyen and Martin Gruebele scientists from the University of Illinois at Urbana-Champaign who co-authored the Nature study."

(Folding@home Scientists Report First Distributed Computing Success)

Re:What about global warming?

[cxreg]

I don't pretend to understand all of the science, but I assume the people at Stanford know what they're doing. The results page has some things that sound interesting

Re:What about global warming?

[cxreg]

Of course we have no idea WHAT we are folding. We could be helping create the next superbug for all we know.

Heh. The client does tell you what protein you're looking at, if you care enough to investigate it. My understanding is that the most likely benefit to disease research would be finding how "bad folds" happen, which are responsible for things like Alzheimers.

Re:What about global warming?

[usrusr]

Folding on an idle PC that is used for CPU-easy things like typing /. comments and the like is one thing, the PC would be running anyways. But i don't (yet, likely to change with the increasing media-center role game consoles are supposed to take) see why a PS3 should be running at times when it is not running at full CPU demand (e.g. gaming). There should only be full throttle gaming or power off. Or can it still fold in the background while emulating PS2 games?

An exception is the scenario where somebody would be heating electrically anyways, then it's always a good idea to pipe the energy through some circuits before enjoying the inevitable temperature increase. If you shake up that enthalpy, shake it up in the most funky way you can. But then it's still much better to avoid electrical heating at all...

Re:What about global warming?

[GrievousMistake]

They have a Results page showing some of the problems that Folding@Home have been applied on. Not unbiased, but it seems like they're putting it to use.
As to useful results, it's just a distributed supercomputer. Why would its results be more feel-good and less meaningful than those of any other ~500 TFLOP computer? It's not like researchers can ever get enough processing power. Molecular folding is a processor intensive and parallelizable research problem with real applicability, and I'd rather see people spend power on this than on SETI@home.

Re:What about global warming?

[OddThinking]

I hate to repeat a post, but for additional information:

In case anyone is wondering about what the project has acheived so far, here is the link.

Concerning global warming, the processing statistics imply the PS3 is by far the most efficient. At 380 watts, using the statistics given (which are said to be conservative in the case of the PS3), that puts the PS3 at 63 teraFLOPS/megawatt, or 16.5 kilowatts/teraFLOPS. I'm not really familiar with this, but isn't that fairly good? It's definately better than using PCs. Blue Gene/L, which is supposed to be very efficient, will deliver 240,000 FLOPS/watt, or about 0.24 teraFLOPS/megawatt. My calculations may be off, but that suggests the PS3 is highly efficient and a better use of power than a supercomputer.

I'm sure I'm missing some important considerations, so can someone through a little knowledge at this?

Re:What about global warming?

[qbwiz]

The numbers in that article can't be right. 240000 flops/watt*1.5 Megawatts = 360 gigaflops, not 360 teraflops. It looks like it should really be 240000000 flops/watt (or 240 teraflops/megawatt), which is relatively consistent with their explanation of 5.6 gigaflops per 12 watts per chip (that number is slightly higher, but it doesn't consider RAM and other components). This is therefore better than the PS3's 65 teraflops/megawatt.

Re:What about global warming?

[ScrewMaster]

True ... but "better" is relative for the supercomputer owner. Anyone with a conventional supercomputer pays for the power himself. In the PS3 Folding@Home project, not only the processing is highly distributed: the electric bill is as well.

Re:What about global warming?

[drix]

No, doof. Whatever benevolent alien race we finally make contact with should be more than happy to bootstrap us with their limitless free energy generation technology. All part of the master plan.

Curing cancer... pfft. Like that's gonna help anyone.

That's all nice and good but

[Rosco P. Coltrane]

As we've previously discussed, the Folding@Home client is now available on the PS3, and already some early results are in [CC].

When will the SNES version finally be available?

impressive

[mastershake_phd]

Impressive, but I wonder if this interest among PS3 owners will drop off. Especially when GTA IV comes out, or they get next months power bill.

The Real Question Now

[Shadow Wrought]

Is if you can write-off your PS3 as a charitable purpose since its spending the bulk of its time volunteering;-)

For 64bit floats, the PS3 is a powerhouse

[maynard]

Cell is very optimized toward one data type for calculation: 64bit floats. If you want to efficiently use the PS3 in a cluster, just be aware that your code must:

a) use primarily 64bit floating point

b) either:
            - fit code and data segments within 256K for each SPU
            - crunch long enough between streamed data blocks such that DMA latency doesn't kill performance

c) have the entire calculation broken down into no more than six parts for streaming (one per SPU)

There are SPU userspace threading models that run cooperatively (similar to the old userspace pthreads, I guess), but the thread manager consumes valuable SPU RAM. Also, SPUs don't support a supervisor bit for memory protection... so... bad things happen when threaded code running on SPU goes tits up.

If you want to calculate 128bit floats, ints, or have lots of branch logic... buy a quad core2duo; cell don't do you any good.

BTW: Anyone here hacking GEANT or BLAST for Cell?

Re:For 64bit floats, the PS3 is a powerhouse

[adisakp]

You obviously have no idea what you're talking about since you're wrong on every point you make. I'm a PS3 developer but all of the points I make below are well known, publicly released information.

Cell is very optimized toward one data type for calculation: 64bit floats

Wrong!!! Cell is optimized towards 4-float vectors of 32-bit floats. All the vector math operations on the SPU operate this way and it's capable of doing 8 32-bit operations per cycle (FMAC = 32-bit multiply + 32-bit add X 4 wide). On the other hand, 64-bit operations are scalar and non-pipelined. The take a minimum of 7 cycles for throughput and can have longer latency (13 clock cycles). The maximum 32-bit FP Single Precision (SP) rate is at least 28 times faster than the DP rate.

b) either:
- fit code and data segments within 256K for each SPU
- crunch long enough between streamed data blocks such that DMA latency doesn't kill performance

Wrong!!! Actually, a single code processing step and data should fit in considerably less than 256K. Preferably around 128K. Then you can double-buffer your DMA's for input and output. When you do this the DMA latency doesn't even matter unless your processing occurs faster than the DMA transfers since your DMA's are completely asynchonous to the SPU processing. This simple method of programming can hide nearly all DMA latency -- especially for code that repetitively iterates on multiple data blocks.

c) have the entire calculation broken down into no more than six parts for streaming (one per SPU)

Wrong!!! You can break the calculation into many more parts than six. As a matter of fact you could have 100 calculation parts on on chunk of data and simply swap in new code and work on old data. You can arbitrarily schedule more than a single task per SPU. Sony (and even IBM on non PS3 Cells) have libraries that allow you to share SPUs between many different tasks with only a very small minimal overhead incurred in switching between a task on the SPU.

Also, SPUs don't support a supervisor bit for memory protection

Wrong!!! The SPU's can only directly access their own local memory. All other accesses go through a protected external memory interface (the SPU DMA to main memory) and are controlled by memory protection. It is possibly to virtualize and lock-out SPU's from the rest of the system and run them in "safe mode". If the SPU's could run rampantly and access the entire memory there wouldn't be much point to Sony running a hypervisor to keep you out of their system space on the PS3 linux project and still give you access to the SPU. Also, it wouldn't make much sense not to have memory protection from IBM's point of view to use Cell's as CPUs for clustered supercomputers.

bad things happen when threaded code running on SPU goes tits up

Wrong!!! There is no reason on the CELL hardware why it shouldn't be possible to kill SPU threads / processes and the SPU rescheduled by the OS if necessary. This way a single task can no more take down an SPU than the PPSU. It is possible to even swap out an entire SPU programs pre-emptively on the fly and restore their state. This incurs a much higher swap cost (full SPU threading) than a more simple task manager because you have to save and restore the entire context of the SPU (including 128 16-byte registers and 256K memory region) but your implied limitation of the SPU's is definitely incorrect here.

If you want to calculate 128bit floats, ints, or have lots of branch logic... buy a quad core2duo

So quad-core2duo can do 128-bit floats ? If you're thinking SSE (4 X 32-bit floats) then the SPU's do the same thing. SPU's can run integer code albeit more slowly since most of the integer operations are scalar but still running integer code in parallel on an SPU can sometimes be faster than a Core2Duo - If you align scalar-processed integers to 16-bits for preferred slot l

Re:For 64bit floats, the PS3 is a powerhouse

[maynard]

I was already corrected on the issue of floats. I appear to have been misinformed. However, on many of these issues you appear to be splitting hairs. For example, you appear to argue that an SPU does not lack a supervisor bit - but instead go on a tangent about how SPUs don't have local memory access, but must instead perform a DMA transfer. Well, as it turns out, the SPU does NOT have a supervisor bit for memory protection.

Your other arguments boil down to: we don't program Cell like that. In particular, you're discussing how to double buffer and/or stream many code/data segments on Cell with low performance penalty. I'm sure you're right for your particular task (which I assume is game programming). However, that is not the best choice for coding up a Monte Carlo sim. Particularly since it might actually be possible to cram the algorithm and data segment straight into LS. And regarding your userspace threading argument: when a simulation that just spent CPU days (or weeks) in compute goes tits up, bad things do happen. Heh.

Re:For 64bit floats, the PS3 is a powerhouse

[adisakp]

SPU's do have local memory access. However, local memory access is not the same as unprotected memory access. All accesses to external memory (DMA) can be protected. DMA commands use the same type of translation and protection governed by the page and segment tables of the Power Architecture as the PPU (indeed there is MMU management for each Memory Flow Controller for each SPU). I suggest you read this paper to see the memory management on the SPU: http://www-128.ibm.com/developerworks/power/librar y/pa-celldmas/

The SPU's can run in a completely safe and sand-boxed environment with actual SPU threads and processes. In fact, you can set a secure "vault" mode in which the SPU is completely disconnected from external access and it's possible to ensure that SPU's are not capable of touching anything other than what is in their local memory (and nothing external can touch the SPU's local store in this mode).

If during normal SPU something goes wrong (i.e. access to a protected area), it's possible to kill the curren SPU process and restart the SPU with a new process -- neither that specific SPU nore the Cell processor need to be hung up by some bad SPU code. In this respect, it's possible to build a secure OS that allows SPU access. If something bad happens on an SPU, you can kill the offending process -- just like if something goes bad on a single core on a Core2Duo process, the linux kernel can kill the process. The machine won't go "tits up" -- just your process will get killed. But this is true of any bad program on any "secure" OS.

I don't know what you're claiming as a "supervisor" bit for memory protection. Perhaps you're referring to read/modify/execute (RWX) page settings on the page-table (and cached in the TLB) with virtualized memory access. The SPU's do not have a TLB for local store (although it is mapped into the PPU address space and that TLB). However, they can set up "privileged" areas in local memory (and local register address space - i.e. DMA regs) that can be used to accomplish memory protection for local store and access to non-allowed areas can cause an exception which the OS can then deal with appropriately. Additionally, there is an Local Store Limit Register (LSLR) to mask the available region of local store. Now witin the normally accessible area of local store, the SPU can store instructions and execute them allowing for self-modifying code. Before DEP on windows (and/or if your CPU is more than a year or two old and doesn't support execution page protection), you could make self-modfying code run in any region of writeable memory -- one reason why windows buffer overflows are so easy to exploit.

As far as the "this is how we program SPU's", sure you can program stuff to only run only 6 fixed SPU tasks with no swapping and use 256K data blocks (minus code overhead) so you have DMA latencies. However, you implied by stating those as limitations that was the *ONLY* way to program them which is wrong. All the papers out there from IBM and Sony suggest PREFERRED METHODS which are different than your implied limitation. They advocate cooperatively running multiple small task (dozens or even hundreds) rather than dedicating cores to fixed tasks and to double-buffer data (or even code) to mask DMA latencies. Singular task SPU usage can lead to idle SPU's very easily. Additionally, the double-buffer method can hide DMA latencies which can double the speed of your code if you're roughly equal on memory (DMA) and SPU compute time. We prefer to program SPU's in the manner in which they will be utilized much more fully and run up to 2X faster. I think anyone rewriting code for the SPU (since you have to retarget for the SPU anyway, there's no reason to write your code in a manner that will deliberately underperform).

What's so impressive here...

[MarcoAtWork]

... besides the number of PS3 owners that are running this? The PS3 seems to be significantly slower than the GPU client for example

GPU: 41tflop 697cpus
PLAYSTATION®3 346tflop 14138cpus

so basically the GPUs are 2.4x as powerful as the PS3s.

Re:What's so impressive here...

[ucblockhead]

The combination of the numbers and the power. Folding@home just tripled its power. If you think Folding@Home is important, that's important.

If, on the other hand, you're only worried about arguments about what machine is "better", then yeah, you're right.

But not everything in life is about CPU dick-waving contests.

Re:What's so impressive here...

[MarcoAtWork]

I like that folding got a significant speedup, don't misunderstand me, I just don't particularly care for the fanboyism that's happening with people going on on how much more powerful the PS3 is than a PC and yadda yadda yadda. I really appreciate that the PS3 crowd is contributing so much, although I think this might die off a bit when the novelty factor won't be there anymore.

Although the folding folks were smart to put on an animated 3d screensaver with the client which will make it that much more likely that folks will keep this running, I wonder about burn-in of the white-on-dark text line statistics that I've seen on the various videos: I wonder if those could be disabled (or if they orbit around the screen).

Re:What's so impressive here...

[EGSonikku]

After being idle a few minutes the stats go away and it's just the (moving) protein and the (moving) globe in the background.

Conversely, you could just turn off your TV.

Re:What's so impressive here...

[Phil Wilkins]

This is correct, although to achieve that, the GPU client requires twice as many FLOPs to process the same amount of work.

power bill

[DogDude]

Somehow, I doubt that people buying a $600 game system will care if their power bill goes up $1 (or $10 or $20) a month. Power is one of those things that most people ignore and simply pay unless it's completely out of whack. My commercial power bill fluctuates by sometimes as much as a hundred bucks a month, but even that's not enough to make it worth my time to figure out what might be causing it.

Re:power bill

[EjectButton]

yeah most probably wouldn't notice
according to this
http://www.hardcoreware.net/reviews/review-356-2.h tm
the ps3 uses about 200watts maximum
and if you look at the cost per kwh around the US http://www.eia.doe.gov/neic/brochure/electricity/e lectricity.html
and round up for the sake of argument, so say you run it for 24 hours a day, you never play any games on it, and you are paying $0.10 /kwh, that's $14.60/month

more realistically say you pay $0.10/kwh and only run f@h when you are asleep, so 8 hours a day, less than $5 a month more than you would have paid otherwise.

Re:power bill

[ucblockhead]

380 watts == 9.12 kWh per day. Here in California, I pay about $0.22 per kWh for "above baseline" power, which means it'd cost me almost $2/day, or $60/month to run.

I wonder if that can be written off as a charitable expense.

Different work units?

[pavon]

When the GPU client first came out, it was pointed out that it was actually using different work units than the normal PC version and so the numbers weren't directly comaparable. I don't know what the situation is for the PS3, but it may not be using the same work units as either the GPU version or the PC version, and thus not directly comparable to either.

/. team number

[4g1vn]

Slashdot team# 11326 Now go get your PS3 and start crunching numbers.

GPU performance

[Panzergheist]

Also, since I haven't seen anyone mention this yet, the GPU client on the F@H site are all ATI X1900s. The work units performed by GPU clients and Cell clients are of a different type than those performed by general purpose CPUs. Check the F@H FAQs for more information.

Re:From much less CPU's too

[gardyloo]

Is that a slim ps2 or a old ps2, and what's the longest you've had it up before? Ooh. Slashdot come-on lines.

Re:F@H ps3 vs 360

[rustalot42684]

Well with the Wii, you can actually fold the proteins yourself using the innovative new motion-sensing controller!