Slashdot Mirror
A Three-Way AMD Opteron Server
Abdul tips a thin little review up at The Inquirer of the Themis Slice. "The Slice is a three socket Opteron machine with two PCIe slots and two Infiniband 4x ports... Why would you want three sockets rather than four? Easy, latency. Any CPU in a 3S system is one hop away from any other CPU. In a 4S system, you can be two hops away. This adds latency, and more importantly, you take a big hit on cache coherency latency. This kills performance."
Disclaimer, I work for IBM.
The IBM System x3755 has offered this feature since it came out as well. Instead of the fourth processor card you install a pass through card and it turns it into a three way. We've done a few benchmarks (warning pdf) with the Pass Through card and what it could do between 3CPU and 4CPU operations.
pretty cool ability for a few things.
As a rock-in-roll Physicist once said, No matter where you go, there you are.
This is also a problem on FSB systems, as all CPUs need to snoop the bus for cache coherency information. On Intels dual-bus systems, this information needs to go across busses. The Intel 4 FSB systems are even worse. AFAIK, Opteron is the only x86 chip that would support 6 cores (12 cores with Barcelona) with a single hop.
> Why would you want three sockets rather than four? Easy, latency. Any CPU in a 3S system is one hop away from any other CPU. In a 4S system, you can be two hops away. This adds latency, and more importantly, you take a big hit on cache coherency latency. This kills performance."
Lawrence: Three chips at the same time, man.
Peter: That's it? If you had a million dollars, you'd use three sockets at the same time?
Lawrence: Damn straight. I always wanted to do that, man. And I think if I worked at AMD I could hook that up, too; 'cause I hate motherboard layouts with latency.
Peter: Well, not all layouts.
Lawrence: Well, the type of chips that'd triple up on a board like that would.
Peter: Good point.
Lawrence: Well, what about you now? what would you do?
Peter: Besides three chips at the same time?
Lawrence: Well, yeah.
Peter: Idle.
Lawrence: Idle, huh? Peter: I would relax... I would sit on my ass all day... I would idle.
Lawrence: Well, you don't need a million dollars to idle, man. Take a look at that fourth chip: it's two hops away, don't do shit.
Yes, it's possible. The main problem in general is that cost scales in proportion to the factorial of the number of nodes. The main problem in the specific case of Opterons is that each chip needs one HyperTransport controller per other CPU. Current Opterons come with up to three HT connections, and you need one for connecting to the PCIe bus, and other peripherals, leaving two for CPU-to-CPU connections.
I am TheRaven on Soylent News
I mean how to convince the wife that we need a three-way?
"Waste not one watt!" - CZ
x
/|\
/ x \ . . \
/ | \
/
x---------x
All generalizations are false, including this one. Mark Twain
Yes, the quad-core chips will have the fourth link. In addition, the chips will be able to split their 16-bit HT links into dual 8-bit HT links, allowing for 8-way CPU configurations without hops (8 x 8-bit HT links per socket). In reality, this is the reason why AMD is pushing the new HyperTransport 3.0: so they can cut the bus lines to 8 without sacrificing too much bandwidth.
Check it out here.
Man is the animal that laughs.
And occasionally whores for Karma.