Slashdot Mirror
Open Compute 'Group Hug' Board Allows Swappable CPUs In Servers
Nerval's Lobster writes "AMD, Intel, ARM: for years, their respective CPU architectures required separate sockets, separate motherboards, and in effect, separate servers. But no longer: Facebook and the Open Compute Summit have announced a common daughtercard specification that can link virtually any processor to the motherboard. AMD, Applied Micro, Intel, and Calxeda have already embraced the new board, dubbed 'Group Hug.' Hardware designs based on the technology will reportedly appear at the show. The Group Hug card will be connected via a simple x8 PCI Express connector to the main motherboard. But Frank Frankovsky, director of hardware design and supply chain operations at Facebook, also told an audience at the Summit that, while a standard has been provided, it may be some time before the real-world appearance of servers built on the technology."
And all you need, is a server case to fit it in. A 21" "new" standard to go nicely between already-existing 19" and 23" telco specs, which is just as well because in this day and age, metric still is too newfangled for teh zuck. Obxkcd and all that.
I don't get it. Are we redesigning the whole computer archetecture so we have a different group specing out the north bridge (etc) that all CPU manufactures will use? Or are they just adding an additional cpu into the current architecture the same way we do with graphics cards? And then offloading work onto it, like people have been doing for a while with GPGPUs?
I'm having trouble finding technical information about this design, and I'm curious how much of the motherboard logic has to move onto this daughterboard. For example, is memory still on the main board? If so, an x8 PCIe channel doesn't seem adequate.
who gives a fuck?
seriously. it's a stupid standard. why would I want to swap the CPU for another architecture? Why would I want ARM in a high performance server? Why would I want "easy" replacement of a CPU for another kind, when the rest of the motherboard isn't able to interface with it?
Why should I care for a "standard" connection where pins will be outdated 2 years from now?
Why are "high performance, low cost" servers socketed, instead of processors soldered to a motherboard? What dies is the motherboard, not the CPU. When the motherboard dies, the CPU is so outdated it doesn't even make sense to keep it. Why are we talking about socketed CPUs when a soldered-on will do just fine?
Why do we keep insisting in this new, useless, "proprietary open" standard that NO ONE will use (BTX anyone? wasn't it supposed to be the next great thing and solve everything? Why not focus, say, in a "heatsink landing" standard so i can fit ANY motherboard in a case (1U rackmount cases where the lid almost touches the processor) and have it touch the heatsink. Even make it easier to watercool if you want.
Still trying to figure out what's the deal with all this. Still trying to figure out why racks are limited to 42U or so, instead of less dense but "taller" racks (pretty sure a custom-made datacenter like google's or facebook's could get away with it. WAIT. Google already does!)
Really, let facebook fuck off and die already. It'll probably be dead by the time this "standard" hits the streets.
I can understand an organisation on the scale of Facebook wanting the ability to take advantage of bargains to buy processors in bulk and swap them out. I am not sure how widely applicable this is though.
The cool thing about ARM is the lack of complexity and therefore a potentially cheaper cost and greater energy efficiency. The daughter board seems to go against that by adding complexity, if you swap out an ARM chip, which might be passively cooled or have low powered fan, with some high end Intel server chip, you will also need to change the cooling and PSU.
My little Linux and tech blog
Maybe this would be useful in some HPC environments where applications can be written to maximize the use of CPU cache, but the bandwidth of a PCI /8 or /16 slot is a fraction of what is available to a socketed CPU.
A core i7 has been clocked at 37GB/sec bandwidth, while PCIe /8 is good for 1.6GB/sec, and /16 is good for 3.2GB/sec
Is replacing the CPU socket with a PCIe card really worth giving up 90% of the memory bandwidth? I've never upgraded a CPU on a motherboard even when new generation CPU's are backwards compatible with the old motherboard since if I'm going to buy an expensive new CPU, I may as well spend the extra $75 and get a new motherboard to go along with it.
Likewise, by the time I'm ready to retire a 3 or 4 year old server in the datacenter, it's going to take more than a CPU upgrade to make it worth keeping.
Indeed. Why not stick an Itanium on it too?
My little Linux and tech blog
I think I experienced something like this almost two decades ago. It didn't pan out. No one wanted to be limited to the least common denominator.
Anyway, the little detail I see in the two linked articles is that it's simply a standardized socket for a CPU mezzanine card that will mount to a motherboard that is little more than a x8 PCI express bus with some other peripheral devices/interfaces installed and power.
These comments are my own and do not necessarily reflect the views or opinions of my employer or colleagues...
A PCIe x8 slot is pathetically slow compared to the memory channels used by CPUs today. These CPUs are going to have to be used like GPUs, sent specific workloads on specific datasets to be useful. Any kind of non-cached memory access is going to cause major thread stalls and probably kill any performance benefits.
A general purpose compute card is probably useful in cases where GPUs aren't a good fit but you want more cores per RU than you can normally get, but I see this as a niche application for the foreseeable future.
I read the internet for the articles.
When they say '8x' PCIe, if that is accurate and the entiriety of the connectivity, what it really means is a standardized IO board for video, ethernet, maybe storage. The CPU daughtercard becomes the new 'motherboard' in terms of cost. You might marginally improve service time for the relatively rare board replace case (CPU replace case is already easy for repair action), but having that board segregated will drive up cost a bit.
They could mean that 8x is the data path and some amount of I2C and Power is allowed through as well to make it more like a 'daughter' board, but the high cost board will still be the daughter board (CPU is pricy and the layers required for a beefy memory subsystem drive up the actual board cost). Meanwhile you have a mess for management, thermal, and power management, likely resulting in having to be wasteful with fans.
Unless they make the daughtercard a behemoth, the design is a non starter for GPGPU and Infiniband applications (unless daughtercard has own PCIE slot allowed, 8x isn't enough for even one of those applications). The daughtercard must hold the memory, either meaning it will be large and expensive, or mandate soldered memory significantly restricting the customization potential in building out a system.
XML is like violence. If it doesn't solve the problem, use more.
Suppose you have a machine with an Intel x86 cpu in it and you swap it out for an ARM cpu. What happens?
The BIOS won't even boot! It's instructions are written in x86 machine language. Even if you somehow detect the new CPU architecture, and your BIOS has a copy of code that will run on it, then your OS won't boot!
What a novel but dumb idea.
They've rediscovered the backplane.
"A person is smart. People are dumb, panicky dangerous animals and you know it." - K
So i just ended up on wikipedia, good grief you where not even joking about that new 21" standard. :'(
http://en.wikipedia.org/wiki/19-inch_rack#Open_Rack
19" = 48,26cm wide.
21" = 53,7cm wide.
23" = 58cm wide.
48U = 200cm high.
So 58cm 48U racks it is, dully noted.
Hivemind harvest in progress..
PCI x8? 4Gb's a sec? That like a Pentium 4's 133Mhz quad pumped bus. Congrats on inventing something that transfers as fast as something from 2002!
What a POS.
Intel doesn't want to be put in a situation where its chips and AMD's (and now various vendor's ARMs) can be shoved in the same machine. That would make the server wars too real - it would be absolutely a race to the bottom, with daughtercards being replaced with the best *OPS/watt instead of whole servers being replaced.
It will be a really damned long time before this hardware ever meets production, if at all.
I think the 42U rack height comes from the standard 7-foot door/elevator height. Some datacenters definitely have custom tall racks already, and I wouldn't be surprised to see more of that in the future.
With a 8x pcie interface this sounds like most of the motherboard is on the card. You would be pressed to get storage IO and network IO though that interface forget memory. It's not going to handle many 40gb infiniband adapters or they will move onto the card. This sound alot like the old industrial single board computers of the 80's and 90's.
No sir I dont like it.
how does it work for HTX cpus? ram linked to cpus? QPI CPUS?
Now both AMD and Intel have Memory controller build in the cpus.
all AMD cpus have HTX to link them to the chipset / other cpu. Also that did plan for HTX slots at some time as well.
intel uses QPI in there higher end desktop systems and more then 1 cpu severs / workstations.
So the CPU is on the daughterboard. Everything that's specific to the CPU type (North/South bridges, etc) would have to go on there as well. Likely memory as well.
Congratulations. They've re-discovered the backplane and single board computer.
And how is this any different from the PICMG 1.x standards?
http://www.picmg.org/v2internal/resourcepage2.cfm?id=8
Lots of people have been building systems around this technology for years using passive backplanes.
Okay this didn't work in the x186 days, so I guess it's been long enough that it needs a redux. I remember when they were selling these machines which were essentially a backplane and one of the things you could plug into it was your CPU daughtercard. These were being sold as a way to "futureproof" (remember that buzzword?) your system. Turns out that was only good for a few CPU cycles...
http://www.opencompute.org/wp/wp-content/uploads/2013/01/Open_Compute_Project_Micro-Server_Card_Specification_v0.5.pdf
It uses that connector but the signals are ethernet, SATA, etc. RAM and optionally mSATA are on board.
they should be in HyperTransport slots / on the HTX bus
They want their industrial PC architecture back.
That change left the /proc/cpuinfo file insisting that I had one Intel and one AMD processor in the system.
[It didn't fix the problem - nothing ever did. I had it shut-down after 2 years of persistent problems and no useful work. No-one ever did explain why it would totally freeze (system clock stationary) for 60*n seconds]
It's PCIe x4 + 2*SATA2 + 1*GbE + 1*USB2.0 + 1*RS232 + power ... over a physical PCIe x8 connector.
So yes, it IS PCIe. And some other stuff.
== yet another fucking pointless host board "standard".
I wonder if the machine will stay up without rebooting while trying to isolate the failed CPU like how things work today.
We have some machines were you can reduce or expand memory on an online way, but I don't see a kernel behaving normally if one of the processors goes down for failure reasons
The purpose of this connector is to allow the connection of signal-processing co-processors. Two-dimensional video signal processing, similar to that sold at much cost by Texas Memory Systems, is greatly useful to Facebook, Google+, and all other cloud properties which track identity by facial recognition.
The image-processing algorithms are not as easily distributed as the search indexing. An approach which cuts through the problems at much better cost is the dedicated image processing SIMD pipeline style of processor.
In other words, the purpose of Mezzanine connectors is so that you can have a motherboard with, e.g., two AMD processors on it. You can then add your choice of accelerator via the Mezznine:
*A 2-d signal (image) processor
*A video compression/decompression accelerator
*A SHA256 / bitcoin collision accelerator ASIC
*An extremely high-bandwidth interface for a data acquisition peripheral
The purpose of Mezzanine connections has historically been just this: to provide heterogenous processor expansion to a motherboard.