SeaMicro Unveils 512 Atom-Based Server

1sockchuck writes "Stealthy startup SeaMicro has unveiled its new low-power server, which incorporates 512 Intel Atom CPUs, a load balancer and interconnection fabric into a 10u server. SeaMicro, which received a $9.3 million government grant from DOE to develop its technology, says its server uses less than 2 kilowatts of energy — suggesting that a single rack with four SeaMicro units and 2,048 CPUs could draw just 8 kilowatts of power. Check out the technical overview, plus additional coverage from Wired, GigaOm and VentureBeat."

Comments

Vitual center

[steelcobra]

Wow. Just think how many servers you could run in VMWare on that. A hundred would be a decent functional number.

Re:Vitual center

[robthebloke]

Absolute tosh. You need CPU hardware virtualisation support to do those things you speak of. Those features are no where to be found in the Atom. You will not be running VM's on any Atom based system because they are simply not up to the job. (I speak from experience here)

Re:Vitual center

[AigariusDebian]

Actually it is much more interesting to handle each of them as you would handle an individual virtual machine - so you have 512 nice low-powered virtual servers with each of them having a fixed and dedicated processor.

In fact such a load-out would be very useful for hosting companies - you can have a ton of small clients with minimal management or scheduling burden.

Re:Vitual center

[sleeping143]

I think the more important element here is the performance/energy consumption ratio. Atoms might be slow, but they're not so slow that their minuscule power consumption can't make up for it.

Re:Vitual center

[asdf7890]

SMP will only bring you so far - i'll bet 8 VCPU VMs on Atoms will be beat by a 2 VCPU VM on a Core 2 Duo.

Perhaps not, depending on the other load the system is working on. Because of the way VCPUs are scheduled (at least in VMWare) that 8-vCPU VM won't get a time-slice until such time as there are 8 real cores available for the duration of that slice. If your task is CPU intensive and can be easily separated into distinct tasks not overly chatty (i.e. cross VM latency is not going to be a major issue) and the host has gobs of RAM available, you are often better off having several VMs with one cVPU each than one VM with several vCPUs. This may be much less of a problem on a many-CPU monster like the 512 core unit being discussed than it is on 2/4/8-core boxes, but I expect the balance to still be in favour of multiple single-vCPU VMs in cases where the task can be efficiently split between them.

Re:Vitual center

[BitZtream]

Thats funny cause I have a Intel D410 Mobo that runs a couple virtual box instances on top of FreeBSD.

Virtualization of the x86 existed before Intel added special support for it.

Re:Vitual center

[Anonymous Coward]

check your facts - the atom Z530 which they use does have VT-x. and people did virtualize before that , you know.

Re:Vitual center

[Josh Triplett]

SMP will only bring you so far - i'll bet 8 VCPU VMs on Atoms will be beat by a 2 VCPU VM on a Core 2 Duo.

Perhaps not, depending on the other load the system is working on. Because of the way VCPUs are scheduled (at least in VMWare) that 8-vCPU VM won't get a time-slice until such time as there are 8 real cores available for the duration of that slice.

Not all virtualization systems have that limitation. In a modern VM, each VCPU gets scheduled separately on the physical CPUs, rather than using gang-scheduling, for exactly the reason you described. That way, if you have a pile of N-cpu VMs, each of which just has one or two CPUs waking up periodically rather than all doing intensive computation, they can all share relatively few hardware CPUs and run efficiently.

Re:Vitual center

[Jah-Wren Ryel]

You don't need hardware support for paravirtualization and paravirtualization will handle everything you listed. It won't let you run different OSes but there are plenty, potentially even a majority, of cases where that's not necessary.

Re:Vitual center

[afidel]

Which would you rather have in 8kw, 2048 atom cores or 768 56xx cores? Because that's what HP can cram into 4xc7000's in the same power and floor space envelope (not sure about capital outlay though). Personally for my workloads I'd take the 56xx cores, but YMMV.

What's the "bang for the buck"?

[TeXMaster]

The question is, how good is the performance for, say, intensive numerical computations? Is the gigaflop per watt convenient?

Re:What's the "bang for the buck"?

[Nadaka]

The FLOPS are not all that important for this device. It isn't designed to crunch big numbers. It is designed as a web|web application server with the goal of serving far more connections per watt than a traditional server.

Re:What's the "bang for the buck"?

[jandrese]

But traditional web servers aren't CPU bound, they're IO bound at high connection rates. It might help if you need to do a whole lot of https traffic, but even then this smells of overkill. If you're really planning to use this as a webserver, I'd be a whole lot more interested in the IO backplane and the available IO ports to the server.

Re:What's the "bang for the buck"?

[fuzzyfuzzyfungus]

Depends on your storage medium. A hogshead'sfull of class-6 SD cards will be nice and zippy. If your enterprise data pedlar stiffs you with a bunch of class-2s, it'll take a lot longer.

Re:What's the "bang for the buck"?

[robthebloke]

Almost, but not quite. The things that suck about the atom:

1. double precision. Use a double, and the Atom will grind to a halt.
2. division. Use rcp + mul instead.
3. sqrt. Same as division.
All of those produce unacceptable stalls, and annihilate your performance immediately. So don't use them!

Now, you'd imagine those are insurmountable, but you'd be wrong. If you use the Intel compiler, restrict yourself to float or int based SSE instuctions only, avoid the list of things that kill performance, and make extreme use of OpenMP, they really can start punching above their weight. Sure they'll never come close to an i7, but they aren't *that* bad if you tune your code carefully. Infact, the biggest problem I've found with my Atom330 system is not the CPU itself, but good old fashioned memory bandwidth. The memory bandwidth appears to be about half that of Core2 (which makes sense since it doesn't support dual channel memory), and for most people that will cripple the performance long before the CPU runs out of grunt.

The biggest problem with them right now is that they are so different architecturally from any other x86/x64 CPU that all apps need to be re-compiled with relevant compiler switches for them. Code optimised for a Core2 or i7 performs terribly on the atom.

Re:What's the "bang for the buck"?

[drinkypoo]

Then they screwed up, and they should have used ARMs, because a great deal of Atom's performance lies in its multimedia instruction set. Or in other words, if you're not pushing flops, you have a lot of hardware lying around unused. Atom delivers a lot of flops (or iops, for that matter) but doesn't shovel data any more efficiently than anyone else.

Am I the only one who....

[mark-t]

...misread the headline as saying that somebody had made a server out of only 512 atoms (as in the particle, not the cpu)?

Imagine a beowulf cluster of...

[Chas]

Oh, wait. Nevermind.

Re:Imagine a beowulf cluster of...

[CAIMLAS]

It would seem that the 'beowulf cluster' is starting to fall out of style, doesn't it? :P We're getting to the point where such concepts are as quaint as a "Cray supercomputer" were just a couple years ago.

Re:Imagine a beowulf cluster of...

[TheRaven64]

Clusters are still very much alive. They're cheap to build and give you a lot of computing power to play with. If anyone mentions Beowulf when describing them, however, it's a good clue that they have no idea what they are talking about.

For their next trick

[polaris20]

They're going to power a Ferrari out of 34.5 Vespa scooter engines.

Re:For their next trick

[by (1706743)]

Or power a motorcycle with 24 chainsaw engines.

What does a normal rack consume?

[Call Me Black Cloud]

I don't know if 8 kW is a lot or a little less than a normal rack would draw. The article doesn't say...how much power would a "normal" rack consume? Isn't there a measure of computing power vs power used?

Re:What does a normal rack consume?

[ircmaxell]

A typical 1 U server (2 processor 4 cores each) can consume between 300 and 500 watts. So 42 of them (the number in a rack) would give 336 cores at between 12kw and 21kw. As a rough number for reasonable performance servers, 300 to 500 watts per U is about what you'll find. Sure, you can find more powerful and less powerful servers, but that's a decent figure...

Re:What does a normal rack consume?

[geekboybt]

That's not too hard to calculate. I usually budget 2 A for a dual-CPU 1u server. At 120 V, that's 240 W. 8 kW divided by 240 W = 33u of servers. Multiplied by 12 cores (allowing for the new X5600 series, ignoring hyperthreading on both Atom and Xeon), you get 396 total cores of standard Xeon in 8 kW. Meanwhile, they're advertising 2048 Atom cores in the same amount of power. So, the real question becomes how powerful ~5 Atom cores are compared to 1 Xeon core.

Re:What does a normal rack consume?

[ircmaxell]

Whoops! Good point. So that brings our total down to about 4 Watts per core for the Atom... Even further strengthens the point. Thanks!

Operating system not mentioned?

[dragisha]

Or it's only me who can't find it?

"No changes to software" or something like that.... And only tons of RFC* and "funny acronyms"... What software needs no change?

Low power, really?

[TheRaven64]

In all of the benchmarks that I've seen, clock for clock a Core 2 gets about twice the score of an Atom, sometimes more. The Core 2 uses a bit more than twice as much power, but if you have two Atoms you also need twice as many north-bridge chips and this pushes the power usage up to over what the Core 2 will consume. The newer Xeons do even better.

The first benchmark results I found that compared the two were PassMark benchmarks, where a 2GHz Atom scored 386 and a Intel Xeon X5680 at 3.33GHz scored 10620. The fastest Atom, the D510 at 1.66GHz, scored 662. Even if your code scales linearly, you need more than 16 of the fastest Atom that you can buy to replace one Xeon. Or, to put it another way, this 512-Atom machine is about as powerful as a 32-CPU Xeon.

A single Atom D520 draws around 13W, so 16 of them draw 208W. The Xeon will draw 130W. Drawing under 2KW for 512 Atoms means that they probably aren't using the fastest available ones. Actually, it means that they're drawing under 4W per Atom, which means that they're probably using Z-series Atoms, getting about half the performance of the D-series ones, so you'd only need about 16 Xeons for the same performance.

For most workloads, if the server is not busy, you'll get much better power usage from the Xeon as well. Power usage drops off dramatically when the CPU is not 100% busy. Unless you are turning individual atoms off completely, you can't scale back power usage nearly as well with the Atoms, as single processes that would not be CPU-bound on the Xeon will require an Atom core to run at full speed.

In other words, it sounds a lot more like greenwashing than anything that's actually sensible.

Re:Low power, really?

[Jeng]

Considering how many articles were linked I don't know if you rtfa'd or not.

The Core 2 uses a bit more than twice as much power, but if you have two Atoms you also need twice as many north-bridge chips and this pushes the power usage up to over what the Core 2 will consume.

This is from the wired article.

Just changing the CPU to a low power chip, though, isn't enough says SeaMicro. The trick lies in creating a new architecture that can pull all the chips together and manage their power requirements.

"If you just replace the chips in a traditional server with Atom processors, the power consumption actually goes up," says Feldman.

Integrating features such as storage, networking and server management into a single ASIC (application specific integrated circuit) helps manage power better, says the company. It has also virtualized the CPU input-output so those modules that would have otherwise occupied space on a board and consumed power don't anymore.

Re:Low power, really?

[Skal Tura]

Because you are thinking serial while they are thinking parallel.

How many simultaneous operations can do 512 atoms VS. say total 128 Xeon cores?

What happens when single operation is extremely small, but there are extremely high volume of them?

What happens to a CPU core while it's waiting for RAM or other I/O? Yea, that's right: It waits.

What happens to memory IOPS when you have 512 channels versus 128 dual-channels? Yup, it's vastly higher, but not actually just twice, but quadruple (dual channel is for dual bandwidth, not dual IOPS afaik)

Re:Low power, really?

[Big_Mamma]

The 130W parts are usually only sold to the people who don't care about both performance per watt tho, X5670 is a better choice. Using Intels numbers on power usage, as the 4W/CPU claimed including rest of the system is just ridiculous, the numbers are: 386/8W or 50 points per watt for Atom, or 9356/95 or 98 points per watt for Xeon X5670. Xeons win the race with a 100% margin. Even the X5680 scores 80p/W. If you rig the numbers and use 386/4W, the Xeon still wins. What saving?

Also, no matter how much power they claim to save with this beast, it's nothing compared to virtualizing a rack or more into 3 ESX hosts. With this Atom "solution", you'll just have tons of nodes on 0.00 load again like before, instead of having all the physical servers at a load you're comfortable with. To make things worse, if you run a job on the Atoms, it will run in slow motion because of the "performance" of that CPU - you cannot burst to the full 2/4/8 vCPU allocated, @x3 speed +turbo when needed. It's *much* harder to use 512 slow CPU's than fewer, faster ones.

And never mind running any commercial software on it. The per core/socket license will make it impossible.

Other options (in 10U, 240 opteron cores can fit)

[IYagami]

SeaMicro: 512 Atom processors in 10U rack
Dell PowerEdge R815: 4 Opteron processors, each one up to 12 cores, in 2U rack. In a 10U you can include 5 of these servers, which will bring 4 x 12 x 5 = 240 Opteron cores

More info here

System Specs

[dlapine]

This is a good start- SM10000 System Overview

Interconnect is 1.28 Tbps or 2.5 Gbps per core.

I/O includes a minimum or 8 gige or 2 10-gige, which can be increased to 64 gige or 16 10-gige links per chassis.

This unit runs as 512 system images using stock 32 bit OS's. Each CPU may have 1 or 2 GB's of ram and up to 64 local drives may be installed and divided among the CPU's with the included management software. The unit supports PXE boot, so the system images may run off local disk or from a ram image.

Just to note, the Atom z530 is a single core, 32 bit only CPU, if that matters.

I couldn't tell you if the 16 10-gige links would seriously limit this box or not. You'd have to show me a data center with more than 160 Gbps of internet connectivity first. :) And that's assuming you only purchased one of these suckers, because you'd need that much per chassis.

Re:Other options (in 10U, 240 opteron cores can fi

[washu_k]

I've seen this posted before in other Atom stories and it's 100% BS. The Atom is a full CPU, they did not shift any features into the northbridge as you claim. The 945 chip often used on Atom motherboards has been around for a while, and was used in systems before the Atom even existed. The 945 is just not very power efficient so that's why it needs more cooling than the CPU.

The Atom can be paired with other northbridge chips, notably Nvidia's ION. If part of the CPU was in the 945 chip as you claim this would be impossible.