Slashdot Mirror
There's A Cluster of 750 Raspberry Pi's at Los Alamos National Lab (insidehpc.com)
Slashdot reader overheardinpdx shares a video from the SC17 supercomputing conference where Bruce Tulloch from BitScope "describes a low-cost Rasberry Pi cluster that Los Alamos National Lab is using to simulate large-scale supercomputers." Slashdot reader mspohr describes them as "five rack-mount Bitscope Cluster Modules, each with 150 Raspberry Pi boards with integrated network switches."
With each of the 750 chips packing four cores, it offers a 3,000-core highly parallelizable platform that emulates an ARM-based supercomputer, allowing researchers to test development code without requiring a power-hungry machine at significant cost to the taxpayer. The full 750-node cluster, running 2-3 W per processor, runs at 1000W idle, 3000W at typical and 4000W at peak (with the switches) and is substantially cheaper, if also computationally a lot slower. After development using the Pi clusters, frameworks can then be ported to the larger scale supercomputers available at Los Alamos National Lab, such as Trinity and Crossroads.
BitScope's Tulloch points out the cluster is fully integrated with the network switching infrastructure at Los Alamos National Lab, and applauds the Raspberry Bi cluster as "affordable, scalable, highly parallel testbed for high-performance-computing system-software developers."
BitScope's Tulloch points out the cluster is fully integrated with the network switching infrastructure at Los Alamos National Lab, and applauds the Raspberry Bi cluster as "affordable, scalable, highly parallel testbed for high-performance-computing system-software developers."
Did they make a Beowulf cluster of those?
Fuck Beta!
When somebody buys 750 all at once.
It was my experience that pi's are hard to buy so i gave up trying to get one. Mind you when people use ancient rasbian os and make 'secure' email servers on port 26 and then get called out for issues it is good to see that somebody is using them properly instead of poorly.
As in bidirectional communication I assume!
Twinstiq, game news
You are missing the point! The idea is not to have an super computer but to emulate one. Writing code for stuff like thus is hard and running it on the real deal is expensive. This way the can emulate 750 core system at an fraction of the cost.
Apparently the point is to simulate a powerful machine with many cores, so that people can develop and optimize their code without requiring CPU time on the actual (very expensive) machine.
If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
If you cannot get 1000's of slow cpu's to scale, then wasting debug time on the big fast server is really a waste. Today's programmers need to learn how it used to be. Even with using RPI's they have an advantage. The network is much faster than what we had 20 or 30 years years ago. Internal busses are faster, ram/memory is faster, caches are faster. This is a smart way to spend money for a bringup development environment on the cheap.
You get effect of network latency to induce concurrency paradoxes that wouldn't happen on a shared memory system.
ObCarAnalogy: a single bus can move a lot of people, but if you're modeling highway traffic, you want to use many independent cars.
Not really, but it does show that there are a lot more idiots like you coming here, and a lot fewer of the people who belong here. My very first thought was "Holy shit! Something that actually belongs on Slashdot on Slashdot!" If your thought was "meh" then I have no idea why you even come here.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
You are missing the point! This way the can emulate 750 core system at an fraction of the cost.
So, what point am I missing? The Xeon phi 7290 is 4k$ and has 72 cores, you can get 10 of those and get way more speed, shared memory benefit etc...
Entirely different architecture. The point of this scale model is to have a cluster of compute nodes with TCP/IP communication between them.
So, what point am I missing? The Xeon phi 7290 is 4k$ and has 72 cores, you can get 10 of those and get way more speed, shared memory benefit etc...
The shared memory is a detrator not a benefit if you're trying to have something which emulates an expensive distributed architeture. The point isn't to get lots of speed, it's to get a bunch of cores distributed over a local network in order to get a cheap test bed emulation of a much larger machine.
SJW n. One who posts facts.
You don't need a supercomputer to figure out that the headline is poor usage. The Chicago Manual of Style will do that for you.
10 CPUs with 72 cores each is 720 cores.
750 SOCs with 4 cores each is 3,000 cores (and RAM and motherboards included).
The point is to have a massive number of cores in a large number of machines, to simulate a large number of machines, at the budget point. Your idea would have 75% fewer cores.
> shared memory
Yep, that's another problem with your idea. It would no longer be an accurate simulation. Well except your plan doesn't include any RAM at all. Or motherboards, networking, etc. You're going to need to buy 750 network cards to simulate 750 machines, motherboards each capable of holding 18 cards, a number of storage devices, etc. So maybe FIVE 7290 CPUs with exotic motherboards plus RAM, network cards, storage, etc. Five 7290s would provide 360 cores, vs the 3,000 cores they got with the Pis.
Now AFTER the research yields fruit, in a couple years someone might want to put the ideas into production using fifty 72-core processors which may cost $2,000 each.
This happens so often that I think we need a new mod:
Score: -1 Wrong topic, you idiot
#DeleteFacebook
That was 20 years ago. Today we don't get those kinds of performance leaps anymore.
#DeleteFacebook
Here you can unplug a node to simulate a hardware failure. The latency is more real world between nodes. Cache levels are more similar (L1 L2 RAM) , hardware levels (nic, bridge, CPU). It's a cheap approximation. Leave it at that.
Thereâ(TM)s A Cluster of 750 Raspberry Piâ(TM)s at Los Alamos National Lab
There. Happy now?
#DeleteFacebook
Not useless if you're debugging queue systems, schedulers etc.
ROFL. We're not talking about debugging the scientific number crunching code that will run on the actual cluster, but the cluster management software. The actual jobs to run may very well just be doing sleep(10000*rand()); if rand()0.1 call WriteAllTheDiskSpace; else if rand() 0.2 then call segfault_horribly(); else return SUCCESS;. etc.;, one should probably add in a few more "bad things", MPI calls etc.
No, there isn't a square-boxed question mark.
Your analogies have zero relevance to how HPCs work.
They are relevant. What is zero is your willingness to learn, or at least accept that other people know what they are doing.
1440 is NOT 3000!!! AND
your power budget went to
hell in a hand basket.
One other thing, your being
a DICKWAD.
The only difference between a cluster and and a shared memory machine is you pay by the minute on the cluster. If you haven't realized that consumption fee is the same whether you are debugging or doing actual science, then you are the moron who can't see the point of using a less powerful less costly cluster to do mockups.
The RPi modules are 4-core, so the cluster is 3000 cores.
Awesome furniture, accessories and cabinetry in Santa Rosa, CA: http://humanity-home.com/
Interconnects don't matter much. Whether you use InfiniBand, GigE or serial, you're just pumping TCP packets.
Custom electronics and digital signage for your business: www.evcircuits.com
I don't think it is acceptable to make an understandable and relevant car analogy.
I wonder how many Commodore 64's I could emulate at once on a decent sized workstation. Maybe 500?
“Common sense is not so common.” — Voltaire
You could do all that by setting up a bunch of VMs on a shared memory machine.
That will give you different latencies and different bottlenecks. The point of this system is not to crunch data, but to serve as a testbed for parallel software development. It is possible that they also use VMs, but that would be in addition to this cluster rather than a replacement.
1kW at idle is a lot. You could cut that down by shutting down Pis in banks as they went unused, and firing them up again as needed. It wouldn't require very much more hardware, just some microrelay boards which can be driven by some of the Pis themselves.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
There's A Cluster of 750 Raspberry Pi's at Los Alamos National Lab
I saw a bunch of them at the grocery store before Thanksgiving, next to the apple ones.
It must have been something you assimilated. . . .
Emulating the cores would falsify what they are testing since this would reduce a lot of possible race conditions (among other things). Virtualization is nice but it's not an end all solution.
Purely out of academic interest, how fast is this thing? How does it compete with, say, a 16 core Xeon or Threadripper workstation?
The dangers of excessive individualism are nothing compared to the oppressiveness of excessive collectivism
And running 1.2GHz 4 core STB processors over 10/100 ethernet is going to be similar to clusters of dual socket 3GHz 54 core processors with 25+ Gbps interconnects? (aka Cavium ThunderX2 CPUs. Nobody is planning on building an ARM based supercomputer with only one CPU per node, let alone with IO limited smartphone/tablet/set-top-box oriented SoCs)
This will obviously be used to verify global warming, so it belongs here. Let's argue politics!
Have you read my blog lately?
You raang?
I am Audience.
Oh yes, true. Now why on earth didn't the folks Los Alamos think of that!? They must be complete idiots. You should write to them and explain your ideas - they might give you a job as their chief architect, or maybe their Head of Cost Cutting.
I think this was also on slashdot last year:
https://robertmcgrath.wordpress.com/tag/the-megaprocessor-laughs-at-your-puny-integrated-circuits-stephen-cass/
Tracy Johnson
Old fashioned text games hosted below:
http://empire.openmpe.com/
BT
moderation is for assholes , but ...
here you don't get moderated, you get rated ...
one might say you seem to be over reacting a bit
but thats fine, i divide my days between standard and less bad too, if this is your biggest problem today then it can't be that bad its not like you get money or anything for it, right ?
i like the "green computing" approach here btw
Free speech was meant to be free for all... how can anyone grow up in a nanny state ?
They are testing how their software scales to a massive amount of cores. This you cannot do on a single Xeon Phi. The speed and available bandwidth is irrelevant for that, it is of course relevant for other test cases but that is not what they test here.
No because if a single core emulated 10 other cores there will i.e never be a situation where those 10 cores execute an instruction all at the same time. The laws of physics you know.
Someone should invent software for emulating a CPU, that way you could use one machine to emulate many.
I'd call it a virtual machine.
And you cannot (as of yet) effectively simulate the kind of massive scale out that places like this code for.