How To Build a Supercomputer In 24 Hours

An anonymous reader writes with a link to this "time lapse video of students and postdocs at the University of Zurich constructing the zBox4 supercomputer. The machine has a theoretical compute capacity of ~1% of the human brain and will be used for simulating the formation of stars, planets and galaxies." That rack has "3,072 2.2GHz Intel Xeon cores and over 12TB of RAM." Also notable: for once, several of the YouTube comments are worth reading for more details on the construction and specs.

Comments

Pretty sure

[Anonymous Coward]

that my old palm pre has more computing power than most human brains on this planet.

Re:Pretty sure

[bertok]

Computers have surpassed that level a loooooong time ago

Doubtful.

The computational requirements for simulating the human brain have been severely, even hilariously, underestimated in the past. To quote Wikipedia: One estimate puts the human brain at about 100 billion neurons and 100 trillion synapses.

That's... a lot.

First off, a lot of people think that 1 FLOP = 1 Neuron, which is not even close. The active points are the synapses, of which there are about a thousand per neuron! Each may receive an impulse over ten times a second, and involve dozens of parameters, such as the recent history of firings, neurotransmitter levels, hormone levels, membrane potentials, etc... A very conservative estimate would be that a single neuron, receiving impulses at around 10 Hz on 1000 synapses would require on the order of 1 megaflop to simulate. That's ONE neuron. Now multiply that by 100 billion, and you get a picture of what's required: about 100 petaflops, minimum. Storage is nothing to sneeze at either. Assuming a mere 50 single-precision floating point values per synapse to store all simulation state, you're looking at almost 18 petabytes of memory! That's over $100M for the memory sticks alone, even with a deep bulk-purchase discount. Unlike most server or HPC workloads those 18 petabytes would have to completely read out, processed, and possibly updated again at least ten times a second.

Second, consider that the first simulations won't be very optimized. We still don't really know what's relevant, and what can be simplified away. Hence, I suspect that the first attempts will be much less efficient, and may require 10x or even 100x as much computer power compared to later attempts. For example, neurons don't just fire impulses, they also grow and change shape. I don't think there's even a good model for how that works in the complex 3D environment of the brain!

We are getting closer, but expect to wait at least a decade or two before people start talking seriously about a full human brain simulation.

Re:Pretty sure

[AK Marc]

So these things happen on different (connected) machines if you will. Every neuron is a processor in itself anyway.

I think that's part of why computing power greatly surpassed humans long ago, and will not reach human levels for many years. The brain isn't digital. It holds an "infinite" number of analogue states, simultaneously. With massive errors and gaps filled in with guesses made from other parts, without even an minor error check that indicates that he information being determined to be "true" is 100% interpolation with 0% fact or actual memory. The very idea of an error check that was wrong more than right and kept no indication of where the result actually came from is so incredible that nobody would ever create a computer capable of operating that way. It won't be until we have computers many millions times more powerful where we can remake a "perfect" brain, until then, we'll never be able to match the capabilities of the human brain.

Re:Pretty sure

[Anonymous Coward]

Doing equivalent computation to a human brain, and simulating a human brain, are two very different problems.

You have to solve a non-linear coupled differential equation at 1MHz to simulate a 5kHz sawtooth wave generator in Spice. It's about 30 MFLOPS. But, functionally, all you're doing is generating a 5kHz sawtooth wave, which is 15 kFLOPS of work. This is a 2000:1 efficiency difference between simulating an analog system and running a direct digital equivalent implementation.

So divide that 100 PFLOPS by the fundamental inefficiencies of simulating the analog domain in the digital domain, and you get a more reasonable figure for when a computer can functionally compete with the human brain.

Re:Pretty sure

[zbox4]

your assumptions are close to mine when i estimated the ~1% compute capability of the brain. individual neurons send an outgoing signal depending on the amount and rate of incoming signals), but i am an astrophysicist, not a neuroscientist ;) the zBox4 can calculate at over 10 petaflops.

Re:Pretty sure

[Anonymous Coward]

We are getting closer, but expect to wait at least a decade or two before people start talking seriously about a full human brain simulation.

Boats and submarines don't "simulate" fish, but they still swim.

Airplanes don't "simulate" birds, but they still fly.

Artifical intelligence may not need to "simulate" the brain to reach human level.

Just sayin'.

Re:Pretty sure

[__aaltlg1547]

Google's cars say different.

Looking at physical complexity:
A human brain has about 86 billion neurons. An Intel Core-i7 process has 731 million transistors. A neuron is more complex than a transistor. Let's say it does a job, for the sake or argument, that would take about 16 transistors. So say the Core-i7 has the equivalent of about 45 million neuron-equivalents. That's a factor of about 1900 in physical complexity.

But the brain manages to pull off a clock cycle about 200 Hz, based on the neuron's firing rate. Maybe 1000 Hz at most. The clock rate of the CPU is 3.2 GHz. It is 16 million times faster than your brain.. Since the computer can execute programs of arbitrary complexity, it can simulate your brain's operation -- if properly programmed, with a much smaller hardware set running much faster. In raw computational capacity, it apparently has 16 million / 2000 = 8000 times the computational capacity of your brain. So even if its' simulation were quite computationally inefficient, it should still be able to do the job of a number of brains, if programmed to do so.

In short, exceeding the capacity of a human brain isn't a hardware problem any more. It hasn't been for years. It's a programming exercise, albeit a particularly challenging one.

Re:Pretty sure

[raftpeople]

I think you are making some gigantic assumptions about how the brain works if you think you can even calculate a comparison of the brain to any computer at this point in time. Are you aware of the following (for example)?
1 - White matter (neuroscientists are discovering) is actively involved in computation and it changes due to learning - there are 10x more glial cells than neurons
2 - "Type" of dendrite (in some cases) influences how the signal is transmitted and at what rate and this function plays a part in learning, independent of the synapse
3 - In some cases firing is triggered purely by the electrical field the neurons are surrounded by - no synapse
4 - Brain waves transmit information from one region to another

My impression of reading neuroscientists is that they don't think they can accurately estimate brain processing power - everything is just very simplistic ballpark guesses that vary by multiple orders of magnitude.

Re:Pretty sure

[pitchpipe]

The brain isn't digital. It holds an "infinite" number of analogue states, simultaneously.

You can't approximate a very large number with infinity. The difference between a very large number and infinity is ... well ... infinity.

Re:Pretty sure

[Nemyst]

That's a rather simplistic analysis. The problem is that we don't even know how the brain fully works. If, for instance, neurons make use of quantum properties (a very real possibility), then classical computers may be hard-pressed to replicate even a mouse brain. We don't know whether quantum computers are more powerful than classical computers. Many believe that they are, but we have not found proof of this. They could be anywhere from equal to exponentially faster, and you know what exponentially faster means with numbers like what you've been pulling.

Your entire post is based on the premise that a neuron and a transistor are even comparable. That's one hell of an assumption.

Title could be

[hcs_$reboot]

"How to spend $800,000 in one day"
Price, from comments:

Just under 750,000 Swiss Francs, or about $800,000

Re:Title could be

[Gaygirlie]

"How to spend $800,000 in one day"

Challenge accepted! Now to seek someone to finance the challenge!

Re:Title could be

[zbox4]

we benchmarked various configurations with our codes. then we made a tender for the motherboards, memory, cpu's. we got a pretty good price ;)

Re:quantum entanglement -- my brain to wind ?

[edibobb]

Why do you use 16th century English? Is this really how you speak, or did you run this through an obfuscator? What are tares, and what does "lest while ye" mean?

Yes, but...

[jerry.tk]

... can you build a Beowulf cluster of those?

Re:Yes, but...

[ls671]

Didn't you notice the modern miniaturized version of on board Beowulf cluster integrated in a chip chips they put it there? I would guess they put about 50 of them in the rack but since it was in fast forward, I couldn't count accurately. Anyway, if they build a Beowulf cluster of those, we will end up with a Beowulf cluster of Beowulf clusters.

Re:quantum entanglement -- my brain to wind ?

[Gaygirlie]

What are tares

"Any of several weedy plants that grow in grain fields." -- http://www.thefreedictionary.com/tare

and what does "lest while ye" mean?

The sentence would be approximately "But he said: 'No, out of fear that while you root the weeds you also root up the wheat with them.'" Ie. "lest" is used to denote the fear or danger of something happening.

Re:It's already done.

[DMiax]

Not if you want to have a result in your lifetime. Cloud-based systems can be more powerful than your desktop, but are terribly slow compared to even an average supercomputer like this one (they are using ethernet cables for the interconnects...). The stuff you want to compute on these machines requires a high level of communication between the processes, else you would simply run them on several decoupled machines in parallel. You cannot access the required speed for inter-process communication on the cloud.

Also notable:

[tbird81]

"for once, several of the YouTube comments are worth reading for more details on the construction and specs."

Yeah, unlike the impeccably high standard of comments you see on Slashdot. Mod me up if you hate Bieber!

Re:Also notable:

[Anonymous Coward]

Racist comments don't routinely stay modded up on slashdot as they do on many parts on YouTube. Use of dubious debating techniques such as the strawman usually gets noticed here. Unpopular viewpoints are often modded up to +5 Interesting if they are sufficiently well argued.

Slashdot ain't what it used to be, but it still has standards.

That pesky static discharge

[hardtofindanick]

only seems to bother EE majors and everyone else seems to be immune to it.

Re:That pesky static discharge

[thegarbz]

The risk of damage depends on the sensitivity of the component. For instance good luck wreaking a standard BJT with ESD. On many ICs the I/O is either suitably protected against ESD, or the nature of the design makes it less vulnerable to the effects of a sudden static spike.

It's a very different story for say working with high frequency MOSFETs, uncut silicon wafers, or pretty much any RF gear. I remember in our lab one of the PHD students was working on a silicon wafer. The instrument had the wafer suspended on a cushion of air. One of the other students rushed into the room suddenly and the wind from opening the door made the wafer float towards the edge of the instrument and was stopped instinctively with a hand that prevented it from falling off. 2/3rds of the transistors failed testing after that, the survivors were on the opposite side of the hand that steadied it.

Also you think EE majors are bad you should try Hams. A Motorola technician nearly bit my head off when he saw me dismantle one of their repeaters without an static strap.

Re:It's already done.

[DMiax]

Yes, and it is nice. the question is whether you value having the hardware or not, i.e. for how long your money will get you running on those servers rather than a one-time payment and keeping the cluster with you. More in general, I take issue with the attitude that the best solution to any problem is always paying some company to do the work for you. It is, if you have the same needs as everyone and a dedicated company can lower the costs, but you should always ask yourself if this is the case.

OCZ Rebates

[DarwinSurvivor]

So who's job was it to mail in all the OCZ rebate forms?

Re:Headline is stupid

[zbox4]

it took ~year to acquire the funds, benchmark tests, fix the design, make the tender for the parts etc, but all the construction was done in 3x8hr shifts

Re:Software

[zbox4]

we use various astrophysics simulation codes, i.e. GASOLINE, PKDGRAV, RAMSES etc. some are developed by us. they are all MPI and solve the coupled gravitational and hydrodynamic equations that can describe the dark matter and baryons evolving in the expanding universe. memory and speed of the computer limit the resolution that can be attained, so various "sub-grid" physical processes have to be treated carefully. for cosmological simulations we know the initial conditions - those are the fluctuations that we can read off the microwave background. they show the universe was hot, dense and smooth early on. the codes follow the perturbations into the non-linear regime when dark matter haloes, stars and galaxies form. we can then compare the properties of simulated structures with observational data etc.

Re:Headline is awesome

[Barny]

I will ask the inevitable questions, as a system builder.

How many parts were DOA?

How many failed inside of the first month?

Re:Headline is awesome

[zbox4]

surprisingly few - a couple of bad motherboards (or static ;). its only been up for a week or so and we are still testing/installing stuff before making user queues live.

WTF?! have these kids never heard of ESD?

[metaforest]

NO grounding straps.
NO signs of any ESD precautions!!!

Lacking from the video is the debugging process.

Sure they built it in 2 days... but how many nodes came ready?

I was cringing through the whole video over their lack of concern for basic ESD prevention. They don't need to be wearing bunny suits or anything that extreme, but FFS.... could ya show a little bit of respect for the hardware? Heck even clipping the freaking base-plates to ground during assembly would have been more than adequate.

That video was like watching "OW MY BALLS" for geeks!

Comment removed

[account_deleted]

Comment removed based on user account deletion

But what about all the Garbage?

[Grumpinuts]

Used to do this kind of stuff when I was with IBM about 10 years ago, we had a group in XSeries Manufacturing who specialised in quick turnaround configuration of HPC rack systems just like this. Funnily enough, one of the major logistical elements was dunnage, ie the empty cardboard/foam and plastic that all the option parts arrive in. When running full out we used to have 1-2 guys per shift just to move the rubbish out to the big compactors out back. You wouldn't believe just how much packaging even a comparatively small cluster like that can generate.

Re:1% of the human brain ?

[Thiez]

> The computing power of the human brain is infinite, by some standard definitions, as it is analogue (at least partially, according to some theories) and there are an infinite number of analogue states.

Just because the brain theoretically has a practically infinite number of possible states does not mean all (or even most) of those states are meaningful and important. People lose thousands of neurons each day without changing significantly (with respect to both personality and intelligence). Obviously the brain contains a lot of detail that it doesn't really need. There is no reason to believe it is impossible to create a significantly less complex model/simulation of the brain that is both functionally equivalent and finite (except our ego, which seems to prefer to think of the human brain as infinitely complex).

Re:A lot of CPUs != Supercomputer

[symbolset]

Yeah, like infiniband, Linux, high performance high volume storage. Which if you read further you will find they also have.