A Million Node Supercomputer

An anonymous reader writes "Veteran of microcomputing Steve Furber, in his role as ICL Professor of Computer Engineering in the School of Computer Science at the University of Manchester, has called upon some old friends for his latest project: a brain-simulating supercomputer based on more than a million ARM processors." More detailed information can be found in the research paper.

Comments

Re:Tremendous overhead

[creat3d]

FTFA: "To overcome this limitation, Furber - along with Andrew Brown, from the University of Southampton - suggests an innovative system architecture that takes its cues from nature. "The approach taken towards this goal is to develop a massively-parallel computer architecture based on Multi-Processor System-on-Chip technology capable of modelling a billion spiking neurons in biological real time, with biologically realistic levels of connectivity between the neurons," Furber explains in a white paper outlining the project."

ARM processors?

[Yvan256]

Shouldn't they be using BRAIN processors? /duck

Re:ARM processors?

[cashman73]

No, but it costs an ARM and a LEG!

Build a mouse brain first.

[Animats]

OK, a mouse brain has about 1/1000 the mass of a human brain. So build a mouse brain with 1000 ARM CPUs, which ought to fit in one rack, and demonstrate the full range of mouse behavior, from vision to motor control.

I read the paper. It's a "build it and they will come" design. There's no insight into how to get intelligence out of the thing, just faith that if we hook enough nodes together, something will happen.

About 20 years ago, I went to hear Rodney Brooks (the artificial insect guy from MIT) talk about the next project after his reactive insects. He was talking about getting to human-level AI by building Cog, a robot head and hand that was supposed to "act human". I asked him why, since he'd already done insect-level AI, he didn't try mouse-level AI next, since that might be within reach. He said "Because I don't want to go down in history as the man who created the world's best artificial mouse".

Cog turned out to be a dead end. It was rather embarrassing to all concerned. As one grad student said, "It just sits there. That's all it does."

Re:Build a mouse brain first.

[Nidi62]

As one grad student said, "It just sits there. That's all it does."

Sounds like it actually modeled most of human behavior fairly well then.

Re:Build a mouse brain first.

[Flyerman]

Without the need to breath, eat, procreate, keep my brain occupied etc... I would certainly do the same.

Re:Build a mouse brain first.

[drolli]

Yeah, well. As an interested bystander (a physicist), to me it also seems AI ran into several dead ends because of setting the aims too high. I remember for as long as i can read people predicted that machines which would replace human translators to be around the corner.

IHMO the fundamental mistake is to define something like intelligence on one hand to be very specific, and then hope to reach it by general methods, but taking a few shortcuts, which are not even known yet...

The general rule of thumb which the physicists learned the hard way is that assuming something general and hoping that the shortcuts you need to take to get a result magically fall out once you randomly scatter enough phd students at a subject, is futile. Literally all abstract theories were created the other way round, as a systematically, gradually developed answer to many, many specific problems.

A more practical note about this system is the following: why an ARM processor? to me it seems more efficient to use the block memory in FPGAs and local logic using minimalistic state engines should suffice to simulate this. also the BW should be higher in that case.

Re:Build a mouse brain first.

[Animats]

The general rule of thumb which the physicists learned the hard way is that assuming something general and hoping that the shortcuts you need to take to get a result magically fall out once you randomly scatter enough phd students at a subject, is futile.

I heard that exact approach proposed more than once at Stanford in the 1980s.

AI is no longer a hardware problem. Any major data center probably has enough power to do a human brain, if we only knew how to program it. In some areas, like vision, sheer compute power has helped in a big way. Some problems can be hammered into the ground with machine learning techniques and CPU time. In other areas, we're still stuck. There's been almost no progress on "common sense reasoning" in years.

On the other hand, having lived through the "AI Winter" (from 1985 or so, when it became clear expert systems were a dead end, to the late 1990s, when Bayesian statistics and machine learning started to take over), it's nice to see real progress being made again.

It's not clear that we need a different hardware architecture for AI. There used to be much enthusiasm for neural nets, but it turns out that modern machine learning techniques do better on the few problems neural nets can do. The modern approaches are all matrix algebra, and you usually work in Matlab. Much of that stuff is parallelizable, and what you want is more like a GPU than neurons.

Re:Tremendous overhead

[betterunixthanunix]

On the other hand, if you are simulation a brain, I suspect that you don't really need to have fast communication between any two nodes; localized subclusters should communicate quickly, with slower communication between clusters. This wouldn't work for *all* problems, but for the specific problem they mentioned it seems to be a workable solution.

Re:Oblig. Question

[Eunuchswear]

Well, since he's ICL professor of computer engineering I'd expect it to run George 3 or VME/B.

But, then again, maybe it'd run BBC Basic?

Putting the cart before the horse

[Okian Warrior]

This makes sense, how?

It's like trying to simulate a computer by wiring 5 million transistors. Without a deep understanding of how computers work and a plan for implementation, the result will be worthless.

I see this all the time in AI strategies. Without no deep understanding of AI, the project implements bad assumptions.

Some examples: no way to encode the adjacency information, a fixed internal encoding system which cannot change (ie - a chess program that can't learn checkers), linear input->process->output models, and so on.

Before building a system with a million processors capable of simulating the brain, how about we design an algorithm that embodies the simplest possible AI?

Re:Putting the cart before the horse

[PPH]

Typical IT project philosophy: I'll go to the customer and try to get some requirements. The rest of you, start coding.

Re:Oblig. Question

[Psion]

No, no, no! Given the intended purpose, the question is: Will it run me?

Re:Tremendous overhead

[xkuehn]

I am not a neuroscientist. As a grad student I do study artificial neural networks, which means that I must also have a little knowledge of neuroscience.

The brain is not a fully connected network. It is divided into many sub-networks. I think it's estimated at about 500k, but don't quote me on that number. These sub-networks are often layered, so if you have a three-layer feed-forward sub-network of 5 cells in each layer, each of these cells has only 5 inputs except for the 5 nodes in the input layer, which connects to other sub-networks. (If there are connections from later layers back to earlier layers, the network is said to be a 'feedback' rather than feed-forward network.) These sorts of networks can be simulated very efficiently on parallel hardware, as a cell mostly gets information from the cells that are close to it.

In short, your suspicion is entirely correct. Moreover, you not only don't need fast connections between many of your processing nodes, most of them don't need to be connected to each other at all.

This is the reason why neural networks are interesting in the first place: that they can be simulated on parallel hardware when we don't know a good parallel algorithm with conventional computing techniques. (If it interests you: another name for neural networks is 'parallel distributed computing'.)

There is a hard limit on the 'order' (think of it as function complexity) of functions that can be computed with a given network. To compute a function beyond that limit, you need to have a larger number of inputs to some cells, thereby increasing the order of the network but making it less parallel. Most everyday things are in fact of surprisingly low order. Fukushima's neucognitron can perform tasks like handwriting recognition with only highly local information.