Slashdot Mirror
Flex Logix Says It's Solved Deep Learning's DRAM Problem (ieee.org)
An anonymous reader quotes a report from IEEE Spectrum: Deep learning has a DRAM problem. Systems designed to do difficult things in real time, such as telling a cat from a kid in a car's backup camera video stream, are continuously shuttling the data that makes up the neural network's guts from memory to the processor. The problem, according to startup Flex Logix, isn't a lack of storage for that data; it's a lack of bandwidth between the processor and memory. Some systems need four or even eight DRAM chips to sling the 100s of gigabits to the processor, which adds a lot of space and consumes considerable power. Flex Logix says that the interconnect technology and tile-based architecture it developed for reconfigurable chips will lead to AI systems that need the bandwidth of only a single DRAM chip and consume one-tenth the power.
Mountain View-based Flex Logix had started to commercialize a new architecture for embedded field programmable gate arrays (eFPGAs). But after some exploration, one of the founders, Cheng C. Wang, realized the technology could speed neural networks. A neural network is made up of connections and "weights" that denote how strong those connections are. A good AI chip needs two things, explains the other founder Geoff Tate. One is a lot of circuits that do the critical "inferencing" computation, called multiply and accumulate. "But what's even harder is that you have to be very good at bringing in all these weights, so that the multipliers always have the data they need in order to do the math that's required. [Wang] realized that the technology that we have in the interconnect of our FPGA, he could adapt to make an architecture that was extremely good at loading weights rapidly and efficiently, giving high performance and low power."
Mountain View-based Flex Logix had started to commercialize a new architecture for embedded field programmable gate arrays (eFPGAs). But after some exploration, one of the founders, Cheng C. Wang, realized the technology could speed neural networks. A neural network is made up of connections and "weights" that denote how strong those connections are. A good AI chip needs two things, explains the other founder Geoff Tate. One is a lot of circuits that do the critical "inferencing" computation, called multiply and accumulate. "But what's even harder is that you have to be very good at bringing in all these weights, so that the multipliers always have the data they need in order to do the math that's required. [Wang] realized that the technology that we have in the interconnect of our FPGA, he could adapt to make an architecture that was extremely good at loading weights rapidly and efficiently, giving high performance and low power."
designed to do difficult things in real time, such as telling a cat from a kid in a car's backup camera video stream
So now I'm really curious which one they think it's OK to back over.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Seems that's where logic design is going: distribute processing logic around, closer to storage.
Kind of like HBM with Ultrascale Virtex, or Nvidia's Tesla.
From what little I remember from when neural networks made their first buzzword splash back in the 1990s I think all the buzzwords in the summary are basically saying that they need an architecture that is really fast at doing multiplication of large matrices. Yes? If so, this really is not in any way a new problem - fast matrix math has been a staple of high performance computing since day 1, and these guys are just saying (I think) they want to build a processor designed just for that purpose. Or am I missing something, blinded by the sheer wonderfulness of their choice of buzz-ness?
It's just that cars are harder to hit. You need to get them with the tire whereas with the kid the middle of the bumber is better.
get a load of this retard
Is that it is not any "bandwidth problem". It is that deep learning is actually pretty bad at solving classification problems. These are just some more people trying to get rich before the ugly truth becomes impossible to ignore.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Work on your buzzword density.
I remember when I used to get IEEE Spectrum mailed to me.
I don't know why they started doing that or how they got my address. I don't know why they stopped. All I remember is getting it and tossing it in the trash every single time.
Can you please explain? You keep posting this shit. I want to know why.
don't need a startup to know about deep learning and neural networks. 30 years ago we knew if was about classification. just a lot of comparison crunching... just think about how we learned as humans what a rock was, a ball, a tree... how one tree was like another tree, and so on... whereas a bush was brush or was it a tree? hmmmm.. now the plot thickens... the FPGA solution is classic as well.. use assembly on hardware to get things done. *** really fast.. ***
I mean, the competitors go up to 8. But theirs goes to 11, it slings more bits right?
Geez that article was painful to read.
Yes, heâ(TM)s a fat virgin living is he moms basement and so he has nothing better to do than this.
Weights are always needed in an order known well in advance, and are typically reused a lot of times.
The solution is to have tiny caches inside or very near to multipliers, so that rather than saying A*B, you instead say A*next_cached_value.
For many use cases, even caching a single value would dramatically speed things up.
That I can buy right now? If not, then they haven't "solved the problem", but, "think they have a possible way of solving the problem that is yet to be proven to work".
A possible solution is Quantum Machine Learning.
Quantum Neural Training could accelerate itself.
All that is needed is using operators for Hadamard matrices.
These neural weights could be slightly imprecise but acceptable.
FPGAs could be useful for implementing these operators with matrices's multipliers/adders.
The problem is how much accuracy is required for representing a floating number of 1 / sqrt(2) [ better sqrt(2) / 2 ].
The brains of a bird, fish, crayfish, fruit fly or even a mosquito defy so called AI brute force solutions. Less is more is the solution.
cat from a kid
Yeah, because killing cats instead of kids is a great goal to have. How about just avoiding killing ANYTHING?!
Oh yeah because to do that you need to go slowly and rich people with self driving cars want to go fast.
These do not do the inferior "deep" learning. They do proper learning where the neural network is designed for the task. Of course they perform better.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Can you please explain? You keep posting this shit. I want to know why.
Just a mental disease. Ignore, unless you're his physician.
When your problem calls for an expensive fab that you don't have funding for, FPGA seems like the solution. Again and again.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Good algorithms haven't been doing serialized demand load since the first CPU with sixteen whole lines of cache memory was attached to a split-transaction memory bus.
For the record, that was also one of the first microcoded CPUs.
With proper data orchestration, matrix multiply is far more of an aggregate bandwidth problem than a latency problem.
A pair of 64Ã--64 matrices fit into 64 kB of L1 cache. That's a good 250,000 MACs, right there (by the simple N^3 algorithm).
Suppose your 5 GHz core performs 4 double-precision MACs per clock cycle (40 GFLOPs).
2^18 / (40e9 Hz) = 6.55 microseconds
I don't regard streaming out your 32 kB answer to main memory (bypassing cache) in 6 us as straining at the latency bit.
Large, square matrices are rarely even a bandwidth problem.
The 1xN * Nx1 case (for large N) is a bandwidth problem, however. For this case you require two 8-byte memory bus reads and one 8-byte memory bus write per MAC. Probably not gonna happen at 20 GHz (though it might get close, on your single core i7 with three memory channels, running yesterday's AVX).
For skinny matrices, you need to keep your servers blade thin.
The first phrase extracted from the article for the blurb on a real nerd site would have been "folded Benes network".
Also, on a real nerd site, it would have rendered the S-with-caron properly, as well.
Why can't we have nice things?