Intel Launches Movidius Neural Compute Stick: 'Deep Learning and AI' On a $79 USB Stick (anandtech.com)

Nate Oh, writing for AnandTech: Today Intel subsidiary Movidius is launching their Neural Compute Stick (NCS), a version of which was showcased earlier this year at CES 2017. The Movidius NCS adds to Intel's deep learning and AI development portfolio, building off of Movidius' April 2016 launch of the Fathom NCS and Intel's later acquisition of Movidius itself in September 2016. As Intel states, the Movidius NCS is "the world's first self-contained AI accelerator in a USB format," and is designed to allow host devices to process deep neural networks natively -- or in other words, at the edge. In turn, this provides developers and researchers with a low power and low cost method to develop and optimize various offline AI applications. Movidius's NCS is powered by their Myriad 2 vision processing unit (VPU), and, according to the company, can reach over 100 GFLOPs of performance within an nominal 1W of power consumption. Under the hood, the Movidius NCS works by translating a standard, trained Caffe-based convolutional neural network (CNN) into an embedded neural network that then runs on the VPU. In production workloads, the NCS can be used as a discrete accelerator for speeding up or offloading neural network tasks. Otherwise for development workloads, the company offers several developer-centric features, including layer-by-layer neural networks metrics to allow developers to analyze and optimize performance and power, and validation scripts to allow developers to compare the output of the NCS against the original PC model in order to ensure the accuracy of the NCS's model. According to Gary Brown, VP of Marketing at Movidius, this 'Acceleration mode' is one of several features that differentiate the Movidius NCS from the Fathom NCS. The Movidius NCS also comes with a new "Multi-Stick mode" that allows multiple sticks in one host to work in conjunction in offloading work from the CPU. For multiple stick configurations, Movidius claims that they have confirmed linear performance increases up to 4 sticks in lab tests, and are currently validating 6 and 8 stick configurations. Importantly, the company believes that there is no theoretical maximum, and they expect that they can achieve similar linear behavior for more devices. Though ultimately scalability will depend at least somewhat with the neural network itself, and developers trying to use the feature will want to play around with it to determine how well they can reasonably scale. As for the technical specifications, the Movidius Neural Compute Stick features a 4Gb LPDDR3 on-chip memory, and a USB 3.0 Type A interface.

Comments

Proprietary all the way down.

[Gravis Zero]

So I was interested in what drives this thing, the Myriad 2 VPU and found out this is right up Intel's ally because it's proprietary from top to bottom. Everything needs software only they can provide and naturally comes with conditions. I found a presentation which clearly shows what their priorities are.

Their big claims to fame:

- 8+ years of heritage. Close to $60M invested into technology development
- Proven architecture. 100% internally developed. Strong IP position

Buy into the lock-in now! -_-

Intel's answer to a GPU

[clifwlkr]

Seems most people don't understand what this is doing. It looks like it is using Caffe standard neural network libraries. It mentions 'limited' layer support, but not by how much. Specifically it says it will support convolutional neural networks, which are decent image detectors. They could be used for object detection, handwriting recognition, etc.

You then cross compile your network using their toolkit to run on this device, and much like GPUs and tensorflow, you get high powered processing of your network. When married with a low power CPU, this could allow you to do CNN processing on devices that were not otherwise up to the task.

That said, exactly how performant this is remains to be seen. Although at only $80, it is a pretty cheap experiment and somewhat interesting as an idea.

I wonder if you can plug it into your Edison, though? :-)