Slashdot Mirror


New Chip Offers Artificial Intelligence On A USB Stick (pcmag.com)

An anonymous reader writes: "Pretty much any device with a USB port will be able to use advanced neural networks," reports PC Magazine, announcing the new Fathom Neural Compute Stick from chip-maker (and Google supplier) Movidius. "Once it's plugged into a Linux-powered device, it will enable that device to perform neural network functions like language comprehension, image recognition, and pattern detection," and without even using an external power supply.

Device manufacturers could now move AI-level processing from the cloud down to end users, PC Magazine reports, with one New York computer science professor saying the technology means that now "every robot, big and small, can now have state-of-the-art vision capabilities."

The article argues that this standalone, ultra-low power neural network could start the creation of a whole new category of next-generation consumer technologies.

81 comments

  1. Prototype as far as I can see by Mostly+a+lurker · · Score: 2

    This is all very interesting. However, there is no indication of when the sticks will become generally available. Their website indicates that they intend to create 1000 sticks shortly for use by selected customers. It is difficult to know how real this is, actually.

    1. Re:Prototype as far as I can see by ShanghaiBill · · Score: 3, Insightful

      This is all very interesting. However, there is no indication of when the sticks will become generally available.

      There also seems to be very little actual information about it. How much memory does it have? How many FLOPS? The product sheet says it uses 16 bit floats, which are generally good enough for NNs. But can it do FP32 and FP64 at all? The power consumption is ~1W, so I doubt if it can do much with that. The USB interface would be a major bottleneck, as you fed information in, and pulled results out. A GPU on a PCIe bus would be way faster at that ... and nearly all computers already have a GPU. I think I will continue to run my NNs on a Tesla K80.

    2. Re:Prototype as far as I can see by samkass · · Score: 1

      This is all very interesting. However, there is no indication of when the sticks will become generally available. Their website indicates that they intend to create 1000 sticks shortly for use by selected customers. It is difficult to know how real this is, actually.

      Wouldn't be surprised if this is a "please buy us out!" advertisement-product. I could see Apple buying them and integrating their chip into the next A-series processor to do client-side Siri among other things.

      --
      E pluribus unum
    3. Re:Prototype as far as I can see by Anonymous Coward · · Score: 0

      You'll have one in your consumer-grade Internet of Things devices ten years after the NSA has a Beowulf cluster of them reading your e-mail. By the point you actually get one in your IoT-run smart home, the NSA will have moved on to quantum USB sticks to analyze all the data hoovered from your smart home.

    4. Re:Prototype as far as I can see by caferace · · Score: 1

      I suspect (yes, I'm guessing) that this may be somewhat less expensive than a Tesla K80.

    5. Re:Prototype as far as I can see by ceoyoyo · · Score: 2

      It's just a mobile gpu chip with a USB interface.

    6. Re:Prototype as far as I can see by ShanghaiBill · · Score: 3, Insightful

      I suspect (yes, I'm guessing) that this may be somewhat less expensive than a Tesla K80.

      Sure, but it is more expensive than the GPU already included in your computer, which has a marginal cost of $0 since you already have it. So why should you buy something that is far slower and less capable than something that is effectively free?

      Also, you don't need to buy a Tesla K80. You can rent them by the minute from AWS.

    7. Re:Prototype as far as I can see by Anonymous Coward · · Score: 0

      It's basically just a bitcoin miner/video card with different drivers. Any number of devices could be bought and used for this purpose. I expect both AMD and NVidia to have "OpenAI" type frameworks ala OpenCL to offload neuronet workloads within a few years. The same way physX did for physics workloads. I bet one even offers to by Movidius within 10 years.

      I have no doubt its very real, but its not some fundamental addition or change in tech.

    8. Re:Prototype as far as I can see by Anonymous Coward · · Score: 0

      This is all very interesting. However, there is no indication of when the sticks will become generally available.

      There also seems to be very little actual information about it. How much memory does it have? How many FLOPS? The product sheet says it uses 16 bit floats, which are generally good enough for NNs. But can it do FP32 and FP64 at all? The power consumption is ~1W, so I doubt if it can do much with that. The USB interface would be a major bottleneck, as you fed information in, and pulled results out. A GPU on a PCIe bus would be way faster at that ... and nearly all computers already have a GPU. I think I will continue to run my NNs on a Tesla K80.

      I respectfully disagree. First of all, USB3 can pretty much guarrantee a 1 way stream of 0.5 GigaBytes/Sec. That's quite a lot of data to process, until communication becomes a bottleneck. And, you can stream your inputs in this processor, whereas you have to memcpy the whole memory to process stuff in a GPU.

      It can afford to be incredibly power frugal, because it has very simple pipelines, focused on efficient vector processing, a centralised SRAM cache, and no power hungry external memories. Yes it has fewer "cores" than a GPU, but you'd be surprised how little complexity you need to perform some vector instructions quickly.

      Which means, you can run quite complex NNs, on the go with pretty much just a windows/linux tablet and this stick, on a total budget of say, 10Watts. That's quite impressive.

      -K

    9. Re:Prototype as far as I can see by GiganticLyingMouth · · Score: 1

      It has FP32, but no FP64. And most of the hardware filters, e.g. for convolutions are FP16 only. And yes, GPUs are faster (I've written code for GPUs and the Myraid2), but the Myraid2 does pretty well from the perspective of processing power to power consumption ratio

    10. Re:Prototype as far as I can see by instinct71 · · Score: 1

      The datasheet on their website claims 1TOps/s within 0.5W! Any idea how that is being achieved/reported ?

  2. Only for Linux? by johnsmithperson123 · · Score: 2, Funny

    Well, first time I've seen that in a long time. Sounds like this Xkcd (https://xkcd.com/644/) might have something to do with it.

  3. Bullshit by geek · · Score: 5, Insightful

    We should first create AI before we start selling it on fucking USB sticks.

    1. Re:Bullshit by Applehu+Akbar · · Score: 1

      "We should first create AI before we start selling it on fucking USB sticks."

      It's a neural network on a stick. It's up to you too try to make it usable as an AI.

    2. Re:Bullshit by The+Other+White+Meat · · Score: 1

      This is precisely the type of HARDWARE that AI SOFTWARE needs to reach maximum performance. They are making this technology more accessible, which means more developers have the tools they need to start working in the AI field. How is this a bad thing?

      --

      --- Generation X: The first generation to have SIG lines inferior to their parents... ---
    3. Re:Bullshit by __aaclcg7560 · · Score: 1

      Just another IT guy who's hoping that AI isn't really here. Sorry, your fear is justified: AI will take your job away soon, if it hasn't already.

      I've been hearing that argument for years. If a robot does replaces my IT job, I'm going to become the guy who repairs robots. Most people don't have a situational awareness that their job might go away someday and prepare for the possibility that they might have to do something else for a living.

    4. Re:Bullshit by mschwanke97402 · · Score: 2

      I agree. I'm tired of AI this and AI that. At best we're getting to expert systems that are tied to speech and sight recognition. When one of these "AI" thingies can come up with an original idea and implement new behavior as a result, we might be getting there.

    5. Re:Bullshit by Anonymous Coward · · Score: 0

      If a robot does replaces my IT job, I'm going to become the guy who repairs robots.

      Except that "robot repairers" - if not robot themselves - will be no more than 1/10 the jobs lost. Banks are replacing MBAs with robo-advisors right now, and you think you can keep your job as a sysadmin or whatever? Enjoy capitalism.

    6. Re:Bullshit by Anonymous Coward · · Score: 0

      Get real, if an AI Robot can do you job then an AI Robot can repair the faulty robots too...

    7. Re:Bullshit by Midnight+Thunder · · Score: 1

      Don't worry someone will come from the future and destroy them, to prevent a war that hasn't yet taken place.

      --
      Jumpstart the tartan drive.
    8. Re:Bullshit by ShanghaiBill · · Score: 1

      We should first create AI before we start selling it on fucking USB sticks.

      We have created AI. What we haven't done (yet) is create human level general purpose AI. But you don't need GP-AI to do things like object recognition, basic natural language processing, machine learning, etc. All of that is still artificial intelligence.

    9. Re:Bullshit by __aaclcg7560 · · Score: 1

      Except that "robot repairers" - if not robot themselves - will be no more than 1/10 the jobs lost.

      Correct. If I recognized that I'm being replaced by a robot and make the preparations to become a robot repairer, I'm going to be the person who gets the job. Meanwhile, the 90% who got laid off will wonder what happen, blame someone else for their problems and demand that the government fix it for them.

      [...] you think you can keep your job as a sysadmin [...]

      Change my current job, let the robots have it.

      Enjoy capitalism.

      With every career change, I make more and more money.

    10. Re:Bullshit by HiThere · · Score: 1

      You should read the Go masters commentary on the recent match. These "AI" thingies can come up with original ideas and implement them. And their original ideas can be better than those of any human expert. (They aren't always, of course.)

      What all current AIs I've heard of are weak on is layered hierarchies of goals.

      --

      I think we've pushed this "anyone can grow up to be president" thing too far.
    11. Re:Bullshit by __aaclcg7560 · · Score: 1

      Get real, if an AI Robot can do you job then an AI Robot can repair the faulty robots too.

      Who will repair the repair robots?

    12. Re:Bullshit by Anonymous Coward · · Score: 1

      With every career change, I make more and more money.

      Oh, sure. Looking at your photo in your website, I think that robots definitely won't be able to physically "replace" you. Do you also have advice for people who don't eat a whole grocery store every week?

    13. Re:Bullshit by Anonymous Coward · · Score: 0

      Repairing robots will be like repairing anything else...follow a checklist and swap out the needed part(s), or scrap it for parts and ship a new one. It'll be a minimum wage job, and will likely be outsourced.

      The only future in robots is designing/improving robotic hardware/software. Maintaining them will not be a career...not a good one, anyway.

    14. Re:Bullshit by 110010001000 · · Score: 1

      Object recognition is not AI. Neither is language processing. Christ.

    15. Re:Bullshit by __aaclcg7560 · · Score: 1

      Do you also have advice for people who don't eat a whole grocery store every week?

      What makes you think I eat a whole grocery store? I actually eat less than most skinny people since I'm on a low carb diet (150 grams/1,500 calories per day).

    16. Re:Bullshit by 110010001000 · · Score: 2

      I can replace most MBAs with a dirty sock. That doesn't mean we have AI.

    17. Re: Bullshit by Anonymous Coward · · Score: 0

      Who treats doctors when they get sick?

    18. Re: Bullshit by Anonymous Coward · · Score: 0

      Then you're genetically defective. An abomination. Immolate yourself now so that you cannot infect the gene pool.

    19. Re: Bullshit by __aaclcg7560 · · Score: 2

      Who treats doctors when they get sick?

      Their mothers — and robots don't have mothers.

    20. Re: Bullshit by WarJolt · · Score: 2

      You can train a machine to classify cat photos. I'm not sure if I'd call that intelligence.

      Scientists use the words machine learning. This device enables ML. At this time ML is by far the most useful aspect related to AI, but it's misleading to say this device enables AI. Additionally, mimicking human intelligence doesn't alway involve learned behavior.

      You might disagree with my terminology, but machine leaning enables better AI. Machine learning isn't AI. There are plenty of machines that have been trained that do completely unintelligent things.

    21. Re:Bullshit by Anonymous Coward · · Score: 0

      The Maytag Robot Repairman!

    22. Re: Bullshit by HiThere · · Score: 1

      Well, if you're talking about this particular device...I've no idea what it would be useful for, or why they picked a USB stick form-factor. If you're comparing Google's Go machine to something that classifies cat pictures, I don't think you understand the problem...or what the problem that was being solved is. Or if you're being dismissive of Go (i.e., "I don't like Go, therefore playing it isn't intelligent."), then perhaps you don't understand what was being done or why.

      I'm trying to read your comment as being more insightful, and failing. If you're feeling threatened, this is a valid feeling, but denial isn't a useful response. OTOH, there's probably several years before programmers jobs are threatened. I would no longer say several decades.

      --

      I think we've pushed this "anyone can grow up to be president" thing too far.
    23. Re:Bullshit by dcollins117 · · Score: 2

      f I recognized that I'm being replaced by a robot and make the preparations to become a robot repairer, I'm going to be the person who gets the job.

      And I'll be the guy who breaks them. Job security for us both!

    24. Re: Bullshit by Anonymous Coward · · Score: 0

      Both of those are indeed artificial intelligence. Christ. Even the phrase "object recognition" sounds, on its face, like something that is non-trivial are requires some level of intelligence. Ieshua ben Ioseph.

    25. Re:Bullshit by angel'o'sphere · · Score: 1

      Perhaps you should go back to school, as I told you already several times: both are AI
      It is not you who decides what AI is. It is the people actually working in AI

      It took decades of research to figure how a computer can recognize objects, that was once a hard AI problem.

      Language and image processing/recognition is done by a subset of aI called "cognitive systems".

      Sorry binary number, you know absolutely nothing about AI.

      --
      Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
    26. Re:Bullshit by mschwanke97402 · · Score: 1

      Sorry, I would disagree that the Go playing computer is coming up with any original ideas. It is a very complex expert system. It might even remember tactics used in the game against it and "learn". It is all still the result of pre-programmed algorithms. One could examine the inputs, and knowing the programming, predict the computer's behavior. When the Go playing computer decides to take the day off or engage in some other behavior that was never contemplated by its programming we can talk.

    27. Re:Bullshit by Anonymous Coward · · Score: 0

      > Sorry binary number

      How do you he's a binary number? You're letting your preconceived notions of what a binary number is define his identity! He could be decimal for all we know! TRIGGERED!

    28. Re:Bullshit by Anonymous Coward · · Score: 0

      I can replace most MBAs with a dirty sock. That doesn't mean we have AI.

      Hello 110010001000, Remember a few weeks back you tried to "School" me on how Watson and Deep blue and the concepts of Connectionist, Behavioralist and Functionalist philosophies were somehow "Not AI" in your words?

      Well, You likened my argument to the movie Chappie, and I had not seen it, so your argument was kind of a poor one, but I didn't realize that you were using a straw man argument in the middle of your meandering argument.. So having seen the movie I am going to put that right here, despite the fact that you have made 0 effort to try and back up your points to any degree whatsoever. You and I operate differently!

      Chappie, is a fairytale movie, so that is why, while the first half shows a somewhat realistic, yet Dumbed down AI in a robot, the point later in the movie about transferring human consciousness into a robot and vice versa.. is ridiculous. I hadn't seen the movie so I didn't realize that that was the straw man argument you were using. So, no to answer your question about my thinking our understanding of AI, Robotics or our knowledge of how the human brain works (By itself or as a way to mimic human consciousness in an AI) is nothing like the movie chappie. I said so, but you kept hammering the point and not listening so I will say it again here as I clearly pointed out in my first post on the topic and responses to your childish trolling.. Building AI into robots and getting a "Chappie" or a "C3-PO" or any other type of AI character is not an immediate goal of the research, it is understanding how to build intelligent machines and there are multiple schools of thought on how that can be achieved (AS I POINTED OUT AND YOU SO DUTIFULLY IGNORED BECAUSE IT DIDNT SUIT YOUR STRAW MAN ARGUMENT) and I pointed them out.. and was modded up and you were modded down.. Apparently the /. crowd knows the difference between Bullshit and scholarly level research when they see it.

      I suggest if you are going to comment on AI you ask questions more so that you can cure your ignorance rather than spouting out luddite, pretentious idiocy. I tried to help you but apparently you have to help us help you if curing your ignorance is your goal. I have to assume from your responses that it is not.

      Bye.

    29. Re:Bullshit by Anonymous Coward · · Score: 0

      Object recognition is not AI. Neither is language processing. Christ.

      Neither are "no true scotsman fallacies" part of a valid argument along the lines of your point. Christ on a fucking stick!

      Idiot!

  4. DOA if the API isn't well documented by Anonymous Coward · · Score: 1

    I can see a niche market for such a thing(aerial robotics, marketing research, statistics data collection for some other purpose such as at a kiosk) but the question is: is it less effort to integrate than rolling your own with other frameworks? Unless there is a severe power budget or weight limitation of the application(IE aerial drones), the notion of "plug and play" is dependent on how much the device does, and how easy it makes it to add the capabilities it doesn't.

  5. artificial intelligence ? by Anonymous Coward · · Score: 0

    more like a magic 8-ball with a USB connector.

  6. Wasted potential... by __aaclcg7560 · · Score: 2

    This is AI on a USB stick is smart enough to know that it was go into the drawer with all the non-AI USB sticks that I don't use anymore?

    1. Re: Wasted potential... by Anonymous Coward · · Score: 0

      I'm sure the AI would speak better English than you. Idiot.

    2. Re: Wasted potential... by __aaclcg7560 · · Score: 1

      I'm sure the AI would speak better English than you. Idiot.

      An idiot is someone who isn't open to correction. I'm always open to correction. Since you're not offering correction but an insult, I must presume you're the idiot.

    3. Re: Wasted potential... by Anonymous Coward · · Score: 0

      Speaking of correction, creimer, I'm the same AC who insulted you. You're right, that was very rude of me. I apologize for insulting you. That was wrong of me. Please accept my apology.

      As for your OP, I still don't understand what you meant to say. You may want to rephrase that.

  7. No external power supply? by wonkey_monkey · · Score: 1

    and without even using an external power supply.

    What are the 5 volts provided by the USB bus if not "external power"?

    --
    systemd is Roko's Basilisk.
    1. Re:No external power supply? by Anonymous Coward · · Score: 0

      You are a moron. What it is supposed to run on free energy? It obviously means its using only the USB power and is not a device that connects data to USB but has to have an external power supply, like a printer for example.

  8. Re: Linux Foundation job interview question by The+Other+White+Meat · · Score: 1

    That question is absurd. A straight linux nerd? Please.

    --

    --- Generation X: The first generation to have SIG lines inferior to their parents... ---
  9. Can anyone explain to me... by Anonymous Coward · · Score: 0

    Can anyone explain to me why I should care about this?

    I doubt it.

    1. Re:Can anyone explain to me... by wonkey_monkey · · Score: 1

      Probably not.

      What's really inexplicable is that you actually took time out to write about how little you care about it. Why would anyone do that?

      --
      systemd is Roko's Basilisk.
  10. Why? by Anonymous Coward · · Score: 0

    Why would a neural network on a stick be preferable to any of the dozens of free and open neural network libraries already available without dedicated hardware requirements?

    1. Re:Why? by wonkey_monkey · · Score: 1

      It's faster.

      --
      systemd is Roko's Basilisk.
    2. Re:Why? by Anonymous Coward · · Score: 0

      I honestly doubt that a 1 watt neural network on a USB bus is going to be faster than a GPU accelerated neural network.

    3. Re:Why? by ceoyoyo · · Score: 1

      It's a mobile GPU in a USB stick. Kinda handy if you've got some device with a USB port but no GPU.

      No, I couldn't think of any examples either.

    4. Re:Why? by Gavagai80 · · Score: 1

      It's for Linux. Most of don't have working GPU drivers.

      --
      This space intentionally left blank
    5. Re:Why? by ceoyoyo · · Score: 1

      Sure. All the deep learning libraries that support GPU computation on Linux beg to differ with you. TensorFlow doesn't run on Windows at all.

    6. Re:Why? by HornWumpus · · Score: 1

      So it's one of these: http://www.newegg.com/USB-Disp... plus bullshit?

      --
      John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
    7. Re:Why? by ceoyoyo · · Score: 1

      Kind of like that, except without the video out.

    8. Re: Why? by Anonymous Coward · · Score: 0

      Exactly

  11. Re: Linux Foundation job interview question by __aaclcg7560 · · Score: 1

    Cut a small hole into a coconut and name it Jane.

  12. Maybe by ArchieBunker · · Score: 2

    This USB stick will do a better job of editing Slashdot than the humans,

    --
    Only the State obtains its revenue by coercion. - Murray Rothbard
  13. Artificial Intelligence by JustAnotherOldGuy · · Score: 1

    "New Chip Offers Artificial Intelligence..."

    They keep using that phrase, but I do not think that phrase means what they think that phrase means.

    --
    Just cruising through this digital world at 33 1/3 rpm...
    1. Re:Artificial Intelligence by Anonymous Coward · · Score: 0

      I find it humorous how many slashdot experts there are in the field of unattainable, undefinable AI.

  14. I have to create an account to view the API? by Viol8 · · Score: 3, Insightful

    Fuck off. If you want me to have even a passing interest in this i want to see how easy it will be to use and port applications BEFORE I give you my details.

  15. Do you want Skynet? by U2xhc2hkb3QgU3Vja3M · · Score: 1

    Because that's how you get Skynet. - Archer

  16. Read this: by Anonymous Coward · · Score: 0

    http://uploads.movidius.com/1441734401-Myriad-2-product-brief.pdf

    Seems to provide a comprehensive list of features of the chip. I still can't find any documentation on the memory included in the USB stick itself, however..

  17. WARNING by Anonymous Coward · · Score: 0

    It also means that soon ever USB stick will have that or something similar in it, which means you won't be able to rely on them for keeping your data secure, and not under control of others.

    But, maybe in the short term we can take advantage of this as a way to establish convert communications channels.

  18. NATURAL INTELLIGENCE Sufficient by Anonymous Coward · · Score: 0

    For example:

    * Tell people what kind of monsters the lefties are. They taught Hitler how to be a brute in Russia. The wholesale killing of the Romanovs was just the start of their multi million killings.

    * Expose Mohammedism. It is as evil as Communism. And it has similar goals, including world domination.

    * Throw out the TV. It is a programming machine using your ears and your eyes. If they could, they would also electrically generate smell for you. Well, for them, so that you are one of their obendient slaves.

    * Blast their covert aid for the Mohammedist War in Syria. Stop believing that they 'fight' ISIS. No, the opposite is true.

    So - these are some things you can do. Start today and help many other people to make the world a better place.

    One "troll" comment at a time.

  19. Yes, Bill Gates by Anonymous Coward · · Score: 0

    You and your 1% folks are so evil, you openly discuss plans to assassinate Donald Trump. One of your NYTimes hacks did it.

    And of course you try to smear Linux with the Sodom and Gomorrea practices you promote yourself via 1% orgs.

  20. Only by Anonymous Coward · · Score: 0

    ...if you are stupid enough to hand them relevant data on a silver plate. Like using ZuckBook, G-Mail and the like.

    Patriots use an RPI and ssh. And similar things running on the RPI.

    And for top secret patriotic information, we use a petrol carriage to meet other people in the woods. They cannot bug up all the forests.

    1. Re: Only by Anonymous Coward · · Score: 0

      Here's your AI on a stick! What else is on it? Um, nothing, why do you ask?

  21. Groovy by Anonymous Coward · · Score: 0

    Now if we can get 90% of Americans to have a USB port coming out of their head...

  22. perform neural network functions like language... by matbury · · Score: 1

    perform neural network functions like language comprehension

    They let on it was bullshit with this one. Ain't no AI that can do language comprehension (natural language processing doesn't actually comprehend language) let alone one on a USB stick. If it can correctly answer questions like:

    "The city councilmen refused the demonstrators a permit because they feared/advocated violence. Who feared/advocated violence?"

    ...then I'll believe they're getting closer to AI that can comprehend language.

  23. Entirely nonsense by Anonymous Coward · · Score: 0

    I'm surprised Slashdot commentators aren't catching this one. Everything in the article is nonsense. Neural networks, even deep NNs, are handled easily with either cloud computing or simply a bit of time. The article mentions bandwidth problems which makes no sense, as trained neural network require very little in terms of space or computational power at runtime. The bottleneck for deep NNs are quality training sets, which this device doesn't solve.

    If anything, it creates more problems than it could possibly solve by removing the crowd-sourcing capabilities of training through a class of devices and completely localizing training to a single device, which is incredibly limited in the amount of data it can provide to train the network.

    Just, it's all complete nonsense.

  24. The new Turbo by Ferocitus · · Score: 1

    I want an "AI button" on the front of the case, just like the old Turbo button
    which gave incredible improvements in performance.

    --
    USB, USB, USB!
  25. Who has any spare ports available? by Keybounce · · Score: 1

    So who has any spare ports available anyways?

    Most of these "sticks" I've seen are so wide that they block adjacent ports, so that means it will take up a pair (at least, all the USB ports I've seen have been a pair here, a pair there).

    You have devices that need to be powered by the computer, and cannot go into a hub.
    You have your high-speed devices that take up a full port.
    You have printers that refuse to work properly through a hub.

    By the time you're done? I'm glad that USB is hot swappable, because I'm constantly swapping already.