Slashdot Mirror


RightHand Robotics Automates a New Type of Warehouse Work: Recognizing, Picking Up Items From Boxes (qz.com)

An anonymous reader quotes a report from Quartz: A startup called RightHand robotics recently began piloting technology that automates a task robots have previously struggled to master: recognizing and picking up items from boxes. RightHand can't say which companies are part of its pilot project and Amazon didn't reply to a request for comment. But the new technology could help the ecommerce giant with a problem that has long vexed it. Like robots elsewhere, Amazon's robots retrieve entire shelves and transport them to humans who pick out items from them. They can find and move a shelf that holds a box of shirts, but they aren't capable of removing the single shirt from that box to be packed into an order. In order to pick items from boxes, robots need to master the more complex task of identifying a wide range of objects and adjusting their grips accordingly. RightHand robotics, which was started by a team of researchers from Harvard Biorobotics Lab, the Yale Grab Lab, and MIT, built a solution called RightPick that, according to co-founder Leif Jentoft, can pick items at a rate of 500 to 600 per hour -- a speed on par with a human worker. It uses a machine learning background and a sensitized robot hand to recognize and handle thousands of items.

56 comments

  1. Re:Smash the Robots by TWX · · Score: 1

    Right? Because then we have a lucrative business opportunity for technically minded people to fix the smashed robots!

    Back from fantasy-land for a moment, my grandfather was a machinist at the Whirlpool manufacturing plant that used to be in southern Indiana. He didn't build washing machines or refrigerators or ranges, but he worked on the machines that the assemblers used to do those tasks. Every time there was a strike there were knuckleheads that would break the assembly line machinery as they left, and inevitably the machinists were the first ones paid when the strike ended because they had to put the plant back into order before any of the assemblers could come back to work. Sometimes the terms ending the strike paid everyone even if the plant wasn't running, but not always, especially when management had evidence of damage on the part of workers causing the delay in restarting production.

    --
    Do not look into laser with remaining eye.
  2. Could put people out work by Anonymous Coward · · Score: 0

    Such as these gentlemen

  3. Re:Smash the Robots by roman_mir · · Score: 0

    Don't you know, picking things up and putting them down is a great field to be in for any human...

    Seriously though, I wonder if a vacuum cleaner like device (what a pneumatic tube used to do) can be used for dealing with specific types of uses, how about picking fruits?

    On an unrelated note I can come up for a few more things that can be automated by a 'right hand robot'...

  4. Re:Smash the Robots by Anonymous Coward · · Score: 0

    Nice. Now tell the one about how Sarah Connor's father lost his job at the mattress factory to a machine.

    Oh sorry, was that a detour into fantasy land? Do you not understand the writing on the wall?

    There won't be any jobs for machinists or assemblers. There won't be jobs for anyone. There won't be anyone working except billionaire Bezos telling you to buy shit from robots. When you can't afford to buy, the economy won't need you anymore, and then some enterprising innovator will send robots to fucking kill you.

  5. Re:Smash the Robots by Hylandr · · Score: 1

    On an unrelated note I can come up for a few more things that can be automated by a 'right hand robot'...

    Then shut up and do it.

    --
    ~ People that think they are better than anyone else for any reason are the cause of all the strife in the world.
  6. Right hand robotics? by Anonymous Coward · · Score: 1

    I can't be the only one who thought of wanking...

  7. Re:Smash the Robots by Hylandr · · Score: 1

    The economy or electoral college won't need you anymore, and then some enterprising dictator will send robots to fucking kill you.

    FTFY

    --
    ~ People that think they are better than anyone else for any reason are the cause of all the strife in the world.
  8. Haw-haw! by Anonymous Coward · · Score: 0

    Losers! Didn't keep their skills up. Good thing the future is bright for devops and always will be. There's no way automation can combine developing and operating and marketing and public relating and accounting and human resourcing all into one role and fire all the devops. Never!

  9. Re:Smash the Robots by Anonymous Coward · · Score: 0

    Tell me my sperm count, talking fleshlight.

  10. Re:Smash the Robots by Anonymous Coward · · Score: 0

    Nice. Now tell the one about how Sarah Connor's father lost his job at the mattress factory to a machine.

    Oh sorry, was that a detour into fantasy land? Do you not understand the writing on the wall?

    There won't be any jobs for machinists or assemblers. There won't be jobs for anyone. There won't be anyone working except billionaire Bezos telling you to buy shit from robots. When you can't afford to buy, the economy won't need you anymore, and then some enterprising innovator will send robots to fucking kill you.

    oh yes indeed every human's greatest fantasy is to spend 8 hours a day in a sweaty factory for low wages, it's truly nirvana on earth and we need to do all we can to protect these jobs

    maybe it might be possible for you to imagine a world beyond capitalism but maybe not

  11. Re:Smash the Robots by Anonymous Coward · · Score: 0

    maybe it might be possible for you to imagine a world beyond capitalism but maybe not

    Did you even read this shit:

    Smash every robot until we get our Basic Income.

  12. Re:Smash the Robots by Anonymous Coward · · Score: 0

    Too late, your fleshlight eloped with a vibrator.

  13. Re:Smash the Robots by FatdogHaiku · · Score: 2

    Right? Because then we have a lucrative business opportunity for technically minded people to fix the smashed robots!

    You would think so but NO! For I have perfected the robot smashing robot!
    Humans could never keep up with this titian, only my smashed robot repairing robot will do!

    Note to self:
    Hide smashed robot repairing robot from robot smashing robot this time...

    --
    You have the right to remain sentient. If you give up the right to remain sentient, you will be elected to public office
  14. Robotic Pickers by mentil · · Score: 1

    My understanding is that the main job for humans in Amazon warehouses is for 'pickers', that these machines are claimed to be able to replace (no word on accuracy, however). The article mentions that packing items in boxes is still done by hand, and I imagine loading/unloading trucks is still done with humans. However I can foresee completely-automated Amazon warehouses in the near future. With self-driving trucks, and completely-automated factories, there will likely soon be some products whose packaging are unseen by a human until they reach a consumer's doorstep. Completely-automated retail. From what I could find here, at least 10.1 million Americans work jobs that'd be replaced with automated retail. The American Trucking Association claims 3.4 million American truck drivers. So that adds up to 13.5 million jobs between retail and trucking, add in other driving jobs and it'd be 10% of all jobs.

    For comparison, about 2.5 million new jobs are created in the US each year. In the unlikely event every driver and retail supply chain worker were laid off at once, it'd take ~5 years for new jobs to be created to absorb them (assuming an equal number of vacancies.) That's ignoring the fact that many of these 'new jobs' are in the driving and retail sectors. Another 5.4 million Americans work as food preparers/waiters; as minimum wage increases I wonder how many restaurants will increase automation. I know many restaurants won't fully-automate due to tradition or being high-class, but most restaurants aren't too high-class and will automate if it's either that or go out of business.

    --
    Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
    1. Re:Robotic Pickers by kobaz · · Score: 2

      The year is 2025... There's going to be such a massive amount of items that a household orders from Amazon, packed so furiously by our robot overlords, that in order to maintain our sanity, and ease the strain of endlessly ordering, receiving, categorizing, putting away, retrieving, and consuming, said items ordered from Amazon, we as a human race will need to acquire robot helpers.

      So soon, our houses will be whirring with robots, ordering items, preparing items, and giving us our items, shipped from the Amazon robot assembly line. When one breaks, the others will automatically issue a RMA back to Amazon to return the defective unit, which will be received, repaired, assembled, and shipped again by our robot overlords. This new replacement robot will be automatically delivered, unpackaged, and activated by the already present backup household robots and will immediately join the crew, back where it left off.

      And now that the human race finally has food growing, food harvesting, food preparation, construction, health care, and everything else completely automated, us humans can finally sit back, relax, and start to enjoy what we were really put here on Earth to do...

      ... to build more robots.

      --

      The goal of computer science is to build something that will last at least until we've finished building it.
    2. Re:Robotic Pickers by Anonymous Coward · · Score: 0

      and global warming and climate change don't exist....

      add it all up and we've been down that road before.

      watch that future documentary, wall-e, for a reminder of the road we're traveling down.

    3. Re:Robotic Pickers by aberglas · · Score: 1

      It is robots that will build the more robots, not people.

    4. Re:Robotic Pickers by Godwin+O'Hitler · · Score: 1

      I feel like I was put on earth to spend hours in front of a screen looking for the item I want.
      I'd much rather spend the time outside fruitlessly searching. We need robots to do the Web searching.

      --
      No, your children are not the special ones. Nor are your pets.
    5. Re:Robotic Pickers by Maxo-Texas · · Score: 1

      Now add 90% of fast food jobs (3.3 million) and 90% of retail clerk/cashier jobs (another 3 million plus) and you are sitting at 15% of jobs destroyed in a 12 year period.

      And people who lose fast food and retail jobs are not going to be going to work as robot designers.

      It's at least 6 years off- but by 12 years from now it should be wide spread and in your face.

      I'm thinking universal basic income or civil unrest.

      --
      She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
    6. Re:Robotic Pickers by Osgeld · · Score: 1

      yea and I look forward to being taxed so hard that my lower middle class income becomes less than the ubi

    7. Re:Robotic Pickers by Anonymous Coward · · Score: 0

      Yes, by all means, watch children's cartoons instead of using your brain to try to understand things.

    8. Re:Robotic Pickers by Anonymous Coward · · Score: 0

      No, prices just come down until most things are free. Basic supply and demand.

    9. Re:Robotic Pickers by Maxo-Texas · · Score: 1

      Yea.. that's why so many items are up by 50% over the last 6 years alone.

      Prices are targeted at the top 20% who make big money. Not at the bottom 80% who do not.

      If they can sell 3 items for $21 why would they destroy their brand by selling 10 items for $20?

      --
      She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
    10. Re:Robotic Pickers by Nethead · · Score: 1

      Fine, as long as one of those robots is able to break down all my damn Amazon boxes and put them in the recycling for me. They will also need to pop and dispose of the air pillows too.

      Added points for a robot that tracks down and gives a wedgie to anyone that ships stuff with packing peanuts.

      --
      -- I have a private email server in my basement.
    11. Re: Robotic Pickers by Anonymous Coward · · Score: 0

      Because raw materials are free?

    12. Re: Robotic Pickers by Anonymous Coward · · Score: 0

      Of those jobs created each year a significant fraction are needed to accommodate new entrants to the job market. Many jobs are also dependent on the wages of others, either generally or directly, e.g. truck stop diners, so it is unlikely it would be absorbed within five years. However it's not like 10 million people will wake up one morning to find they are all out of work, and so if the change is sufficiently gradual then some truckers will have retired, some moved on, and truck stop diners moved on to satisfying leisure travellers. But there's no certainty, of course.

    13. Re:Robotic Pickers by Talderas · · Score: 1

      The article mentions that packing items in boxes is still done by hand, and I imagine loading/unloading trucks is still done with humans. However I can foresee completely-automated Amazon warehouses in the near future.

      Amazon would need to have more tightly entwined work with UPS and FedEx or push their own truck delivery system which is probably why they're putting effort into drone delivery. There's technology out there for "smart shipping" which will pack stuff into a delivery vehicle which can cause the vehicle to be packed based on the route that it will take. This being optimal as you can reduce the volume required since you no longer need to have space for the driver to get into the storage portion of the vehicle and search through the packages for the one to deliver. This means either smaller delivery vehicles or more packages in the delivery vehicle. It also optimizes the delivery process a bit more for automated warehouse to door shipping.

      To do this you need to be able to stock your vehicle to a reasonable fullness which means packages need to sit in a staging area for the loading dock until some combination of time and volume quantity has been achieved.

      Amazon is probably using the drone delivery right now because automated vehicles are still a restricted technology or they may be assuming that UPS and FedEx and the like are also at least doing some element of research into automated delivery systems in order to not be obsoleted by it.

      My feeling is that picking and shipping, while done by humans, are going to be solved by the same technology. The big issue with picking is making sure the goods aren't damaged by the robot hand when selecting various goods from within a container. Shipping should be a quick to solve problem once the picker problem is solved as the part about picking items out of the container to pack into a box is solved and at that point its a computation problem of the dimensions of the products and optimizing the order to package them as well as any packaging.

      --
      "Lack of speed can be overcome. In the worst case by patience." --Znork
  15. Re:Smash the Robots by FatdogHaiku · · Score: 1

    Don't you know, picking things up and putting them down is a great field to be in for any human...

    Seriously though, I wonder if a vacuum cleaner like device (what a pneumatic tube used to do) can be used for dealing with specific types of uses, how about picking fruits?

    Some fruits could be picked with a suction device but transporting things via pneumatic tubes comes with a "sudden stop issue" that is incompatible with fruits that must be picked ripe.
    http://food.unl.edu/fruits-continue-ripen-after-theyre-picked

    On an unrelated note I can come up for a few more things that can be automated by a 'right hand robot'...

    Paging Mr. Wolowitz...
    https://www.youtube.com/watch?v=Tb627xDlqBs

    --
    You have the right to remain sentient. If you give up the right to remain sentient, you will be elected to public office
  16. Real world problems - warehouse sized overlords. by mistr · · Score: 1

    At least, having an AI solve real world problems with advanced sensory feedback and complex appendages is doing more for robotics than silly animatronic detours like https://www.youtube.com/watch?....

    Imho, these kind of solutions is what is the real driving force of the future of robotics. Now to have a coffee and think about how our warehouse sized, multiped overlords will shape the future.

  17. Re:Smash the Robots by Anonymous Coward · · Score: 0

    Fleshlight recently got open source support (aka hax) so that guys for once can do teledildonics. Why is it that chicks are the target of teledildonics when really it's only the guys who are into that kind of stuff?

  18. robot discrimination! by Anonymous Coward · · Score: 1

    leftie persecution. they're going smash lefties till they use their right hands.

  19. well, damn. by Anonymous Coward · · Score: 0

    there goes my best job prospect for when i lose my bus driving gig.

  20. This is THE big change by aberglas · · Score: 5, Insightful

    Robots that can see. And respond in sensible ways.

    "Bin Picking" requires recognizing which objects are which, and what their orientation ("pose") is. Then plan a way to move to collect them.

    That is an order of magnitude more sophisticate than simply moving in rigid, predefined ways to work on things that have been precisely positioned in advanced.

    It opens up whole new fields of automation.

    And it is not that new, bin picking robots have been around for a while. They are getting better. This is only a story because Amazon is doing it now.

    1. Re:This is THE big change by Anonymous Coward · · Score: 0

      If they can do this, then it wouldn't be too difficult to get a system to go through refuse waste dumps and sort things out into metals, glass, fabrics and paper. I imagine a system with rotating scoop wheel, a conveyor belt, electromagnets and a cryogenic cooler to freeze material to make it brittle, then a tumbler to shatter the items into small fragments, which are then sorted.

    2. Re:This is THE big change by SNRatio · · Score: 1

      "Bin Picking" requires recognizing which objects are which, and what their orientation ("pose") is. Then plan a way to move to collect them.

      For big retail operations, wouldn't requiring the wholesalers to package their items so that they are easy to pick and place be easier? A QR code, standardized grab points, etc. It won't work for all operations (like reshelving clothes after they have been tried on), but it would be easy for anything that currently comes in a blister pack.

    3. Re:This is THE big change by dinfinity · · Score: 1

      Robots are about to beat us (and all animals) at hand-eye coordination. Our set of unmatched skills is dwindling every day.

  21. Re:Smash the Robots by Calydor · · Score: 1

    So they make an all-purpose robot repairing machine, one of which can also repair another robot of the same model.

    NOW what do you do?

    --
    -=This sig has nothing to do with my comment. Move along now=-
  22. Somewhere out there by Provocateur · · Score: 1

    There is a nerd that decided he wanted to enter the exciting future of robotics, yet he would much rather prefer becoming a graduate of the Yale Grab Lab so he can, you know, pick up chicks instead.

    --
    WARNING: Smartphones have side effects--most of them undocumented.
  23. Robotics is currently a joke by 110010001000 · · Score: 1

    It is always amazing how people get so worked up over this. The fact is, the current state of robotics is a joke. Yes, Amazon robots can locate a shelf of products and retrieve it (as long as the shelf is uniform, and labeled clearly). That is releatively easy to do. But pick an arbitrary item out of a box and then pack it? Very very hard. In fact, might not be possible to get it to work reliably. CPU technology isn't going to progress in leaps and bounds anymore, so we might be getting close to the fastest digital processors we will ever see. Of course people scoff and say "Moore's Law", but the fact is that Moore's Law is dead. It is obvious by just looking at the current state of computing that it is dead. That is why there is a rush to multi-core. Everything is dependent on progress of digital processors. And they haven't been progressing at the same rate as they have been historically. Not even close.

    1. Re:Robotics is currently a joke by Gravis+Zero · · Score: 1

      Yes, Amazon robots can locate a shelf of products and retrieve it (as long as the shelf is uniform, and labeled clearly). That is releatively easy to do. But pick an arbitrary item out of a box and then pack it? Very very hard. In fact, might not be possible to get it to work reliably.

      FTFA:

      "The robots still aren’t as good as humans. They get stumped by stuff wrapped in plastic or things partly obscured by other items, for instance."
      "RightHand’s Jentoft said robots in the pilot programs are being used mostly to pick items from boxes."

      You rarely solve a complex problem all at once, it's usually solving parts of a problem until you have solved the whole thing. They will get there in time.

      CPU technology isn't going to progress in leaps and bounds anymore, so we might be getting close to the fastest digital processors we will ever see. Of course people scoff and say "Moore's Law", but the fact is that Moore's Law is dead. It is obvious by just looking at the current state of computing that it is dead. That is why there is a rush to multi-core. Everything is dependent on progress of digital processors. And they haven't been progressing at the same rate as they have been historically. Not even close.

      1) Moore's Law is an observation about the number of transistors in CPUs, not their computational power or power draw. It has no direct correlation to performance.
      2) GPUs massively accelerate computer vision solutions and there is plenty of processing power.
      3) Neural networks are ultimately more suited for this type of task since they work like brains, so a shift to using NN chips is inevitable.
      4) If I've told you once, I've told you a billion times, DO NOT PROVOKE THE BORG.

      --
      Anons need not reply. Questions end with a question mark.
    2. Re:Robotics is currently a joke by rkordmaa · · Score: 1

      CPU prowess is hardly the limitation, the problem is reliability of vision algorithms. Getting vision to work reliably is hard.

    3. Re:Robotics is currently a joke by 110010001000 · · Score: 1

      No they won't get there in time. That is my point. The entire idea hinges on growth of processors. There is an end to everything. Progress is not guarenteed.

    4. Re:Robotics is currently a joke by 110010001000 · · Score: 1

      Yes it is the limitation. There is no magical undiscovered algorithm. Currently everything hinges on digital processing capability. And that has peaked (basically).

    5. Re:Robotics is currently a joke by Gravis+Zero · · Score: 1

      The entire idea hinges on growth of processors.

      Not at all. If anything, it depends on the ratio of cost to computational throughput. We can always add more processing power to a robot, it just requires more electricity and hardware which cost money. Neural networks are the masters of efficiency when properly configured, which is probably why they occur in nature. I have no idea why you think we are going to fall short when IBM put an entire mouse brain into a collection of NN processors just a couple years ago.

      There is an end to everything. Progress is not guarenteed.

      Why do you think anyone would just give up because of Moore's law? If we needed, we would use a different medium than silicon. Hell, we already know how to make superconductive circuits/memory and space is a nice cool environment to keep them superconductive but we don't even need that thanks to massively parallel computing. I think you are declaring failure far too quickly.

      --
      Anons need not reply. Questions end with a question mark.
  24. Re:Smash the Robots by Anonymous Coward · · Score: 0

    Smash every robot until we get our Basic Income.

    Right? Because then we have a lucrative business opportunity for technically minded people to fix the smashed robots!

    LUDDITES are smashing bots...wait, that kinda goes without saying, doesn't it? Lemme try this again.

    LUDDITES are fixing smashed bots ruining robot-repairbots by making humans repair bots smashed by LUDDITE humans! ONLY bots can fix bots, NOT LUDDITE humans!

    Bots!

  25. Sorting by Weight by JimSadler · · Score: 1

    A simple digital scale can sort many items by weight. An XL shirt for example will have less weight than a size M shirt. So if the shirts are stored such that all are the same except by sizes the robot can work its way through the bin until the correct size if located. There have been several other useful ways to sort items for quite some time. But the real question is whether robots can sort items that are very diverse and mixed together. So far that is difficult. But go to a junk yard that has one of those machines that can grind a car into small chunks and watch them pick up the steel pieces with an electro magnet and leave all else behind. It is efficient but not perfect as a tiny bit of cloth could be impaled upon a sharp piece of steel. Sorting can be easy or difficult depending upon each situation.

  26. Re:Smash the Robots by Anonymous Coward · · Score: 0

    Burn the fields until we get our grain!

    This is how barbarians think.

  27. Re:Smash the Robots by Anonymous Coward · · Score: 0

    No, prices will simply come down until everything is free. This is what always happens when things become fully automated. Or even just almost fully automated.

    I mean, you don't pay to have "robots" serve your shitposts onto slashdot, do you?

  28. or till you go to jail / prison and you get better by Joe_Dragon · · Score: 1

    or till you go to jail / prison and you get better doctors then the ER

  29. Vexing problem by JustAnotherOldGuy · · Score: 1

    "But the new technology could help the ecommerce giant with a problem that has long vexed it."

    And that problem is....human employees.

    Human employees are always demanding stuff like food, shelter, bathroom breaks, medical care, adequate lighting, temperatures above freezing....if we could just get rid of them everything would be wonderful and we'd be living in Utopia!

    --
    Just cruising through this digital world at 33 1/3 rpm...
  30. Then explain.... by Anonymous Coward · · Score: 0

    ...why things from the vending machine cost more?

    1. Re:Then explain.... by BronsCon · · Score: 1

      Stocking the machines is far from an automated process, combined with the fact that the owners of those machines pay considerable rent to place them.

      --
      APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
  31. Re:Smash the Robots by pete6677 · · Score: 1

    This will finally allow Amazon to realize their long-awaited goal of operating their warehouses with NO human employees at all! Then they won't even have to worry about sabotaged machinery, since there will be no workers in the place at all.

  32. A parable about structural unemployment and BI by Paul+Fernhout · · Score: 1

    "There won't be any jobs for machinists or assemblers. There won't be jobs for anyone. There won't be anyone working except billionaire Bezos telling you to buy shit from robots. When you can't afford to buy, the economy won't need you anymore ..."

    I made this parable in 2010 that is about a world that develops along those lines:
    "The Richest Man in the World: A parable about structural unemployment and a basic income "
    https://www.youtube.com/watch?...

    --
    A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.
  33. Re: Smash the Robots by Anonymous Coward · · Score: 0

    Already done: https://www.festo.com/group/en/cms/10241.htm