A new system enables robots to understand objects much better, allowing them to manipulate objects more accurately. For example, the system allows robots to pick up an object by a specific part, such as a teddy bear by its ear or a shoe by its lip.
Researchers have long been working on software that enables robots to successfully pick up unknown objects. For example, MIT, in collaboration with Princeton University, previously developed a system that can pick up a specific object from a bin full of random objects, without having seen this object before. The robot then analyses this object, compares the object with objects it has seen before and, based on this, determines in which bin the object should be placed.
Dense Object Nets
However, the new Dense Object Nets (DON) system from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) goes a lot further. This system allows robots to perform much more specific commands. Think 'lift the red teddy bear by its left arm' or 'lift the toy car by the red part'.
"Many manipulation methods are unable to identify specific parts of an object in the many different orientations an object can have," says CSAIL graduate student Lucas Manuelli. Manuelli co-authored a paper on DON with professor Russ Tedrake and graduate student Pete Florence. "For example, existing algorithms would not be able to pick up a head by its ear, especially if the head can have different orientations such as standing or on its side."
Visual map
This is possible because DON converts an object into a collection of points, which function as a visual map. Using these points, the system can create a 3D model of the object. If a person then specifies a particular part of the object, the robot can use this model to identify this part and pick up the object here.
The researchers conducted several experiments with a KUKA robotic arm equipped with DON in which this robot had to perform various tasks. For instance, during one of the experiments, the robotic arm successfully managed to pick up a toy caterpillar by its ear. During another experiment, the robot managed to pick up a specific cap from a bin full of caps, without ever having seen this cap - not even in training data.
Applications
The researchers see potential applications for DON in several environments, including manufacturing. "In factories, robots often need complex 'part feeders' to work reliably," Florence explains. "A system like this that understands the orientation of objects can simply take a picture and then be able to pick up an object and manipulate it."
The research team additionally points to commercial robots for home use on MIT's news website. For example, users could have a robot using DON to clean their house by simply showing a picture of their home in a tidy state.
Manuelli and Florence's paper titled 'Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation' can be found here.
Author: Wouter Hoeffnagel
Source: MIT
Source photo: Pixabay / jarmoluk
