A new robot called RFusion can locate and pick up targeted objects. This includes objects that are directly in sight, but also objects that are completely covered under other objects, for example. The robot was developed by Tara Boroushaki, Isaac Perper, Mergen Nachin, Alberto Rodriguez and Fadel Adib from the Massachusetts Institute of Technology (MIT).
Robotic arms that can autonomously locate and pick up objects have been around for some time. Often, these systems use cameras, which in many cases perform well. "However, if the desired object is completely covered, robotic systems that rely on cameras cannot detect it. They then fall back on a time-consuming search process," the researchers write in their paper.
Using radio frequencies (RF) can help. Since RF waves can move through objects, RF can help locate objects under a pile of other objects, for example. However, existing systems that use RF to pick up objects often make limiting assumptions about the object and their surroundings in doing so. This limits their usefulness in practice, according to the researchers.
Combining techniques
RFusion therefore combines several techniques. The robot consists of a robotic arm equipped on its gripper with both a camera and an antenna. Both are used to locate specific objects. In doing so, RFusion uses two new techniques: RF visual sensing and RF visual reinforcement learning. "We present RFusion, a fully integrated robot that enables practical and efficient picking up of objects in direct vision, out of direct vision and fully covered conditions," the researchers write.
The system works using standard RFID tags, which are applied to objects. Using these RFID tags, the robot can detect objects. Initially, in doing so, the robot uses the antenna on its gripper, which RFusion uses to communicate with the RFID tag. The response of this tag is analysed. In this way, the robot maps how long the signal takes to travel from the antenna to the tag and back.
However, this information is not enough to effectively locate an object. The robot therefore combines the measurement data with additional RF and visual information. This combination enables RFusion to locate an object, move towards it and pick up the desired object.
Dense RF-Visual Geometric Fusion
This process takes place in three steps in phases. The first step the researchers call 'Dense RF-Visual Geometric Fusion'. In this, RFusion uses its antenna to measure the distance to an object. It combines this information with visual data collected using its built-in camera. The combination leads to a list of potential locations where the searched object might be located.
RF-Visual Reinforcement Learning
The second stage is 'RF-Visual Reinforcement Learning'. In this phase, the robot takes its gripper to a new location, where it collects new RF and visual data. For this, the researchers deploy a so-called reinforcement learning network. Reinforcement learning is part of machine learning. Here, agents of an AI network learn to take action independently to maximise their results.
This network analyses previous RF and visual measurements to determine the optimal next location for its gripper. And thus that of its antenna and camera. Thanks to these measurements, RFusion gets an increasingly better idea of the approximate location of the searched object. The robot repeats the process until there is sufficient certainty about this.
RF-Visual Grasping
Once the robot is sufficiently sure of the location of the object it is looking for, the process moves on to the next phase. In this phase, the robotic arm picks up the object. Once the object has been picked up, the robot uses its antenna to check via additional RFID measurements whether the desired object has indeed been successfully picked up. If not, the process is repeated. Just until the robot detects the object in its gripper.
Accuracy
A prototype of RFusion has been tested in real-world situations by the researchers. The researchers point to RFusion's accuracy. For example, a demonstration of the robot's ability to locate objects with centimetre-level accuracy shows that it can do so. Fully covered objects can be picked up by the robot with a 96% success rate. Even if they are under a pile of other objects.
"The system paves the way for innovative picking up tasks for robots in complex environments such as warehouses, production sites and smart homes," the MIT researchers write in a paper.
Further information is available in the research paper.
Author: Wouter Hoeffnagel
Image: Screenshot from YouTube video
