Not only transparent, but also highly reflective and 'jet-black' objects are difficult for robots to detect in practice. This limits the deployment of robots in some industrial environments. A new sensor offers a solution and allows robots to capture these objects more accurately in 3D.
The sensor was developed by Germany's Fraunhofer Institute for Applied Optics and Precision Engineering (IOF). Among other things, stickers can regularly be found on large glass windows to warn birds and thus prevent collisions. Robots are also familiar with this problem. This limits their use in automation in industrial environments. The Fraunhofer Institute's new sensor should provide a solution.
Laser projection combined with thermography
The sensor is called the MWIR 3D Sensor and is also called 'Glass360Dgree'. The sensor is capable of reliably detecting reflective or light-absorbing objects. The sensor does this by combining infrared laser projection with thermography. In this process, Glass360Dgree locally heats up the object, after which a pair of thermal cameras records the resulting distribution of heat on the object's surface. Additional measurement aids or temporary markings are not required here, unlike many conventional sensors.
Warming takes place using a CO2 laser. Using lenses, a line is projected onto the object, which moves incrementally over the object in a fraction of a second. In the process, the object absorbs the laser's energy and heats up, which is recorded using two sensitive thermal cameras. In this process, the heat emitted by the laser is generally less than 3 °C, which does not cause any damage to the object.
Software analysis
The images collected by thermal cameras are analysed using specially developed software. In doing so, the software looks specifically at the heat left by the laser as it moved over the object. This involves using two different viewing angles. The system looks specifically at distortion to collect spatial coordinates. The software then merges this data into the exact dimensions of the object.
A key requirement for the effective use of such a sensor in an industrial environment is the speed at which objects can be scanned. After all, many manufacturing processes revolve around a combination of speed and high quality. If the system flawlessly produces more products per minute, this increases production profitability. This has therefore been a key focus in the development of the MWIR 3D Sensor.
Cooperation with industry
"Our research group has been cooperating for years with companies from a wide range of manufacturing areas," says Dr Stefan Heist, head of the 3D Sensors research group at Fraunhofer IFO. "Among them are companies in the semiconductor, automotive and aerospace industries. We look at past joint projects and analyse how our 3D glass sensor could have produced better results than conventional sensors in various application areas."
Heist added: "We managed to identify three key starting points to make our system suitable for potential use in a large-scale industrial manufacturing facility." The team therefore set out to find the ideal balance between the required resolution and detail of a scan, on the one hand, and the duration of a measurement, on the other. They also explored different options for the optical configuration of their 3D sensor. This allows the researchers to adapt measurements to the task at hand and the space available in a production facility.
The research team also worked closely with researchers at Ilmenau University of Technology. Together, the parties they optimised the data processing of the measurement system.
Author: Wouter Hoeffnagel
