Researchers at the Fraunhofer Institute for Factory Automation (IFF) have developed cognitive robotics capabilities that can perform complex tasks in manufacturing that were previously impossible to automate. They also reveal patented speed and distance monitoring and computer-aided safety (CAS).
Cognitive robotics
Researchers at Fraunhofer IFF are using new AI-based solutions to give robots the cognitive skills they need to operate autonomously in unstructured, changing environments. Cognitive robotics make it possible to automate complex processes, such as assembly and disassembly in industrial environments or object handling in healthcare.
Projection- and camera-based safety technologies
Projection- and camera-based safety technologies enable robots with AI-based motion control to react reliably to changes, adapt to new tasks and operate an application safely. This opens up a wide range of new application areas previously excluded from conventional robotics, which is limited to specific, narrowly defined tasks. Cognitive robotics allows robots to learn from experience, make decisions independently and adapt to different scenarios.
Pick-and-place
For pick-and-place tasks that involve picking up components and placing them in the right place, a cognitive robot no longer needs to learn what individual work pieces look like before it can grab them. Instead, it uses a camera to register the size, shape, texture and condition of the object and adjusts its behaviour accordingly. In doing so, it can deal with different environmental conditions and even different packaging materials," said Magnus Hanses, head of the Cognitive Robotics group at Fraunhofer IFF.
The cognitive robot uses a camera to register the size, shape, texture and condition of the object and adjusts its behaviour accordingly. In doing so, it can deal with different environmental conditions and even different packaging materials.Magnus Hanses, head of the Cognitive Robotics group at Fraunhofer IFF.
Cognitive robotics in use. Photo: Fraunhofer IFF, Anne Bornkessel
Training AI models
The experts use simulated environments to train the AI models used. For example, they simulate assembly and disassembly processes, such as removing a motherboard from a computer. An unlimited number of virtual robots can work simultaneously and much faster in digital space, without safety risks. Learning in digital simulation offers many advantages, but also has one vulnerability. The virtual learning environment is never 100 per cent the same as the real world.
Sim2real gap
The challenge for the researchers is to close this gap between reality and reality, also known as the Sim2Real gap, as much as possible. There are two possible approaches. The simulation can be designed as realistically as possible, or it can include as wide a range of real-world versions as possible so that the neural network used for the AI learns to generalise and find its way in unfamiliar environments.
Domain randomisation
One way the researchers are achieving this is through domain randomisation. This approach allows them to create a large number of simulated environments with random properties and train a model that works in all of them. "There are many different parameters, such as illumination, that affect the simulation. We can change these parameters during training. The robot does not learn to solve the exact simulation. Instead, it learns to understand the abstract concept behind it. Reality becomes, so to speak, just another version of a simulation for the AI," Hanses explains.
PARU - patented speed and distance monitoring
But cognitive robotics faces another challenge. Currently, there is no way to ensure the safety of AI-generated robot movements in line with safety standards. To ensure that AI-based robots can interact with humans in a safe environment, researchers at Fraunhofer IFF have developed PARU, a patented new technology for monitoring workspaces.
Projector and camera technology
PARU uses advanced projector and camera technology to project visible warning and protection fields directly around the machine and recognise when humans enter safety zones. "After the projector and the two cameras are calibrated, the first step is to generate virtual expectation images. The projector then projects a visible light curtain around the robot and the part to be picked up, according to the distance formula defined in the ISO/TS 15066 standard. This light curtain acts as a safety line and visualises for employees the protective space that people must keep clear," explains Norbert Elkmann, head of the Robotics Department at Fraunhofer IFF.
Fostering trust
Safety zones are dynamically adapted to the machine's movements, as PARU always takes into account the robot's current state. This makes it ideal for use in cognitive robotics. "Our technology is unique. No other system allows a smaller distance between human and robot, while meeting the specifications of applicable standards and at the same time requiring so little space. This is possible because the cameras and sensors recognise not only torsos, arms and heads, but even fingers," Elkmann said.
Projection
Another advantage is that the projection can also show the worker where the robot will move next. This increases confidence in working with machines. The additional coded, visible safety lines work independently of lighting conditions and conditions. If the cameras or projectors stop working, the entire system is automatically switched off.
CAS: intelligent software solutions for adaptive robotic systems
Fraunhofer IFF has launched Computer Aided Safety (CAS), a suite of digital safety solutions. It makes human-robot collaboration (HRC) efficient, cost-effective and safe . Ready-made software modules are available for efficient calculation of safe distances and speeds. Digital assistants support risk assessment and safety approval processes and make it easier, especially for newcomers, to accurately and efficiently comply with all obligations of the EU Machinery Directive.
Digital
Unlike the collision measurement function, the safety approval tool works completely digitally. It takes parameters such as impact force and pain threshold into account to determine the maximum permissible speed of the robot. The modules can optionally be integrated into any type of robot controller or existing simulation environment for planning purposes, to accurately match economic specifications to applicable safety requirements. This prevents planning errors and saves on engineering costs.
Opening photo:
Patented PARU safety technology generates visible light curtains around the area where the robot works. The illuminated blue safety lines dynamically adjust to the machine's movements, enabling safe interaction between humans and AI-controlled robots (Photo: Fraunhofer IFF, Anne Bornkessel)
Also read: TOP 5 trends in robotics in 2025
