For anyone who has lost a hand, a functional prosthetic hand is a huge advantage in everyday activities. This is why researchers at Fraunhofer are working as part of an EU research project to improve the control of prosthetic hands down to individual fingers. Instead of conventional electrodes that detect nerve impulses in muscle tissue in the arm, they rely on ultrasonic sensors. This means commands can be executed much more accurately and sensitively. In the next phase, the researchers aim to make the design bidirectional, with the brain also receiving sensory stimuli from the prosthesis.
Together with the partners in the project, Fraunhofer researchers have shown that controlling prosthetic hands can be significantly improved. This is possible through the use of ultrasonic sensors. For example, someone who has lost a hand after an accident might be able to control individual fingers on the prosthesis even better. It is also possible to move the fingers even more precisely than was previously possible with myoelectric prostheses. Myoelectric prostheses usually work with electrodes on the skin that pick up electrical signals from muscle contractions and transmit them to an electronics module, which in turn controls the prosthesis. With the so-called SOMA project, scientists at the Fraunhofer Institute for Biomedical Engineering IBMT in Sulzbach have opted for a new approach using ultrasonic sensors.
Ultrasonic sensors
The researchers use ultrasonic sensors that continuously send sound pulses to the muscle tissue in the forearm. Unlike electrical impulses, sound waves are reflected by tissue. The time it takes for the reflected signals to propagate provides information about the physical depth of the muscle strand reflecting the corresponding sound wave. As a result, contractions in muscle tissue triggered by nerve impulses in the brain can be studied in great detail. This means that identification of typical activation patterns in the muscle such as hand or finger movements is possible.
AI-controlled software
The aim of the project is for AI-controlled software in a compact electronics box to take over the task of identification. This box wears the patient on the body. The electronics could send the decoded signals as a command to the actuators in the prosthetic hand. This causes the prosthetic fingers to move. The control commands are detected, analysed and sent in real time.
Laboratory phase
This EU basic research project is currently still in the laboratory phase. Ultrasonic transducers and electronics generate signals and decode the sound waves that are reflected back. This data is then transmitted to a PC where the AI starts analysing. The electronics then send the decoded signals as commands to the actuators in the prosthetic hand. This triggers the finger movement. In fact, the benefits of this technology are already evident. "The ultrasonic control works with greater sensitivity and accuracy than would be possible with electrodes. The sensors are able to detect different degrees of freedom, such as bending, stretching or twisting." So says Dr Marc Fournelle, head of the Sensors & Actuators group at Fraunhofer IBMT. He is responsible for the development of the SOMA ultrasonic sensors within the project.
Time differences
It is important to achieve high precision and reliability. Therefore, piezoelectric sound transducers send impulses dozens of times per second into the muscle tissue at a frequency between 1 and 4 MHz. Moreover, at least 20 sensors are interconnected. In addition to depth information, each sensor also provides data on the position of the muscle strand that has just returned a wave. The collected data on the location and depth of the signals are pre-sorted before the AI goes to work. "The AI then has to analyse the ultrasound signals. It then identifies an activation pattern. Finally, AI converts this into a control command that is sent to the corresponding finger on the prosthesis. From an engineering perspective, the AI analyses the amplitude and time profile of the electrical voltages delivered by each sensor module," Fournelle explains.
Bracelet
The sensors are integrated into an armband that can be inserted into the shaft of the prosthetic hand at a later stage. To correctly link the muscle signals to the correct finger and the desired movement, the subjects have to complete a short training session. This involves them trying to move different parts of the hand and fingers. The activity patterns created in this way are stored by the system as a basic reference. This means that a link can be established between the corresponding finger or part of the hand and the desired movement. The training takes only a few minutes.
The technology works
Andreas Schneider-Ickert, project manager in the Active Implants unit and innovation manager at Fraunhofer IBMT, says: "Experiments with test subjects have shown that the technology works. It is very easy to use and non-invasive. We are now working on making the system even more unobtrusive."
Project partners in five countries
A total of seven partners from five countries are working together as part of the SOMA consortium. Fraunhofer IBMT experts contribute their experience in sensor development and in the field of neuroprosthetics and implants, among others. The team developed the specially adapted ultrasonic transducers and the electronics box. The Imperial College of Science Technology and Medicine in London, together with the Fraunhofer researchers, developed the AI process for recognising movement patterns and conducted the first tests on test subjects. "We have also been working very closely with the Università Campus Bio-Medico di Roma (UCBM) for several years. They coordinate the whole SOMA project and approached us with the idea for the sensors," explains Schneider-Ickert.
Next phase
Following the proof of concept and positive feedback from the test subjects, work on SOMA continues steadily. In the next phase, the researchers want to further improve the temporal resolution of the sensors and make the electronics smaller. This will allow even more precise and comfortable control of the prosthesis. The sensor bracelet will be hidden in the cuff of the prosthetic hand. With a view to improving its suitability for daily use, it is also conceivable that the AI and control software will one day be integrated into a smartphone. After the signals are decoded by the electronics box, they can be sent to the smartphone and back again via Bluetooth, for example.
Sensory feedback of the prosthetic hand
The consortium is also working on making the system bidirectional. The prosthetic hand should not only be able to execute commands,. It should additionally be able to provide feedback that the person wearing the prosthesis can sense as a sensory stimulus and respond to. "When a person who has not lost his hand picks up a glass of water and holds it to his mouth, he gets constant feedback from his fingers on how tightly to hold the glass. As a result, on the one hand, the glass does not slip out of his hands. On the other hand, it also does not shatter because it has been squeezed too hard. "Such functionality is something we are researching within SOMA," explains Schneider-Ickert.
User acceptance and usability
Usability and user acceptance are crucial factors at every stage of the project. The SOMA project team has collected feedback from subjects in each phase. In the current phase, these are subjects who still have their hands. "The subjects' feedback helps us make this innovative prosthetic hand even better. People who have lost a hand have endured a long period of suffering. A functioning prosthetic hand is a huge advantage in everyday activities and also restores some of the quality of life," explains Schneider-Ickert.
Improved comfort
The development of the innovative hand prosthesis also gives a noticeable boost to the market for myoelectric prostheses. Worldwide, an estimated three million people have had an arm or hand amputated and this number continues to grow. These people should benefit from this improvement in myoelectric prostheses in terms of functionality and comfort.
Source: Fraunhofer; Image: Demonstration sensor wristband used to measure muscle activity related to hand movements in subjects (Photo: Fraunhofer IBMT)
