More and more companies are developing and testing autonomous vehicles. These vehicles drive fully automatically and make their own decisions. To a large extent, users therefore put their lives in the hands of a computer. To do this safely and responsibly, it is necessary to ensure that the algorithms on which autonomous vehicles base their decisions contain as few errors as possible. After all, an error made by an autonomous vehicle could have deadly consequences in the worst case scenario. A new testing framework called DeepXplore detects errors in algorithms of self-driving cars.
Autonomous cars make choices independently, relying on algorithms. These algorithms are created independently by computer systems through a form of machine learning called deep learning. To do this, it uses an artificial neural network, which is based on the human brain. Such a network consists of several layers of chips, which mimic neurons. Each layer makes a different analysis of the data being analysed and passes on its results to the next layer of neurons.
Independent learning
Deep learning networks can therefore learn autonomously and get smarter and smarter as they analyse more data. The technology has the significant advantage that algorithms can become significantly smarter than they could ever be programmed by a human programmer. A disadvantage, however, is that no one knows exactly how these algorithms are written and how they respond to exceptional situations.
"Critical safety and security systems based on deep learning, like traditional software, need to be systematically tested for various exceptions to ideally detect and remedy any potential error or undesirable behaviour. This presents a new problem, as automated and systematic testing of large-scale deep learning systems with thousands of neurons and millions of parameters is extremely challenging," Kexin Pei, Junfeng Yang and Suman Jana from Columbia University and Yinzhi Cao from Lehigh University write in a research paper.
Things do go wrong sometimes, unfortunately
The researchers cite some examples of situations where things have already gone wrong. For example, the researchers point to a recent accident involving a self-driving car from Google, which assumed that a bus would yield the right of way. However, this turned out not to be the case, resulting in an accident.
Another example is a Tesla car that was driving in autopilot mode and collided with a truck. This accident could have occurred because the autopilot system could not recognise the trailer as an obstacle, as the white colour of the trailer was difficult to distinguish. The system proved unable to distinguish the white trailer from the brightly lit sky and was additionally confused by the trailer's high ride height.
Traditional testing
Traditional testing for deep learning systems involves two methods. In the first method, manually generated test images are randomly presented to a deep learning system until an image makes a wrong decision. The second method uses test images that are generated automatically. Here, a test image is continuously modified in small increments, just until an image causes the system to make the wrong decision. However, the researchers argue that these methods can never test more than a minimal amount of all possible exceptions.
"The main challenges in automated and systematic testing of large-scale deep learning systems are twofold: (1) how to generate enough input that triggers different parts of the logic of deep learning systems and discover different types of incorrect behaviour, and (2) how to identify incorrect behaviour of a deep learning system without manually applying labels and performing checks," the researchers said.
DeepXplore
To overcome these challenges, researchers have developed the new testing framework DeepXplore. This system deploys the deep learning network being tested to generate test images that have a high probability of causing different layers of neurons to make conflicting decisions. This way of working makes it possible to test many more exceptions than is possible with traditional testing methods.
By the way, DeepXplore can not only detect errors in deep learning systems, but also correct them. If the different layers of the neural network make conflicting decisions, DeepXplore goes to work to correct this error. This involves retraining the neural network to better recognise the picture that leads to problems. DeepXplore thus increases the accuracy of the system and prevents the system from going wrong again.
Increase accuracy by 3%
Researchers put DeepXplore to the test and used the testing framework to test 15 deep learning algorithms trained using five datasets. In this test, DeepXplore managed to detect thousands of errors in the deep learning algorithms that were not identified by previous tests using other techniques. The researchers also report using DeepXplore to increase the accuracy of some networks by 3% by re-training the network based on DeepXplore's input.
Incidentally, DeepXplore is not beatific. The testing method can be used to significantly reduce the number of errors in deep learning systems, but the testing framework cannot guarantee that a deep learning system meets a certain safety standard. Incidentally, there are already systems available that are capable of this. However, these systems are currently only capable of controlling small-scale deep learning systems and are not suitable for the large-scale networks used in autonomous cars.
Author: Wouter Hoeffnagel
Source: Columbia University
Source: arXiv
Source: Pixabay / markusspiske
