Robots of all shapes and sizes are getting smarter thanks to the implementation of artificial intelligence (AI). This technology enables robots to draw their own conclusions based on observations or data and act accordingly. However, who is responsible if a robot equipped with AI causes a fatality?
This question has been explored by John Kington, professor of Information Systems and Business Computing at the UK's University of Brighton. In a paper, Kingston sets out his findings. In his paper, the professor points to previous research from 2010 by Gabriel Hallevy, a professor affiliated with the Faculty of Law at Israel's Ono Academic College. In his paper, Hallevy outlined several legal models for the legal assessment of trespassing by AI systems.
1. Indirect perpetrator (perpetrator-via-another)
If the law is broken by a mentally incompetent person such as a child or an animal, the offender is presumed innocent as he or she does not have the mental capacity to form a 'malicious intent' (mens rea). However, in this case, if the offender was instructed by a third person to commit an offence, that third person is legally responsible.
According to this model, an AI programme can be labelled mentally incompetent, Hallevy concludes. However, the developer or user of an AI algorithm could be held responsible for the offence committed by the AI.
2. Natural-probable-consequence (natural-probable-consequence)
This model applies if part of an AI programme developed for legitimate purposes is improperly deployed and thereby commits an offence. As an example, Hallevy cites an employee at a motorbike manufacturing plant who is killed by the robot with AI working alongside him. In this case, the robot appears to have identified the employees as a threat to its mission and concluded that pushing the employee into an adjacent machine was the most efficient method to eliminate this challenge. Using its hydraulic arm, the robot pushed the unsuspecting employee into this machine, killing the employee.
Natural and probable consequence is usually applied to prosecute accomplices for a crime. For example, US law allows for holding a defendant responsible if a law violation is the natural and probable consequence of an action that the accomplice encouraged or supported. A condition for this, however, is that the accomplice was aware that the act performed was illegal.
This model, according to Hallevy, allows a user or programmer to be held responsible if they were aware that a breach of law was the natural and likely consequence of the programme they used or developed. Hallevy does argue that a distinction should be made here between AI programmes that 'know' that the act they are performing is illegal (if, for example, the programme was specifically written to perform a criminal act) and AI programmes that do not know this (if the programme was programmed for a different purpose). Hallevy suspects that the latter group cannot be sued for offences requiring 'malicious intent'.
3. Direct liability
Direct liability, according to Hallevy, refers to both the act (actus rea) and the intention of the perpetrator (mens rea). Hallevy argues that the actus rea can be assigned to an AI system relatively easily, for example in situations where the system performs an act that results in a violation of the law or, on the contrary, the system fails to perform an action when it should be performed.
However, assigning human rea to an AI system is much more difficult, according to the researcher. AI systems are therefore expected to be held liable mainly in cases where it does not matter whether an offence was deliberately made. As an example, Hallevy cites a situation in which a self-driving car exceeds the speed limit, arguably committing a law violation. In this case, according to the researcher, the legal responsibility for this lies entirely with the AI programme that controlled the car during this drive, and not with the user.
Defence
The paper also looks at how an AI system could defend itself against allegations in court. For example, Hallevy points to several cybercrime court cases in which it was successfully put forward as a defence that malicious software had taken control of the defendant's computer. The criminal act would have been committed by this malicious software and therefore not the responsibility of the owner of the system.
Based on the legal models outlined by Hallevy, Kingston concludes that the legal responsibility of AI systems depends on three factors:
- Whether an AI is a product or a service. This is not clearly defined in law; different experts have different views on this.
- Which human rea required to classify an act as a criminal offence. According to Kingston, it is unlikely that an AI programme could be found guilty of violating a law that requires the perpetrator to have knowingly performed a criminal act. However, Kingston does call it possible for AI systems to be found guilty of breaking the law if they perform an action that a "reasonable man would have known" would result in a law violation. Kingston also argues that AI systems can be found guilty with great certainty in cases involving liability for the consequences of certain actions.
- Whether the limitations of AI systems are communicated to the buyer. Some AI systems have both general and specific limitations. In court cases, Kingston says the specific words used in warnings about these limitations can be referred to.
Who is responsible?
Finally, the question arises as to who is responsible for violations committed by AI systems. According to Kingston, this depends on which of Hallevy's three models are applied:
- Indirect perpetrator (perpetrator-via-another): the person who instructs the AI system to perform an act is likely to be held responsible, according to Kingston. This could be the user or the programmer.
- Natural-probable-consequence: In this case, a person who could have foreseen that the AI product would be deployed in this way can be held responsible, according to the professor. This could be the programmer, product vendor or service provider. In this case, Kingston says the user is less likely to be held responsible unless the instructions provided with the system clearly state the limitations of the system and discuss in detail the possible consequences of misuse.
- Direct liability: AI programmes can be held liable for the consequences of actions taken. In practice, this is likely to mean that the programmer who built the AI programme will be held responsible.
Kingston, incidentally, notes that in all cases where the programmer is held responsible, it can be debated whether the fault lies with the programmer, the designer of the AI programme, the expert who provided the required knowledge or the manager who appointed an incompetent designer or programmer.
More information can be found in Kingston's paper 'Artificial Intelligence and Legal Liability'.
Author: Wouter Hoeffnagel
Source: Paper 'Artificial Intelligence and Legal Liability by John Kingston
Source photo: Pixabay / jarmoluk
