Meaningful human control: Designing safety into automated driving systems

Regulations demand that the driver of an automated vehicle must always be able to take over manual control, and in such a scenario, it is vital they are alert and aware of the surrounding conditions. The individual’s ability to act is impaired, however, if they are inattentive due to focusing on non-driver related tasks like using their phone, eating, or resting. Currently, the driver’s role is not clearly defined or understood, and the notion that more automation means improved safety is not accurate as there is insufficient understanding of human factors and behaviour when using ADS.

Dr Heikoop’s research has explored peoples’ behaviour when driving an automated vehicle. He is currently involved in the Meaningful Human Control (MHC) project, a collaboration with psychologists, traffic engineers, and philosophers that addresses some of the challenges raised by ADS. The MHC project also considers the mismatch between a decrease in requirements for driver skills and rule-based behaviour, and a rise in untrained behaviour, due to the novel tasks introduced by driving with an ADS.

Understanding ADS
An explanatory model that describes how drivers interact with their vehicle is beneficial. As we see advancements in vehicle automation technology, even more research is vital to provide an updated safety framework. To close this gap, during his PhD Dr Heikoop studied available literature on driving automation and conducted a range of experiments. The research has led to an updated psychological model which provides an informed basis for further developments in the field of knowledge on the relationship between ADS and driver behaviour. This helps to highlight any discrepancies and mismatches, and makes recommendations for future research and design strategies.

As regulations demand that the driver of an automated vehicle can take over manual control,
awareness of surrounding conditions is vital.

Platoon driving
One way to address gaps in knowledge about automated driving is to design and conduct research, and the team carried out an experiment to test driver behaviour when driving in a platoon. A platoon, or convoy, involves a tightly spaced group of vehicles travelling together, a configuration that is attractive in terms of safety and fuel efficiency. Whilst legislation does not clearly define the driver’s role, and much of the time they are exempt from control, they must have the capacity to gain control in a critical incident. Concern has been expressed by researchers on the psychological state of the driver and their ability to monitor task conditions, as they are likely to lose their situational focus, particularly on a long journey. Previous experiments have not directly tested the type of attention required by ADS and so, to test driver alertness when in a platoon, an experiment was conducted.

Participants were placed in a driving simulator and given three different task instructions; No Task, Voluntary Task (where they could do what they choose, for example read, eat, check their phone), and Detection Task (where they were asked to detect red cars among traffic). In all tasks they were asked to watch the road and intervene if needed; heart rate was monitored to measure stress, heart rate variability for workload, and eye tracking documented the percentage of eye closures as an indicator of attention. Interestingly, half of participants in the Voluntary Task chose to detect red cars to ensure they remained attentive. The participants reported that Detection Task and the Voluntary Task were less worrisome than No Task, where subjects had nothing to focus their attention on. However, the Detection Task was found to be the most workload-inducing. This simulator-based experiment indicates that not only the type of task, but also time-on-task affects the driver. As a means to increase performance and reduce stress, these findings should be considered in system design.

Testing situation awareness
Another experiment in the simulator required participants to do a task of low demand (involving ‘thinking out loud’ at timed intervals, saying whatever they were thinking of), a medium demand task (requiring them to do the first task and also encouraging them to repeat a consonant that had been uttered two letters ago), and a high demand task (requiring them to do all the experimental tasks). The findings indicate that although drivers may be able to combat fatigue and improve alertness by performing verbal tasks, this does not imply that they are aware of the situation around them. For policy makers and designers, the outcomes suggest that situation awareness alone is insufficient and that some level of mental demand, such as that requested in the Medium Demand Task condition should be requested so that drivers maintain an acceptable level of situation awareness.

In a similar test to the simulator experiment, but this time on a UK motorway, participants were monitored for the effects of driving over a period of time in a real-world setting. These participants all worked for a large automotive company and already had enhanced driver training experience, but no ADS experience. Heart rate and eye tracking were monitored to measure stress and workload, and the test car, a Tesla Model S 90D with Autopilot, followed a lead car. A safety driver sat in the passenger seat at all times but only intervened if the participant acted inappropriately or unsafely. The experiment consisted of two drives: in one, participants were instructed to press a button when driving under a bridge, in the other, there was no task. The findings indicated an acclimatisation to automation, which was seen in the decreased heart rate and self-reported workload over time.

Meaningful human control is central to ensuring automated driving systems are designed with safety and accountability in mind.

Indicators for further research
Dr Heikoop’s work gives useful indications for future research and design, and importantly, can help to address the lack of clear driver training guidelines and legal guidelines. Future research on safety-critical situations, rather than just on drivers’ psychological state, is required. For example, if a driver inappropriately leaves a platoon situation, design features and sufficient training may need to be developed to counteract this variable. Human behaviour in the event of an unexpected incident, such as an automation malfunction, also needs consideration.

The range of experiments conducted suggest that keeping the driver alert through verbal and mental tasks may not equate to the situational awareness needed if an emergency occurs. Further, the requirement for awareness is of differing magnitudes – taking over manual control when exiting a platoon requires lower levels of awareness than coping with a complex incident. A balanced level of situational awareness that ranges across low, medium and high levels of alertness is likely to be appropriate. Further research into on-road settings could test various scenarios such as driving in rush hour, using differing levels of automation, like adaptive cruise control, and driving without a safety driver in the passenger seat. Meaningful human control is central to ensuring automated driving systems are designed with safety and accountability in mind. The research provides results that can be utilised by designers, manufacturers, lawyers and policy makers, as the use of ADS increases.

For more information about the Meaningful Human Control over Automated Driving Systems project, visit https://www.tudelft.nl/ceg/mhc-ads