Behavioural Sciences

Meaningful human control: Designing safety into automated driving systems

Automated vehicles hold great potential for improving traffic flow, fuel consumption and road safety, while reducing workload and stress for drivers. However, the vehicle still requires human supervision, with the human-machine interface improving the human’s ability to supervise the automated driving system (ADS). Simply putting the human in control of supervising such an automated system, however, is not meaningful. Humans have been trained to drive a vehicle, not to monitor an automated system. Therefore, when vehicles become more and more automated, the tasks for the human driver change. But how many tasks, and to what extent? Dr Daniël Heikoop, Delft University of Technology, is developing a framework for eliciting meaningful human control over automated vehicles.

Regulations demand that the driver of an automated vehicle must always be able to take over manual control, and in such a scenario, it is vital they are alert and aware of the surrounding conditions. The individual’s ability to act is impaired, however, if they are inattentive due to focusing on non-driver related tasks like using their phone, eating, or resting. Currently, the driver’s role is not clearly defined or understood, and the notion that more automation means improved safety is not accurate as there is insufficient understanding of human factors and behaviour when using ADS.

Dr Heikoop’s research has explored peoples’ behaviour when driving an automated vehicle. He is currently involved in the Meaningful Human Control (MHC) project, a collaboration with psychologists, traffic engineers, and philosophers that addresses some of the challenges raised by ADS. The MHC project also considers the mismatch between a decrease in requirements for driver skills and rule-based behaviour, and a rise in untrained behaviour, due to the novel tasks introduced by driving with an ADS.


Understanding ADS
An explanatory model that describes how drivers interact with their vehicle is beneficial. As we see advancements in vehicle automation technology, even more research is vital to provide an updated safety framework. To close this gap, during his PhD Dr Heikoop studied available literature on driving automation and conducted a range of experiments. The research has led to an updated psychological model which provides an informed basis for further developments in the field of knowledge on the relationship between ADS and driver behaviour. This helps to highlight any discrepancies and mismatches, and makes recommendations for future research and design strategies.

As regulations demand that the driver of an automated vehicle can take over manual control,
awareness of surrounding conditions is vital.


Platoon driving
One way to address gaps in knowledge about automated driving is to design and conduct research, and the team carried out an experiment to test driver behaviour when driving in a platoon. A platoon, or convoy, involves a tightly spaced group of vehicles travelling together, a configuration that is attractive in terms of safety and fuel efficiency. Whilst legislation does not clearly define the driver’s role, and much of the time they are exempt from control, they must have the capacity to gain control in a critical incident. Concern has been expressed by researchers on the psychological state of the driver and their ability to monitor task conditions, as they are likely to lose their situational focus, particularly on a long journey. Previous experiments have not directly tested the type of attention required by ADS and so, to test driver alertness when in a platoon, an experiment was conducted.

Automated vehicles hold great potential for improving traffic flow, fuel consumption and road safety, while reducing workload and stress for drivers. metamorworks/

Participants were placed in a driving simulator and given three different task instructions; No Task, Voluntary Task (where they could do what they choose, for example read, eat, check their phone), and Detection Task (where they were asked to detect red cars among traffic). In all tasks they were asked to watch the road and intervene if needed; heart rate was monitored to measure stress, heart rate variability for workload, and eye tracking documented the percentage of eye closures as an indicator of attention. Interestingly, half of participants in the Voluntary Task chose to detect red cars to ensure they remained attentive. The participants reported that Detection Task and the Voluntary Task were less worrisome than No Task, where subjects had nothing to focus their attention on. However, the Detection Task was found to be the most workload-inducing. This simulator-based experiment indicates that not only the type of task, but also time-on-task affects the driver. As a means to increase performance and reduce stress, these findings should be considered in system design.

Dr Heikoop, Delft University of Technology, is developing a framework for eliciting meaningful human control over automated vehicles. Andrey Suslov/

Testing situation awareness
Another experiment in the simulator required participants to do a task of low demand (involving ‘thinking out loud’ at timed intervals, saying whatever they were thinking of), a medium demand task (requiring them to do the first task and also encouraging them to repeat a consonant that had been uttered two letters ago), and a high demand task (requiring them to do all the experimental tasks). The findings indicate that although drivers may be able to combat fatigue and improve alertness by performing verbal tasks, this does not imply that they are aware of the situation around them. For policy makers and designers, the outcomes suggest that situation awareness alone is insufficient and that some level of mental demand, such as that requested in the Medium Demand Task condition should be requested so that drivers maintain an acceptable level of situation awareness.

Future research on safety-critical situations, rather than just on drivers’ psychological state, are required. Zapp2Photo/

In a similar test to the simulator experiment, but this time on a UK motorway, participants were monitored for the effects of driving over a period of time in a real-world setting. These participants all worked for a large automotive company and already had enhanced driver training experience, but no ADS experience. Heart rate and eye tracking were monitored to measure stress and workload, and the test car, a Tesla Model S 90D with Autopilot, followed a lead car. A safety driver sat in the passenger seat at all times but only intervened if the participant acted inappropriately or unsafely. The experiment consisted of two drives: in one, participants were instructed to press a button when driving under a bridge, in the other, there was no task. The findings indicated an acclimatisation to automation, which was seen in the decreased heart rate and self-reported workload over time.

Meaningful human control is central to ensuring automated driving systems are designed with safety and accountability in mind.

Indicators for further research
Dr Heikoop’s work gives useful indications for future research and design, and importantly, can help to address the lack of clear driver training guidelines and legal guidelines. Future research on safety-critical situations, rather than just on drivers’ psychological state, is required. For example, if a driver inappropriately leaves a platoon situation, design features and sufficient training may need to be developed to counteract this variable. Human behaviour in the event of an unexpected incident, such as an automation malfunction, also needs consideration.

This research offers vital indications for future research and design; importantly it can help to address the lack of clear driver training guidelines and legal guidelines. Scharfsinn/

The range of experiments conducted suggest that keeping the driver alert through verbal and mental tasks may not equate to the situational awareness needed if an emergency occurs. Further, the requirement for awareness is of differing magnitudes – taking over manual control when exiting a platoon requires lower levels of awareness than coping with a complex incident. A balanced level of situational awareness that ranges across low, medium and high levels of alertness is likely to be appropriate. Further research into on-road settings could test various scenarios such as driving in rush hour, using differing levels of automation, like adaptive cruise control, and driving without a safety driver in the passenger seat. Meaningful human control is central to ensuring automated driving systems are designed with safety and accountability in mind. The research provides results that can be utilised by designers, manufacturers, lawyers and policy makers, as the use of ADS increases.

Who is Danny/

For more information about the Meaningful Human Control over Automated Driving Systems project, visit

Personal Response

What do you see as being the emerging issues in ADS that you would like to focus future research on?

<>Clearly, the emerging issues are the introduction of SAE level 2/3 automated driving systems onto the consumer market. Regular people are not trained to drive with such systems, as these systems introduce a plethora of novel tasks, such as supervising an autonomous system, and collaborating with such a system. My future research focus aims to develop guidelines for driver training with automated driving systems, to ensure safety in a future where we will drive with increasingly automated driving systems.

This feature article was created with the approval of the research team featured. This is a collaborative production, supported by those featured to aid free of charge, global distribution.

Want to read more articles like this?

Sign up to our mailing list and read about the topics that matter to you the most.
Sign Up!

Leave a Reply

Your email address will not be published. Required fields are marked *

Thank you for expressing interest in joining our mailing list and community. Below you can select how you’d like us to interact with you and we’ll keep you updated with our latest content.

You can change your preferences or unsubscribe by clicking the unsubscribe link in the footer of any email you receive from us, or by contacting us at at any time and if you have any questions about how we handle your data, please review our privacy agreement.

Would you like to learn more about our services?

We use MailChimp as our marketing automation platform. By clicking below to submit this form, you acknowledge that the information you provide will be transferred to MailChimp for processing in accordance with their Privacy Policy and Terms.

Subscribe to our FREE PUBLICATION