AUTOMATION MYTH BUSTING SERIES - PAPER #2 - By Dr Itiel Dror, University College London (UCL) and Jayne Yeo, Project Officer, ERA
Automation as the holistic or partial takeover of the control of tasks and regulation of processes has great potential in our everyday lives: It can increase productivity and safety, and improve health and wellbeing in many ways, as well as help tackle many issues in our lives and society. However, automation, per se, by itself, does not necessarily achieve any of those benefits. To fulfil the potential benefits of automation it must take into account the human element. Too often automation is developed and deployed in a vacuum, without properly considering it in light of human and organisational factors (HOF). In such cases automation may not only not improve things but can actually have a negative and detrimental impact.
To move forward, utilising and maximising the benefits that automation offers, we first need to begin with doing away with the myths that surround automation. These myths blind us and misguide our efforts, so we must first engage in ‘myth busting’. Already in the early 1980s, researcher Lisanne Bainbridge pointed out these "ironies of automation"1, yet many of the myths associated with them persist in practice today. Within this introductory outline, we use examples from everyday life to give first insights into how automation has already changed our lives and thus already challenges some existing myths.
The number of existing myths is large, and they persist for a long time. Back in the 1960s, with emerging new technologies and increased automation, there was a ‘human concern’: What will people do with all their free time? There was much talk about the need to develop new hobbies and recreational activities so people will have things to do in their free time as automation will take over and reduce the need for people. And thus we introduce the first general automation myth namely that automation will give us much more free time whereas the reality is that most of us have very little free time and are busier than ever. With huge advances in technology and automation over the past 50 years, people are still incredibly busy, and in many ways they have only made us more busy, more hectic, and if anything, we have less free downtime than before.
A myth that remains persistently in people's minds: Automation will make people redundant and not needed. This perception is gaining renewed popularity with advances in Artificial Intelligence (AI) and increased computational capabilities with new technologies. These types of automation are particularly powerful, as they involve Cognitive Technologies. Such technologies do not improve our physical abilities, but support, take part in, and even take over, some of our cognitive activities, our actual thinking processes. However, will we humans not be needed anymore?
One the one hand, humans can give over some functions to the technology so as to reduce the cognitive load off from the human and free up their cognitive resources to carry out other operations. The simplest examples of that, is letting technology carry out mathematical operations for us, keeping track of appointments and alerting us when a scheduled appointment is coming up, remembering phone numbers for us, etc.
It would also be possible that the technology predominantly takes over, leaving the human to supervise and monitor. Between the two extremes, there are many intermediate modes of distributed cognition, whereby the human and automation collaborate as partners. In this collaboration, tasks should be distributed in such a way that the strengths of each partner are used optimally. For example, machines can solve complex analyses much faster and more correctly and store data for a long time, while humans excel in creative problem-solving tasks or spontaneity.
If you take driving for example, to illustrate the different modes of collaboration Offloading happens when drivers no longer need to remember to put on seatbelts, as the car will remind them, similarly, drivers no longer need to remember to turn on the lights at night, as the car does that for them – these operations are offloaded and nowadays are carried out by the car itself. At higher levels, the car works with us to achieve a variety of operations, from parking (whereby the car has sensors that tell us how near we are to an object or the car parks automatically under human supervision) to take part in emergency breaking (whereby anti-lock brakes automatically are deployed rather than the driver having to vary the pressure on the brake pedal), and GPS systems which advise the driver how to navigate.
These automations are no doubt very useful and can increase safety. However, they can also be a cause of accidents. As automation does more for us, as humans are left to only supervise and monitor, we are less engaged with the task at hand and are more likely to be distracted by other stimuli. These control tasks permanently require a high level of attention or vigilance which humans can only maintain to a certain level over a longer period of time. Accordingly, increasing automation can lead to accidents if tasks are not adequately designed. This addresses another myth: Automation will increase safety. Automation can increase safety, but it might not, again, it depends if HOF have been properly taken into account. If HOF are not taken into account this can lead to automation abuse – the disuse and misuse of automation by human operators. In particular, too little and too much trust in automated systems do not allow for any gain, but rather a reduction in safety.
The role of humans changes with automation - from active control to passive observation. It is not only limited to issues of safety, but also has a profound impact on human skills and even human intelligence. Hence another myth: Automation increases our abilities. In fact, automation may actually reduce human capacity. This is due to neuronal plasticity of the human brain, in a nutshell: use it or lose it. Automation ‘spoils us’, makes our brains lazy, as more and more is done by machines. As automation does more, as we do less, our skills and ability degrade. We actually think less! Take for example photography, before digital photography, when you had to buy a roll of film for your camera, which cost money and had only 36 exposures, and then pay to develop it into negatives, and pay more to have them printed, then you would think twice before taking any picture. You had to consider carefully which pictures to take and how to take them. Nowadays, with digital photography and smartphones, we do not think twice, we do not even think once, we just take photos without really thinking that much about which ones to take or how.
These are examples of how automation changes the way we act and think and, as a consequence, certain abilities and unused skills diminish over time. Going back to the driving a car example, even if one was very skilful at parallel parking, after relying on parking automation it is no longer easy to parallel park–that skill has degraded, it was not used and was therefore lost. The outcome of this skill loss is a possible increase in errors.
Another myth is: Technology and automation will eliminate bias. In reality often automation just makes the bias hidden. Remember that automation has been created by humans. Furthermore, automation can actually not only just hide existing biases, but can even create new biases. Research has been carried out on “automation bias” which is a specific class of human errors in highly automated decision-making contexts. Incorrect automation instructions or directions are simply followed without further verification or errors of the automation are not recognized and identified.
We need to be very careful about our beliefs about automation and what it achieves, some of them are naïve and are just myths. In the five automation myths highlighted (and there are many others), it has been shown that automation can be good, but it is not necessarily always good, it can also be bad. How good it is, how bad it is, depends much on if and how it fits and interacts with humans. If we properly consider the human and organisational factors in development and deployment of automation, we are more likely to enhance the benefits of automation and reduce its downsides.
This introductory outline shows that humans still matter, even as automation continues to advance. Automation myths still exist today in a wide variety of contexts, certainly also in rail transport. With the advancing automation and technical developments in railways, it is important to consider specific myths, which will now be done in a series of in-depth Automation myth busting papers. In this series, different researchers will specifically address selected myths, place them in the context of rail transport and further discuss them based on scientific evidence. This Automation Myth Busting Series will be published by the European Union Agency for Railways (ERA) in collaboration with the German Centre for Rail Traffic Research (DZSF) at the Federal Railway Authority.
1 Bainbridge, L. (1983). Ironies of automation. Automatica, 19 (6), 775-779. https://doi.org/10.1016/0005-1098(83)90046-8