Analysis of Automation Mode Confusion with Brazilian Airline Pilots

ABSTRACT

Automation is an important feature in-flight operations and its implementation in aviation has brought numerous benefits, such as reducing the workload and optimizing operations. However, there are difficulties in crew interaction with automation, such as autopilot mode confusion. To expose which mode confusions are usually prevalent and recurrent with Brazilian pilots, a survey-style questionnaire was administered to airline pilots. The survey results, answered by 145 pilots, reveal that the average frequency of mode confusion is 2.01 ± 0.40 occurrences per year and less experienced pilots are more likely to experience the phenomenon. This research points out that the prevalence of the phenomenon of mode confusion occurs in the final stages of flight, the vertical navigation (VNAV) modes being the most complex and confusing to pilots. The research also presents the main causes of mode confusion, as perceived by the pilots, their consequences, and mitigating measures. Data collected in this study can be used in future studies and by airlines to develop mitigating actions to manage the risk of mode confusion.

Keywords
Automation mode; Automatic flight control; Confusion; Situational awareness

INTRODUCTION

Human error has historically been credited for 65–80% of air transport accidents, as a whole or in part. Thus, one of the motivations for increased automation in transport aircraft has been manufacturers’ and operators’ aim to decrease the incidence of human errors by automating more pilot actions (Billings 1996).

The integration of the area navigation system with autopilot in the late 1970s was a step toward advanced automation. During this period, the first issues regarding the safety of cabin automation appeared, as did studies on the subject. The research on accidents and incidents revealed that while integrating cockpit automation, proper human factors concepts should be followed (Parasuraman et al. 1992).

Regardless of these issues, automation technology continues to be developed and widely adopted. The incorporation of management devices for automated systems was a significant advancement. These new automated technologies offer horizontal and vertical navigation (VNAV) as well as automatic guidance, providing pilots with improved flight paths and optimal fuel efficiency (Parasuraman et al. 1992).

For many years, automated systems have been employed successfully, contributing significantly to advances in safety, operational efficiency, and accurate flight path management. However, pilot use of and interaction with automated systems revealed vulnerabilities in some areas. Evidence of potential safety issues, such as accidents, incidents, crew reports, training, operational challenges, research, and surveys, point to flaws in this area (FAA 2013).

FAA’s Commercial Airline Safety Team (CAST) and Performance Based Aviation Rule-making Committee (PARC) identified difficulties that revealed vulnerabilities in the interaction of flight crews with autopilot mode and situational awareness. Concerns related with automation included: (i) Pilots sometimes rely too much on automated systems and may be reluctant to intervene; (ii) Autoflight mode confusion errors continue to occur, and can cause automation surprises; (iii) The use of information automation is increasing, including implementations that may result in errors and confusion (FAA 2013).

Automation surprises happen when the automation does not act as the pilots have anticipated, for example, when the autopilot has detached and the pilots are not aware of it (Mumaw et al. 2001). These surprises are notably widely documented in commercial airplane cockpits, and several deadly accidents and other occurrences have been linked to issues with the flight crew automation interaction (Rushby et al. 1999).

According to cognitive scientists, humans create mental models of their surroundings. Operators of an automated system, particularly, create models of system behavior and use them to orient their actions with the system. When the actual behavior of a system differs from the projected model of its operator, it is referred to as an automation surprise (Rushby et al. 1999).

Sophisticated systems are frequently divided into modes (e.g., an aircraft’s flight control system may have several modes for cruising, descent, and landing), and their behavior can vary considerably across modes. When a system is in a different mode compared to the one anticipated by its operator, confusion occurs. This circumstance may result in automation surprises, as the operator might interact with the system using a mental model that is incompatible with the system’s present mode (Rushby et al. 1999).

This article is intended to address the following research question: Which autopilot mode confusions are usually prevalent and recurrent with Brazilian pilots? By means of a research survey, another intention is to find out if there are specific phases of the flight that are more vulnerable to mode confusion and if it varies with the pilot’s experience.

There are few scientific papers in Brazil on the subject, which is, therefore, a field that needs to be explored, especially regarding the experience of Brazilian pilots with the situation of mode confusion in commercial aircraft.

The motivation came from a similar survey carried out by researchers in Europe (De Boer and Dekker 2017). The application of the research to the Brazilian reality can benefit the understanding of the main modes which are susceptible to this phenomenon, in which phases of flight they occur, and the main risks associated with mode confusion.

This article is expected to expose the reality of Brazilian pilots concerning surprise and confusion between automation modes, serving as a basis for future studies on mitigating measures applied to aeronautical operation, training, and manufacturing.

A survey-style questionnaire was developed to verify the prevalence of automation mode confusion with Brazilian pilots and contrast these data with studies and concepts in the literature.

Structure of Automation Modes

Nowadays, the amount of automation, integration, and sophistication of flying systems suggests that pilots have higher cognitive needs. The crew must master several complicated dynamically-interacting systems, which frequently operate at varying levels of automation. These systems frequently have numerous modes of operation to allow flexibility during the many phases of flight (Miller et al. 2002).

Modes are a method of organizing complexity and providing the operator with a range of control styles, called operating modes. Combinations of the major modes and submodes of glass cockpit airplanes provide pilots with a variety of alternatives for flying the aircraft: manual, semiautomatic, or completely automated. Additionally, they enable operators to customize automation to their specific scenarios and personal preferences. Unfortunately, flexibility, power, and choice all have their drawbacks. While modes enable users to achieve more, they eventually result in confusion and errors (Chappell et al. 1997).

An aircraft’s autopilot system generally divides its operation into vertical and horizontal components. Flying a preprogrammed route utilizing lateral navigation (LNAV) mode or flying a chosen heading are examples of horizontal guidance modes. The horizontal axis movement is determined by the aircraft’s banking. Flying level hold (Altitude Hold), maintaining a selected vertical speed (V/S), or a preprogrammed path utilizing a VNAV mode are examples of typical vertical automation modes. Along with the aircraft’s speed, the vertical flight path is controlled by a combination of engine thrust and elevator actuation (Vakil et al. 1995).

According to Vakil et al. (1995), a mode is a distinct state of the aircraft’s automation at a particular time. A mode is thus defined as a collection of attributes (heading, speed, V/S, pitch, and altitude) required to control the aircraft, as well as the actuators used to perform it (aileron, elevator, engine thrust).

Transitions among modes are categorized as commanded, uncommanded, or automatic conditional. A commanded transition happens when the automation mode is initiated immediately upon the pilot’s selection. For instance, pushing the V/S button immediately activates this mode. Uncommanded transitions are those that occur without direct pilot engagement; these transitions are often used to preserve the aircraft’s envelope. A transition triggered by overspeed protection is an example. Automatic conditional transitions happen when a flight condition is met after arming the autopilot. For instance, when the airplane crosses the glide slope trajectory, Glide Slope Capture automatically switches to approach mode (Vakil et al. 1995).

Automatic flight systems alternate among two basic operating modes known as base-mode and macro-mode. Base-modes, also known as basic modes, are used under stable conditions and have a constant set of characteristics and objectives. The V/S mode is an example of a base-mode, in which the aircraft maintains a V/S by controlling the pitch and air speed with engine thrust (Vakil et al. 1995).

According to Vakil et al. (1995), macro-modes, on the other hand, are an associated sequence of base-modes. Each macro-mode base-mode has its own set of characteristics, implying that they vary across the macro-mode. The Autoland sequence exemplifies the macro-mode, as there is a transition between the base-modes during landing: Flight Level Change, Glideslope Capture, Flare, and Rollout. As a result, each base-mode has a unique set of characteristics and objectives.

Automation Mode Error, Confusion, and Awareness

Mode confusion is a form of automation surprise that arises whenever the flight crew assumes the automation is operating in a different mode than it actually is, and thus reacts incorrectly. Mode confusion can also emerge when crews are unfamiliar with the behavior of automation in specific modes, as well as their interactions and transitions, i.e., when the crew has a different mental model than the aircraft’s actual automation (Miller et al. 2002).

According to Kanazaki and Hori (2003), mode confusion also happens when the crew is unable to understand what action the autopilot system is performing just by reading the shortened letters displaying the airplane’s modes on the panel. As a result, pilots are frequently confused and unable to comprehend the what, why, and what follows next of the autopilot modes. As a result, mode confusion is a contributing factor in a significant number of aviation accidents and incidents.

These events and fatalities indicate that the confusion is a recurring issue, and aircrew might be perplexed by what the cockpit automation is doing. Accident statistics show that the challenge of mode confusion extends far beyond inadequate training and a lack of ergonomics in airplanes (Butler et al. 1998). Human factors experts are aware of this phenomenon. According to Charles E. Billings (1996, p. 63):

[...] today’s flight management systems are mode rich, and it is often difficult for pilots to control them... The second problem involves pilots’ lack of understanding of the architecture and internal logic of the system, and therefore what the machine is doing, why, and what it will do next.

Mode confusion additionally contributes to the occurrence of mode errors. Norman (1988, p. 179) addresses errors by explaining that one approach to enhance the possibility of error is to “... change the rules. When something is done one way in a certain mode and another way in another.” Mode errors arise when a command for one mode is issued when the system is in another automation mode. A mode error can also relate to the failure to perform a required action or intervention in automated operations.

In basic automation equipment, most system actions are controlled by the user. Consequently, the pilot needs to act for an error to happen. On the other hand, in more sophisticated systems, each mode is a self-contained automated function that, once triggered, may perform extensive sequences of tasks without the intervention of human pilots (Sarter et al. 1997).

This development in automation results in scenarios where mode changes can occur indirectly, without explicit orders from the crew. This circumstance indicates a growing need for automation mode awareness (Sarter et al. 1997). “Mode awareness is the ability of a supervisor to monitor and anticipate the behavior of automated systems” (Sarter and Woods 1994). Mode awareness failures might result in automation surprises and mistakes, limiting operators’ capacity to notice and react in uncontrolled or unwanted system activities.

Advanced technology’s versatility enables automation engineers to create more complicated, mode-rich systems. As aeronautical engineers added various levels of automation and techniques for executing flying operations, modes multiplied. Because of this phenomenon, many indicators of the automated system’s condition and behavior are generated and transmitted by several flight monitors. Not only the number of modes has grown over time, but also the complexity of their interactions (Sarter et al. 1997).

As automation autonomy grows, the time delay between the pilot’s command and feedback on the system’s behavior increases. These extended feedback loops intensify the difficulty of detecting and recovering from mistakes and create a demand on the human capacity to retain situational awareness of active modes and its interactions.

Therefore, pilots are limited by the interface between the cockpit display and the crew, as well as the mental models they have built up during their years of training and operating experience, and particular challenges, such as unknown situations, time pressure, and changes in plans. This makes it harder for them to follow and predict the actions of automated systems (Huettig et al. 1999).

Monitoring mode annunciators on the aircraft panel and communicating such changes, known as callouts, are recognized as essential for establishing and preserving mode awareness in airplane cabins. The Flight Mode Annunciator (FMA) is positioned at the top of the Primary Flight Display (PFD), and its purpose is to display mode changes and the current mode in which the airplane is flying. It shows a range of two to four-letter codes (for example, V/S, VNAV) that give information about automated activity (Björklund et al. 2006). The representation of an FMA can be seen in Fig. 1.

Figure 1
Representation of the PFD and FMA.

Mumaw et al. (2001) revealed in their study that flight crews do not pay much attention to the FMA (the usual fixation time is less than 5%), and that the pilots generally do not value what the FMA is showing, despite manufacturers’ assertions that it is the only trustable and expected source of automation mode information.

Similarly, Polson and Javaux (2001) show that most of these monitoring errors are not the result of deliberate infractions of company policies. When the crew engage an automation mode, additional data from other displays and the airplane’s behavior, in addition to the FMA, indicate successful mode activation.

The PFD and thrust indicator changes, as well as throttle inputs, suggest the mode change and the beginning of the maneuver. Empirical learning mechanisms enable flight crews to retain highly reliable representations of the multisensory patterns and cues associated with the engagement of a mode. These sensory patterns are used in conjunction with FMA to confirm the mode engagement.

Automatic Flight System And Flight Management Systems (FMS)

Pilots can select from a variety of automation modes established by the FMS during autonomous flight. The Control Display Unit (CDU) and Mode Control Panel (MCP) are the primary interfaces between the flight crew and the FMS (Duan et al. 2015).

One of the FMS’s primary goals is to enable three-dimensional navigation in the airspace. Route navigation is classified into two control modes in FMS: LNAV and VNAV. The LNAV is in charge of the aircraft’s horizontal movement in space, while VNAV maintains control of both altitude and speed. The crew configures the FMS by selecting a predetermined route or a collection of spatial coordinates (waypoints) that describe the intended flight path (Chappell et al. 1997).

A monitor and a multipurpose keyboard comprise the CDU. The pilot programs the flight path, the departure and destination airports, waypoints, altitude, and speed limitations using that keyboard and a set of function keys. Once the autopilot, LNAV, and VNAV modes are activated, the flight management system then manages the aircraft to fly between points while following the defined parameters (Chappell et al. 1997).

The MCP is used to choose and engage the autopilot, automation modes, and flight director, whereas the CDU is utilized to configure the airplane’s flight plan and performance parameters. There are three types of automation modes: autothrottle, banking mode (roll mode), and pitch mode. Autothrottle mode regulates engine power configurations to sustain a selected or required speed by the pilot or Flight Management Computer (FMC) (Duan et al. 2015). Figure 2 illustrates the MCP.

Figure 2
Representation of the Boeing 737 MCP.

Vertical movement is guided by pitch modes, whereas horizontal movement is guided by banking (roll) modes. Many of these modes are enabled through a single button push on the MCP; for instance, on Boeing 737, there are the Altitude Hold (ALT HLD), V/S, Heading Mode Select (HDG SEL), and Level Change modes (LVL CHG). Certain modes, on the other hand, can be armed with a simple button click but cannot be activated until a set of flight circumstances is met (e.g., flight path intercept or Instrument Landing System signal capture). LNAV, VNAV, Localizer (LOC), glideslope (G/S), approach (APP), and FLARE are all examples of these modes. The notification of a transition between modes, as well as a visual representation of the armed and engaged modes, are presented at the top of the PFDs in the FMA (Duan et al. 2015).

On autopilot there are numerous control modes available, each with a different amount of automation. The flight management system’s LNAV and VNAV features provide control at the highest level. These modes carry out the flight according to the flight plan entered the FMS through the CDU. Pilots utilize the MCP to choose lower degrees of automation by entering values to be followed by automation, including altitude, direction, and vertical and horizontal speed. HDG SEL and ALT HLD are examples of low-level automation modes: the first instructs the aircraft to fly on the defined heading, the second maintains the flight level. The lower automation levels may ignore the FMC-programmed flight plan for a short period. Until the crew updates these values or switches the automation mode, the airplane will preserve the settings set in the MCP (Chappell et al. 1997).

Vertical Automation Mode (VNAV)

Vakil et al. (1995) evaluated 300 reports in the Aviation Safety Report (ASR), a program of the US Federal Aviation Administration (FAA) that collects voluntary and anonymous reports from pilots and other aviation professionals with the objective of enhancing aviation safety. Of all the reports, 184 were classified as having difficulty with recognition or mode confusion. These complaints were then classified according to the believed source of the problem and the affected flight path (horizontal, vertical/velocity, or both).

A summary of the causes of the analyzed incidents is found in Fig. 3.

Figure 3
Perceived causes and flight path.

Vertical trajectory/velocity issues are noted to predominate in many categories. A total 62.7% of the classified reports have been concerned with the vertical flight path in conjunction with velocity. Vertical/velocity issues, especially, dominate the category of Mode Transition Problems. Additionally, results from the Insufficient Understanding of Automation category indicate a lack of awareness of automation in the vertical flight sector (Vakil et al. 1995).

VNAV (also known as PROF on Airbus aircraft) is therefore a mode that is used very often by pilots and is more subject to mode confusion. This vertical mode is divided into three submodes, and the transition between them is controlled by a sophisticated transition logic which is not clearly described or mentioned in flight manuals. There are inconsistencies between the pilots’ conceptualization of VNAV and their actual behavior. Sometimes, cockpit designers, training developers, and researchers are unaware of them as well (Duan et al. 2015).

A frequent source of confusion among pilots about vertical flight paths is the condition of VNAV Speed Reversion. According to Duan and Haag (2015), this occurrence happens when an unplanned and unknown wind (or an incorrect estimate) leads the vertical automation mode to switch abruptly from VNAV PTH to VNAV SPD, leading the airplane to diverge out of its intended path as specified in the FMS. After taking into consideration the wind prediction along the trip, the VNAV’s trajectory is plotted. Thus, if the intensity or direction of the expected wind differs from the real wind, the VNAV trajectory may be too short or too steep for the proper execution of the flight profile.

When in an unexpected tailwind, the airplane could accelerate to a specific speed 11 knots under the maximum operating speed VMO or 15 knots above the designed FMS speed, depending on whether the airplane is above or below the transition altitude to attempt to maintain its path, and the VNAV accepts a fluctuation of up to 150 feet above trajectory. Without further acceleration, if the VNAV is unable to sustain the 150-feet deviation from the optimum trajectory, the VNAV changes from VNAV PTH to VNAV SPD, overriding the FMS airspeed and disregarding the prior VNAV flight path.

Likewise, when an unplanned headwind occurs and the airplane is not utilizing the autothrottle, the aircraft will seek to fly below the optimal flying path to avoid wind deceleration. The VNAV then is changed from VNAV PTH to VNAV SPD if the VNAV is unable to keep the aircraft within 150 feet of its ideal trajectory without further deceleration (Duan and Haag 2015).

Whenever a VNAV speed reversion takes place, the crew’s workload increases significantly, as the pilots are required to restore the airplane to its optimal vertical trajectory following a sequence of actions to stabilize the speed. This condition may result in altitude restriction violations and is typically linked with the last phases of flight, descent, approach, and landing. Thus, a go-around is a frequent speed reversion-related consequence, since an altitude violation generally results in an unstabilized approach (Duan and Haag 2015).

The speed reversion frequently leads to mode confusion; Duan and Haag (2015) attribute this to the inadequate layout of the VNAV human-machine interface and the cockpit’s absence of a predictive warning system. The CDU displays the phrases DRAG REQUIRED and THRUST REQUIRED to inform the aircrew of the VNAV mode reversal condition. However, these warnings are not explicitly communicated to the pilots and are reactive in nature. Thereby, the flight crew’s workload is inherently elevated, and their period to react to abnormal conditions is reduced, as a result, the possibility of an accident increases.

METHODOLOGY

A survey-style questionnaire was developed to explore the different automation mode confusions encountered by Brazilian pilots. It was made available through the Google Forms platform and has 19 questions, including 17 multiple-choice and 2 optional descriptive questions. The questions are based on the goal to identify the main types of automation mode confusion that occur among pilots, the flight phases when they are most likely to occur, and the pilot’s experience with this type of phenomenon. The questions are similar to the survey used in the research conducted by De Boer and Dekker (2017) and were validated by a doctor from the Brazilian Aeronautical Technological Institute and two Boeing 737 captains.

The pilots’ participation was voluntary, completely anonymous, and convoked through the website and social networks of the Association of Aeronauts of Gol (ASAGOL) and message groups for airline crew members. The survey was answered by 145 pilots.

The questionnaire was divided into three parts: information about the pilot, details about the occurrences with mode confusion, and the pilots’ perception of the measures to mitigate mode confusion.

RESULTS AND DISCUSSION

Characteristics of the Participants

The survey was made available in June 2021 through the Google Forms platform, and in total, there were responses submitted by 145 aircraft pilots. Respondents’ age ranges from 18 to over 60 years, with an average of 41.5 years. Captains are 53.8% (78) of the participants, while Copilots are 46.2% (67). Regarding the type of aircraft currently operated, pilots cite (in order of frequency): Boeing 737 (81%), Airbus A319, A320, A321 (8%), Embraer 170/175/190/195 (5%), Airbus A330 (2%), Boeing 747 (1%), Boeing 777 (1%), Phenom 100 (1%), BE9L (1%) and King Air 250 (1%). The experience of participants ranges from 200 flight hours to more than 15,000 flight hours, with an average of 7,547 hours of flight time. Data on the type of aircraft flown can be observed in Fig. 4.

Figure 4
Type of aircraft operated by pilots.

Incidence of Automation Mode Confusion

The questionnaire includes two measures to assess the frequency of occurrence of the phenomenon of mode confusion. First, it is asked whether the pilot has ever experienced confusion in automation mode in his career, using a five-point frequency Likert scale, ranging from never to extremely frequent. Of the total responses, 11 (8%) pilots reported never having experienced the situation of confusion, 105 (72%) had experienced the phenomenon rarely, 28 (19%) occasionally and 1 (1%) said it was extremely frequent.

The answers are separated into three categories according to the pilots’ experience: inexperienced pilots (200–4,000 flight hours), experienced pilots (4,000–12,000 flight hours), and very experienced pilots (more than 12,000 flight hours). They are also segmented in relation to the manufacturer of the operated aircraft: Embraer, Airbus, and Boeing. This breakdown aims to identify trends or discrepancies between the groups.

Figure 5 shows that pilots of planes manufactured by Embraer and Airbus present a higher frequency of negative responses to the experience with mode confusion (“No, never”) when compared to pilots of planes manufactured by Boeing. The responses rarely and occasionally are more frequent in Boeing pilots (74% and 20%) when compared to Airbus (73% and 13%) and Embraer (57% and 14%), respectively. This result may suggest that planes manufactured by Boeing are more susceptible to generating automation mode confusion in pilots.

Figure 5 also shows a higher frequency of negative responses to the experience with mode confusion (“No, never”) in more experienced pilots. The “Yes, occasionally” option presents greater adherence in less experienced participants. This result reveals that automation mode confusion occurs or is perceived more frequently by pilots with fewer flight hours in the operated aircraft.

Figure 5
Incidence of automation mode confusion.

Similar to the first question, the participant was asked the number of occurrences of automation mode confusion they had in the last operation year. Responses vary in intervals from 0 to more than 20 events in the same year and are shown in Fig. 6. Of the total responses, 48 (33%) pilots report 0 episodes of mode confusion in the last year of operations, 75 (52%) report from 1 to 3 occurrences, 17 (12%) from 4 to 6, 3 (2.1%) from 6 to 10, 1 (1%) from 10 to 15, 0 (0%) from 15 to 20 and 1 (1%) say there were more than 20 cases in the year. The average number of automation mode confusion episodes in a year of operations is 2.01 ± 0.40 occurrences, with a confidence level of 95%. occurrences.

Figure 6
Frequency of automation mode confusion.

The answers are also separated according to the experience of the pilots and according to the aircraft manufacturer. It can be seen from the analysis of Fig. 6, that 60% of the pilots flying planes manufactured by Airbus cite zero cases of confusion in one year, a much higher percentage than the pilots of Embraer (43%) and Boeing aircraft (29%). The average number of cases for the manufacturers Embraer, Boeing and Airbus is 3.07 ± 3.11, 2.04 ± 0.43 e and 1.2 ± 0.87 cases per year, with a confidence level of 95%. This result would indicate that Embraer pilots are the most susceptible to experiencing the phenomenon of mode confusion per year and those of the manufacturer Airbus the least susceptible. However, the evidence is inconclusive due to the low number of respondents who fly Airbus (12) and Embraer (7), when compared to the sample of pilots who operate Boeing aircraft (117). So, any variation in the answer can generate a large percentage in the figure, not clearly representing the behavior of the population.

Some atypical data were observed in the data collection. Among the 145 respondents, only 1 pilot responded that it was extremely frequent and reported 10 to 15 occurrences in a year of operations. Similarly, another pilot responded that he rarely witnessed the phenomenon, but reported more than 20 occurrences in a year. It is not possible to state whether these two abnormal responses are real or due to errors in interpretation or marking, since the survey is anonymous and respondents did not use the descriptive response space.

In terms of the number of cases of confusion based on pilot experience, more experienced pilots (53%) have a higher number of responses of none occurrences per year than experienced (39%) and less experienced (12%) pilots. This result reveals that automation mode confusion occurs more frequently in a year of operations for pilots with fewer flight hours.

These results corroborate the research by De Boer and Dekker (2017), whose conclusion suggests that less experienced pilots are more susceptible to automation surprises and consequently observe these phenomena more frequently.

Phases and Flight Path

Participants were asked which flight phase they believe is more prone to automation mode confusion. The possible answers cover all the usual stages of a flight: Ground Operations and Taxi, Take-off, Climb, Cruise, Descent, Holding, Approach, Go-around and Landing. Figure 7 summarizes the results found.

Figure 7
Flight phases most susceptible to mode confusion.

Most responses focus on the final stages of flight, approach (36%), go-around (23%) and descent (22%), stages in which there is a greater concentration of functions and workload is high. Factors such as flight fatigue and complexity of operations also contribute to the incidents. The phases of ground operations, taxi and cruise flight are not mentioned by any pilot. Meanwhile, the climb, landing and take-off are pointed out by 6%, 5%, and 3% of the participants, respectively.

Pilots were also asked which flight path has the most complex modes and which are more susceptible to confusion of automation mode, the results are seen in Fig. 8.

Figure 8
Flight paths most susceptible to mode confusion.

Most pilots (129, 89%) choose the vertical trajectory coupled with velocity as the trajectory that has the most complex modes and is more subject to mode confusion. These research results are consistent with the findings of Vakil et al. (1995), whose study shows that more than 85% of the problems associated with mode transition reported in ASRs are relative to the vertical flight path coupled with speed. It appears, therefore, that the prevalence of the phenomenon of mode confusion occurs in the final phases of flight, approach, go-around and descent and in the vertical flight path.

Automation Modes

In order to answer the research question, pilots were asked, “What are the most complex automation modes to understand?” All the Boeing 737-800 automation modes and some specific Airbus aircraft modes are listed for the pilots to choose from; the option none and the option to write in information manually were also available. More than one answer was possible. Automation modes and responses are available in Fig. 9.

It is observed that the three most cited modes are related to the vertical flight path and are submodes of the VNAV. The VNAV SPD mode is the most cited, with 47 (21%) responses, followed by VNAV PTH with 40 (18%) and VNAV ALT with 22 (10%). Indicating that these modes are more complex and difficult to understand. These responses confirm the thesis presented by Duan et al. (2015) who stated that the transitions between VNAV submodes are determined by complex logics and that they are not well documented in flight manuals.

The Alpha Floor mode, characteristic of Airbus aircraft, is the fourth most cited, with 21 (9%) responses. About 6% (13) of pilots reported that there is no mode in which they have difficulty understanding.

Figure 9
Automation modes that are more difficult to understand and which have already resulted in a mode confusion

Similarly, pilots were asked in which modes they have experienced a mode confusion or a surprise. The modes available for selection were the same as in the previous question, the option none and the option to write in information manually were also available to choose from. The results can also be seen in Fig. 9.

The results are similar, the three modes with the most mode confusion are the VNAV submodes. The VNAV SPD mode is the most cited, with 63 (25%) responses, followed by VNAV PTH with 54 (22%) and VNAV ALT with 36 (14%).

These results suggest that the VNAV submodes are the most susceptible to mode confusion and can be considered more complex and less understood by pilots. This can be explained by the complex and poorly documented transition between them and the occurrence of VNAV Speed Reversion. According to Duan and Haag (2015), this phenomenon occurs when an unexpected and unknown wind (or a wrong estimate) causes the sudden change of the vertical automation mode from VNAV PTH to VNAV SPD, causing the aircraft to deviate from its reference trajectory.

Probable Causes and Consequences for the Flight

The results regarding the factors that lead to mode confusion, the consequences for the flight and how the phenomenon is usually discovered, are shown in Figs. 10–12, respectively.

Figure 10
Factors that lead to mode confusion.

It can be seen in Fig. 10 that conditions related to human factors are more prominent causes for the occurrence of mode confusion. The factors of high workload and lack of situational awareness of modes are the most cited, with 41 responses each, followed by factors followed by factors such as fatigue (31), lack of knowledge about modes (30) and system malfunction (27).

Figure 11 shows that, in general, the problem of mode confusion does not significantly affect the continuity of the flight, being quickly noticed, and corrected. Among the consequences generated by mode confusion are the correction of the vertical trajectory, cited by 40 pilots, nonstabilized approach, and consequent go-around (12), reversion to manual flight (4), correction of the lateral trajectory (4) and reprogramming of the FMS (3). There was an accident citation, but no information was given by the participant in the descriptive answer space. This result reinforces that vertical trajectory modes are among the most likely to generate confusion and can lead to an unstabilized or go-around approach, generating costs for the company and affecting operational safety.

Figure 11
Consequences for the flight.

Figure 12 reveals that most mode confusions are noticed by the pilots themselves (81% of cases) or are pointed out by the other crew member (17%). Alert systems only account for 1% of notifications and support or notifications from air traffic control is not a significant contributor for alerting automation mode confusion (0%).

This scenario suggests that the lack or lapse of mode consciousness is momentary, being identified by the pilot himself or by another crew member. Mode awareness is enhanced by monitoring the mode annunciators.

Figure 12
How mode confusion is discovered.

Security Perception and Mitigation

Mitigation measures are important to reduce the risk of mode confusion. It can be seen in Fig. 13 that about 35% of pilots consider FMA monitoring and callouts as the best mitigating measure for mode confusion, reinforcing the work of Björklund et al. (2006), whose study states that monitoring mode annunciators is the main tool for maintaining mode awareness.

Again, it is observed that the human factor is more prominent in the cause of mode confusion, 86% of the measures cited are related to the improvement of human actions, in contrast to 14% of responses related to the improvement of equipment.

Figure 13
Mitigating measures for mode confusion.

CONCLUSION

Aviation automation has been developed as an aid to the human factor inside the cabin to reduce workload, increase accuracy and reduce error. However, its use can cause some side effects such as complacency, automation surprises and automation mode confusion.

The bibliography on the subject was explored to expose key concepts regarding automation, automation surprise, mode confusion, aircraft modes and the use of FMA to mitigate associated risks.

It is observed that automation mode confusion is a recurring problem in Boeing 737 pilots. The identification of factors such as the frequency in which they occur, the phases of flight, the modes, causes, consequences, and mitigating measures are relevant for the correct management and correction of the problem.

In order to identify these elements and answer the research question about which mode confusions are usually prevalent and recurrent in Brazilian pilots, a survey-style questionnaire research study was developed, distributed to Brazilian airline pilots, and an analysis was conducted based on the collected data.

The survey results reveal that the average frequency of mode confusion is 2.01 ± 0.40 occurrences per year and less experienced pilots are more likely to witness the phenomenon. It is verified that the prevalence of the mode confusion phenomenon occurs in the final phases of flight, approach, go-around, and descent and in the vertical flight path, with the VNAV modes and its sub-modes being the most complex and confusing for pilots.

The research also indicates that the top four causes of mode confusion perceived by pilots are related to human factors, being high workload, lack of mode awareness, fatigue, and lack of mode knowledge. In a similar way, pilots perceive that the most important mitigating measures are related to the improvement of the human link and reinforce the relevance of FMA monitoring and callouts.

The mode confusion is most often noticed and resolved quickly, which causes minimal consequences for the flight. However, when they are not promptly corrected, they end up generating vertical trajectory corrections and unstabilized approaches with go-around, producing extra costs for companies and possibly compromising operational safety.

It should be noted that the data collected in the research is important for understanding the interaction between the crew and the automation modes in the national scenario. They can, therefore, serve as a basis for future studies on mode confusions and for airlines and manufacturers to direct their efforts towards more complex and less understood modes for mitigating the associated risks.

Since the survey was administered to groups and associations of pilots that operate the Boeing 737 aircraft, little input was obtained from pilots of other aircraft, such as those manufactured by Airbus and Embraer. Therefore, although the data gathered from these pilots is significant for the research, it was not possible to draw conclusions regarding the experience with mode confusion segregated by aircraft type.

Some suggestions for future research are listed as follows: More extensive research on mode confusion, with larger samples of pilots who operate aircraft from other manufacturers; conduction of field research in aircraft cabins, to monitor the correct performance of callouts in the FMA; initiate research analyzing the startle effect in mode confusion situations.

ACKNOWLEDGEMENTS

Not applicable.

• Peer Review History: Single-Blind Peer Review.

• DATA AVAILABILITY STATEMENT

  The data will be available upon request.

• FUNDING

  Not applicable.

REFERENCES

• Billings CE (1996) Human-centered aviation automation: Principles and guidelines. California: NASA.
• Björklund CM, Alfredson J, Dekker SWA (2006) Mode monitoring and call-outs: An eye-tracking study of two-crew automated flight deck operations. Int J Aviat Psychol 16:263-275.
• Boeing (2019) 737-700 Flight crew operations manual. Seattle: Boeing Commercial Airplane Group.
• Butler RW, Miller SP, Potts JN, Carreno VA (1998) A formal methods approach to the analysis of mode confusion. Paper presented 17th DASC. AIAA/IEEE/SAE. Digital Avionics Systems Conference. Proceedings, Bellevue, WA, USA. https://doi.org/10.1109/DASC.1998.741497
  » https://doi.org/10.1109/DASC.1998.741497
• Chappell AR, Crowther EG, Mitchell CM, Govindaraj T (1997) The VNAV Tutor: Addressing a mode awareness difficulty for pilots of glass cockpit aircraft. IEEE Trans Syst Man Cybern A Syst Hum 27(3):372-385. https://doi.org/10.1109/3468.568746
  » https://doi.org/10.1109/3468.568746
• De Boer RJ, Dekker SWA (2017) Models of automation surprise: Results of a field survey in aviation. Safety 3(3)20. https://doi.org/10.3390/safety3030020
  » https://doi.org/10.3390/safety3030020
• Duan P, Haag MU (2015) A multiple hypothesis predictive alerting (MHPA) method for improved aircraft state awareness. Paper presented 2015 IEEE/AIAA 34th Digital Avionics Systems Conference (DASC) p. 6A4-1-6A4-15.
• Duan P, Miltner M, Haag MU (2015) Improving mode awareness of the VNAV function with a Multiple Hypothesis Prediction method. Paper presented 2015 IEEE Aerospace Conference, Big Sky, MT, USA, p. 1-16. https://doi.org/10.1109/AERO.2015.7118881
  » https://doi.org/10.1109/AERO.2015.7118881
• [EASA] European Union Aviation Safety Agency (2013) EASA automation policy: Bridging design and training principles. Cologne: EASA.
• [FAA] Federal Aviation Administration (2013) Federal Aviation Administration report of the PARC/CAST flight deck automation WG. Operational use of flight path management systems. Washington: FAA.
• Huettig G, Anders G, Tautz A (1999) Mode awareness in a modern glass cockpit attention allocation to mode information. In: Jensen R, editor. Proceedings of the 1999 Ohio State University Aviation Psychology Conference. Dayton, OH: Ohio State University.
• Kanazaki H, Hori K (2003) Human interface reducing pilot’s confusion on autopilot system. SICE 2003 Annual Conference, Fukui, Japan, 2:1918-1923.
• Miller SP, Barber S, Carlson TM, Lempia DL, Tribble AC (2002) A methodology for improving mode awareness in flight guidance design. Paper presented Proceedings of The 21st Digital Avionics Systems Conference, Irvine, CA, USA, p. 11. https://doi.org/10.1109/DASC.2002.1052928
  » https://doi.org/10.1109/DASC.2002.1052928
• Mumaw RJ, Sarter NB, Wickens CD (2001) Analysis of pilots’ monitoring and performance on an automated flight deck. Paper presented 11th International Symposium on Aviation Psychology, Columbus, OH, USA, p. 1-7.
• National Transportation Safety Board (2014) Descent below visual glidepath and impact with seawall, Asiana Airlines Flight 214, Boeing 777-200ER, HL7742, San Francisco, California, July 6, 2013. Aircraft Accident Report NTSB/AAR-14/01. Washington, DC.
• Norman DA (1988) The psychology of everyday things. New York: Basic Books.
• Parasuraman R, Bahri T, Deaton JE, Morrison JG, Barnes M (1992) Theory and design of adaptive automation in aviation systems. Warminster, PA: Air Vehicle and Crew Systems Technology Department / Naval Air Warfare Center - Aircraft Division.
• Polson PG, Javaux D (2001) A model-based analysis of why pilots do not always look at the FMA. Paper presented Proceedings of the Eleventh Symposium on Aviation Psychology. Columbus, OH: Ohio State University.
• Rushby J, Crow J, Palmer E (1999) An automated method to detect potential mode confusions. Gateway to the New Millennium. Paper presented 18th Digital Avionics Systems Conference. Proceedings, St. Louis, MO, USA, p. 6, Oct. 29, 1999. https://doi.org/10.1109/DASC.1999.863725
  » https://doi.org/10.1109/DASC.1999.863725
• Sarter NB, Woods DD, Billings C (1997) Automation surprises. In: Handbook of human factors & ergonomics. 2nd ed. Wiley: New York.
• Sarter NB, Woods DD (1994) Pilot interaction with cockpit automation II: An experimental study of pilots’ model and awareness of the flight management and guidance system. Int J Aviat Psychol 4(1):1-28. https://doi.org/10.1207/s15327108ijap0401_1
  » https://doi.org/10.1207/s15327108ijap0401_1
• Vakil SS, Hansman Jr RJ, Midkiff AHM, Vaneck TV (1995) Mode awareness in advanced autoflight systems. Paper presented 6th IFAC Man-Machine Systems, Cambridge, Massachusetts, USA, p. 6, January 1, 1995.

Edited by

Section Editor: Joana Ribeiro

Publication Dates

• Publication in this collection
  28 Nov 2022
• Date of issue
  2022

History

• Received
  30 May 2022
• Accepted
  13 Sept 2022