automation bias automation bias automation bias is the propensity for humans to favor suggestions from automated decision-making systems and to ignore contradictory information made without automation, even if it is correct. automation bias stems from the social psychology literature that found a bias in human-human interaction that showed that people assign more positive evaluations to decisions made by humans than to a neutral object. the same type of positivity bias has been found for human-automation interaction, where the automated decisions are rated more positively than neutral. this has become a growing problem for decision making as intensive care units, nuclear power plants, and aircraft cockpits have increasingly integrated computerized system monitors and decision aids to mostly factor out possible human error. errors of automation bias tend to occur when decision-making is dependent on computers or other automated aids and the human is in an observatory role but able to make decisions. examples of automation bias range from urgent matters like flying a plane on automatic pilot to such mundane matters as the use of spell-checking programs. part of a series onautomation automation in general banking building home highway system laboratory library broadcast console pool cleaner pop music reasoning semi-automation telephone attendant switchboard teller machine vehicular vending machine robotics and robots autonomous research domestic guided vehicle industrial lawn mower paint impact of automation manumation ool bias self-driving cars technological unemployment jobless recovery post-work society threat trade shows and awards asp-dac dac date ieee robotics and automation award iccad disuse and misuse an operator's trust in the system can also lead to different interactions with the system, including system use, misuse, disuse, and abuse. the tendency toward overreliance on automated aids is known as "automation misuse". misuse of automation can be seen when a user fails to properly monitor an automated system, or when the automated system is used when it should not be. this is in contrast to disuse, where the user does not properly utilize the automation either by turning it off or ignoring it. both misuse and disuse can be problematic, but automation bias is directly related to misuse of the automation through either too much trust in the abilities of the system, or defaulting to using heuristics. misuse can lead to lack of monitoring of the automated system or blind agreement with an automation suggestion, categorized by two types of errors, errors of omission and errors of commission, respectively. automation use and disuse can also influence stages of information processing: information acquisition, information analysis, decision making and action selection, and action implementation. for example, information acquisition, the first step in information processing, is the process by which a user registers input via the senses. an automated engine gauge might assist the user with information acquisition through simple interface features—such as highlighting changes in the engine's performance—thereby directing the user's selective attention. when faced with issues originating from an aircraft, pilots may tend to overtrust an aircraft's engine gauges, losing sight of other possible malfunctions not related to the engine. this attitude is a form of automation complacency and misuse. if, however, the pilot devotes time to interpret the engine gauge, and manipulate the aircraft accordingly, only to discover that the flight turbulence has not changed, the pilot may be inclined to ignore future error recommendations conveyed by an engine gauge—a form of automation complacency leading to disuse. errors of commission and omission automation bias can take the form of commission errors, which occur when users follow an automated directive without taking into account other sources of information. conversely, omission errors occur when automated devices fail to detect or indicate problems and the user does not notice because they are not properly monitoring the system. errors of omission have been shown to result from cognitive vigilance decrements, while errors of commission result from a combination of a failure to take information into account and an excessive trust in the reliability of automated aids. errors of commission occur for three reasons: (1) overt redirection of attention away from the automated aid; (2) diminished attention to the aid; (3) active discounting of information that counters the aid's recommendations. omission errors occur when the human decision-maker fails to notice an automation failure, either due to low vigilance or overtrust in the system. for example, a spell-checking program incorrectly marking a word as misspelled and suggesting an alternative would be an error of commission, and a spell-checking program failing to notice a misspelled word would be an error of omission. in these cases, automation bias could be observed by a user accepting the alternative word without consulting a dictionary, or a user not noticing the incorrectly misspelled word and assuming all the words are correct without reviewing the words. training that focused on the reduction of automation bias and related problems has been shown to lower the rate of commission errors, but not of omission errors. factors the presence of automatic aids, as one source puts it, "diminishes the likelihood that decision makers will either make the cognitive effort to seek other diagnostic information or process all available information in cognitively complex ways." it also renders users more likely to conclude their assessment of a situation too hastily after being prompted by an automatic aid to take a specific course of action. according to one source, there are three main factors that lead to automation bias. first, the human tendency to choose the least cognitive approach to decision-making, which is called the cognitive miser hypothesis. second, the tendency of humans to view automated aids as having an analytical ability superior to their own. third, the tendency of humans to reduce their own effort when sharing tasks, either with another person or with an automated aid. other factors leading to an over-reliance on automation and thus to automation bias include inexperience in a task (though inexperienced users tend to be most benefited by automated decision support systems), lack of confidence in one's own abilities, a lack of readily available alternative information, or desire to save time and effort on complex tasks or high workloads. it has been shown that people who have greater confidence in their own decision-making abilities tend to be less reliant on external automated support, while those with more trust in decision support systems (dss) were more dependent upon it. screen design one study, published in the journal of the american medical informatics association, found that the position and prominence of advice on a screen can impact the likelihood of automation bias, with prominently displayed advice, correct or not, is more likely to be followed; another study, however, seemed to discount the importance of this factor. according to another study, a greater amount of on-screen detail can make users less "conservative" and thus increase the likelihood of automation bias. one study showed that making individuals accountable for their performance or the accuracy of their decisions reduced automation bias. availability "the availability of automated decision aids," states one study by linda skitka, "can sometimes feed into the general human tendency to travel the road of least cognitive effort." awareness of process one study also found that when users are made aware of the reasoning process employed by a decision support system, they are likely to adjust their reliance accordingly, thus reducing automation bias. team vs. individual the performance of jobs by crews instead of individuals acting alone does not necessarily eliminate automation bias. one study has shown that when automated devices failed to detect system irregularities, teams were no more successful than solo performers at responding to those irregularities. training training that focuses on automation bias in aviation has succeeded in reducing omission errors by student pilots. automation failure and "learned carelessness" it has been shown that automation failure is followed by a drop in operator trust, which in turn is succeeded by a slow recovery of trust. the decline in trust after an initial automation failure has been described as the first-failure effect. by the same token, if automated aids prove to be highly reliable over time, the result is likely to be a heightened level of automation bias. this is called "learned carelessness." provision of system confidence information in cases where system confidence information is provided to users, that information itself can become a factor in automation bias. external pressures studies have shown that the more external pressures are exerted on an individual's cognitive capacity, the more he or she may rely on external support. definitional problems although automation bias has been the subject of many studies, there continue to be complaints that automation bias remains ill-defined and that reporting of incidents involving automation bias is unsystematic. a review of various automation bias studies categorized the different types of tasks where automated aids were used as well as what function the automated aids served. tasks where automated aids were used were categorized as monitoring tasks, diagnosis tasks, or treatment tasks. types of automated assistance were listed as alerting automation, which track important changes and alert the user, decision support automation, which may provide a diagnosis or recommendation, or implementation automation, where the automated aid performs a specified task. automation-induced complacency the concept of automation bias is viewed as overlapping with automation-induced complacency, also known more simply as automation complacency. like automation bias, it is a consequence of the misuse of automation and involves problems of attention. while automation bias involves a tendency to trust decision-support systems, automation complacency involves insufficient attention to and monitoring of automation output, usually because that output is viewed as reliable. "although the concepts of complacency and automation bias have been discussed separately as if they were independent," writes one expert, "they share several commonalities, suggesting they reflect different aspects of the same kind of automation misuse." it has been proposed, indeed, that the concepts of complacency and automation bias be combined into a single "integrative concept" because these two concepts "might represent different manifestations of overlapping automation-induced phenomena" and because "automation-induced complacency and automation bias represent closely linked theoretical concepts that show considerable overlap with respect to the underlying processes." automation complacency has been defined as "poorer detection of system malfunctions under automation compared with under manual control." nasa's aviation safety reporting system (asrs) defines complacency as "self-satisfaction that may result in non-vigilance based on an unjustified assumption of satisfactory system state." several studies have indicated that it occurs most often when operators are engaged in both manual and automated tasks at the same time. in turn, the operators' perceptions of the automated system's reliability can influence the way in which the operator interacts with the system. endsley (2017) describes how high system reliability can lead users to disengage from monitoring systems, thereby increasing monitoring errors, decreasing situational awareness, and interfering with an operator's ability to re-assume control of the system in the event performance limitations have been exceeded. this complacency can be sharply reduced when automation reliability varies over time instead of remaining constant, but is not reduced by experience and practice. both expert and inexpert participants can exhibit automation bias as well as automation complacency. neither of these problems can be easily overcome by training. the term "automation complacency" was first used in connection with aviation accidents or incidents in which pilots, air-traffic controllers, or other workers failed to check systems sufficiently, assuming that everything was fine when, in reality, an accident was about to occur. operator complacency, whether or not automation-related, has long been recognized as a leading factor in air accidents. as such, perceptions of reliability, in general, can result in a form of automation irony, in which more automation can decrease cognitive workload but increase the opportunity for monitoring errors. in contrast, low automation can increase workload but decrease the opportunity for monitoring errors. take, for example, a pilot flying through inclement weather, in which continuous thunder interferes with the pilot's ability to understand information transmitted by an air traffic controller (atc). despite how much effort is allocated to understanding information transmitted by atc, the pilot's performance is limited by the source of information needed for the task. the pilot therefore has to rely on automated gauges in the cockpit to understand flight path information. if the pilot perceives the automated gauges to be highly reliable, the amount of effort needed to understand atc and automated gauges may